Distributed system increment testing method and device guided by model inspection

Abstract

The invention discloses a distributed system incremental testing method and device guided by model inspection, and belongs to the technical field of software. The method comprises the following steps: firstly, carrying out static analysis on a TLA + form protocol and a system implementation code of a distributed system, and identifying changes generated in an evolution process; then, according to a predefined incremental test mode, identifying affected nodes and edges in the new state diagram; then, applying an increment-based test case generation algorithm to generate a test case covering affected state transition; and finally, executing the test case, controlling the execution sequence of the system, and verifying the consistency between the execution sequence and the expected state and conversion. According to the method, only the changed part in the evolutionary process of the distributed system is tested in an incremental test mode, so that comprehensive test on the state space of the whole system is avoided, and the test efficiency and accuracy are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of software, and particularly relates to a method and device for incremental testing of a distributed system guided by model checking. Background Art

[0002] In the era of big data and cloud computing, distributed systems have become the cornerstone of many important applications. Distributed systems represented by the distributed storage system HDFS, the cluster management service YARN, the distributed computing framework MapReduce, the distributed coordination service ZooKeeper, etc. are widely deployed and applied in large Internet companies. These systems must correctly handle various non-deterministic events, such as user requests, network messages, and external failures, which makes the design and implementation of distributed systems extremely complex. Due to their complexity, the complex errors that may occur in distributed systems are not only difficult to discover, but may also cause serious consequences under specific event sequences, such as service interruption, resulting in huge economic losses.

[0003] Software testing is an important technical means to discover errors in distributed systems. However, the testing of distributed systems faces multiple challenges. First, the huge state space generated by non-deterministic events makes it difficult to generate various test inputs and systematically explore the entire state space. Second, for a large number of system states, there is currently a lack of a general solution to determine the correctness of each state, that is, to establish test oracles. In addition, controlling non-determinism during the testing process to guide the system under test (SUT) towards potential error states also faces major challenges.

[0004] To address the above problems, various model checking guided testing (MCGT) methods have been proposed in recent years, such as Mocket (Dong Wang et al., "Model checking guided testing for distributed systems", Proc. EuroSys 2023.) and SandTable (Ruize Tang et al., "SandTable: Scalable distributed system model checking with specification level state exploration", Proc. EuroSys 2024.), to systematically test distributed systems. These methods automatically generate test cases by traversing the abstract state space generated from the formal specification of the distributed system and check whether the behavior of the target system is correct during the testing process. For example, the Mocket method uses the TLA+ formal language to model the high-level specification of the SUT and utilizes the TLC model checker to verify the TLA+ specification, obtaining a verification state graph containing all possible system states and behaviors. Then, Mocket traverses this state graph to automatically generate test cases and controls the execution order of all modeled system events during the testing process to ensure its consistency with the test cases.

[0005] Although MCGT methods have achieved certain results in distributed system testing, there are still many problems in practical applications. First, the testing cost of MCGT methods is relatively high, and it usually takes several weeks to complete the testing of a distributed system. For example, using Mocket to test ZooKeeper may take more than three weeks. In addition, when the distributed system changes, such as introducing new features or fixing bugs, it is necessary to regenerate test cases and re-run the entire testing process, which makes MCGT methods particularly inefficient during the system evolution process.

[0006] In addition, the existing MCGT methods still have limitations in dealing with the non-determinism and uncontrollability of distributed systems. For example, the Mocket method mainly relies on deterministic control of system states, and non-deterministic events and uncontrollable system behaviors in distributed systems may lead to inaccurate test results. In addition, the existing MCGT methods usually require annotation and mapping of the system implementation, which not only increases the complexity of testing but also may introduce additional errors.

[0007] In summary, although the existing MCGT method provides a systematic testing means in the testing of distributed systems, it faces significant problems during the evolution of distributed systems. First, whenever the distributed system changes, such as introducing new functions or fixing bugs, the MCGT method needs to regenerate test cases and re-run the entire testing process. This process is not only time-consuming and resource-intensive, but also severely limits the testing efficiency and response speed in the case of frequent system updates. Second, the existing MCGT methods lack flexibility and pertinence in dealing with incremental changes during system evolution, and are unable to effectively identify and test those states and state transitions that are only affected by the changes, resulting in a large amount of redundant testing and resource waste. Therefore, the existing MCGT methods still have certain limitations during the evolution of distributed systems. Summary of the Invention

[0008] The object of the present invention is to provide an incremental testing method for distributed systems guided by model checking to address the problems of high testing cost and low efficiency of the MCGT method during the evolution of distributed systems. This method extracts changes from formal specifications and system implementations, identifies the affected states in the abstract state space, generates incremental test cases to cover these states, and executes tests to verify system changes. This method can efficiently generate and execute test cases during the evolution of distributed systems, significantly reducing testing costs and time.

[0009] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] An incremental testing method for distributed systems guided by model checking, comprising the following steps:

[0011] 1) Perform static analysis on the TLA+ formal specification and system implementation code of the distributed system to identify modifications to variables, actions, and logical structures in the specification, as well as changes to mapped actions and adjustments to action concurrency relationships in the system implementation code; compare the original and modified specifications and code, record the addition, deletion, and modification of variables, basic blocks, and logical relationships, and generate change information;

[0012] 2) Based on predefined incremental testing patterns, analyze the change information to identify the affected state nodes and edges in the new state diagram; according to different types of changes, filter the states and state transitions to be tested, and generate a set of affected states and transitions;

[0013] 3) Based on the incremental test case generation algorithm, construct a test path starting from the affected states; extend the test path to first cover the affected edges until all affected transitions are covered, and generate test cases;

[0014] 4) Execute the test cases and perform the mapping actions in sequence according to the generated test paths; during the testing process, compare the system state with the expected state in real time, record the abnormal behaviors that deviate from the test cases, and generate a test report.

[0015] Furthermore, the method for identifying variable modifications in step 1) is as follows: Parse the variable declarations in the TLA+ specification, compare the original and modified specifications, identify the addition, deletion, and modification of variables, and analyze their impacts on the system state and transitions.

[0016] Furthermore, the method for identifying action modifications in step 1) is as follows: Analyze the actions in the TLA+ specification, identify the addition, deletion, and modification of basic blocks, record the changes in logical conditions and execution statements, and check their impacts on the system behavior.

[0017] Furthermore, the method for identifying logical structure modifications in step 1) is as follows: Analyze the initial state, concurrency relationships, and parameter adjustments of the specification, and evaluate the impacts of these modifications on the system execution order, consistency, and potential errors.

[0018] Furthermore, the ways to identify the affected state nodes and edges in the new state diagram in step 2) include the following 7 modes:

[0019] P1 mode: When a new basic block is added to the TLA+ specification, identify the actions associated with the new basic block, search for the relevant state transitions in the new state diagram, and mark the affected states and edges.

[0020] P2 mode: When a basic block is deleted from the TLA+ specification, identify the actions associated with the deleted basic block, search for the corresponding transitions in the original state diagram; check whether the transitions still exist in the new state diagram, and if not, mark the predecessor transitions as affected edges.

[0021] P3 mode: When a new variable is added to the TLA+ specification, combine P1 and P2 modes, test the relevant state transitions, and identify the state nodes affected by the new variable.

[0022] P4 mode: When a variable is deleted from the TLA+ specification, combine P1 and P2 modes, test the relevant state transitions, and identify the state nodes affected by the deleted variable.

[0023] P5 mode: When the mapping actions in the system implementation code are modified, identify the TLA+ variables involved in the modified action code, search for the relevant state transitions in the new state diagram, and mark the affected edges.

[0024] P6 mode: When the modified implementation introduces a new executable action sequence, identify the sequence, search for the state transition paths in the new state diagram that match the sequence, and mark the affected edges.

[0025] P7 mode: When the modified implementation blocks certain originally executable action sequences, identify the prohibited action sequences, search for relevant transitions in the new state diagram, and mark the affected edges.

[0026] Further, the steps of extending the test path in step 3) include: in the new state diagram, identify all edges marked as affected; select the edges that have not been covered by any test path as the current extension starting point; and select critical edges for extension according to the importance of the edges or the test coverage strategy.

[0027] Further, in step 3), the integrity of the test path is ensured by forward and backward traversals, where:

[0028] The forward traversal method is as follows: First, select the path containing the affected edge for extension. Starting from the end state of the affected edge, extend the path forward until reaching an end state or all successor nodes have been visited. If the current state has multiple successor states, select the unvisited successor node to continue traversing according to the priority strategy. If all successor nodes have been visited, randomly select one to continue extending. When reaching an end state or there is no path to extend, end the forward traversal.

[0029] The backward traversal method is as follows: First, select the path containing the affected edge for extension. Starting from the start state of the affected edge, trace back along the state diagram until reaching the initial state. If the current state has multiple predecessor states, select the unvisited predecessor node to continue traversing according to the priority strategy. If all predecessor nodes have been visited, randomly select one to continue extending. When tracing back to the initial state or there is no path to extend, end the backward traversal.

[0030] Further, in step 4), the Mocket tool is used to execute the test cases. According to the state transition sequence defined in the test cases, the mapping actions of the system are triggered in sequence, and the actual state, variable values, output information, and internal state changes of the system are recorded.

[0031] Further, the test report in step 4) includes: the number and type of errors found, the execution status of each test case, and the specific system state and action execution records.

[0032] A computer device includes a memory and a processor. The memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the above method.

[0033] A computer-readable storage medium is used to store a computer program. When the computer program is executed by a processor, it implements the steps of the above method.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] 1. The present invention conducts tests only on the parts that have changed during the evolution of the distributed system through incremental testing, avoiding comprehensive testing of the entire system state space. Compared with the traditional MCGT method, the present invention significantly reduces the number of test cases and testing time, and remarkably improves the testing efficiency.

[0036] 2. The present invention can accurately identify the affected states and state transitions during the evolution of the distributed system, and generate highly targeted incremental test cases. Through a meticulous incremental testing mode and test case generation algorithm, it ensures comprehensive coverage and accurate verification of system changes, effectively improving the accuracy of test results.

[0037] 3. The present invention is applicable to distributed system evolution scenarios, including refinement of specifications, addition of new functions, and bug fixing, etc. Its incremental testing strategy and test case generation algorithm have good adaptability, can be adjusted according to different change types and testing requirements, and have broad applicability and good scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is the flowchart of the model checking-guided incremental testing method for distributed systems of the present invention.

[0039] Figure 2 is the architecture diagram of the model checking-guided incremental testing method for distributed systems of the present invention.

[0040] Figure 3 is an example diagram of TLA+ specification.

[0041] Figure 4 is a partial state diagram generated by model checking. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings. Obviously, the described embodiments are only specific embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] The embodiments of the present invention specifically disclose a model checking-guided incremental testing method for distributed systems, which is used to efficiently test the changes during the evolution of distributed systems. Its processing flow and framework are as Figure 1 and Figure 2 shown. The specific steps are as follows:

[0044] (1) Perform static analysis on the TLA+ formal specification of the distributed system to identify modifications to the specification during the evolution process, including adjustments to TLA+ variables, changes to actions, and modifications to the logical structure. At the same time, analyze the changes in the system implementation code, with a focus on the adjustments to the code within the mapped actions and the changes to the action concurrency relationship. By comparing the original and modified specifications and implementation code, extract the change information and record the impact of these changes on TLA+ variables and system behavior.

[0045] 1) Identify variable modifications: Parse the variable declaration section in the TLA+ specification to extract the names and types of all variables. Compare the original and modified specifications to determine the addition, deletion, or modification of variables. For example, if a new variable is added in the modified specification, record its name and type and pay attention to its impact on the system state in subsequent tests. If a variable is deleted, check all states and state transitions that depend on this variable to ensure that the tests can correctly cover the relevant impacts.

[0046] 2) Identify action modifications: Analyze the actions in the TLA+ specification to identify the addition, deletion, and modification of basic blocks. Basic blocks are the logical execution units of actions, representing different execution paths. By comparing the original and modified specifications, determine the newly added, deleted, or modified basic blocks. For example, when a new basic block is added in the modified specification, record its logical conditions and execution statements and verify its impact on system behavior in the tests. When a basic block is deleted, check the validity of the relevant state transitions to detect potential errors.

[0047] 3) Identify modifications to the logical structure: Analyze the logical structure of the TLA+ specification to identify modifications to the initial state, concurrency relationship, and parameters. Modifications to the initial state may affect the initial behavior of the system, and modifications to the concurrency relationship and parameters may affect the execution order and conditions of actions. For example, if the variable assignment in the initial state is modified, verify in the tests whether the system can correctly start execution from the new initial state. If the concurrency relationship of actions is adjusted, such as adding or removing synchronization constraints between actions, check whether these changes affect the consistency of the system or introduce new errors.

[0048] 4) Analyze changes in the system implementation code: Compare the original and modified implementation code to extract the code adjustments within the mapped actions and record the impact of these adjustments on TLA+ variables. For example, if code for modifying TLA+ variables is added or modified within the mapped action, record the relevant variables and pay special attention to their state changes in the tests. In addition, clearly record the adjustments to action concurrency, including newly added or removed allowed concurrent execution sequences, in order to verify their impact on system behavior in the tests.

[0049] (2) After extracting the change information, identify the affected state nodes and edges in the new state diagram according to the predefined incremental testing pattern. The incremental testing pattern describes the impact of different types of changes on the state diagram and guides how to select the states and state transitions that need to be tested. For example, when a new basic block is added to the TLA+ specification, new state nodes and their connecting edges will be introduced in the new state diagram. At this time, all state transitions related to this basic block and their direct successor transitions need to be tested to verify whether there are incorrect states or inconsistent actions. If a basic block is deleted, some state nodes and edges will be removed, and it is necessary to check whether the predecessor transitions of the deleted states still exist to detect whether unexpected inconsistencies are introduced. For the addition or deletion of variables, since all state nodes will be affected, the changes in the basic blocks need to be considered comprehensively, and the corresponding incremental testing pattern should be applied to identify the affected states and transitions. For example, if a new variable is added and the basic block is modified, multiple patterns need to be combined for testing to comprehensively evaluate the impact of the changes on the state diagram.

[0050] Identify the affected state nodes and edges in the new state diagram according to the incremental testing pattern, including:

[0051] 1) P1 mode (adding a basic block): When a new basic block is added to the TLA+ specification, new state nodes and their connecting edges will be added to the new state diagram. Specifically, first identify the actions associated with the newly added basic block and find the corresponding state transitions in the new state diagram; then, mark these state transitions and their direct successor transitions as affected edges. For example, if the newly added basic block introduces a new state transition path, all state transitions on this path need to be tested to verify whether there are incorrect states or missing inconsistent actions.

[0052] 2) P2 mode (deleting a basic block): When a basic block is deleted from the TLA+ specification, the corresponding state nodes and their associated edges will be removed from the new state diagram. Specifically, first identify the actions associated with the deleted basic block and find the corresponding state transitions in the original state diagram; then, check whether these transitions still exist in the new state diagram. If not, mark their predecessor transitions as affected edges. For example, if deleting a basic block causes some state transitions to no longer occur, it is necessary to test its predecessor states to check whether unexpected action inconsistencies are introduced.

[0053] 3) P3 mode (adding a variable): When a new variable is added to the TLA+ specification, all state nodes will be affected because each state needs to contain the value of this variable. In addition, the addition of variables is usually accompanied by adjustments to the action logic, which may lead to the addition or deletion of basic blocks. Therefore, it is necessary to combine P1 and P2 modes to identify the affected states and transitions. For example, if the basic block is adjusted after adding a new variable, the relevant state transitions need to be tested, and it is necessary to verify whether the introduction of the variable results in a new state transition path.

[0054] 4) P4 mode (Delete variable): When the TLA+ specification deletes a variable, all state nodes will be affected because the value of this variable needs to be removed from each state. At the same time, variable deletion may affect the logic of basic blocks, making some state transitions invalid or adding other state transition paths. Therefore, it is necessary to combine P1 and P2 modes to identify the affected states and transitions.

[0055] 5) P5 mode (Modify action code): When the mapping action in the system implementation code is modified, it is necessary to test the state transitions that match the modified action logic. Specifically, identify the TLA+ variables involved in the modified action code, and find the state transitions related to the changes of these variables in the new state diagram, and mark them as affected edges. For example, if the modified action code updates the values of some TLA+ variables, it is necessary to test the state transitions affected by the variable changes to verify whether there are incorrect states or missing action inconsistencies.

[0056] 6) P6 mode (Allow action sequence): When the modified implementation introduces a new executable action sequence (such as action B can be executed after action A), it is necessary to test all state transition paths in the new state diagram that match this action sequence. Specifically, identify the state transitions corresponding to actions A and B, and mark these transitions and their direct successor transitions as affected edges. For example, if the new action sequence allows action A to be executed first and then action B in a certain state, it is necessary to test all possible paths of this sequence to detect whether there are inconsistencies in the action execution order.

[0057] 7) P7 mode (Prohibit action sequence): When the modified implementation prevents some originally executable action sequences (such as action B cannot be executed after action A), it is necessary to test all state transitions after action A in the new state diagram to detect whether there are unexpected action inconsistencies. Specifically, identify the state transition corresponding to action A, and mark all subsequent action transitions as affected edges. For example, if the new implementation prohibits action B from being executed immediately after action A in a certain state, it is necessary to test all possible transitions after action A to ensure that the system behavior meets the expectations.

[0058] (3) After obtaining the affected states in the state diagram, the present invention designs an incremental test case generation algorithm to cover the identified affected state transitions. The goal of this algorithm is to minimize the number of test paths while ensuring that all affected edges are tested, that is, to reduce the redundancy of test cases. The algorithm starts from the affected states and gradually expands the test paths until all affected edges are covered. During the path expansion process, paths containing affected edges are preferentially selected to ensure the comprehensiveness and effectiveness of the test. For example, when a certain state transition is identified as an affected edge, the algorithm will preferentially select the path containing this transition for expansion to generate the corresponding test case. In addition, the algorithm also considers the diversity and representativeness of test cases to avoid generating duplicate or redundant test paths. Through this strategy, the generated test cases can effectively evaluate the impact of system changes on the state diagram and provide a solid foundation for the execution of subsequent tests.

[0059] The incremental test case generation algorithm includes the following steps:

[0060] 1) Initialize the test path set: Create an empty test path set to store the generated incremental test cases. Each test path starts from the initial state, goes through a series of state transitions, and finally reaches an end state. The purpose of initializing the test path set is to ensure that all test paths can be systematically recorded and managed during the subsequent test case generation process.

[0061] 2) Select affected edges for path expansion: In the new state diagram, identify all edges marked as affected and cover them one by one. First, select an un-covered affected edge as the starting point for expanding the current path. If there is already a test path in the current path set that covers this edge, skip this edge and select the next un-covered affected edge. The selection priority can be based on the importance of the edge or the test coverage requirement to ensure that all affected state transitions can be effectively incorporated into the test paths.

[0062] 3) Perform forward and backward traversals of the path:

[0063] Backward traversal: Starting from the starting state of the selected affected edge, trace back forward along the state diagram until reaching the initial state. During the traversal process, preferentially select paths containing affected edges for backtracking. If there are multiple predecessor states for the current state, select an un-visited predecessor node for traversal according to the priority strategy; if all predecessor nodes have been visited, randomly select a predecessor node to continue the traversal. The purpose of backward traversal is to ensure that the generated test paths can start from the initial state and finally reach the affected edge.

[0064] Forward traversal: Starting from the end state of the selected affected edge, expand forward along the state diagram until reaching the end state or all successor nodes have been visited. During the traversal, paths containing the affected edge are also preferentially selected for expansion. If there are multiple successor states for the current state, an unvisited successor node is selected according to the priority strategy for continued traversal; if all successor nodes have been visited, a successor node is randomly selected for continued traversal. The purpose of forward traversal is to ensure that the test path can continue to extend from the affected edge to a valid end state.

[0065] 4) Generate a complete test path: Combine the path segments obtained from backward traversal and forward traversal to form a complete test path. This path starts from the initial state, passes through the selected affected edge, and finally reaches an end state. The generated test path will be added to the test path set. If an unexpandable situation is encountered during the traversal (e.g., there are no available predecessor or successor nodes), the generation of the current path is abandoned, and the next uncovered affected edge is selected to re-expand the path.

[0066] 5) Repeat the path expansion process until all affected edges are covered: Check whether the test path set has covered all affected edges. If there are still uncovered affected edges, repeat steps 2) to 4) to continue expanding the path and generating new test cases. This process continues until all affected state transitions are covered by the test paths, ensuring that the changes occurring during the system evolution are comprehensively verified.

[0067] (4) After generating the test cases, the present invention uses Mocket to perform actual system testing to control the mapped action sequence and verify whether the system execution is consistent with the expected states and transitions defined in the test cases. During the testing process, Mocket precisely controls the execution order of the system according to the generated incremental test cases to ensure that each action is executed in the order specified by the test cases. For example, for the state transition sequence in a certain test case, Mocket triggers the corresponding actions in sequence, monitors the changes in the system state, and compares it with the expected state. If the system state is consistent with the expected state, the test case passes; otherwise, the test failure information is recorded for subsequent analysis and repair. In this way, the correctness of the distributed system during evolution can be effectively verified, and potential errors introduced by system changes can be timely discovered and located.

[0068] The specific steps for Mocket to perform system testing are as follows:

[0069] 1) Load test cases and execute the mapped action sequence: Load the generated incremental test cases into the Mocket test tool. According to the state transition sequence defined in the test cases, Mocket gradually triggers the mapped actions in the system implementation and strictly controls the execution order of the actions to ensure that they are executed in the expected order of the test cases. For example, if the state transition sequence in a certain test case is A → B → C, Mocket first triggers A, then triggers B after A is executed, and finally executes C. During the execution of each action, Mocket records the actual state of the system and the action execution results, including the values of system variables, system output information, internal state changes, etc., for subsequent comparison and analysis.

[0070] 2) Real-time verification of the correctness of system state and transition: After each action is executed, Mocket compares the actual state of the system with the expected state in the test case to check whether the system variable values match and whether the system is in the expected state node. At the same time, Mocket verifies whether the execution of the action conforms to the expected state transition, including checking: ① whether the triggering condition of the action is met; ② whether the system successfully transitions to the expected state after the action is executed. For example, if the test case expects that after executing action B, the system should transition from state A to state C, Mocket checks whether the system correctly enters state C after actual execution. Mocket continuously tracks the execution of each state transition during the test process, and discovers and records any abnormal behavior that deviates from the expectation in real time for subsequent analysis and correction.

[0071] 3) Test result analysis and report generation: After all test cases are executed, Mocket analyzes and summarizes the test results, including: ① the number and type of errors found; ② the execution status of each test case; ③ the specific system state and action execution records. Finally, Mocket generates a detailed test report, listing the execution details of each test case, the abnormal detection situation, and the relevant system behavior data to support subsequent problem fixing and system optimization.

[0072] Through the above processing steps, this method analyzes the changes in the system specification and implementation, identifies the affected states and state transitions, incrementally generates test cases, and uses the Mocket tool to execute the tests to verify the correctness of the system behavior. The following combines Figure 3 and Figure 4 to illustrate this method in detail with a specific example, where Figure 3 the code marked in red is the added part for introducing new functions, Figure 4 the nodes and edges represented by the dotted lines are the parts introduced due to the newly added code in the specification.

[0073] First, perform static analysis on the TLA+ formal specification and system implementation code of the distributed system to identify the changes that occurred during the evolution. Taking Figure 3 as an example, assume that a simple communication protocol is defined in the original specification, including variables msg, cache, and stage, as well as related actions Request, MaxRespond, and Process. During the evolution of the specification, a new action MinRespond may be added to respond with the minimum value. By comparing the original and modified specifications, extract the addition information of the new action MinRespond, as well as the possible changes in variables and state transitions involved. For example, the MinRespond action may affect the value of the cache variable and introduce new state transition paths.

[0074] Second, based on the extracted change information and predefined incremental test patterns, identify the affected nodes and edges in the new state diagram. Since the new action MinRespond is added, new state nodes and edges connecting these nodes will be introduced into the state diagram. For example, assume that in the original state diagram, the state node sinit reaches the state s1 through the Request action, and then reaches the state s2 through the MaxRespond action. After adding MinRespond, a transition edge from the state s1 to the new state s3 may appear, representing the process of executing the MinRespond action. By applying the incremental test pattern, identify these newly added state nodes and transition edges, as well as the original state transition paths that may be affected, to ensure comprehensive coverage of the impact brought by system changes.

[0075] Third, use an incremental-based test case generation algorithm to generate test cases that cover the affected state transitions. Starting from the affected state transitions, perform forward and backward traversals step by step to generate test paths. For example, in Figure 4 , starting from the newly added state transition s1->s6, perform forward and backward traversals to generate a complete test case. When traversing forward, starting from the state s1, it can be traced back to the initial state sinit, forming the path sinit->s1. When traversing backward, starting from the state s6, it can be extended to subsequent states. By Figure 4After reaching state s6, state s3 can be reached, forming a path s6->s3. If there are subsequent states after state s3, such as state s5, continue traversing backward, forming a path s3->s5, until the user-defined end state is reached or all outgoing edges of the reached state have been traversed. Combine the paths obtained from forward and backward traversals to generate a complete test case path sinit->s1->s6->s3->s5. This test case covers the entire process from the initial state, through the state transitions introduced by the new action MinRespond, and subsequent states, and can effectively verify the correctness of the new action MinRespond and its impact on the system state. In the test case, special attention is paid to checking relevant variables to ensure that after executing the MinRespond action, the system state correctly reflects the minimum value response. In this way, the generated test case can comprehensively cover the impact of system changes on the state diagram, providing a solid foundation for subsequent test execution and ensuring that the overall behavior and state changes of the system after the introduction of the new action can be accurately evaluated.

[0076] Among them, starting from the affected state transition, the specific steps of forward and backward traversals are as follows: The forward traversal process starts from the end state of the current edge and explores the state diagram forward to generate the second half of the test case. It checks whether the end state of the current edge is one of the end states specified by the tester or whether all its subsequent edges have been visited. If not, according to the priority policy, select an unvisited affected edge or randomly select a subsequent edge, append it to the end of the current path, and continue recursively traversing forward until the end state is reached or all subsequent edges have been visited. The backward traversal process starts from the start state of the current edge and explores the state diagram backward to generate the first half of the test case. It checks whether the start state of the current edge is the initial state. If not, according to the priority policy, select an unvisited affected edge or randomly select a predecessor edge, append it to the beginning of the current path, and continue recursively traversing backward until the initial state is reached or all predecessor edges have been visited.

[0077] Finally, use the Mocket tool to execute the generated incremental test cases. Mocket triggers the actions mapped in the system implementation in sequence according to the state transition sequence in the test cases and monitors the execution of the system in real time. For example, for the generated test cases, Mocket will first trigger the Request action, and then trigger the MinRespond action, the Process action, and the Request action respectively. After each action is executed, Mocket compares and validates the actual state of the system with the expected state in the test case, checks whether the value of the actual state is consistent with the expected value, and whether the system has successfully transitioned to the expected state. If any error information inconsistent with the expectation is found during the test, Mocket will record it in time and generate a test report at the end of the test. The test report will list the execution status of each test case and the error information found, providing a clear view of the test results for testers and helping them quickly locate and fix errors in the system. By executing the tests, the correctness and stability of the distributed system during evolution can be accurately evaluated to ensure that the introduction of the new action MinRespond will not have a negative impact on the overall function of the system.

[0078] Although specific implementation processes and example drawings of the present invention are disclosed for illustrative purposes, aiming to help understand the content of the present invention and implement it accordingly, those skilled in the art can understand that: within the spirit and scope of the present invention and the appended claims, various substitutions, changes, and modifications are possible. Therefore, the present invention should not be limited to the content disclosed in the shown implementation processes and example drawings.

Claims

1. A distributed system incremental testing method guided by model checking, characterized in that: The following steps are involved: 1) Perform static analysis on the TLA+ formal specification and system implementation code of the distributed system, identify the modification of variables, actions and logical structures in the specification, as well as the change of mapping actions and adjustment of action concurrency relations in the system implementation code; compare the original and modified specifications and codes, record the addition, deletion and modification of variables, basic blocks and logical relations, and generate change information; 2) Based on the predefined incremental test mode, analyze the change information and identify the affected state nodes and edges in the new state graph; according to different types of changes, filter the states and state transitions to be tested and generate a set of affected states and transitions; 3) Based on the incremental test case generation algorithm, the test path is constructed starting from the affected state; Expand the test path, first cover the affected edges, until all affected transitions are covered, and generate test cases; 4) Execute the test case and perform the mapping actions in sequence according to the generated test path; During the test, the system status is compared with the expected status in real time, abnormal behaviors that deviate from the test cases are recorded, and a test report is generated.

2. The method according to claim 1, characterized in that The method for identifying variable modifications in step 1) is to parse the variable declarations of the TLA+ specification, compare the original and modified specifications, identify the addition, deletion and modification of variables, and analyze their impact on system states and transitions.

3. The method according to claim 1, characterized in that The method for identifying action modifications in step 1) is: analyzing the actions in the TLA+ specification, identifying the addition, deletion and modification of basic blocks, recording the changes in logical conditions and execution statements, and checking their impact on system behavior.

4. The method according to claim 1, characterized in that The method for identifying logical structure modifications in step 1) is to analyze the initial state, concurrency relationship and parameter adjustment of the specification, and evaluate the impact of these modifications on the system execution order, consistency and potential errors.

5. The method according to claim 1, characterized in that The methods for identifying the affected state nodes and edges in the new state graph in step 2) include the following seven modes: P1 mode: When a new basic block is added to the TLA+ specification, identify the actions associated with the new basic block, find the relevant state transitions in the new state graph, and mark the affected states and edges; P2 mode: When the TLA+ specification deletes a basic block, identify the actions associated with the deleted basic block and find the corresponding transition in the original state graph; check whether the transition still exists in the new state graph. If not, mark the predecessor transition as the affected edge; P3 mode: When a new variable is added to the TLA+ specification, the P1 and P2 modes are combined to test the relevant state transitions and identify the state nodes affected by the new variable; P4 mode: When the TLA+ specification deletes variables, it combines P1 and P2 modes to test related state transitions and identify the state nodes affected by the deleted variables; P5 mode: When a mapping action in the system implementation code is modified, identify the TLA+ variables involved in the modified action code, find the relevant state transitions in the new state diagram, and mark the affected edges; P6 mode: When the modified implementation introduces a new executable action sequence, identify the sequence and find the state transition path in the new state diagram that matches the sequence and mark the affected edges; P7 mode: When the modified implementation prevents some action sequences that were originally executable, identify the prohibited action sequences, find the relevant transitions in the new state diagram, and mark the affected edges.

6. The method according to claim 1, characterized in that The step of extending the test path in step 3) includes: in the new state graph, identifying all edges marked as affected; selecting edges that have not been covered by any test path as the current extension starting point; and selecting key edges for extension based on the importance of the edges or the test coverage strategy.

7. The method according to claim 1, characterized in that In step 3), the integrity of the test path is ensured by forward and backward traversal, where: The forward traversal method is as follows: first, select the path containing the affected edge for expansion, start from the end state of the affected edge, and expand the path forward until an end state is reached or all successor nodes have been visited; if the current state has multiple successor states, select the unvisited successor node according to the priority strategy to continue traversal; if all successor nodes have been visited, randomly select one to continue expansion; when the end state is reached or there is no expandable path, the forward traversal ends; The backward traversal method is: first select the path containing the affected edge for expansion, start from the starting state of the affected edge, and trace back along the state graph until the initial state is reached; if the current state has multiple predecessor states, select the unvisited predecessor node according to the priority strategy to continue traversing; if all predecessor nodes have been visited, randomly select one to continue expanding; when tracing back to the initial state or there is no expandable path, end the backward traversal.

8. The method according to claim 1, characterized in that In step 4), the Mocket tool is used to execute the test case, trigger the system's mapping actions in sequence according to the state transition sequence defined in the test case, and record the system's actual state, variable values, output information, and internal state changes.

9. The method according to claim 1, characterized in that The test report in step 4) includes: the number and type of errors found, the execution status of each test case, the specific system status and action execution records.

10. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the method according to any one of claims 1 to 9.

Images