Remote procedure call fault injection test method and device, equipment and medium
By employing iterative RPC discovery and batch fault injection techniques, the coverage and efficiency issues of existing test suites in RPC fault injection testing are resolved. This enables comprehensive detection of RPCs and discovery of dependencies in distributed systems, improving test accuracy and system stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEKING UNIV
- Filing Date
- 2026-04-10
- Publication Date
- 2026-05-08
AI Technical Summary
Existing unit testing methods are difficult to effectively simulate the response and interaction of RPC peers, and the workload of manually identifying and writing test cases is large, resulting in limited coverage of RPC fault injection tests in distributed systems, which cannot fully detect error handling logic.
By employing iterative RPC discovery and batch fault injection techniques, we retrieve the preconditions of RPCs and inject faults under empty preconditions. This allows us to discover RPCs triggered by error handling logic and conduct fault injection tests through multiple rounds of iteration. We also build a fault library and a precondition library to record test status and dependencies.
It improves the coverage and efficiency of RPC fault injection testing in distributed systems, ensuring the reliability and stability of the system in the face of various faults, and enhancing the accuracy and efficiency of testing.
Smart Images

Figure CN121996445A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of software testing, specifically relating to a remote procedure call fault injection testing method, apparatus, equipment, and medium. Background Technology
[0002] In recent years, distributed systems have become increasingly large-scale, with more components and more complex interactions. They require frequent communication over networks to coordinate tasks and share data. Remote Procedure Call (RPC) facilitates this process by allowing components to invoke methods on remote servers with the semantics of local function calls. RPC has become ubiquitous in modern distributed systems, such as microservices, distributed file systems, distributed databases, and distributed computing frameworks. RPC frameworks such as gRPC, Apache Thrift, and RESTful APIs provide a transparent communication interface, shielding the complexity of underlying network connections and data transmission. Despite these advantages, RPC can also introduce vulnerabilities and failures into systems. Remote Procedure Calls (RPC) may fail due to transient errors such as network jitter and server overload. These errors can disrupt communication workflows, propagate exceptions to application logic, or cause system stagnation. Such errors are unavoidable in distributed systems, and handling them correctly is not easy.
[0003] One reason is that distributed systems have a large number of RPCs, each with its own error handling requirements. Ensuring that all RPCs have adequate error handling logic for all possible errors is not easy. Another reason is that error handling is highly sensitive to system logic and error types. For example, retry strategies are a common practice for handling unavailability errors. However, retry strategies are not suitable for non-idempotent RPCs because retries to non-idempotent requests can lead to unexpected side effects such as data duplication or loss. Therefore, systematically testing the error handling mechanisms of distributed systems under various transient RPC errors is crucial.
[0004] Unit testing is a common reliability assurance technique. It is widely used in distributed systems to test the correctness of individual modules. However, it falls short in testing RPC (Remote Procedure Call) error handling for two main reasons.
[0005] First, simulating RPC peers in unit testing is very difficult. Simulating RPC peers requires simulating complex responses and interactions between distributed components, which is not only challenging but also error-prone.
[0006] Secondly, in large-scale systems, the workload required to manually identify and write test cases for each RPC is enormous, inevitably leading to gaps in coverage. Therefore, a more practical approach is fault injection testing. This involves injecting transient RPC errors into the terminal during system operation and checking whether the system behaves correctly. However, due to coverage gaps, efficiency bottlenecks, and the complexity of fault identification, implementing effective RPC fault injection in distributed systems is not easy.
[0007] First, RPC calls are distributed across various parts of the system, including both normal execution logic and error handling logic. Existing test suites struggle to trigger error handling logic, thus failing to cover RPCs initiated from error handling logic. Second, after a single test, it's impossible to identify the faulty RPCs that caused the test failure for further testing and exploration of new RPCs, making it difficult to improve the efficiency of RPC fault injection testing in distributed systems. Summary of the Invention
[0008] The purpose of this application is to provide a remote procedure call fault injection testing method, apparatus, device, and medium that can solve the above-mentioned problems.
[0009] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a remote procedure call fault injection testing method, the method comprising: Retrieve the preconditions for the RPC in the current iteration, where the preconditions for the RPC in the current iteration are failures previously injected into other RPCs that triggered the occurrence of the RPC in the current iteration; If the preconditions for the current iteration of the RPC are empty, a fault is injected into the current iteration of the RPC and a fault injection test is performed. The empty preconditions indicate that the current iteration of the RPC can be triggered in normal processing logic without any preconditions. During the fault injection test of the RPC in the current iteration, an RPC triggered by error handling logic was found, which was triggered by the error handling logic of the RPC in the current iteration that was injected with fault. The RPC of the current iteration is used as a prerequisite for the RPC triggered by the error handling logic. The RPC triggered by the error handling logic is used as the RPC of the next iteration and the fault injection test of the next iteration is executed. In order to discover the RPC triggered by the error handling logic through multiple iterations and perform fault injection tests on the discovered RPC triggered by the error handling logic.
[0010] Optionally, before retrieving the preconditions for the RPC in the current iteration, the method further includes: determining the RPC in the test state as the RPC in the current iteration based on multiple fault entries contained in a pre-built fault library, wherein the fault library contains multiple fault entries, and each fault entry represents the test state of an RPC under a fault. After discovering the RPC triggered by the error handling logic, the method further includes: for the RPC triggered by the error handling logic, creating multiple fault entries in the fault database based on multiple faults, and setting the test status of the fault entry corresponding to the RPC triggered by the error handling logic to be tested. After setting the RPC of the current iteration as a prerequisite for the RPC triggered by the error handling logic, setting the RPC triggered by the error handling logic as the RPC of the next iteration, and executing the fault injection test of the next iteration, the method further includes: setting the test status of the fault entry corresponding to the fault injection test of the next iteration among the multiple fault entries corresponding to the RPC triggered by the error handling logic to be completed.
[0011] Optionally, retrieving the preconditions for the RPC in the current iteration includes determining the preconditions for the RPC in the current iteration from multiple precondition entries in a pre-built precondition library, wherein the precondition library contains multiple precondition entries, and each precondition entry represents an RPC that can be triggered under a certain precondition. After discovering an RPC triggered by error handling logic, the method further includes: for the RPC triggered by error handling logic, based on the preconditions of the RPC in the current iteration and the fault injected into the RPC in the current iteration, creating a precondition entry corresponding to the RPC triggered by error handling logic in the precondition library.
[0012] Optionally, the method further includes: Based on the number of times the error handling logic is triggered in the preconditions, determine the error level of each RPC triggered by the error handling logic; Set the preset error level threshold; For the RPC triggered by the error handling logic, multiple fault entries are created in the fault database based on multiple faults, including: for RPCs triggered by the error handling logic with an error level lower than the preset error level threshold, multiple fault entries are created in the fault database based on multiple faults. For the RPC triggered by the error handling logic, based on the preconditions of the current iteration of the RPC and the fault injected into the current iteration of the RPC, a precondition entry corresponding to the RPC triggered by the error handling logic is created in the precondition library. This includes: for RPCs triggered by the error handling logic whose error level is less than the preset error level threshold, a precondition entry is created in the precondition library based on the preconditions of the current iteration of the RPC and the fault injected into the current iteration of the RPC.
[0013] Optionally, the method further includes: Based on the precondition library and the fault library, the number of RPCs to be tested corresponding to each precondition entry is determined. The RPC to be tested represents an RPC triggered by error handling logic that can be triggered simultaneously under a precondition and whose test status is pending. The step of using the RPC triggered by the error handling logic as the RPC for the next iteration and performing the fault injection test for the next iteration includes: using multiple RPCs triggered by the error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested as the RPCs for the next iteration and performing the fault injection test for the next iteration.
[0014] Optionally, after selecting the RPCs triggered by error handling logic corresponding to the precondition entries with the largest number of RPCs to be tested as the RPCs for the next iteration and performing the fault injection test for the next iteration, the method further includes: Determine whether there is an abnormal RPC among the multiple RPCs triggered by error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested. The abnormal RPC represents an RPC triggered by error handling logic that did not occur after the injection of fault. In the presence of an abnormal RPC, the test status of the fault entry corresponding to the abnormal RPC in the fault database is kept as pending test. In addition, for the abnormal RPC, a new precondition entry is created in the precondition database. The new precondition entry indicates that the abnormal RPC is triggered alone under the original precondition. The fault injection test is re-executed on the anomalous RPC based on the new precondition entries.
[0015] Optionally, after re-performing the fault injection test on the anomalous RPC according to the new precondition entries, the method further includes: Determine whether the abnormal RPC occurs after the fault injection test is re-executed; If the abnormal RPC does not occur after the fault injection test is re-executed, the new precondition entry in the precondition library is updated for the abnormal RPC to obtain an updated precondition entry. The updated precondition entry indicates that the abnormal RPC is triggered individually under other preconditions. The fault injection test is re-executed on the anomalous RPC based on the updated precondition entries.
[0016] Secondly, embodiments of this application provide a remote procedure call fault injection testing apparatus, the apparatus comprising: The precondition retrieval module is used to retrieve the preconditions of the RPC in the current iteration. The preconditions of the RPC in the current iteration are the faults previously injected into other RPCs that triggered the occurrence of the RPC in the current iteration. The first fault injection test module is used to inject faults into the RPC of the current iteration and perform fault injection tests when the precondition of the RPC of the current iteration is empty. The empty precondition indicates that the RPC of the current iteration can be triggered in normal processing logic without any precondition faults. An RPC detection module is used to detect RPCs triggered by error handling logic during fault injection testing of the current iteration of the RPC, wherein the RPCs triggered by error handling logic are triggered by the error handling logic of the current iteration of the RPC that has been injected with faults. The iterative fault injection test module is used to take the RPC of the current iteration as a prerequisite for the RPC triggered by the error handling logic, take the RPC triggered by the error handling logic as the RPC of the next iteration, and execute the fault injection test of the next iteration, so as to discover the RPC triggered by the error handling logic through multiple iterations and perform fault injection tests on the discovered RPC triggered by the error handling logic.
[0017] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps in the method described in the first aspect.
[0018] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps in the method described in the first aspect.
[0019] The technical solution of this application effectively overcomes the limitations of existing test suites in exploring RPCs and injecting faults within error handling logic through multi-round iterative fault injection testing. This application retrieves the preconditions of the RPCs in the current iteration and injects faults when the preconditions are empty. This triggers RPCs that can be called in normal processing logic. By injecting faults into these normal processing logic RPCs, it discovers RPCs triggered by error handling logic. Then, using the RPCs in the current iteration as preconditions, it treats the RPCs triggered by error handling logic as RPCs for the next iteration and injects faults. Through multi-round iterative fault injection testing, it can progressively and deeply discover each RPC and the dependencies between them, effectively overcoming the limitations of existing test suites in RPC discovery and fault injection. It can comprehensively probe the RPCs within both normal and error handling logic in distributed systems, enhancing test coverage, improving test accuracy and efficiency, and ensuring the reliability and stability of distributed systems in the face of various faults. Attached Figure Description
[0020] Figure 1 This is a complete structural diagram of a remote procedure call fault injection testing method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a remote procedure call fault injection testing method according to an embodiment of this application; Figure 3 This is a flowchart illustrating the iterative RPC probing process in a remote procedure call fault injection testing method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the process of detecting an abnormal RPC during the iterative RPC detection in a remote procedure call fault injection testing method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the framework of a remote procedure call fault injection testing device provided in an embodiment of this application; Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. Detailed Implementation
[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without prior creative effort are within the scope of protection of this application.
[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0023] The following description, in conjunction with the accompanying drawings, details a remote procedure call fault injection testing method, apparatus, device, and medium provided in this application through specific embodiments and application scenarios.
[0024] Figure 1 This is a complete structural diagram of a remote procedure call fault injection testing method provided in an embodiment of this application, referencing... Figure 1 This application is driven by two key technologies: (1) Iterative RPC discovery technology: used to discover RPCs initiated from normal execution logic and error handling logic; (2) Batch fault injection technology: used to improve the testing efficiency for a large number of RPCs.
[0025] Specifically, during the testing phase, this application iteratively executes two processes: RPC probing and fault injection. First, an existing end-to-end test suite is run to collect RPCs from the normal execution logic. Then, faults are prepared for these RPCs in the normal execution logic, batched, and injected simultaneously during the test run.
[0026] Meanwhile, during testing, newly detected RPCs are collected. These newly detected RPCs are triggered by the error handling logic of previously injected faults. Then, in the next iteration, new faults are prepared for the newly detected RPCs, and a new injection scheme containing batch faults is generated. This iterative process establishes dependencies between RPCs, thereby determining which RPCs depend on the faults of other RPCs. Simultaneously, during testing, it is checked whether each RPC behaves correctly under the injected fault conditions in each test run.
[0027] The key to this iterative RPC discovery lies in systematically probing RPCs within error handling logic that cannot be directly triggered by the test suite. By iteratively injecting faults into the detected RPCs and monitoring any newly triggered RPCs, these new RPCs become the targets of subsequent test iterations. In this way, not only RPCs in normal processing logic can be discovered, but also those in error handling logic. Furthermore, the dependencies between RPC faults and RPC occurrences are monitored and analyzed to identify the triggering conditions for each RPC.
[0028] This paper improves testing efficiency by batch processing faults from multiple RPCs and injecting them simultaneously in a single test run. During each test run, if a newly triggered RPC is detected, a fault is injected into these newly triggered RPCs in subsequent test runs. This allows for the detection of more RPC triggers within the error handling logic of these newly triggered RPCs. Traditional single-function test cases, which inject a fault into only one RPC per test run ("inject one fault per run"), fail to fully utilize testing opportunities in complex end-to-end test cases, resulting in excessive test runs and high testing overhead. In contrast, this application batch processes faults, injecting multiple faults into a single test run to test multiple RPCs, reducing the total number of test runs and further improving testing efficiency.
[0029] To facilitate the maintenance and recording of the testing process for each RPC, the following two types of libraries are built during the iteration process to maintain the test status: a fault library, used to track which faults have been injected into which RPCs; and a precondition library, used to determine how to trigger each RPC.
[0030] Batch RPC fault injection utilizes an online greedy algorithm to dynamically combine injection schemes containing batch faults based on the currently discovered RPCs and the triggering preconditions, aiming to cover as many faults as possible in each test run.
[0031] In addition, new injection schemes are dynamically constructed to cover previously missed faults, thereby resolving the discrepancy between the injection scheme and the actual injection result.
[0032] The technical solutions of this application will now be described in detail with reference to various embodiments.
[0033] Figure 2 This is a flowchart illustrating a remote procedure call fault injection testing method provided in one embodiment of this application.
[0034] This application provides a remote procedure call (RPC) fault injection testing method. It improves the testing efficiency for fault injection testing of a large number of RPCs in a distributed system by iteratively executing two processes: injecting faults into detected RPCs and detecting whether new RPCs are triggered after fault injection. (See reference...) Figure 2 The method specifically includes steps S11 to S14: Step S11: Retrieve the preconditions of the RPC in the current iteration, wherein the preconditions of the RPC in the current iteration are the failures previously injected into other RPCs that triggered the occurrence of the RPC in the current iteration.
[0035] In this embodiment, in the initial stage of testing the distributed system, the RPCs in the normal processing logic (i.e. without injecting faults) contained in the distributed system are first collected by running an end-to-end test suite. Then, the two processes of fault injection for the detected RPCs and detection of whether new RPCs are triggered after fault injection are started in each iteration.
[0036] Before executing the current iteration, it is necessary to retrieve the RPC for the current iteration and its preconditions from the multiple detected RPCs. The RPC for the current iteration refers to the internal RPC that needs to be injected during the current iteration to detect whether there is an RPC triggered by the injection of a fault into that RPC, i.e., an RPC triggered by error handling logic. The preconditions of the RPC for the current iteration refer to the faults that occurred in the RPCs that were previously injected into other RPCs to trigger the current iteration.
[0037] Step S12: If the precondition of the current iteration RPC is empty, inject a fault into the current iteration RPC and perform a fault injection test. The empty precondition indicates that the current iteration RPC can be triggered in normal processing logic without any precondition faults.
[0038] If the preconditions for the RPC in the current iteration are empty, it means that the RPC in the current iteration is the RPC in the normal processing logic collected by running the end-to-end test suite in the initial stage of the test. Therefore, a fault will be injected into the RPC in the current iteration with empty preconditions and a fault injection test will be performed.
[0039] Step S13: During the fault injection test of the RPC of the current iteration, an RPC triggered by error handling logic is found. The RPC triggered by error handling logic is triggered by the error handling logic of the RPC of the current iteration that is injected with fault.
[0040] In this embodiment, after injecting a fault into the RPC of the current iteration with empty preconditions and performing a fault injection test, it is determined whether the RPC of the current iteration behaves normally after the fault is injected. If it behaves normally, it indicates that the RPC of the current iteration has actually failed due to the fault injection. At this time, it is necessary to monitor whether a new RPC has been triggered. After it is determined that a new RPC has been triggered, these newly triggered RPCs are collected to obtain the RPCs triggered by the error handling logic. The RPCs triggered by the error handling logic are the internal RPCs triggered by the error handling logic generated by the RPC of the current iteration due to the actual failure.
[0041] Step S14: Use the RPC of the current iteration as a prerequisite for the RPC triggered by the error handling logic, use the RPC triggered by the error handling logic as the RPC of the next iteration, and perform fault injection test for the next iteration. This is done through multiple iterations to discover RPCs triggered by the error handling logic and perform fault injection test on the discovered RPCs triggered by the error handling logic.
[0042] In this embodiment, after obtaining the RPC triggered by the error handling logic, the fault injected into the RPC of the current iteration in step S12 is used as a precondition for the RPC triggered by the error handling logic obtained in step S13. Then, the RPC triggered by the error handling logic is used as the RPC of the next iteration and the fault injection test of the next iteration is executed. During the fault injection process of the fault injection test of the next iteration, it is necessary to simultaneously inject the fault in the precondition of the RPC of the current iteration in step S12, and inject new faults into the RPC of the next iteration, in order to determine whether there is a new RPC triggered by the injection of preconditions and new faults. For the triggered new RPC, the fault injection test process of the next iteration will continue. In this way, the fault injection test is performed on the discovered RPCs triggered by the error handling logic through multiple iterations to continuously detect RPCs triggered by the error handling logic in the distributed system that cannot be directly triggered by the test suite.
[0043] For example, first, collect the normal processing logic RPCs contained in the distributed system, such as RPC R1, by running an end-to-end test suite; Then, during the first iteration, a fault is injected into RPC R1 and a fault injection test is performed, such as an outbound unavailability fault. The injected fault RPC R1 is then monitored, and it is determined that RPC R1 triggered an RPC triggered by error handling logic after the fault is injected, i.e., RPC R2. This indicates that RPC R2 is triggered based on the error handling logic of RPC R1 after the outbound unavailability fault is injected.
[0044] Therefore, RPC R2 is continued as the RPC for the second iteration. The fault injected into RPC R1 during the first iteration (outbound failure) is used as a precondition for RPC R2. An outbound failure is injected into RPC R1. At the same time, a fault, such as inbound unavailability, is injected into RPC R2. The RPC R2 with injected faults is monitored. Through monitoring, it is found that RPC R2 triggers RPC R3. Therefore, it is determined that injecting a fault into RPC R1 triggers RPC R2, and injecting a fault into RPC R2 triggers RPC R3. By analogy, RPC R3 can be used as the RPC for the third iteration. The fault injected into RPC R1 during the first iteration (outbound failure) and the fault injected into RPC R2 during the second iteration (inbound failure) are used as preconditions for RPC R3. After injecting a fault into RPC R3, fault injection testing is performed. In this way, through multiple iterations of testing, RPC faults in the normal call logic and error handling logic of the distributed system are discovered.
[0045] The technical solution described above, through multi-round iterative fault injection testing, effectively solves the limitations of existing test suites in exploring RPCs and fault injection within error handling logic. This application retrieves the preconditions of the RPCs in the current iteration and injects faults when the preconditions are empty. This triggers RPCs that can be called in normal processing logic. By injecting faults into these normal processing logic RPCs, RPCs triggered by error handling logic are discovered. Then, using the RPCs in the current iteration as their preconditions, the RPCs triggered by error handling logic are used as the RPCs in the next iteration and fault injection is performed. Through multi-round iterative fault injection testing, the various RPCs and their dependencies can be gradually and deeply discovered. This effectively solves the limitations of existing test suites in RPC discovery and fault injection, enabling comprehensive detection of RPCs within both normal and error handling logic in distributed systems. This enhances test coverage, improves test accuracy and efficiency, and ensures the reliability and stability of distributed systems in the face of various faults.
[0046] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for remote procedure call fault injection testing, in which: Before performing the "retrieve the preconditions for the RPC in the current iteration" in step S11, the method further includes: step S01, determining the RPC whose test state is to be tested as the RPC in the current iteration based on multiple fault entries contained in a pre-built fault library, wherein the fault library contains multiple fault entries, and each fault entry represents the test state of an RPC under a fault.
[0047] In this embodiment, in order to facilitate the recording of the test status of various fault conditions corresponding to each detected RPC, we pre-created a fault database data structure. The fault database records multiple fault entries to record the test status of each detected RPC under different fault types.
[0048] The test status is divided into "completed" and "pending test". "Completed" means that an RPC has completed the test under a certain failure condition. "Pending test" means that an RPC has not completed the test or the test has failed and needs to be retested under a certain failure condition.
[0049] Each fault entry corresponds to the test status of an RPC under a certain fault. The fault entry is in the form of a triple, i.e. (RPC, fault type, test status), for example (R1, Outbound Unavailable, Todo), which means that the "unavailable" fault of RPC R1 in the outbound path has not been tested.
[0050] Before retrieving the preconditions for the RPC in the current iteration, it is necessary to first determine the fault entries whose test status is to be tested based on the test status of the fault entries already recorded in the fault database, and then use the RPC corresponding to the fault entry as the RPC for the current iteration, and use the fault type corresponding to the fault entry as the fault to be injected into the RPC for the current iteration.
[0051] After "detecting the RPC triggered by error handling logic" in step S13, the following steps are also included: Step S21: For the RPC triggered by the error handling logic, create multiple fault entries in the fault database based on multiple faults, and set the test status of the fault entries corresponding to the RPC triggered by the error handling logic to be tested.
[0052] In this embodiment, the initial number of fault entries in the fault database is zero. The fault entries in the fault database are gradually updated during the iterative detection of RPCs and the injection of faults. Specifically, each time an RPC is detected (including RPCs with normal processing logic collected by the test suite and RPCs triggered by error handling logic), multiple fault entries are created in the fault database based on various fault types (such as inbound unavailability, outbound unavailability, timeout, etc.), and the test status of these created fault entries is initialized to be tested.
[0053] After executing step S14, which involves "using the RPC of the current iteration as a precondition for the RPC triggered by the error handling logic, using the RPC triggered by the error handling logic as the RPC of the next iteration, and executing the fault injection test for the next iteration," the process further includes: Step S31: Set the test status of the fault entry corresponding to the fault injection test in the next iteration among the multiple fault entries corresponding to the RPC triggered by the error handling logic to "completed".
[0054] In this embodiment, after one round of fault injection test execution, the test status of the fault entry corresponding to the fault injected by the RPC participating in the fault injection test is updated from pending test to completed.
[0055] Specifically, after the fault injection test of the current iteration is executed in step S13, the test status of the fault entry corresponding to the fault injection test of the current iteration among the multiple fault entries of the current iteration (i.e., the RPC in the normal processing logic) will be set to completed.
[0056] After the fault injection test in the next iteration of step S14 is executed, the test status of the fault entry corresponding to the fault injection test in the next iteration of the fault injection test will be set to completed among the multiple fault entries corresponding to the RPC triggered by the error handling logic.
[0057] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another remote procedure call fault injection testing method. The "retrieve the preconditions of the RPC in the current iteration" in step S11 includes step S11-1, determining the preconditions of the RPC in the current iteration from multiple precondition entries in a pre-built precondition library. The precondition library contains multiple precondition entries, and each precondition entry represents an RPC that can be triggered under a certain precondition.
[0058] In this embodiment, to facilitate the recording of the preconditions corresponding to each detected RPC, we introduce a precondition library data structure. The precondition library contains multiple precondition entries recorded according to the detected RPCs, and each precondition entry represents an RPC that can be triggered under a certain precondition.
[0059] Therefore, after selecting the RPC in the test state as the RPC for the current iteration through the fault library, the precondition entries corresponding to the RPC are determined from the precondition library based on the RPC, and the preconditions that trigger the RPC are determined from the determined precondition entries.
[0060] After "detecting the RPC triggered by error handling logic" in step S12, the following steps are also included: Step S41: For the RPC triggered by the error handling logic, based on the preconditions of the RPC in the current iteration and the fault injected into the RPC in the current iteration, create a precondition entry in the precondition library corresponding to the RPC triggered by the error handling logic.
[0061] In this embodiment, after identifying the RPC triggered by the error handling logic, a precondition entry needs to be created in the precondition library for that RPC. The precondition is in the form of a binary tuple, i.e., (precondition, triggerable RPC), for example, [R1, outbound unavailable,<R2,R3> The message indicates that injecting an "outbound unavailable" fault into RPC R1 will trigger RPC R2 and RPC R3 in R1's error handling logic.
[0062] It should be noted that since some RPCs may share error handling logic, the error handling logic of one RPC may trigger multiple RPCs. A precondition entry may contain multiple triggerable RPCs. At the same time, an RPC may also have multiple different preconditions that trigger it. Therefore, an RPC may correspond to multiple precondition entries with different preconditions in the precondition library.
[0063] Figure 3 This is a flowchart illustrating the iterative RPC detection process in a remote procedure call fault injection testing method provided in an embodiment of this application. It shows the evolution process of iteratively discovering new RPCs by tracing the fault library and the prerequisite library and injecting faults into the detected RPCs.
[0064] refer to Figure 3 This includes the first and second iterations. Each precondition entry in the precondition library includes the precondition and the triggerable RPC. Each fault entry in the fault library includes the RPC, fault type, and test status. OutboundUA represents an "unavailable" fault in the outbound path, and InboundUA represents an "unavailable" fault in the inbound path.
[0065] For example, in Figure 3In the first iteration, one precondition entry in the precondition library is
None R1
R1 InboundUA Done
R1 OutboundUA Todo
[0066] Therefore, combining the precondition entry [None R1] in the precondition library and the fault entry [R1OutboundUA Todo] in the fault library, it is determined that the test for the OutboundUA fault condition of RPC R1 has not yet been injected. Therefore, it is necessary to inject the OutboundUA fault into RPC R1 to detect if there are any new RPCs. After injecting the OutboundUA fault into RPC R1, RPC R2 and RPC R3 are triggered. Therefore, corresponding entries need to be added to the precondition and fault libraries based on RPC R2 and RPC R3. Specifically, if RPC R2 and RPC R3 are triggered with the injection of the OutboundUA fault into RPC R1 as a precondition, it indicates that RPC R2 and RPC R3 are RPCs triggered by error handling logic with the injection of the OutboundUA fault into RPC R1 as a precondition. Therefore, the precondition entry [R1: OutboundUA] is added to the precondition library.<R2,R3> The symbol indicates that when an OutboundUA fault occurs in RPCR1, RPC R2 and RPC R3 can be triggered through the error handling logic of RPC R1. In addition, based on various fault types, fault entries
R2 InboundUA Todo
R3 TimeOutTodo
[0067] In the second iteration, based on the precondition entries [R1: OutboundUA R2, R3] and [R2InboundUA Todo], it will be determined that injecting an OutboundUA fault into RPC R1 is a precondition for fault injection into the InboundUA fault of RPC R2. Therefore, it is necessary to inject both OutboundUA faults into RPC R1 and InboundUA faults into RPC R2 simultaneously. If it is determined that RPC R4 and RPC R5 are triggered after the fault injection, it means that RPC R4 and RPC R5 are RPCs triggered by error handling logic with the preconditions of injecting an OutboundUA fault into RPC R1 and injecting an InboundUA fault into RPC R2. The fault injection tests for RPC R3, RPC R4, and RPC R5 are similar and will not be elaborated further.
[0068] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for remote procedure call fault injection testing, the method further comprising: steps S51 to S52: Step S51: Determine the error level of each RPC triggered by the error handling logic based on the number of times the error handling logic is triggered in the preconditions.
[0069] In this embodiment, considering that in a distributed system, some RPCs are triggered in normal processing logic while others are triggered in error handling logic, we define an RPC hierarchy to represent which layer of error handling logic each RPC belongs to, i.e., the error level. The error handling logic of RPCs in normal processing logic is triggered 0 times, so the error level of RPCs in normal processing logic is level 0. The error handling logic of RPCs in level 0 RPCs is triggered 1 time, so the error level of RPCs in the error handling logic of level 0 RPCs is level 1, and so on.
[0070] For example, the test suite collected a level 0 error for RPC R1 in normal processing logic. After injecting a fault into RPC R1, RPC R2 was triggered, so the level 1 error for RPC R2 was triggered. After injecting faults into both RPC R1 and RPC R2, RPC R3 was triggered, so the level 2 error for RPC R2 was triggered.
[0071] Step S52: Set the preset error level threshold.
[0072] In this embodiment, considering that RPCs appearing in deep error handling logic depend on the simultaneous occurrence of multiple specific faults, which is almost impossible to trigger in real-world scenarios, we set a preset error level threshold (e.g., 2 levels) to limit the error level of RPCs in error handling logic, thereby reducing the testing complexity of distributed systems.
[0073] The step S21, "for the RPC triggered by the error handling logic, create multiple fault entries in the fault database based on multiple faults", specifically includes: step S21-1, for the RPC triggered by the error handling logic with an error level lower than the preset error level threshold, create multiple fault entries in the fault database based on multiple faults. The step S41, "For the RPC triggered by the error handling logic, based on the preconditions of the current iteration RPC and the fault injected into the current iteration RPC, create a precondition entry in the precondition library corresponding to the RPC triggered by the error handling logic", includes: Step S41-1, for the RPC triggered by the error handling logic with an error level lower than the preset error level threshold, create a precondition entry in the precondition library based on the preconditions of the current iteration RPC and the fault injected into the current iteration RPC.
[0074] In this embodiment, specifically, in each iteration of testing, since the newly detected RPC is triggered within the error handling logic of the RPC in the current fault injection scheme, the error level of the newly detected RPC is one level higher than that of the RPC whose error handling logic triggers it. To reduce the testing complexity of the distributed system, we can set a preset error level threshold (e.g., set to 2) and stop further exploring new RPCs triggered by RPCs with an error level of 2 after fault injection. That is, if the error level of the RPC in the current iteration is 2, then no new fault entry will be created in the fault database based on the newly triggered RPC, nor will a new precondition entry be added to the precondition database based on the newly triggered RPC. However, for RPCs triggered by error handling logic with an error level lower than the preset error level threshold, new fault entries and new precondition entries will be created in both the fault database and the precondition database for RPCs triggered by error handling logic with an error level lower than the preset error level threshold.
[0075] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another remote procedure call fault injection testing method, which further includes: Step S61: Based on the precondition library and the fault library, determine the number of RPCs to be tested corresponding to each precondition entry. The RPC to be tested represents an RPC triggered by error handling logic that can be triggered simultaneously under a precondition and whose test status is to be tested.
[0076] Considering that injecting a fault per test run cannot fully utilize the test opportunities in complex end-to-end test cases, resulting in too many test runs and high test costs, we propose a batch RPC fault injection method.
[0077] Specifically, in this embodiment, a greedy batch processing method based on the current state of the fault database and the precondition database is adopted. First, each precondition entry in the precondition database is regarded as a candidate batch, and the RPC to be tested is taken as the test target. Then, according to each fault entry in the fault database, the number of RPCs to be tested contained in each precondition entry is calculated.
[0078] The step S14, "using the RPC triggered by the error handling logic as the RPC for the next iteration and performing the fault injection test for the next iteration", includes: step S14-1, using the multiple RPCs triggered by the error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested as the RPCs for the next iteration and performing the fault injection test for the next iteration.
[0079] In this embodiment, in order to cover as many discovered but untested RPC faults as possible in each test run, before each iterative test, we select the candidate batch based on the precondition entry containing the most RPCs to be tested in each precondition entry. Then, we select the multiple RPCs triggered by the error handling logic corresponding to the precondition entry as the RPCs for the next iteration and execute the fault injection test for the next iteration, ensuring that as many faults as possible can be injected simultaneously during each iterative test.
[0080] Example, reference Figure 3 In the first iteration, after injecting an OutboundUA fault into RPC R1, RPC R2 and RPC R3 were triggered. Therefore, in order to improve testing efficiency, in the second iteration, with the injection of an OutboundUA fault into RPC R1 as a prerequisite, faults were also injected into RPC R2 and RPC R3 to improve testing efficiency.
[0081] By employing the technical solutions described in the above embodiments, this approach maximizes the coverage of a single iteration of testing while minimizing planning overhead, thereby improving testing efficiency.
[0082] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another remote procedure call fault injection testing method. In this method, after executing step S14-1, "taking the multiple RPCs triggered by error handling logic corresponding to the precondition entries with the largest number of RPCs to be tested as the RPCs for the next iteration and executing the fault injection test for the next iteration", the method further includes steps S71 to S73: Step S71: Determine whether there is an abnormal RPC among the multiple RPCs triggered by error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested. The abnormal RPC represents an RPC triggered by error handling logic that did not occur after the injection of fault.
[0083] In this embodiment, considering that although this test method of injecting multiple faults in the same batch ensures the fault injection efficiency of a single iteration test, there may be RPCs that do not fail in actual execution due to the mutual influence between the faults injected at the same time. Therefore, after selecting the precondition items with the largest number of RPCs to be tested and performing the fault injection test, it is also necessary to deal with the gap between planned fault injection and actual fault injection.
[0084] Specifically, after fault injection, it is necessary to determine whether there are any RPCs that were injected but did not occur during the fault injection test of this iteration. These RPCs are called abnormal RPCs.
[0085] Step S72: In the case of an abnormal RPC, the test status of the fault entry corresponding to the abnormal RPC in the fault database is kept as pending test. For the abnormal RPC, a new precondition entry is created in the precondition database. The new precondition entry indicates that the abnormal RPC is triggered alone under the original precondition.
[0086] In this embodiment, when abnormal RPCs are identified, these RPCs that did not fail during actual execution should not be directly ignored. Instead, they need to be retested in subsequent test runs. Since these abnormal RPCs did not occur after the fault was injected, the test status of the corresponding fault entries in the fault database for these abnormal RPCs that did not occur needs to be maintained as "pending test" for the faults. For RPCs that were injected with faults and occurred according to the fault handling logic, the test status of the corresponding fault entries in the fault database will be updated from "pending test" to "completed" according to the faults injected into these RPCs.
[0087] Considering that abnormal RPCs may be caused by interference between multiple injected faults, it is necessary to eliminate interference from other faults before retesting these abnormal RPCs. Based on this, new precondition entries need to be constructed in the precondition library for these abnormal RPCs. The preconditions in the new precondition entries are the same as the preconditions in the original precondition entries of these abnormal RPCs, but no other faults that may cause interference are introduced. Therefore, the RPCs that can be triggered in the new precondition entries are only these abnormal RPCs.
[0088] Step S73: Re-execute the fault injection test on the abnormal RPC according to the new precondition entries.
[0089] In this embodiment, after constructing new precondition entries, when it is necessary to use the abnormal RPCs corresponding to these new precondition entries as the RPCs for the next iteration, it is possible to perform separate fault injection tests on these abnormal RPCs to ensure that there are no missed fault injection tests on abnormal RPCs.
[0090] Figure 4 This is a schematic diagram of the process of detecting abnormal RPCs during the iterative RPC detection process in a remote procedure call fault injection testing method provided in an embodiment of this application.
[0091] Example, reference Figure 4 The precondition library initially contains only two precondition entries (the green part), namely [R1Out.UA].<R2,R3> 】 and 【R1 In.UA <r2>The numbers [R2In.UA Todo] and [R3In.UA Todo] indicate that when RPC R1 experiences an OutboundUA fault, RPC R2 and RPC R3 can be triggered through RPC R1's error handling logic. Similarly, when RPC R1 experiences an InboundUA fault, RPC R2 can be triggered through RPC R1's error handling logic. The fault database initially contains only two fault entries (green text): [R2In.UA Todo] and [R3In.UA Todo]. These represent the test status of RPC R2 and RPC R3 respectively, indicating that they are both pending tests for InboundUA fault injection testing. Since RPC R2 and RPC R3 can be triggered simultaneously when an OutboundUA fault is injected into RPC R1, faults can be injected into both RPC R2 and RPC R3 to improve testing efficiency. Therefore, injecting an OutboundUA fault into RPC R1 is a prerequisite, while injecting InboundUA faults into RPC R2 and RPC R3 respectively. After fault injection, it was found that RPC R2 experienced (corresponding to...) Figure 4 The fault injection was successful, but RPC R3 did not occur (corresponding to...). Figure 4 The fault injection failed), and a new RPC, RPC R4, was triggered. This indicates that RPC R3 was an abnormal RPC, possibly because the fault injected into RPC R2 interfered with RPC R3, causing RPC R3 to fail to occur after the fault injection. Therefore, it is determined that RPC R4 was triggered by the error handling logic of RPC R2. Thus, a new precondition entry is added to the precondition library for RPC R4, namely […].<R1,Out.UA><R2,In.UA> R4 indicates that RPC R4 can be triggered as a prerequisite for an OutboundUA failure in RPC R1 and an InboundUA failure in RPC R2. For various failure types, multiple failure entries are added to the failure database for RPC R4, namely [R4 In.UA Todo] and [R4 Time Out Todo], which respectively indicate that the test status of RPC R4 in the failure injection test of the failure type InboundUA is pending test, and the test status of RPC R4 in the failure injection test of the failure type Time Out is pending test.
[0092] Meanwhile, since the fault injection test for InboundUA in RPC R2 has been completed, the test status of the fault entry [R2 In.UA Todo] in RPC R2 is updated to [R2 In.UA Done] in the fault database. Since the fault injection test for InboundUA in RPC R3 has not occurred, the test status of the fault entry [R3 In.UA Todo] in RPC R3 remains unchanged in the fault database.
[0093] Additionally, new precondition entries can be reconstructed for RPC R3.<R1 Out.UA> R3 indicates that RPC R3 can be triggered independently as a prerequisite for the occurrence of an OutboundUA failure in RPC R1, thereby enabling separate fault injection testing of RPC3 (abnormal RPC) during subsequent testing.
[0094] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for remote procedure call fault injection testing. In this method, after executing step S73, "re-execute the fault injection test on the abnormal RPC according to the new precondition entry", steps S81 to S83 are further included: Step S81: Determine whether the abnormal RPC occurs after re-executing the fault injection test.
[0095] In this embodiment, after re-performing the fault injection test on the abnormal RPC, it is determined whether the abnormal RPC generates corresponding fault handling logic due to the injection fault. If not, it means that the abnormal RPC still did not occur after the retest.
[0096] Step S82: If the abnormal RPC does not occur after the fault injection test is re-executed, the new precondition entry in the precondition library is updated for the abnormal RPC to obtain an updated precondition entry. The updated precondition entry represents the abnormal RPC being triggered individually under other preconditions.
[0097] In this embodiment, if it is determined that the abnormal RPC still does not occur after retesting, the newly created precondition entry needs to be updated. The update process is to replace the original precondition in the new precondition entry with other preconditions. The other preconditions are preconditions that can trigger the abnormal RPC after the injection failure, thereby obtaining the updated precondition entry.
[0098] Step S83: Re-execute the fault injection test on the abnormal RPC according to the updated precondition entries.
[0099] In this embodiment, after obtaining the updated precondition entries, when it is necessary to use the abnormal RPC corresponding to the updated precondition entries as the RPC for the next iteration, it is possible to perform fault injection testing on these abnormal RPCs again based on other preconditions.
[0100] It should be noted that the remote procedure call fault injection testing method provided in this application embodiment can be executed by a remote procedure call fault injection testing device, or by a control module in the remote procedure call fault injection testing device for executing a remote procedure call fault injection testing method. This application embodiment uses the execution of a remote procedure call fault injection testing device to load a remote procedure call fault injection testing method as an example to illustrate the remote procedure call fault injection testing method provided in this application embodiment.
[0101] Figure 5 This is a schematic diagram of the framework of a remote procedure call fault injection testing device according to an embodiment of this application, with reference to... Figure 5 .
[0102] One embodiment of this application provides a remote procedure call fault injection testing apparatus, the apparatus comprising: The precondition retrieval module 11 is used to retrieve the preconditions of the RPC in the current iteration, wherein the preconditions of the RPC in the current iteration are the faults previously injected into other RPCs that triggered the occurrence of the RPC in the current iteration; The first fault injection test module 12 is used to inject faults into the RPC of the current iteration and perform fault injection tests when the precondition of the RPC of the current iteration is empty. The empty precondition indicates that the RPC of the current iteration can be triggered in normal processing logic without any precondition faults. RPC detection module 13 is used to detect RPCs triggered by error handling logic during fault injection testing of the RPCs in the current iteration, wherein the RPCs triggered by error handling logic are triggered by the error handling logic of the RPCs in the current iteration that are injected with faults. The iterative fault injection test module 14 is used to take the RPC of the current iteration as a prerequisite for the RPC triggered by the error handling logic, take the RPC triggered by the error handling logic as the RPC of the next iteration and execute the fault injection test of the next iteration, so as to discover the RPC triggered by the error handling logic through multiple iterations and perform fault injection tests on the discovered RPC triggered by the error handling logic.
[0103] Optionally, the apparatus further includes: a current iteration RPC determination module, used to determine, before retrieving the preconditions of the current iteration RPC, the RPC whose test state is to be tested as the RPC of the current iteration based on multiple fault entries contained in a pre-built fault library, wherein the fault library contains multiple fault entries, and each fault entry represents the test state of an RPC under a fault. The fault entry creation module is used to create multiple fault entries in the fault database based on various faults after discovering an RPC triggered by error handling logic, and to set the test status of the fault entry corresponding to the RPC triggered by error handling logic to be tested. The test status update module is used to set the test status of the fault entry corresponding to the fault injection test of the next iteration among the multiple fault entries corresponding to the RPC triggered by the error handling logic as a precondition, and after taking the RPC triggered by the error handling logic as the RPC of the next iteration and executing the fault injection test of the next iteration, the module sets the test status of the fault entry corresponding to the fault injection test of the next iteration among the multiple fault entries corresponding to the RPC triggered by the error handling logic as "completed".
[0104] Optionally, the precondition retrieval module 11 includes a retrieval unit, used to determine the preconditions of the RPC for the current iteration from multiple precondition entries in a pre-built precondition library, wherein the precondition library contains multiple precondition entries, and each precondition entry represents an RPC that can be triggered under a certain precondition. The apparatus further includes: a precondition entry creation module, configured to, upon discovering an RPC triggered by error handling logic, create a precondition entry in the precondition library corresponding to the RPC triggered by error handling logic, based on the preconditions of the current iteration RPC and the fault injected into the current iteration RPC.
[0105] Optionally, the device further includes: The error level determination module is used to determine the error level of each RPC triggered by the error handling logic based on the number of times the error handling logic is triggered in the preconditions. The preset error level threshold setting module is used to set the preset error level threshold; The fault entry creation module includes: a fault entry creation unit, used to create multiple fault entries in the fault database based on multiple faults for RPCs triggered by error handling logic with an error level lower than the preset error level threshold. The precondition entry creation module includes a precondition entry creation unit, used to create precondition entries in the precondition library for RPCs triggered by error handling logic with an error level lower than the preset error level threshold, based on the preconditions of the current iteration of the RPC and the faults injected into the current iteration of the RPC.
[0106] Optionally, the device further includes: The module for determining RPC data to be tested is used to determine the number of RPCs to be tested corresponding to each precondition entry based on the precondition library and the fault library. The RPC to be tested represents an RPC triggered by error handling logic that can be triggered simultaneously under a precondition and whose test status is to be tested. The iterative fault injection test module 14 includes: an iterative fault injection test unit, which is used to take the RPCs triggered by error handling logic corresponding to the precondition entries with the largest number of RPCs to be tested as the RPCs for the next iteration and execute the fault injection test for the next iteration.
[0107] Optionally, the device further includes: An abnormal RPC determination unit is used to determine whether there is an abnormal RPC among the multiple RPCs triggered by error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested, after taking them as the RPCs of the next iteration and performing the fault injection test of the next iteration. The abnormal RPC represents the RPC triggered by error handling logic that did not occur after the fault was injected. An exception entry construction unit is used to maintain the test status of the corresponding fault entry of the exception RPC in the fault database as pending test when an exception RPC exists, and to create a new precondition entry in the precondition database for the exception RPC, wherein the new precondition entry indicates that the exception RPC is triggered alone under the original precondition. The retest unit is used to re-execute the fault injection test on the abnormal RPC according to the new precondition entries.
[0108] Optionally, the device further includes: An anomaly determination unit is used to determine whether the abnormal RPC has occurred after the fault injection test is re-executed according to the new precondition entries. The precondition entry update unit is used to update the new precondition entry in the precondition library for the abnormal RPC if the abnormal RPC does not occur after the fault injection test is re-executed, so as to obtain the updated precondition entry. The updated precondition entry represents the abnormal RPC being triggered alone under other preconditions. The secondary retest unit is used to re-execute the fault injection test on the abnormal RPC according to the updated precondition entries.
[0109] The remote procedure call fault injection testing device in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the specific devices.
[0110] The remote procedure call fault injection testing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.
[0111] The remote procedure call fault injection testing device provided in this application embodiment can achieve... Figures 1 to 4 The various processes implemented by the remote procedure call fault injection test device in the method embodiment will not be described again here to avoid repetition.
[0112] Optionally, Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. This application also provides an electronic device; it should be noted that the electronic device in this application includes the mobile electronic device and non-mobile electronic device described above.
[0113] The electronic device includes, but is not limited to, components such as: radio frequency unit, network module, audio output unit, input unit, sensor, display unit, user input unit, interface unit, memory, and processor.
[0114] Those skilled in the art will understand that electronic devices may also include power supplies (such as batteries) that supply power to various components. The power supply may be connected to the processor logic through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.
[0115] As an example, such as Figure 6 As shown, the electronic device 600 includes a memory 610 and a processor 620. The memory 610 and the processor 620 are connected via a bus for communication. The memory 610 stores a computer program that can run on the processor 620 to implement the steps in the remote procedure call fault injection test method disclosed in the above embodiments of this application.
[0116] As the apparatus is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment.
[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects.
[0119] Furthermore, this application embodiment also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described remote procedure call fault injection test method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0120] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0121] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described remote procedure call fault injection test method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0122] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0123] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0125] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for testing remote procedure call fault injection, characterized in that, The method includes: Retrieve the preconditions for the RPC in the current iteration, where the preconditions for the RPC in the current iteration are failures previously injected into other RPCs that triggered the occurrence of the RPC in the current iteration; If the preconditions for the current iteration of the RPC are empty, a fault is injected into the current iteration of the RPC and a fault injection test is performed. The empty preconditions indicate that the current iteration of the RPC can be triggered in normal processing logic without any preconditions. During the fault injection test of the RPC in the current iteration, an RPC triggered by error handling logic was found, which was triggered by the error handling logic of the RPC in the current iteration that was injected with fault. The RPC of the current iteration is used as a prerequisite for the RPC triggered by the error handling logic. The RPC triggered by the error handling logic is used as the RPC of the next iteration and the fault injection test of the next iteration is executed. In order to discover the RPC triggered by the error handling logic through multiple iterations and perform fault injection tests on the discovered RPC triggered by the error handling logic.
2. The remote procedure call fault injection testing method according to claim 1, characterized in that, Before retrieving the preconditions for the RPC in the current iteration, the method further includes: determining the RPC whose test state is to be tested as the RPC in the current iteration based on multiple fault entries contained in a pre-built fault library, wherein the fault library contains multiple fault entries, and each fault entry represents the test state of an RPC under a fault. After discovering the RPC triggered by the error handling logic, the method further includes: for the RPC triggered by the error handling logic, creating multiple fault entries in the fault database based on multiple faults, and setting the test status of the fault entry corresponding to the RPC triggered by the error handling logic to be tested. After setting the RPC of the current iteration as a prerequisite for the RPC triggered by the error handling logic, setting the RPC triggered by the error handling logic as the RPC of the next iteration, and executing the fault injection test of the next iteration, the method further includes: setting the test status of the fault entry corresponding to the fault injection test of the next iteration among the multiple fault entries corresponding to the RPC triggered by the error handling logic to be completed.
3. The remote procedure call fault injection testing method according to claim 2, characterized in that, The prerequisites for retrieving the RPC in the current iteration. This includes determining the preconditions for the RPC of the current iteration from multiple precondition entries in a pre-built precondition library, wherein the precondition library contains multiple precondition entries, and each precondition entry represents an RPC that can be triggered under a certain precondition. After discovering an RPC triggered by error handling logic, the method further includes: for the RPC triggered by error handling logic, based on the preconditions of the RPC in the current iteration and the fault injected into the RPC in the current iteration, creating a precondition entry corresponding to the RPC triggered by error handling logic in the precondition library.
4. The remote procedure call fault injection testing method according to claim 3, characterized in that, The method further includes: Based on the number of times the error handling logic is triggered in the preconditions, determine the error level of each RPC triggered by the error handling logic; Set the preset error level threshold; For the RPC triggered by the error handling logic, multiple fault entries are created in the fault database based on multiple faults, including: for RPCs triggered by the error handling logic with an error level lower than the preset error level threshold, multiple fault entries are created in the fault database based on multiple faults. For the RPC triggered by the error handling logic, based on the preconditions of the current iteration of the RPC and the fault injected into the current iteration of the RPC, a precondition entry corresponding to the RPC triggered by the error handling logic is created in the precondition library. This includes: for RPCs triggered by the error handling logic whose error level is less than the preset error level threshold, a precondition entry is created in the precondition library based on the preconditions of the current iteration of the RPC and the fault injected into the current iteration of the RPC.
5. The remote procedure call fault injection testing method according to claim 3, characterized in that, The method further includes: Based on the precondition library and the fault library, the number of RPCs to be tested corresponding to each precondition entry is determined. The RPC to be tested represents an RPC triggered by error handling logic that can be triggered simultaneously under a precondition and whose test status is pending. The step of using the RPC triggered by the error handling logic as the RPC for the next iteration and performing the fault injection test for the next iteration includes: using multiple RPCs triggered by the error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested as the RPCs for the next iteration and performing the fault injection test for the next iteration.
6. The remote procedure call fault injection testing method according to claim 5, characterized in that, After selecting the RPCs triggered by error handling logic corresponding to the precondition entries with the largest number of RPCs to be tested as the RPCs for the next iteration and executing the fault injection test for the next iteration, the following is also included: Determine whether there is an abnormal RPC among the multiple RPCs triggered by error handling logic corresponding to the precondition entry with the largest number of RPCs to be tested. The abnormal RPC represents an RPC triggered by error handling logic that did not occur after the injection of fault. In the presence of an abnormal RPC, the test status of the fault entry corresponding to the abnormal RPC in the fault database is kept as pending test. In addition, for the abnormal RPC, a new precondition entry is created in the precondition database. The new precondition entry indicates that the abnormal RPC is triggered alone under the original precondition. The fault injection test is re-executed on the anomalous RPC based on the new precondition entries.
7. The remote procedure call fault injection testing method according to claim 6, characterized in that, After re-performing the fault injection test on the anomalous RPC according to the new precondition entries, the process also includes: Determine whether the abnormal RPC occurs after the fault injection test is re-executed; If the abnormal RPC does not occur after the fault injection test is re-executed, the new precondition entry in the precondition library is updated for the abnormal RPC to obtain an updated precondition entry. The updated precondition entry indicates that the abnormal RPC is triggered individually under other preconditions. The fault injection test is re-executed on the anomalous RPC based on the updated precondition entries.
8. A remote procedure call fault injection testing device, characterized in that, The device includes: The precondition retrieval module is used to retrieve the preconditions of the RPC in the current iteration. The preconditions of the RPC in the current iteration are the faults previously injected into other RPCs that triggered the occurrence of the RPC in the current iteration. The first fault injection test module is used to inject faults into the RPC of the current iteration and perform fault injection tests when the precondition of the RPC of the current iteration is empty. The empty precondition indicates that the RPC of the current iteration can be triggered in normal processing logic without any precondition faults. An RPC detection module is used to detect RPCs triggered by error handling logic during fault injection testing of the current iteration of the RPC, wherein the RPCs triggered by error handling logic are triggered by the error handling logic of the current iteration of the RPC that has been injected with faults. The iterative fault injection test module is used to take the RPC of the current iteration as a prerequisite for the RPC triggered by the error handling logic, take the RPC triggered by the error handling logic as the RPC of the next iteration, and execute the fault injection test of the next iteration, so as to discover the RPC triggered by the error handling logic through multiple iterations and perform fault injection tests on the discovered RPC triggered by the error handling logic.
9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in any one of claims 1 to 7.
10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Black-box fault injection method, systems, medium and equipment
CN108874663A
Micro-service application oriented failure test tool and method
CN110262972A
Program testing method and device, storage medium and electronic device
CN111538665A
Internet of Things firmware vulnerability mining method and system based on error scene generation
CN112380542A
Test method and device, storage medium and equipment
CN115203008A