Chaotic test method and device, electronic equipment and storage medium

Abstract

The application discloses a kind of chaotic test method, device, electronic equipment and storage medium.The chaotic test method includes: obtaining the basic information of each node in the system to be tested;System application architecture diagram is generated based on the basic information;According to the system application architecture diagram, determine the target fault node on the system application architecture diagram, the target fault node is the node to be executed fault;Determine the test execution information of the target fault node;According to the test execution information, the chaotic test is carried out to the system to be tested.The above technical solution, the basic information of each node in the system to be tested is used to generate system application architecture diagram, and then the target fault node is determined according to the system application architecture diagram, and the chaotic test is carried out to the system to be tested according to the test execution information of the target fault node, without the need for test personnel to input the corresponding command of chaotic test to carry out chaotic test, and the chaotic test can be made more systematic by generating system application architecture diagram.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of system testing technology, and in particular to a chaos testing method, apparatus, electronic device and storage medium. Background Technology

[0002] Chaos testing refers to the process of artificially injecting faults into a system to test its ability to handle and recover from faults. Through chaos testing, the system's service fault tolerance, high availability, monitoring and alarm effectiveness, and active-active architecture effectiveness can be verified, thereby obtaining information about the system's operation under fault conditions.

[0003] In existing technologies, chaos testing can be achieved by deploying testing tools in the application backend and executing corresponding chaos test commands. However, setting these commands may require testers to manually input the necessary information for each node in the system where the fault needs to be injected. This places high demands on testers' understanding of chaos test analysis, case organization logic, and operational skills. Furthermore, this testing method lacks analysis of the overall system architecture, making it impossible to systematically set up a chaos test execution plan or demonstrate the system architecture. Summary of the Invention

[0004] This invention provides a chaos testing method, apparatus, electronic device, and storage medium that can perform chaos testing without requiring testers to input corresponding commands. Furthermore, by generating a system application architecture diagram, chaos testing can be made more systematic.

[0005] In a first aspect, embodiments of the present invention provide a chaos testing method, comprising:

[0006] Obtain basic information about each node in the system under test;

[0007] A system application architecture diagram is generated based on the aforementioned basic information;

[0008] Based on the system application architecture diagram, the target fault node on the system application architecture diagram is determined, and the target fault node is the node to be executed.

[0009] Determine the test execution information for the target fault node;

[0010] The system under test is subjected to chaos testing based on the test execution information.

[0011] Secondly, embodiments of the present invention provide a chaos testing device, comprising:

[0012] The acquisition module is used to acquire basic information about each node in the system under test.

[0013] The generation module is used to generate a system application architecture diagram based on the basic information.

[0014] The first determining module is used to determine the target fault node on the system application architecture diagram according to the system application architecture diagram, wherein the target fault node is the node to be executed.

[0015] The second determining module is used to determine the test execution information of the target fault node;

[0016] The testing module is used to perform chaos testing on the system under test based on the test execution information.

[0017] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0021] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0022] The technical solution of this invention generates a system application architecture diagram by using the basic information of each node in the system under test, then determines the target fault node by using the system application architecture diagram, and performs chaos testing on the system under test based on the test execution information of the target fault node. Chaos testing can be performed without the tester having to input the corresponding commands for chaos testing, and the generation of the system application architecture diagram can make chaos testing more systematic.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a chaos testing method provided in Embodiment 1 of the present invention;

[0026] Figure 2 This is a schematic diagram of a system application architecture provided according to Embodiment 1 of the present invention;

[0027] Figure 3 This is a flowchart of a chaos testing method provided in Embodiment 2 of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a chaos testing device according to Embodiment 3 of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the chaos testing method of this invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0033] Example 1

[0034] Figure 1This is a flowchart of a chaos testing method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where chaos testing is performed on a system. The method can be executed by a chaos testing device, which can be implemented in software and / or hardware and integrated into an electronic device. Further, the electronic device includes, but is not limited to, computers, laptops, smartphones, servers, etc. Figure 1 As shown, the method includes:

[0035] S110. Obtain basic information about each node in the system under test.

[0036] The system under test can refer to the system to which chaos testing will be performed. Chaos testing involves artificially injecting faults into the system to test its ability to handle and recover from those faults.

[0037] Chaos testing can be used to verify the service fault tolerance capability of the system under test. This can include checking whether the faulty node is automatically isolated, whether traffic scheduling is normal, whether the circuit breaker mechanism is effective, estimating the system's fault tolerance threshold, and measuring the system's fault tolerance capability.

[0038] Verifying the high availability of the system under test can include, for example, simulating the killing of service nodes, simulating the unavailability of scheduling nodes, testing whether scheduling nodes automatically migrate to available nodes, simulating primary and backup node failures, and testing whether primary and backup switching is normal.

[0039] Verifying the effectiveness of the monitoring and alarms of the system under test may include: injecting faults into the system to verify whether the monitoring indicators are accurate, whether the monitoring dimensions are complete, whether the alarm thresholds are reasonable, whether the notification channels are available, and whether the alarm receiving system is correct, thereby improving the accuracy and timeliness of alarms.

[0040] Verify the effectiveness of the active-active architecture of the system under test, which may include: verifying the impact of system switching on business continuity, data integrity, and switching latency.

[0041] The system under test can have multiple nodes, and there are no restrictions on the type of nodes. They can be various types of servers, such as application servers, database servers, load balancers, Domain Name System (DNS) servers, or Remote Dictionary Server (Redis) servers.

[0042] Basic information can refer to information that can identify each node. Each node can have corresponding basic information. Through basic information, the corresponding node in the system under test can be uniquely identified, as well as the connection method between the node and other nodes in the system under test.

[0043] Furthermore, basic information includes at least the Internet Protocol address, server type, server name, username, password, parent node, child node, and information indicating whether redundancy is required.

[0044] Among them, the Internet Protocol Address (IP) address can be matched one-to-one with the nodes in the system under test. That is, each node in the system under test has a unique IP address. By querying the IP address, the basic information of the node corresponding to that IP address can be obtained.

[0045] There are no restrictions on the server type; it can be an application server, database server, load balancer, DNS server, or Redis server, etc. There are also no restrictions on the server name. In the system under test, each node can have a unique server name, or multiple nodes can correspond to the same server name. The server name can be set according to the actual application needs. When multiple nodes correspond to the same server name, different nodes can be distinguished by information in the basic information, including, in addition to the server name, such as IP address, parent node, or child node.

[0046] The username and password can correspond to the server name. Under each server name, there can be a corresponding username and password, and the server can be accessed through the username and password.

[0047] For any node in the system under test, the parent node can refer to the node connected to the parent node, and the lower node can refer to the node connected to the parent node. There is no limit to the number of parent and / or lower nodes; they can be zero or one or more, depending on the actual application requirements.

[0048] By using the IP address of any node in the system under test and its corresponding upper-level and lower-level nodes, the nodes connected to that node in the system under test can be determined. Similarly, by using the IP addresses of all nodes in the system under test and their corresponding upper-level and lower-level nodes, the connection relationships between all nodes in the entire system under test can be determined.

[0049] The upper-level nodes and / or lower-level nodes included in the basic information can be identified by their IP addresses. That is, the upper-level nodes and / or lower-level nodes recorded in the basic information can be the IP addresses corresponding to the upper-level nodes and / or lower-level nodes.

[0050] For any node in the system under test, the information indicating whether it is redundant can be understood as indicating whether there are other nodes in the system under test that can perform the corresponding function of that node, or achieve the same effect as that node. If the information indicating whether it is redundant indicates that the node can be redundant, then there are other nodes in the system under test that can perform the corresponding function of that node, or achieve the same effect as that node; conversely, if the information indicating whether it is redundant indicates that the node cannot be redundant, then there are no other nodes in the system under test that can perform the corresponding function of that node, or achieve the same effect as that node.

[0051] The method for obtaining basic information about each node in the system under test is not limited, as long as it is possible to obtain the basic information of each node in the system under test. For example, the basic information of each node in the system under test can be obtained through a human-computer interaction device. The human-computer interaction device can display a basic information setting interface, through which the user can fill in the basic information of each node. Alternatively, the basic information of each node can be preset in a database, and the electronic device can retrieve the basic information of each node from the database when needed. The basic information of each node can be preset according to the actual application needs and saved to the database. The human-computer interaction device can be a device for human-computer interaction by an interactor (i.e., a user). The human-computer interaction device includes, but is not limited to, touch screens and input components (such as keyboards and buttons).

[0052] S120. System application architecture diagram based on basic information generation.

[0053] A system application architecture diagram can refer to the architectural relationship diagram between various nodes in a system.

[0054] In the system application architecture diagram, the system architecture can be divided into multiple layers based on actual application requirements. Each layer can have one or more nodes. When any layer has multiple nodes, the nodes in that layer can be of the same server type and can be used to implement the same function.

[0055] There are no restrictions on the method used to generate a system application architecture diagram based on basic information, as long as it can be generated from the basic information. For example, the connection relationships between nodes in the entire system under test can be determined based on the basic information, and a system application architecture diagram can be generated using diagramming software. The diagramming software itself is not limited, as long as it can be used to generate the system application architecture diagram.

[0056] In one embodiment, the user inputs the basic information of each node in the system through the basic information setting interface in the human-computer interaction device, and draws the system application architecture diagram in the drawing software based on the basic information of each node in the system.

[0057] A system application architecture diagram can visualize the structure of a system, making it easier for users to understand the connection methods between nodes within the system or to determine the data flow between nodes.

[0058] S130. Based on the system application architecture diagram, determine the target fault node on the system application architecture diagram. The target fault node is the node where the fault is to be executed.

[0059] The target fault node is the node to which the fault is to be executed. Faults can be injected into the target fault node to test the system's fault handling and recovery capabilities. There is no limit to the number of target fault nodes; there can be one or more, and the specific number can be set according to the actual application needs.

[0060] The method for determining the target fault node on the system application architecture diagram is not limited, as long as the target fault node on the system application architecture diagram can be determined. For example, any path from the top-level node to the bottom-level node can be selected in the system application architecture diagram, and the target fault node can be set as one or more nodes in the system application architecture diagram excluding the nodes traversed by the path.

[0061] In one embodiment, a target fault node on the system application architecture diagram can be determined using information indicating redundancy included in the basic information. Specifically, if the basic information of a node in the system indicates that the node is redundant, then the node can be set as the target fault node, but it must be ensured that at least one node in the layer where the node resides in the system application architecture diagram is not the target fault node.

[0062] S140. Determine the test execution information for the target fault node.

[0063] Test execution information refers to information about the faults injected into the target fault node. Chaos testing can be performed on the system under test based on this information. For example, the test execution information may indicate the fault type to be injected, the fault execution order, and / or the fault execution time. For any target fault node, the test execution information can indicate one or more fault types; that is, different fault types can be set for a single target fault node according to actual needs.

[0064] The method for determining the test execution information of the target fault node is not limited, as long as the test execution information of the target fault node can be determined. For example, a test execution information setting interface can be displayed in the human-machine interface, through which the user can set the test execution information of the target fault node; another method is to determine the test execution information of the target fault node through a test execution information template. The test execution information template can have multiple different combinations of fault type, fault execution order, and / or fault execution time, and the test execution information template can be saved locally on the electronic device. When it is necessary to determine the test execution information of the target fault node, the electronic device can retrieve the test execution information template saved locally on the electronic device and randomly select one of the combinations of fault type, fault execution order, and / or fault execution time, which is the final determined test execution information of the target fault node.

[0065] S150. Perform chaos testing on the system under test based on the test execution information.

[0066] The method for performing chaos testing on the system under test based on the test execution information is not limited, as long as chaos testing can be performed on the system under test based on the test execution information. For example, the test execution information determines the type of fault that needs to be injected into the target fault node, and the method for performing chaos testing is determined based on the fault type. When the fault type indicates that it can be executed by remotely sending system commands, the electronic device can remotely send system commands to the target fault node, enabling the system to perform chaos testing; when the fault type indicates that it can be executed by script, the electronic device can call the corresponding script to perform chaos testing. The script can be saved locally on the electronic device in advance according to the actual application needs.

[0067] The technical solution of this invention generates a system application architecture diagram by using the basic information of each node in the system under test, then determines the target fault node by using the system application architecture diagram, and performs chaos testing on the system under test based on the test execution information of the target fault node. Chaos testing can be performed without the tester having to input the corresponding commands for chaos testing, and the generation of the system application architecture diagram can make chaos testing more systematic.

[0068] Furthermore, chaos testing methods also include:

[0069] When performing chaos testing on the system under test, the system application architecture diagram is displayed;

[0070] In the system application architecture diagram, identify the target fault node that is currently undergoing fault handling;

[0071] Generate a chaos test report, which reflects the ability of the system under test to provide services when there is a node failure.

[0072] When performing chaos testing on the system under test, the system application architecture diagram can be displayed on the human-computer interaction device, so that users can understand the connection method between each node in the system or determine the data flow between each node in the system through the displayed system application architecture diagram.

[0073] There are no restrictions on how the target fault node undergoing fault handling is identified in the system application architecture diagram. For example, it can be highlighted in the system application architecture diagram; or, different numbers can be assigned to the nodes in the system, with each number corresponding to a unique node. During chaos testing, the number corresponding to the target fault node undergoing fault handling is displayed, thereby identifying the target fault node undergoing fault handling.

[0074] Figure 2 This is a schematic diagram of a system application architecture provided in Embodiment 1 of the present invention, as shown below. Figure 2 As shown in the system application architecture diagram, the system application architecture is divided into 6 layers. From the top to the bottom, the number of nodes in each layer is 1, 2, 3, 1, 2, and 3 respectively. When there are multiple nodes in a certain layer, the nodes included in that layer can perform the same function or achieve the same effect, that is, node redundancy can be achieved. The system application architecture diagram also shows the server name corresponding to each node. The system application architecture diagram also shows the data flow from the top to the bottom.

[0075] exist Figure 2 In the system application architecture diagram shown, the node with the server name pod2 is marked to distinguish it from other nodes in the system application architecture diagram. The marked node indicates that it is the target fault node for which fault handling is being performed.

[0076] A chaos test report can refer to a report reflecting the results of chaos tests, used to demonstrate the ability of the system under test to provide services when node failures occur. There are no restrictions on how chaos test reports are generated; for example, test reports can be generated using plugins. These plugins can be downloaded from electronic devices via a web interface, or they can be user-defined plugins used to generate test reports.

[0077] Chaos test reports can highlight single points of failure in the system, as these failures can affect the entire system's operation. When there are multiple nodes in any layer, ensuring that only one node in that layer is not the target failure node, if that node fails and causes the system to become unavailable, the chaos test report can indicate that the failure of redundant nodes has led to system unavailability. The chaos test report can also indicate the performance of each node in providing continuous service under various failure conditions.

[0078] Example 2

[0079] Figure 3 This is a flowchart of a chaos testing method provided in Embodiment 2 of the present invention. This embodiment is a further refinement based on Embodiment 1 above.

[0080] In this embodiment of the invention, a system application architecture diagram is generated based on basic information, including:

[0081] Based on the Internet Protocol addresses, upstream nodes, and downstream nodes included in the basic information, determine the connection methods between the nodes in the system under test;

[0082] Generate a system application architecture diagram based on the connection methods between nodes in the system under test.

[0083] In this embodiment of the invention, determining the target fault node on the system application architecture diagram according to the system application architecture diagram includes:

[0084] Based on the system application architecture diagram, the shortest path from the top layer to the bottom layer is calculated using the minimum spanning tree algorithm;

[0085] Based on the shortest path, determine the nodes that can be set for failure;

[0086] Select one or more of the configurable fault nodes as the target fault node;

[0087] The number of shortest paths can be one or more.

[0088] In this embodiment of the invention, determining the test execution information of the target fault node includes:

[0089] The test execution information of the target fault node is obtained through the human-computer interaction interface. The test execution information includes: fault type, fault execution sequence and / or fault execution time.

[0090] like Figure 3 As shown, the method includes:

[0091] S110. Obtain basic information about each node in the system under test.

[0092] S121. Based on the Internet Protocol address, superior node, and subordinate node included in the basic information, determine the connection method between each node in the system under test.

[0093] For any node in the system under test, its Internet Protocol (IP) address, included in its basic information, can be used to distinguish it from other nodes in the system. The parent and child nodes listed in the basic information can be used to identify other nodes connected to that node in the system under test. Similarly, by using the IP addresses of all nodes in the system under test, along with their corresponding parent and child nodes, the connection relationships between all nodes in the entire system under test can be determined.

[0094] S122. Generate a system application architecture diagram based on the connection method between each node in the system under test.

[0095] The method for generating the system application architecture diagram based on the connection methods between nodes in the system under test is not limited, as long as it can generate the system application architecture diagram. For example, the system application architecture diagram can be generated using diagramming software based on the connection methods between nodes in the system under test. The diagramming software is not limited, as long as it can be used to generate the system application architecture diagram.

[0096] A system application architecture diagram can visualize the structure of a system, making it easier for users to understand the connection methods between nodes within the system or to determine the data flow between nodes.

[0097] S131. Based on the system application architecture diagram, use the minimum spanning tree algorithm to calculate the shortest path from the top layer to the bottom layer.

[0098] The minimum spanning tree algorithm can be as follows: In a given undirected graph G = (V, E), where V is the set of all vertices and E is the set of all weighted edges, two new sets U and T are set up. Set U stores the vertices in the minimum spanning tree of G, and set T stores the edges in the minimum spanning tree of G. Let (u, v) represent the edge connecting vertex u and vertex v, where u ∈ U, v ∈ VU, and ω(u, v) represent the weight of this edge. If there exists a subset T of E that is acyclic, such that... If the weight ω(T) of the edge in the minimum spanning tree of G is minimized, then this T is the minimum spanning tree of G. Based on the minimum spanning tree algorithm, the shortest path corresponding to the minimum spanning tree can be calculated.

[0099] The shortest path in this embodiment of the invention can be, for example, Figure 2 In the system application architecture diagram shown, there are multiple shortest paths from any node in the top layer to any node in the bottom layer, and each shortest path can enable data in the system under test to be transmitted from the top layer to the bottom layer.

[0100] S132. Determine the fault nodes that can be set based on the shortest path.

[0101] A configurable fault node refers to a node that can be configured to fail. There is no limit to the number of configurable fault nodes; the specific number can be determined based on the actual application requirements.

[0102] Furthermore, based on the shortest path, the nodes that can be set as faulty nodes are determined, including:

[0103] Nodes other than those traversed by the shortest path in the system under test are designated as faultable nodes.

[0104] By designating any node in the system under test that is not traversed by the shortest path as a faultable node, it can be ensured that the system can be connected through the shortest path when all faultable nodes are set to fault.

[0105] S133. Select one or more of the configurable fault nodes as the target fault node.

[0106] There is no limit to the number of target fault nodes; the number can be selected according to actual application needs. One or more configurable fault nodes can be selected as target fault nodes. This can be done by the electronic device randomly selecting from the configurable fault nodes, or by displaying the configurable fault nodes on the human-machine interface, allowing the user to select the target fault node through the human-machine interface.

[0107] S141. Obtain test execution information of the target fault node through the human-computer interaction interface. The test execution information includes: fault type, fault execution sequence and / or fault execution time.

[0108] The fault type refers to the type of fault that the target fault node needs to perform. There are no restrictions on the fault type; examples include increasing Central Processing Unit (CPU) utilization, increasing memory utilization, increasing disk input / output (I / O) reads, filling disk space, simulating network card latency, simulating network packet loss, tampering with Domain Name Exchange (DNX), stopping network card operation, and stopping application node operation. One or more fault types can be set for the same target fault node, without any specific limitations.

[0109] The fault execution sequence can refer to the order in which different target fault nodes execute faults, or the order in which different fault types are executed on the same target fault node.

[0110] Fault execution time can refer to the time it takes for the target fault node to execute different fault types. Fault execution time may include, but is not limited to, fault execution start time and fault execution end time.

[0111] Obtaining the fault type, fault execution sequence, and / or fault execution time of the target fault node through the human-computer interaction interface can be understood as follows: the user can set the fault type, fault execution sequence, and / or fault execution time of the target fault node through the human-computer interaction interface. After the user sets the settings, the electronic device can obtain the fault type, fault execution sequence, and / or fault execution time of the target fault node through the human-computer interaction interface.

[0112] For example, a control for selecting fault types can be set in the human-computer interaction interface. Different fault types can correspond to different controls. When the user clicks the control corresponding to the fault type, the fault type can be set for the target fault node. Another example is that the fault execution order can be automatically generated after the user sets the fault types, and can be displayed in the human-computer interaction interface. Users can customize and modify the fault execution order, such as changing the order in which different target fault nodes execute faults, or changing the order in which different fault types are executed under the same target fault node, or they can leave the fault execution order unchanged and let it execute according to the automatically generated fault execution order. Furthermore, users can input fault execution times in the human-computer interaction interface, such as the fault execution start time and fault execution end time.

[0113] S150. Perform chaos testing on the system under test based on the test execution information.

[0114] In one embodiment, steps S141 and S150 can be understood as follows: the user can set the test execution information of the target fault node through the human-machine interface. After the user sets the information, the electronic device can obtain the test execution information of the target fault node through the human-machine interface. Optionally, a control to start the test can be set in the human-machine interface. When the user clicks the control, the electronic device can perform a chaotic test on the system under test according to the test execution information.

[0115] The technical solution of this invention generates a system application architecture diagram based on the basic information of each node in the system under test. Then, it identifies the target fault node using the system application architecture diagram and obtains the test execution information of the target fault node through a human-computer interaction interface. Based on this test execution information, it performs chaotic testing on the system under test. The human-computer interaction interface enables interaction between the tester (i.e., the user) and the electronic device, allowing the tester to systematically set the test execution information of the target fault node without needing to input specific chaotic test commands.

[0116] The present invention will be described by way of example below:

[0117] The systems involved in chaos testing often include large-scale clusters with multiple architectural layers and numerous nodes. Especially in the case of a dual-active architecture, the system has high requirements for fault tolerance and high availability. Existing chaos testing tools verify the system by setting faults using commands on the node under test (i.e., the target fault node), which is more focused on the verification point and cannot provide a comprehensive and systematic verification.

[0118] Existing chaos testing methods cannot clearly demonstrate the system architecture, lack strong visualization, and have poor display and analysis of monitoring results; they also place high demands on testers' learning of chaos test analysis, case organization logic, and execution operations; and chaos test execution relies on testers manually inputting execution commands.

[0119] This invention proposes a visualization method for the automatic generation, execution, and monitoring of chaos test cases based on a web-based platform. It can automatically generate a system architecture diagram based on input information (i.e., basic information) and analyze the shortest path of the architecture; it can automatically generate chaos test execution case scripts based on system node information (i.e., test execution information of the target fault node), and execute them remotely via a web platform; it monitors the fault execution process on the web platform, returns monitoring logs in the background, and finally generates a system analysis report (i.e., a chaos test report), highlighting single nodes affecting system robustness and the system operation status of each node under various fault states.

[0120] This method can be divided into the following 6 steps:

[0121] 1. Input the IP address, server type, username, password, root node (i.e., parent node), child nodes (i.e., subordinate nodes), and redundancy status (indicating redundancy) for each server in the system architecture. 2. Generate the system application architecture diagram based on the input information (i.e., basic information). 3. Calculate the minimum system architecture and the shortest path to obtain that architecture. 4. Allows setting the fault type for each application node (i.e., the target fault node), and allows customization of node order, fault order (i.e., fault execution order), and execution time (i.e., fault execution time). 5. During continuous test case sending, sequentially execute all application node fault types (e.g., service unavailability, system resource overload, network latency, network card failure, network packet loss, DNS domain name resolution tampering, etc.) to verify system availability. Real-time monitoring and display are available. 6. Generate a chaos test report based on the test results, highlighting single points of failure in the system (i.e., node failure affecting the entire system operation), indicating situations where redundant nodes experience certain failures leading to system unavailability, and the system's performance in providing continuous service under various fault conditions for each other node.

[0122] Step 1: Fill in the specific information of each node in the system in Table 1 (i.e., the tester inputs the basic information of each node in the system through the basic information setting interface in the human-computer interaction device). Table 1 is the basic information setting table displayed in the basic information setting interface.

[0123] Table 1 Basic Information Settings Table

[0124]

[0125] In the IP address field, enter the IP address of any node in the system. In the parent node and child node fields, enter the IP addresses of the corresponding parent and child nodes, respectively.

[0126] Server types include: application servers, database servers, F5 load balancers, DNS domain name servers, Redis servers, etc.

[0127] Testers can input the basic information of each node in the system through the basic information setting interface of the human-computer interaction device, which will enable the electronic device to obtain the basic information of each node in the system through the human-computer interaction device.

[0128] Step 2: Generate a system call relationship architecture diagram based on the IP address of the filled information, as well as the superior and subordinate nodes (i.e., steps S121 and S122).

[0129] Step 3: Calculate the minimum architecture and obtain the shortest path of the architecture using the minimum spanning tree algorithm in graph theory (i.e., step S131).

[0130] During execution, the minimum spanning tree nodes should be able to function normally without simultaneous failures.

[0131] Step 4: Determine the target fault node based on the shortest path (i.e., steps S132 and S133), and customize the fault type for the target fault node, including: increasing CPU utilization, increasing memory utilization, increasing disk I / O read volume, filling disk space, simulating network card latency, simulating network packet loss, tampering with DNX domain name, stopping network card operation, stopping application node operation, etc.; you can also edit the execution node order, fault order (i.e., fault execution order), and set the task to be executed on a timer (i.e., fault execution time) (i.e., step S141).

[0132] Customizing the fault type for the target fault node can be done by testers selecting the fault type through a human-machine interface (HMI). The HMI can display a fault type selection table as shown in Table 2. In Table 2, an IP address can be added to set the fault type for the target fault node corresponding to that IP address. Table 2 also includes controls for selecting fault types; different fault types correspond to different controls. When a tester clicks the corresponding control, that fault type can be set for the target fault node.

[0133] Table 2 Fault Type Selection Table

[0134]

[0135] Depending on the type of fault, the fault can be resolved by remotely sending system commands, such as remotely logging in to execute system commands. For faults that are not handled by system commands but are implemented by scripts, the script template is modified according to the input node description information, including changes to information such as IP address, username, and password. This modified script is then transmitted remotely to the server where the fault operation needs to be performed, and the operation is executed remotely.

[0136] After setting in Table 2, the execution order in Table 3 can be automatically generated. Table 3 shows the fault execution order setting table. Testers can set the fault execution order and fault execution time through Table 3, or execute the fault according to the default generated execution order. The execution time can be determined according to the time of clicking "Start Test".

[0137] Table 3 Fault Execution Sequence Setting Table

[0138]

[0139] Testers can set the fault type, fault execution sequence, and fault execution time through Tables 2 and 3 in the human-machine interface, enabling electronic devices to obtain test execution information of the target fault node through the human-machine interface.

[0140] Furthermore, a control for starting the test can be set in the human-computer interaction interface. When the tester clicks the control, the electronic device can perform a chaos test on the system under test based on the test execution information.

[0141] Step 5: Automatically initiate fault execution operations according to the fault execution sequence, and display the system application architecture diagram graphically on the web interface, highlighting the target fault nodes currently being processed (e.g., ...). Figure 2 As shown in Table 4 below, the execution results are displayed in the execution result table.

[0142] Table 4 Execution Results

[0143]

[0144] Step Six: Based on the test results, generate a chaos test report (i.e., step S150), indicating the single point of failure in the system, i.e., the failure of this node affects the operation of the entire system, indicating the situation where a redundant node fails and causes the system to become unavailable, and the performance of the system in providing continuous service under various failure conditions of each other node.

[0145] The present invention proposes a visualization method for automatically generating, executing, and monitoring chaos test cases based on a web-based platform. Testers do not need to master the scripts for setting up each fault and the writing of commands; they only need to describe the node information according to the system nodes and customize or automate the execution of faults. The test process and real-time node logs can be monitored and displayed on a web page. Detailed test reports can be generated based on the test results, highlighting single points of failure in the system architecture and showing the system's processing capabilities under fault conditions at each node. The report also provides results on system fault tolerance, high availability, alarm and monitoring effectiveness, and the reliability of the active-active architecture.

[0146] Example 3

[0147] Figure 4 This is a schematic diagram of a chaos testing device according to Embodiment 3 of the present invention. This embodiment is applicable to situations where chaos testing is performed on a system. Figure 4 As shown, the specific structure of the device includes:

[0148] Module 21 is used to acquire basic information about each node in the system under test;

[0149] Generation module 22 is used to generate a system application architecture diagram based on basic information;

[0150] The first determining module 23 is used to determine the target fault node on the system application architecture diagram according to the system application architecture diagram. The target fault node is the node to be executed.

[0151] The second determining module 24 is used to determine the test execution information of the target fault node;

[0152] Test module 25 is used to perform chaos testing on the system under test based on the test execution information.

[0153] The chaos testing device provided in this embodiment first acquires the basic information of each node in the system under test through the acquisition module 21; then, the generation module 22 generates a system application architecture diagram based on the basic information; then, the first determination module 23 determines the target fault node on the system application architecture diagram according to the system application architecture diagram, and the target fault node is the node to be executed for fault; then, the second determination module 24 determines the test execution information of the target fault node; finally, the test module 25 performs chaos testing on the system under test according to the test execution information.

[0154] Furthermore, basic information includes at least the Internet Protocol address, server type, server name, username, password, parent node, child node, and information indicating whether redundancy is required.

[0155] Furthermore, the generation module 22 is specifically used for:

[0156] Based on the Internet Protocol addresses, upstream nodes, and downstream nodes included in the basic information, determine the connection methods between the nodes in the system under test;

[0157] Generate a system application architecture diagram based on the connection methods between nodes in the system under test.

[0158] Furthermore, the first determining module 23 is specifically used for:

[0159] Based on the system application architecture diagram, the shortest path from the top layer to the bottom layer is calculated using the minimum spanning tree algorithm.

[0160] Based on the shortest path, determine the nodes that can be set for failure;

[0161] Select one or more of the configurable fault nodes as the target fault node;

[0162] The number of shortest paths can be one or more.

[0163] Furthermore, the first determining module 23 is specifically used for:

[0164] Nodes other than those traversed by the shortest path in the system under test are designated as faultable nodes.

[0165] Furthermore, the second determining module 24 is specifically used for:

[0166] The test execution information of the target fault node is obtained through the human-computer interaction interface. The test execution information includes: fault type, fault execution sequence and / or fault execution time.

[0167] Furthermore, the device also includes:

[0168] The display module is used to display the system application architecture diagram when performing chaos testing on the system under test.

[0169] The identification module is used to identify the target fault node that is undergoing fault handling in the system application architecture diagram;

[0170] The chaos test report generation module is used to generate chaos test reports, which reflect the ability of the system under test to provide services when there is a node failure.

[0171] The chaos testing device provided in the embodiments of the present invention can execute the chaos testing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0172] Example 4

[0173] Figure 5 This is a schematic diagram of the structure of an electronic device implementing the chaos testing method of this invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0174] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0175] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0176] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as chaos testing methods.

[0177] In some embodiments, the chaos testing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the chaos testing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the chaos testing method by any other suitable means (e.g., by means of firmware).

[0178] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0179] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0180] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0181] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0182] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0183] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0184] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0185] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A chaos testing method, characterized in that, include: Obtain basic information about each node in the system under test; A system application architecture diagram is generated based on the aforementioned basic information; Based on the system application architecture diagram, the target fault node on the system application architecture diagram is determined, and the target fault node is the node to be executed. Determine the test execution information for the target fault node; Perform a chaos test on the system under test based on the test execution information; Specifically, determining the target fault node on the system application architecture diagram, based on the system application architecture diagram, includes: Based on the system application architecture diagram, the shortest path from the top layer to the bottom layer is calculated using the minimum spanning tree algorithm. Based on the shortest path, determine the nodes that can be set as fault nodes; Select one or more of the configurable fault nodes as the target fault node; The number of shortest paths is one or more; Among them, determining the fault nodes based on the shortest path includes: Nodes other than those traversed by the shortest path in the system under test are designated as configurable fault nodes.

2. The method according to claim 1, characterized in that, The basic information includes at least the Internet Protocol address, server type, server name, username, password, parent node, child node, and information indicating whether it is redundant.

3. The method according to claim 1 or 2, characterized in that, Based on the aforementioned basic information, a system application architecture diagram is generated, including: Based on the Internet Protocol addresses, upstream nodes, and downstream nodes included in the basic information, determine the connection methods between the nodes in the system under test; A system application architecture diagram is generated based on the connection method between each node in the system under test.

4. The method according to claim 1, characterized in that, The test execution information for determining the target fault node includes: The test execution information of the target fault node is obtained through a human-computer interaction interface. The test execution information includes: fault type, fault execution order and / or fault execution time.

5. The method according to claim 1, characterized in that, Also includes: When performing a chaos test on the system under test, the system application architecture diagram is displayed. The target fault node that is currently undergoing fault handling is identified in the system application architecture diagram. A chaos test report is generated, which reflects the ability of the system under test to provide services when there is a node failure.

6. A chaos testing device, characterized in that, include: The acquisition module is used to acquire basic information about each node in the system under test. The generation module is used to generate a system application architecture diagram based on the basic information. The first determining module is used to determine the target fault node on the system application architecture diagram according to the system application architecture diagram, wherein the target fault node is the node to be executed. The second determining module is used to determine the test execution information of the target fault node; The testing module is used to perform chaos testing on the system under test based on the test execution information. The first determining module is specifically used for: Based on the system application architecture diagram, the shortest path from the top layer to the bottom layer is calculated using the minimum spanning tree algorithm. Based on the shortest path, determine the nodes that can be set as fault nodes; Select one or more of the configurable fault nodes as the target fault node; The number of shortest paths is one or more; The first determining module is specifically used for: Nodes other than those traversed by the shortest path in the system under test are designated as configurable fault nodes.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images