Chaotic testing method, apparatus, device, medium, and program product

Abstract

This application provides a chaos testing method, apparatus, device, storage medium, and program product, which can be applied to the big data field. The method includes: modifying the business data of key nodes based on fault factors in a business knowledge graph to obtain multiple fault nodes. The business knowledge graph is obtained from historical business data, and the key nodes are determined based on their impact on the business. Fault factors include adjustment values ​​for logical judgment conditions and timing delay values. The method also involves extracting the relationships between nodes in the business knowledge graph, and using multiple fault nodes as starting points, traversing the business knowledge graph based on these relationships to obtain multiple fault chains. Finally, the method injects the faults in each of the multiple fault chains into a test system to obtain test results.

Description

Technical Field

[0001] This application relates to the field of big data, and more specifically to a chaos testing method, apparatus, equipment, medium, and program product. Background Technology

[0002] Chaos testing is a testing method that verifies the stability and resilience of a system by simulating system failures. Typical chaos testing methods can only simulate infrastructure failures and cannot simulate deeper failures caused by abnormal business data or errors in business logic. Furthermore, the testing environment for typical chaos tests differs from the actual operating environment, leading to inaccurate test results. In addition, the entire testing process largely relies on manual intervention, resulting in long testing cycles and high labor costs. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a chaos testing method, apparatus, device, medium, and program product.

[0004] According to a first aspect of this application, a chaos testing method is provided, comprising: modifying the business data of key nodes based on fault factors in a business knowledge graph to obtain multiple fault nodes, wherein the business knowledge graph is obtained based on historical business data, the key nodes are determined based on the degree of influence of the nodes on the business, and the fault factors include adjustment values ​​of logical judgment conditions and timing delay values; extracting the association relationships between nodes in the business knowledge graph, and traversing the business knowledge graph based on the association relationships, starting from the multiple fault nodes respectively, to obtain multiple fault links; and injecting the faults in the multiple fault links into a test system to obtain test results.

[0005] According to an embodiment of this application, faults in multiple fault links are injected into the test system to obtain test results, including: obtaining performance indicators associated with the faults in the test system; based on the performance indicators and the system log information of the test system, data reconstruction is performed using an autoencoder to determine abnormal features; and the abnormal features are input into a fault type discrimination model to obtain the fault type.

[0006] According to an embodiment of this application, based on performance indicators and system log information of the test system, an autoencoder is used to reconstruct data and determine abnormal features, including: extracting the temporal features of performance indicators using a temporal convolutional network to obtain a temporal feature sequence; performing structured processing on the system log information and fusing the temporal feature sequence to obtain a fused feature; inputting the fused feature into the autoencoder for data reconstruction; if the deviation between the reconstructed data and the fused feature is greater than a preset threshold, then the fused feature is taken as an abnormal feature.

[0007] According to an embodiment of this application, the method further includes: obtaining fault attribute features from the fault based on the type, the fault attribute features including the scope of influence and the triggering conditions of the fault; matching a repair strategy from a preset fault repair strategy mapping library according to the fault attribute features and the current operating state of the test system to obtain a target repair strategy; and repairing the test system based on the target repair strategy.

[0008] According to an embodiment of this application, a target repair strategy is obtained by matching a repair strategy from a preset fault repair strategy mapping library based on fault attribute features and the current operating state of the test system. This includes: semantically matching the feature tags of the repair strategies in the preset fault repair strategy mapping library with the fault attribute features to obtain at least one candidate repair strategy; determining the feasibility score of the candidate repair strategy based on the current operating state, wherein the feasibility score characterizes the degree of feasibility of the current test system executing the candidate repair strategy; and selecting the candidate repair strategy with a score greater than a preset score as the target repair strategy.

[0009] According to an embodiment of this application, the method further includes: obtaining an environmental fingerprint based on the system configuration of the actual system corresponding to the test system, wherein the environmental fingerprint represents a numerical identifier encoded by the system configuration; parsing the environmental fingerprint to obtain an image matching the system configuration, and building a test environment for the test system.

[0010] According to an embodiment of this application, parsing an environment fingerprint, obtaining an image that matches the system configuration, and building a test environment for the test system includes: decoding and deserializing the environment fingerprint to obtain the image metadata of the system configuration; pulling an image that matches the image metadata from an image repository and injecting the system configuration into the runtime configuration layer of the image to build a test environment.

[0011] According to a second aspect of this application, a chaos testing apparatus is provided, comprising: a fault node acquisition module, used to modify the business data of key nodes based on fault factors in a business knowledge graph to obtain multiple fault nodes, wherein the business knowledge graph is acquired based on historical business data, key nodes are determined based on the degree of influence of nodes on business, and fault factors include adjustment values ​​of logical judgment conditions and timing delay values; a fault link acquisition module, used to extract the association relationships of each node in the business knowledge graph, and traverse the business knowledge graph based on the association relationships, starting from multiple fault nodes, to obtain multiple fault links; and a chaos testing module, used to inject the faults in the multiple fault links into a test system to obtain test results.

[0012] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0013] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0014] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 The illustrations depict application scenarios of chaos testing methods, apparatus, devices, media, and program products according to embodiments of this application.

[0017] Figure 2 A flowchart illustrating a chaos testing method according to an embodiment of this application is shown schematically.

[0018] Figure 3 This illustration schematically shows a fault type determination method according to an embodiment of the present application;

[0019] Figure 4 The illustration shows a schematic diagram of a method for determining abnormal performance indicators according to an embodiment of this application;

[0020] Figure 5 This illustration schematically shows a fault repair method for a test system according to an embodiment of the present application;

[0021] Figure 6 This illustration schematically shows a method for obtaining a target repair strategy according to an embodiment of this application;

[0022] Figure 7 A schematic diagram of a chaos testing apparatus according to an embodiment of this application is shown.

[0023] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a chaos testing method according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0029] In scenarios where personal information is used for automated decision-making, the methods, devices, and systems provided in this application all provide users with corresponding operation entry points for users to choose to agree to or reject the automated decision results; if the user chooses to reject, the process enters the expert decision-making process.

[0030] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0031] An "intelligent agent" is a system or entity capable of autonomously perceiving its environment, making decisions, and executing actions to accomplish a specific task. Large models can provide decision support for intelligent agents, offering them the capabilities of reasoning analysis and task planning. Intelligent agents utilize the analytical results of large models to execute or optimize their decision-making processes. Intelligent agents can integrate multiple large models to handle different types of tasks.

[0032] Figure 1 A schematic diagram of an example environment 100 to which the chaos testing method according to an embodiment of this application can be applied is shown. In this example environment 100, an application 125 is installed on a terminal device 110. A user 140 can interact with the application 125 via the terminal device 110 and / or an attached device to the terminal device 110.

[0033] In some embodiments, application 125 can be downloaded and installed on terminal device 110. In some embodiments, application 125 can also be accessed in other ways, such as through a web page. Figure 1 In environment 100, in response to application 125 being launched, terminal device 110 can display the interface 150 of application 125.

[0034] In some embodiments, terminal device 110 can communicate with server 130 to provide services to application 125. Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 can also support any type of user-facing interface. Application 125 can be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, and computing devices in cloud environments.

[0035] In some embodiments, application 125 may provide interaction capabilities with an intelligent agent. Application 125 may include an application specifically designed to provide services to an intelligent agent, or an application integrated with an intelligent agent. Although Figure 1 The image shows a single application, but in reality, multiple applications can be installed on the terminal device 110.

[0036] In embodiments of this application, multiple intelligent agents 160 can be deployed locally on terminal device 110 or remotely. In the case of remote deployment, terminal device 110 can directly invoke the intelligent agents, or it can invoke the intelligent agents via server 130. Exemplarily, intelligent agents 160 may have intelligent dialogue and task processing capabilities. Terminal device 110 provides an interface 150 that can present interactions with intelligent agents 160. In interface 150, user 140 can initiate task requests to intelligent agents 160 by inputting natural language (e.g., text input or voice input). Optionally, user 140 can upload online or offline file dialogues to instruct intelligent agents 160 to assist in completing various tasks.

[0037] In the embodiments of this application, during interaction with user 140, intelligent agent 160 can respond to user 140's requests and handle tasks instructed by the user. In some embodiments, during task processing, intelligent agent 160 can invoke one or more tools 165 to assist in task execution and the provision of task results as needed. These tools 165 can be any type of tool, such as text generation tools, file reading tools, information search tools, online or offline databases, image processing tools, chart generation tools, web page creation tools, etc.

[0038] In some embodiments, environment 100 may further include a management node for multiple agents 160, which can interact with the agents 160. In some examples, the management node may, in response to a task request from user 140, determine the task requirements corresponding to the task request. The management node may then, based on the task requirements, assign the task request to the agent 160 that matches the task requirements, requesting that agent 160 to perform the task. In other examples, the management node may also determine an execution plan for the task based on the task requirements. The execution plan may indicate one or more subtasks required to complete the task. The management node may assign these one or more subtasks to one or more agents 160, which will then execute their respective subtasks. Regarding the management node, in some examples, the management node may be implemented by one of the multiple agents 160. In other examples, the management node may be implemented by a machine learning model, such as a language model.

[0039] In some embodiments, agent 160 may be constructed based on one or more machine learning models. In some embodiments, the machine learning model on which agent 160 is based may include at least a language model, such as a large language model. In some embodiments, the machine learning model on which agent 160 is based may include a multimodal model capable of handling multiple modal inputs, such as text input, visual input (e.g., images, videos), audio input, etc. These machine learning models may include content-generating models capable of generating corresponding outputs based on model inputs. In some embodiments, the machine learning model may receive text-modal model inputs (e.g., natural language and / or machine language) and / or non-text-modal model inputs (e.g., images, speech, videos, etc.), and may obtain corresponding model outputs based on model inputs and prompts, thereby completing the task execution.

[0040] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and are not intended to limit the scope of this application in any way.

[0041] Figure 2 A flowchart illustrating a chaos testing method according to an embodiment of this application is shown schematically. Figure 2 As shown, the chaos testing method 200 according to an embodiment of this application may include steps S210 to S230.

[0042] In step S210, based on the fault factors of key nodes in the business knowledge graph, the business data of the key nodes are modified to obtain multiple fault nodes.

[0043] In step S220, the relationships between nodes in the business knowledge graph are extracted. Starting from multiple fault nodes, the business knowledge graph is traversed based on the relationships to obtain multiple fault links.

[0044] In step S230, the faults in the multiple faulty links are injected into the test system to obtain the test results.

[0045] In some embodiments, in step S210, the business knowledge graph includes the processing flow of each business. For example, for order business, after a user places an order, the system checks the inventory, deducts the ordered quantity from the inventory, and then generates an order to be paid. If any process in this flow fails, the user may fail to place the order. For example, if the user does not fill in the delivery address when placing the order, the order parameters will be missing, which will cause data verification failure and thus the transaction cannot be completed. For another example, for risk control review business, if the user's application information is incomplete or if null values ​​or overflows occur when cleaning the data, the system will experience processing failure, resulting in business failure. Key nodes are nodes that have a significant impact on the business, such as the verification of order parameters in the order business.

[0046] In some embodiments, bytecode enhancement technology can also be used to dynamically modify the business data of key nodes during system program runtime to simulate changes in business data logic. For example, in balance verification business, bytecode enhancement technology can be used to directly inject error logic into the code to simulate more complex fault scenarios. It can also modify the return value and logical judgment conditions of key nodes. Since bytecode enhancement technology has the following advantages: (1) no source code modification, directly operates Java Virtual Machine (JVM) layer instructions; (2) can modify any method logic; (3) the modification is completed in memory, without logs or exception characteristics; (4) it takes effect dynamically at runtime, without restarting the service; (5) it supports any JVM language; therefore, using bytecode enhancement technology to change the business data of key nodes can improve the efficiency of business logic modification.

[0047] In some embodiments, in step S220, based on other nodes that are associated with the faulty node, the nodes with the associated relationship are linked together according to the business processing logic, which can generate multiple links including the faulty node. These faulty links may be different from the links in the business knowledge graph, and belong to the business processing links derived from the business knowledge graph that conform to the business processing logic. For example, in the order business, it can be seen from the business knowledge graph that user points overflow is closely related to the order amount, the order amount is closely related to the order quantity, and the order quantity is closely related to the remaining inventory. Through this layered relationship, the user points overflow - order amount error - inventory anomaly obtained belongs to the faulty link obtained by the faulty node and the business knowledge graph.

[0048] In some embodiments, in step S230, if the faulty link is a network communication link, the fault may be packet loss, delay, or disconnection; if the faulty link is a call link between servers, the fault may be call timeout, no server response, or incorrect return code.

[0049] In some embodiments, by modifying the business data of key nodes, the faults injected into the test system can be determined. This can simulate not only fault injection scenarios at the infrastructure level, but also deeper fault injection scenarios, such as fault injection at the business logic level. This greatly enhances the scenario of chaos testing and helps improve the reliability of the test system.

[0050] According to embodiments of this application, problems that may occur at the business logic level are simulated by fault factor simulation data. Based on the relationship between nodes in the business knowledge graph, multiple fault links are obtained starting from the fault node. These fault links can be understood as deeper vulnerabilities in the code at the business logic level. The faults in the fault links are injected into the test system for testing. The test system can simulate deeper faults caused by business anomalies and code logic errors, so as to test the system's ability to repair such faults, thereby improving the system's reliability and resilience.

[0051] In some embodiments, the agent may perform steps S210 and S220 and inject the faults in the obtained fault links into the test system to perform chaos testing.

[0052] Figure 3 A schematic diagram of a fault type determination method according to an embodiment of this application is shown.

[0053] like Figure 3 As shown, the fault type determination method includes steps S310 to S330.

[0054] In step S310, the performance metrics associated with the fault in the test system are obtained.

[0055] In step S320, based on performance indicators and system log information of the test system, data reconstruction is performed using an autoencoder to determine abnormal characteristics.

[0056] In step S330, the abnormal features are input into the fault type discrimination model to obtain the fault type.

[0057] In some embodiments, in step S310, if the fault is a continuous container crash, the performance metrics can be the memory growth curve and transaction response time; if the fault is a distributed transaction interruption, the performance metrics can be the communication time between transactions and the transaction status.

[0058] According to the embodiments of this application, by reconstructing the performance data associated with the fault, fault features can be accurately extracted and faults can be quickly identified, improving the accuracy of fault detection. At the same time, this method can filter out interference from invalid performance data, strengthen the test system's ability to perceive faults, and ensure the effectiveness of fault verification and system recovery evaluation in chaotic testing.

[0059] Figure 4The illustration shows a schematic diagram of a method for determining abnormal performance indicators according to an embodiment of this application.

[0060] In some embodiments, based on performance metrics and system log information of the test system, data reconstruction is performed using an autoencoder to determine abnormal features, including: extracting the temporal features of performance metrics using a temporal convolutional network to obtain a temporal feature sequence; performing structured processing on the system log information and fusing the temporal feature sequence to obtain a fused feature; inputting the fused feature into the autoencoder for data reconstruction, and if the deviation between the reconstructed data and the fused feature is greater than a preset threshold, then the fused feature is taken as an abnormal feature.

[0061] like Figure 4 As shown, after the performance index 401 related to the fault is input into the temporal convolutional network 402, a temporal feature sequence 403 can be obtained. Combined with the structured system log information 404, a fusion feature 405 is obtained. The fusion feature 405 is input into the autoencoder 406 for data reconstruction. After the encoding and decoding operations of the autoencoder, the reconstructed data is obtained. If the deviation between the reconstructed data and the fusion feature input into the autoencoder is greater than a preset threshold, it indicates that the fusion feature is abnormal, that is, the performance index 401 is abnormal, and thus the abnormal feature 407 is obtained.

[0062] According to the embodiments of this application, by extracting the temporal features of performance indicators and fusing system log information, the correlation and reconstruction of multi-dimensional data can be achieved, which greatly improves the comprehensiveness and accuracy of abnormal data identification; effectively integrating fault-related temporal features and log context information strengthens the system's ability to perceive and locate anomalies, and provides reliable data support for fault analysis in chaotic testing.

[0063] Figure 5 A schematic diagram of a fault repair method for a test system according to an embodiment of this application is shown.

[0064] like Figure 5 As shown, the fault repair method includes steps S510 to S530.

[0065] In step S510, fault attribute features are obtained from the fault based on the type.

[0066] In step S520, based on the fault attribute characteristics and the current operating status of the test system, a repair strategy is matched from the preset fault repair strategy mapping library to obtain the target repair strategy.

[0067] In step S530, the test system is repaired based on the target repair strategy.

[0068] In some embodiments, in step S510, the fault attribute features include the scope of influence and the triggering conditions of the fault.

[0069] In some embodiments, in step S520, the faults and corresponding repair strategies included in the preset fault repair strategy mapping library may be: for faults with abnormal data consistency, a data rollback strategy can be adopted to roll back to a consistent state using logs or snapshots; for persistent software faults, services can be repeated, abnormal processes can be terminated and initialized to restore service availability; for faults such as disk damage and memory overflow, traffic can be switched to a backup node to isolate faulty resources; for faults such as memory exhaustion, elastic scaling can be performed to dynamically increase resources or distribute the load.

[0070] According to the embodiments of this application, by combining the fault type and the operating status of the test system to formulate a repair strategy, the fault can be repaired accurately, which greatly improves the effectiveness and adaptability of the repair; avoids the blindness of general repair strategies, reduces the additional impact of repair operations on the normal operation of the system, and ensures the stability and efficiency of the system recovery process in chaotic testing.

[0071] Figure 6 This illustration schematically shows a method for obtaining a target repair strategy according to an embodiment of this application;

[0072] In some embodiments, a target repair strategy is obtained by matching a repair strategy from a preset fault repair strategy mapping library based on fault attribute features and the current operating state of the test system. This includes: semantically matching the feature tags of the repair strategies in the preset fault repair strategy mapping library with the fault attribute features to obtain at least one candidate repair strategy; determining the feasibility score of the candidate repair strategy based on the current operating state, wherein the feasibility score characterizes the degree of feasibility of the current test system executing the candidate repair strategy; and selecting the candidate repair strategy with a score greater than a preset score as the target repair strategy.

[0073] like Figure 6 As shown, the feature label 601 of the repair strategy and the fault attribute feature 602 are semantically matched to obtain at least one candidate repair strategy 603. Operation S610 is executed to determine the feasibility score of the candidate repair strategy according to the current running state of the test system, so as to evaluate which candidate repair strategy is more feasible to select on the current test system. The candidate repair strategy with the larger feasibility score is selected as the target repair strategy 604.

[0074] For example, in the order business, assuming the fault is a data calculation overflow, the fault has a small impact and does not trigger a batch of inventory sales. The current test system is in the peak order period. Based on semantic matching, the candidate repair strategies include re-verification of the points deduction logic threshold and interception and rollback of illegal order amount values. Considering the current operating status of the test system, the points deduction logic threshold re-verification method consumes fewer resources, so it is selected as the target repair strategy.

[0075] According to the embodiments of this application, candidate repair strategies are matched from a preset library based on fault attribute characteristics, and the optimal strategy is selected by combining the system operating status. This achieves accurate matching and dynamic adaptation of repair strategies, improves the pertinence and efficiency of fault repair, avoids the system operation risks caused by blind repair, reduces the interference of repair operations on the testing process, and ensures the stability and reliability of system recovery in chaotic testing.

[0076] In some embodiments, method 200 further includes: obtaining an environment fingerprint based on the system configuration of the actual system corresponding to the test system, wherein the environment fingerprint represents a numerical identifier encoded by the system configuration; parsing the environment fingerprint to obtain an image matching the system configuration, and building a test environment for the test system.

[0077] Furthermore, system configuration can be collected through application programming interfaces (APIs), and resources can be built for the test system using Kubernetes.

[0078] According to the embodiments of this application, the test environment can be accurately matched with the actual system, ensuring the authenticity and effectiveness of chaos testing.

[0079] In some embodiments, parsing the environment fingerprint, obtaining an image that matches the system configuration, and building a test environment for the test system includes: decoding and deserializing the environment fingerprint to obtain the image metadata of the system configuration; pulling an image that matches the image metadata from the image repository and injecting the system configuration into the runtime configuration layer of the image to build the test environment.

[0080] According to the embodiments of this application, the environmental fingerprint is decoded and deserialized, and the image is obtained by matching the image metadata. The system configuration is then injected into the runtime configuration layer to build a test environment, thereby achieving a precise replication of the test environment and the actual system, ensuring the authenticity and consistency of the chaos test. Through the standardized image matching and configuration injection process, the efficiency of building the test environment is greatly improved, and the test error caused by environmental inconsistency is reduced.

[0081] In some embodiments, a single agent can be used to acquire faults in the faulty link and perform chaos testing and fault repair using the above methods. Alternatively, multiple agents can be used to implement the above process. For example, a first agent can be used to build a test environment, a second agent can be used to acquire faults, and a third agent can be used to determine the target for fault repair.

[0082] In some embodiments, the above methods of fault injection, chaos testing, and fault repair using target repair strategies can be applied to multiple scenarios. For example, the injected faults are data inconsistencies caused by transaction interruption. If performance indicators show that the transaction timed out and involves fund transfers, a compensation transaction operation is triggered. The system calls the "try-confirm-cancel" compensation framework to roll back the sub-transaction to repair the fault. Another example is in the order placement scenario. When unexpected data is returned, an avalanche may occur. The traditional repair method is to shut down the "user points service," which causes the transfer service to trigger a circuit breaker, and the transaction quickly fails. This also fails to expose deep-seated code defects. However, this application can modify the return value of the "user points service" to a maximum value. When the system processes this maximum value, an integer overflow occurs, leading to a serious internal service anomaly. The thread pool is quickly filled with blocked requests, causing the entire transfer service to become completely unavailable and triggering a cascading avalanche. The intelligent agent detects the avalanche, quickly determines the target repair strategy, and automatically executes data snapshot rollback and service cluster restart, allowing the environment to be fully restored in a short time. This application simulates the failure that occurs when unexpected data is returned in a real business scenario by modifying the return value to unexpected data, and provides a fault repair method. The whole process does not rely on manual intervention and is fully automated.

[0083] Based on the above-described chaos testing method, embodiments of this application also provide a chaos testing apparatus. The following will be combined with... Figure 7 The device is described in detail.

[0084] Figure 7 A schematic block diagram of a chaos testing apparatus according to an embodiment of this application is shown.

[0085] like Figure 7 As shown, the chaos testing device 700 of this embodiment includes a fault node acquisition module 710, a fault link acquisition module 720, and a chaos testing module 730.

[0086] The fault node acquisition module 710 is used to modify the business data of key nodes based on fault factors in a business knowledge graph to obtain multiple fault nodes. The business knowledge graph is obtained from historical business data, and the key nodes are determined based on their impact on the business. Fault factors include adjustment values ​​for logical judgment conditions and timing delay values. In one embodiment, the fault node acquisition module 710 can be used to execute step S210 described above, which will not be repeated here.

[0087] The fault link acquisition module 720 is used to extract the relationships between nodes in the business knowledge graph. Starting from multiple fault nodes, it traverses the business knowledge graph based on the relationships to obtain multiple fault links. In one embodiment, the fault link acquisition module 720 can be used to execute step S220 described above, which will not be repeated here.

[0088] The chaos testing module 730 is used to inject faults from multiple faulty links into the testing system to obtain test results. In one embodiment, the chaos testing module 730 can be used to execute step S230 described above, which will not be repeated here.

[0089] According to embodiments of this application, the device 700 can simulate potential problems at the business logic level using fault factor simulation data. Based on the relationships between nodes in the business knowledge graph, multiple fault links are obtained starting from the fault node. These fault links can be understood as deeper vulnerabilities in the code at the business logic level. By injecting the faults in the fault links into the test system for testing, the test system can simulate deeper faults caused by business anomalies and code logic errors, so as to test the system's ability to repair such faults, thereby improving the system's reliability and resilience.

[0090] In some embodiments, the chaos testing module 730 is specifically used to: obtain performance indicators associated with faults in the testing system; reconstruct data using an autoencoder based on the performance indicators and system log information of the testing system to determine abnormal features; and input the abnormal features into a fault type discrimination model to obtain the type of fault.

[0091] In some embodiments, the chaos testing module 730 is further configured to: extract the temporal features of performance indicators using a temporal convolutional network to obtain a temporal feature sequence; perform structured processing on system log information and fuse the temporal feature sequence to obtain a fused feature; input the fused feature into an autoencoder for data reconstruction; if the deviation between the reconstructed data and the fused feature is greater than a preset threshold, then the fused feature is regarded as an abnormal feature.

[0092] In some embodiments, the apparatus 700 is further configured to: obtain fault attribute features from the fault based on the type, the fault attribute features including the scope of influence and the triggering conditions of the fault; match a repair strategy from a preset fault repair strategy mapping library according to the fault attribute features and the current operating state of the test system to obtain a target repair strategy; and repair the test system based on the target repair strategy.

[0093] In some embodiments, the apparatus 700 is further configured to: semantically match the feature tags of repair strategies in the preset fault repair strategy mapping library with fault attribute features to obtain at least one candidate repair strategy; determine the feasibility score of the candidate repair strategy according to the current operating state, wherein the feasibility score characterizes the degree of feasibility of the current test system executing the candidate repair strategy; and take the candidate repair strategy with a score greater than the preset score as the target repair strategy.

[0094] In some embodiments, the apparatus 700 is further configured to: obtain an environmental fingerprint based on the system configuration of the actual system corresponding to the test system, wherein the environmental fingerprint represents a numerical identifier encoded by the system configuration; parse the environmental fingerprint to obtain an image matching the system configuration, and build a test environment for the test system.

[0095] In some embodiments, the apparatus 700 is further configured to: decode and deserialize the environmental fingerprint to obtain the image metadata of the system configuration; pull the image that matches the image metadata from the image repository and inject the system configuration into the runtime configuration layer of the image to build a test environment.

[0096] According to embodiments of this application, any plurality of modules among the fault node acquisition module 710, fault link acquisition module 720, and chaos testing module 730 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the fault node acquisition module 710, fault link acquisition module 720, and chaos testing module 730 can be at least partially implemented as a hardware circuit, such as a field-programmable gate array, a programmable logic array, a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit, or any other reasonable means of integrating or packaging the circuit, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the fault node acquisition module 710, fault link acquisition module 720, and chaos testing module 730 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0097] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a chaos testing method according to an embodiment of this application.

[0098] like Figure 8As shown, an electronic device 800 according to an embodiment of this application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory 802 or a program loaded from a storage portion 808 into a random access memory 803. The processor 801 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for executing different steps of the chaos testing method flow according to an embodiment of this application.

[0099] Random access memory 803 stores various programs and data required for the operation of electronic device 800. Processor 801, read-only memory 802, and random access memory 803 are interconnected via bus 804. Processor 801 executes various steps of the method flow according to embodiments of this application by executing programs stored in read-only memory 802 and / or random access memory 803. It should be noted that the programs may also be stored in one or more memories other than read-only memory 802 and random access memory 803. Processor 801 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0100] According to embodiments of this application, the electronic device 800 may further include an input / output interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card, such as a local area network card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0101] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the chaos testing method according to the embodiments of this application.

[0102] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 802 described above, and / or random access memory 803, and / or one or more memories other than read-only memory 802 and random access memory 803.

[0103] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0104] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0105] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the chaos testing system, device, apparatus, module, unit, etc., described above can be implemented by computer program modules.

[0106] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0108] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A chaos testing method, characterized in that, The method includes: Based on the fault factors of key nodes in the business knowledge graph, the business data of the key nodes are modified to obtain multiple fault nodes. The business knowledge graph is obtained from historical business data, the key nodes are determined according to the degree of impact of the nodes on the business, and the fault factors include the adjustment value of the logical judgment condition and the time delay value. Extract the relationships between nodes in the business knowledge graph, and take the multiple fault nodes as starting points. Based on the relationships, traverse the business knowledge graph to obtain multiple fault links. The faults in the multiple faulty links are injected into the test system to obtain the test results.

2. The method according to claim 1, characterized in that, The step of injecting faults from the multiple faulty links into the test system to obtain test results includes: Obtain the performance metrics associated with the fault in the test system; Based on the performance indicators and the system log information of the test system, an autoencoder is used to reconstruct the data and determine the abnormal characteristics. The abnormal features are input into the fault type discrimination model to obtain the type of the fault.

3. The method according to claim 2, characterized in that, Based on the performance metrics and the system log information of the test system, an autoencoder is used to reconstruct the data and determine abnormal characteristics, including: Temporal features of the performance metrics are extracted using a temporal convolutional network to obtain a temporal feature sequence. The system log information is structured and fused with the time-series feature sequence to obtain fused features; The fusion feature is input into the autoencoder to reconstruct the data. If the deviation between the reconstructed data and the fusion feature is greater than a preset threshold, the fusion feature is regarded as an abnormal feature.

4. The method according to claim 2, characterized in that, The method further includes: Based on the type, fault attribute features are obtained from the fault, the fault attribute features including the scope of influence and the triggering conditions of the fault; Based on the fault attribute characteristics and the current operating status of the test system, a repair strategy is matched from the preset fault repair strategy mapping library to obtain the target repair strategy; The test system is repaired based on the target repair strategy.

5. The method according to claim 4, characterized in that, The step of matching a repair strategy from a preset fault repair strategy mapping library to obtain a target repair strategy based on the fault attribute characteristics and the current operating state of the test system includes: Semantic matching is performed between the feature labels of the repair strategies in the preset fault repair strategy mapping library and the fault attribute features to obtain at least one candidate repair strategy. Based on the current operating status, the feasibility score of the candidate repair strategy is determined, wherein the feasibility score characterizes the degree of feasibility of the current test system executing the candidate repair strategy; The candidate repair strategies that are greater than the preset score are taken as the target repair strategies.

6. The method according to claim 1, characterized in that, The method further includes: Based on the system configuration of the actual system corresponding to the test system, an environmental fingerprint is obtained, wherein the environmental fingerprint represents a numerical identifier encoded by the system configuration; The environmental fingerprint is parsed to obtain an image that matches the system configuration, and the test environment of the test system is built.

7. The method according to claim 6, characterized in that, The process of parsing the environmental fingerprint, obtaining an image matching the system configuration, and building the test environment for the test system includes: The environmental fingerprint is decoded and deserialized to obtain the image metadata of the system configuration; Pull an image that matches the image metadata from the image repository, and inject the system configuration into the runtime configuration layer of the image to build the test environment.

8. A chaos testing device, characterized in that, The device includes: The fault node acquisition module is used to modify the business data of the key nodes based on the fault factors of the key nodes in the business knowledge graph to obtain multiple fault nodes. The business knowledge graph is obtained based on historical business data, the key nodes are determined based on the degree of impact of the nodes on the business, and the fault factors include the adjustment value of the logical judgment condition and the timing delay value. The fault link acquisition module is used to extract the association relationship of each node in the business knowledge graph, and traverse the business knowledge graph based on the association relationship, taking the multiple fault nodes as the starting point, to obtain multiple fault links. The chaos testing module is used to inject the faults in the multiple fault links into the testing system to obtain the test results.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images