Fault simulation method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to a fault simulation method and device, equipment and a storage medium. The method comprises the steps of determining topological information of a first micro-service in a target micro-service system, wherein the topological information of the first micro-service at least indicates at least one second micro-service associated with the first micro-service in the target micro-service system; determining at least one candidate fault which aims at the first micro-service and relates to the selected second micro-service from the group of candidate faults based on a selection result of the user on the second micro-service indicated by the topological information so as to obtain a first fault arrangement scheme; and performing fault simulation on the first micro-service based on the first fault arrangement scheme to determine a fault simulation result of the first micro-service. In this way, a more accurate fault arrangement scheme can be obtained, fault simulation is executed on the micro-service based on the fault arrangement scheme, and a better test effect can be achieved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to a method, apparatus, device, and storage medium for fault simulation. Background Art

[0002] With the continuous evolution and improvement of microservice architecture, its application scope has gradually expanded from the Internet industry to multiple fields such as manufacturing, energy, communications, Internet of Things, real estate, and even to the financial industry, which is cautious about technology updates. The popularization of microservice architecture has not only significantly improved the efficiency of system development, but also greatly enhanced the scalability of the system. At the same time, it has also effectively reduced the cost of system iteration upgrades and operation and maintenance. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for fault simulation is provided, including: determining topology information of a first microservice in a target microservice system, the topology information of the first microservice at least indicating at least one second microservice associated with the first microservice in the target microservice system; determining, based on a selection result of a user for the second microservice indicated by the topology information, at least one candidate fault for the first microservice and involving the selected second microservice from a group of candidate faults to obtain a first fault orchestration scheme; and performing fault simulation on the first microservice based on the first fault orchestration scheme to determine a fault simulation result of the first microservice.

[0004] In a second aspect of the present disclosure, a device for fault simulation is provided, the device comprising: a topology information determination module, configured to determine topology information of a first microservice in a target microservice system, the topology information of the first microservice at least indicating at least one second microservice associated with the first microservice in the target microservice system; a fault orchestration module, configured to determine, based on a user's selection result of the second microservice indicated by the topology information, at least one candidate fault for the first microservice and involving the selected second microservice from a group of candidate faults, so as to obtain a first fault orchestration scheme; and a fault simulation module, configured to perform fault simulation on the first microservice based on the first fault orchestration scheme, so as to determine a fault simulation result of the first microservice.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer microservice is stored on the computer-readable storage medium, and the computer microservice can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, and when the instructions are executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram illustrating an example environment according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A flowchart showing an example process of a method for fault simulation according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing an example of a fault simulation according to some embodiments of the present disclosure;

[0013] Figure 4 A schematic structural block diagram of an apparatus for model training according to some embodiments of the present disclosure is shown; and

[0014] Figure 5 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0018] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0019] In this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or it is necessary to perform a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0020] As briefly described above, the microservice architecture can not only improve the efficiency of system development, but also enhance the scalability of the system.

[0021] However, the frequent iteration and heterogeneous integration of microservice architectures also bring potential risks to system stability. Since developers can usually only focus on the status of their own services, it is difficult to fully grasp the huge and complex global system. Even if all individual services are running normally, calls and interactions between services may be abnormal, which may lead to cascading failures and endanger the entire system. The inherent uncertainty of this system, coupled with the failures that may occur at any time in the real environment, poses a major challenge to the stability and availability of the system.

[0022] Chaos engineering, as a practical method to test and improve system reliability and stability by systematically introducing failures and uncertainties, aims to simulate real-world failure scenarios, thereby gaining a deeper understanding of system weaknesses, discovering potential problems in a timely manner, and thereby enhancing the robustness and reliability of the system.

[0023] Chaos engineering platforms currently on the market generally have fault injection and recovery functions, allowing users to simulate the risks of failures that systems may encounter in real environments. However, the use process of these platforms is often complicated, making it difficult for users to get started quickly. In actual operations, users often face difficulties such as how to configure simulation scenarios, how to reasonably control the scope of impact, and how to understand steady-state monitoring. Especially in microservice systems, due to the complexity of the system architecture and the large number of microservices involved, it is difficult for business or operation and maintenance personnel as platform users to understand the dependencies between microservices from a global perspective. This leads to blindness in designing simulation scenarios and selecting simulation objects, making users feel uneasy and confused about using chaos engineering platforms, thereby limiting the effective application of chaos engineering in improving the stability of microservice systems.

[0024] In view of this, an embodiment of the present disclosure provides a scheme for fault simulation. According to the scheme, the topology information of a first microservice in a target microservice system is first determined, and the topology information of the first microservice at least indicates at least one second microservice associated with the first microservice in the target microservice system. Then, based on the selection result of the user for the second microservice indicated by the topology information, at least one candidate fault for the first microservice and involving the selected second microservice is determined from a set of candidate faults to obtain a first fault orchestration scheme. Finally, based on the first fault orchestration scheme, a fault simulation is performed on the first microservice to determine the fault simulation result of the first microservice.

[0025] The following description will make it clearer that, in response to the complex use process of existing chaos engineering platforms, this solution provides an intuitive way to automatically build a fault orchestration solution based on the topology information of microservices. Through the topology information of microservices, users can understand the dependencies between microservices from a global perspective, which provides a powerful reference for users when designing simulation scenarios, enabling them to more accurately select simulation objects and design targeted test scenarios. The automatic generation of fault orchestration solutions simplifies the user's operation process and improves work efficiency. In this way, this solution enables users to test microservice systems more efficiently and accurately to ensure the stability and reliability of microservice systems.

[0026] Various example implementations of the solution will be described in detail below in conjunction with the accompanying drawings.

[0027] Figure 1A schematic diagram of an example environment 100 is shown according to some embodiments of the present disclosure. Figure 1 , the example environment 100 may include a terminal device 110 and an electronic device 120 .

[0028] In the exemplary environment 100, an application 130 for interacting with an electronic device 120 is installed in a terminal device 110. A user 140 may interact with the application 130 via the terminal device 110 and / or its attached devices.

[0029] As an example, the user 140 can send a fault simulation request to the electronic device 120 by means of the application 130, and the fault simulation request can indicate the microservice to be tested selected by the user 140 from the N microservices 151-1 to 161-N in the target microservice system 150, that is, the first microservice. It should be noted that, for the sake of clarity, unless otherwise specified, the N microservices 151-1 to 161-N are collectively referred to as microservices 151 below. N is a positive integer greater than or equal to 1.

[0030] After obtaining the fault simulation request, the electronic device 120 determines the topology information of the first microservice based on the microservice 151 associated with the first microservice among the N microservices 151-1 to 161-N. Then, the electronic device 120 can present the topology information to the user 140 with the help of the terminal device 110, and obtain the selection result of the user 140 for the second microservice indicated by the topology information. Then, the electronic device 120 can determine the first fault arrangement scheme based on the second microservice selected by the user 140 and a set of candidate faults, and perform fault simulation on the first microservice.

[0031] In addition, the terminal device 110 may present an interface 160 of the application 130. As an example, the topology information of the first microservice may be presented to the user 140 through the page 160. Optionally, the content presented by the interface 160 may also change according to the specific services provided, the interactive behavior / preset operation of the user 140, and the like.

[0032] In some embodiments, the terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user 140 (such as "wearable" circuits, etc.).

[0033] The electronic device 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The electronic device 120 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. The electronic device 120 may provide background services for the application 130 that supports content presentation in the terminal device 110.

[0034] A communication connection may be established between the electronic device 120 and the terminal device 110. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a universal serial bus connection, a wireless fidelity connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the electronic device 120 and the terminal device 110 may implement signaling interaction through the communication connection between the two.

[0035] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0036] Figure 2 A flowchart of an example process 200 of a method for fault simulation according to some embodiments of the present disclosure is shown. The process 200 may be implemented at the electronic device 120.

[0037] Reference Figure 2 In box 210, the electronic device 120 determines topology information of a first microservice in the target microservice system 150, where the topology information of the first microservice at least indicates at least one second microservice associated with the first microservice in the target microservice system 150.

[0038] In some embodiments, the user 140 can use the electronic device 120 to perform fault simulation tests on one or more microservices 151 in the target microservice system 150. Microservice architecture is a software architecture that allows each service to be developed, deployed and maintained independently. The core concept of microservices is to decompose large and complex applications into several small services. In this way, each small service can focus on implementing specific business functions, thereby improving the efficiency of the development process, the scalability and maintainability of the system.

[0039] In some embodiments, the electronic device 120 can simulate failures of one or more services in the target microservice system 150 based on chaos engineering. Chaos engineering is a practical means of testing and evaluating system reliability and resilience by actively creating failures and abnormal conditions. The core principle of chaos engineering is to deliberately trigger failures in an actual production environment to observe the system's response and self-healing capabilities. In this way, the electronic device 120 can effectively identify the vulnerabilities of the target microservice system 150 and take corresponding countermeasures for optimization.

[0040] In the simulation test based on chaos engineering, the whole process includes at least the fault arrangement stage and the fault simulation stage. The following first describes the fault arrangement stage.

[0041] Figure 3 A schematic diagram of an example 300 of a fault simulation according to some embodiments of the present disclosure is shown. Figure 3 ,right Figure 2 An example process 200 is shown for illustration.

[0042] In some embodiments, the target microservice system 150 may include multiple microservices 151, which may be set on one or more servers. The cluster formed by the combination of these servers is the hardware foundation for carrying the entire target microservice system 150. As an example, in box 301, the electronic device 120 registers the cluster with the electronic device 120. For example, the electronic device 120 may obtain the configuration information (config) of the cluster, and based on the configuration information, register the cluster with the electronic device 120. In this way, the electronic device 120 can collect relevant information about the target microservice system 150 from the cluster according to actual needs.

[0043] As an example, the acquisition of cluster configuration information may be implemented in a variety of ways. For example, the electronic device 120 may obtain the information by accessing an API of a cluster management tool, or directly reading the information from a configuration file of the cluster.

[0044] In some embodiments, the first microservice may be any selected microservice 151 to be tested in the target microservice system 150. As an example, the determination of the first microservice may be based on a variety of factors, including but not limited to its business importance, complexity, historical failure rate, etc. For example, the electronic device 120 may pay special attention to those microservices 151 that have a greater impact on system stability.

[0045] In some embodiments, topology information may refer to connections and dependencies between microservices 151, which reveals how microservices 151 interact with each other and how failures may propagate in the system.

[0046] In some embodiments, the second microservice may refer to other microservices 151 that are directly associated with or dependent on the first microservice. As an example, in the target microservice system 150, a microservice 151 may call or depend on multiple other microservices 151 to complete its business functions, and these called or dependent microservices 151 may be confirmed by the electronic device 120 as second microservices.

[0047] In some embodiments, the electronic device 120 uses a probe component to collect historical operation information related to the first microservice in the target microservice system 150. Then, based on the historical operation information related to the first microservice, the electronic device 120 obtains at least one microservice 151 having a call relationship with the first microservice from the target microservice system 150 to obtain at least one second microservice associated with the first microservice.

[0048] In some embodiments, historical operation information may include, but is not limited to, key indicators such as service call records, response time, and error rate, which provide reference data for the electronic device 120 to analyze the interactions between microservices 151.

[0049] Based on the collected historical operation information, the electronic device 120 can further analyze the calling relationship of the first microservice in the target microservice system 150. The calling relationship here can include the calling of the first microservice to other microservices 151, and can also include the calling of other microservices 151 to the first microservice. By tracing and parsing the calling chain between services, the electronic device 120 can identify one or more microservices 151 that have a direct calling relationship with the first microservice. These called microservices 151 can be determined by the electronic device 120 as second microservices associated with the first microservice.

[0050] In some embodiments, in box 302, the electronic device 120 may install a probe component on an instance of the microservice 151. As an example, each microservice 151 may have multiple instances running to support requirements such as high availability, load balancing, or fault tolerance. These instances may be distributed in different servers or containers, and each instance independently runs the same microservice 151 code but processes different requests or data. In box 303, the electronic device 120 obtains the topology information of the first microservice with the help of the installed probe component.

[0051] In this way, the electronic device 120 can comprehensively monitor each microservice 151 in the target microservice system 150, thereby providing sufficient data support in the fault orchestration stage.

[0052] In some embodiments, the historical operation information is constructed by using the probe component to collect results of a target device provided with the first microservice, and the collection results include at least one of the following: traffic data of the target device, or event data of the target device.

[0053] In some embodiments, the traffic data reveals the network interaction of the target device when the microservice 151 is running. It records in detail the initiator and receiver of each network request, as well as the amount of data transferred between the requests. By analyzing these data, the electronic device 120 can gain insight into the calling relationship and data transmission mode between the microservices 151. For example, when the first microservice calls other microservices 151, corresponding records will be left in the traffic data, including information such as the time of the call, the microservice 151 called, and the amount of data transferred. The event data records various events that occurred during the operation of the microservice 151 on the target device. These events include, but are not limited to, the start, stop, interruption, etc. of the microservice 151. The occurrence of each event will leave a timestamp and event type mark in the event data. By analyzing the event data, the electronic device 120 can understand the events related to the calling process of the first microservice and the association between these events and other microservices 151.

[0054] As an example, the electronic device 120 can use one of the traffic data and the event data to determine the call and dependency relationship between the first microservice and the other microservices 151, or the electronic device 120 can also use the combination of traffic data and event data to determine the call and dependency relationship between the first microservice and the other microservices 151. The specific method can be determined according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0055] In this way, the electronic device 120 can fully understand the position and role of the first microservice in the target microservice system 150 , as well as the complex interactions between it and other microservices 151 .

[0056] In some embodiments, the electronic device 120 can also use the probe component to perform other operations. For example, the electronic device 120 can use the probe component to collect performance indicators of the target device, and the performance indicators can include key indicators such as processor (CPU) usage and memory occupancy. The performance indicators provide the electronic device 120 with information about the consumption of device resources by the microservice 151. In addition, the electronic device 120 can also use the probe component to perform fault injection operations, which will be described in detail below, so it will not be repeated here.

[0057] After obtaining the topology information of the first microservice, the electronic device 120 can further use the information to formulate a fault orchestration plan to test the stability and recovery capability of the first microservice. Specifically, in block 220, the electronic device 120 determines at least one candidate fault for the first microservice and involving the selected second microservice from a set of candidate faults 320 based on the selection result of the second microservice indicated by the topology information by the user 140, so as to obtain a first fault orchestration plan.

[0058] In order to assist the user 140 in making a selection, in some embodiments, the electronic device 120 presents a microservice topology map generated based on the topology information, wherein the microservice topology map at least displays the first microservice, each second microservice in the at least one second microservice, and the calling relationship between the first microservice and each second microservice in the at least one second microservice. Then, the electronic device 120 determines the selection result of the user 140 for the second microservice indicated by the topology information based at least on the operation of the user 140 on the second microservice displayed on the microservice topology map.

[0059] In some embodiments, the microservice topology diagram intuitively displays the calling relationship between the first microservice and each second microservice associated therewith. User 140 can understand the dependency and interaction between the first microservice and each second microservice associated therewith by observing the microservice topology diagram, and thus make an appropriate choice.

[0060] As an example, the user 140 may select one or more of the second microservices displayed on the microservice topology map as the target microservice 151. The electronic device 120 determines these target microservices 151 as the selection result of the second microservice by the user 140. As an example, the user 140 may indicate the selection by clicking or marking a second microservice on the microservice topology map. In block 304, the electronic device 120 may capture these operations and determine the selected second microservice accordingly.

[0061] In this way, the electronic device 120 uses the microservice topology diagram to help the user 140 understand the complex relationship between the microservices 151, thereby providing a basis for subsequent accurate fault orchestration.

[0062] In some embodiments, based on the selection result of the user 140 for the second microservice indicated by the topology information, the electronic device 120 uses a machine learning model to determine at least one candidate fault for the first microservice and involving the selected second microservice from a set of candidate faults 320 to obtain a first fault orchestration plan.

[0063] In some embodiments, once the selected second microservice is determined, the electronic device 120 can use a trained machine learning model for further analysis. The machine learning model can be trained based on features such as historical fault data, service performance data, and call relationships between microservices 151.

[0064] As an example, the trained machine learning model can determine a fault orchestration template that matches the first microservice and the selected second microservice based on the first microservice and the selected second microservice. As an example, the fault orchestration template can be pre-configured with at least one predetermined microservice 151 and one or more specific faults related to the at least one predetermined microservice 151, and the machine learning model can determine a fault orchestration template that matches the first microservice and the selected second microservice based on the similarity between the first microservice and the selected second microservice and at least one predetermined microservice 151 in the fault orchestration template.

[0065] As an example, in a fault orchestration template, one or more specific faults related to at least one predetermined microservice 151 may be determined based on common faults (or high-risk faults) of the predetermined microservices 151 themselves and common faults (or high-risk faults) of the predetermined microservices 151 when calling each other.

[0066] After determining the fault orchestration template, the machine learning model can select, based on the determined fault orchestration template, a candidate fault that is similar to the fault configured in the fault orchestration template from a group of candidate faults 320 as a candidate fault for the first microservice and involving the selected second microservice.

[0067] It should be noted that the above is only an exemplary description of how the machine learning model selects from a set of candidate faults 320, and does not constitute a limitation on the embodiments of the present disclosure. According to actual needs, the machine learning model can also select from a set of candidate faults 320 in other ways. For example, the machine learning model can identify which faults are more likely to cause problems on the first microservice (or in the call process between the first microservice and the selected second microservice), and then select from a set of candidate faults 320 based on the identification results.

[0068] In some embodiments, after selecting at least one candidate fault for the first microservice and involving the selected second microservice from a set of candidate faults 320, in box 305, the electronic device 120 can perform operations such as combination and sorting on the selected candidate faults to construct a fault orchestration scheme (e.g., a first fault orchestration scheme).

[0069] In this way, the electronic device 120 can realize the automatic generation of the fault orchestration plan with the help of the machine learning model. In this process, the user 140 only needs to select the second microservice of interest in the microservice topology map to complete the construction of the fault orchestration plan. As a result, the operation of the user 140 in the fault orchestration stage in chaos engineering is greatly simplified, thereby improving the test work efficiency.

[0070] In some embodiments, a set of candidate faults 320 is constructed based on at least a first fault library 321 , wherein the first fault library 321 includes at least one fault pre-set for the target microservice system 150 .

[0071] As an example, the first fault library 321 may be a fault library pre-set specifically for the target microservice system 150. For example, the first fault library 321 may include common faults or high-risk faults in the process of each microservice 151 in the target microservice system 150 and each microservice 151 calling each other. The setting of the first fault library 321 is conducive to the configuration of the fault arrangement template and other contents described above. As an example, the first fault library 321 may include faults such as delays, downtime, and resource exhaustion.

[0072] In some embodiments, a set of candidate faults 320 is constructed based on a first fault library 321 and a second fault library 322 , wherein the second fault library 322 includes at least one newly added fault recorded after the target microservice system 150 is running.

[0073] In some embodiments, in addition to the first fault library 321, the construction of a set of candidate faults 320 is also combined with a second fault library 322. The newly added faults in the second fault library 322 can be newly discovered during the actual operation of the target microservice system 150, or new fault types that appear as the environment and requirements of the target microservice system 150 change.

[0074] As an example, the second fault library 322 can be constructed in various ways. On the one hand, it can be actively input by the user 140. For example, the electronic device 120 can require the user 140 to input various types and descriptions of faults encountered during the operation of the microservice 151. On the other hand, the second fault library 322 can also be constructed by the electronic device 120 automatically collecting.

[0075] As an example, after the user 140 inputs various fault types and descriptions encountered during the operation of the microservice 151, the electronic device 120 can parse these fault names through the fault analysis model and compare them with the faults in the first fault library 321. If a fault is found that is not included in the first fault library 321, the electronic device 120 will intelligently classify it as a new fault and automatically add it to the second fault library 322. For faults that already exist in the first fault library 321, the device will identify and ignore them to avoid duplicate records.

[0076] As an example, for a newly added fault, the electronic device 120 can further analyze its similarity with the existing faults in the first fault library 321. Based on the analysis results, the electronic device 120 can integrate the newly added fault into the existing fault arrangement template, or build a new fault arrangement template for it. This process reduces the need for manual intervention and improves the automation level of fault collection.

[0077] After the first fault arrangement plan is determined, the fault arrangement phase in chaos engineering is basically completed. Next, the fault simulation phase in chaos engineering is described.

[0078] In block 230 , the electronic device 120 performs a fault simulation on the first microservice based on the first fault orchestration solution to determine a fault simulation result of the first microservice.

[0079] As an example, before performing the fault simulation, the electronic device 120 can set up an isolated simulation environment to ensure that the fault simulation does not affect the actual production environment. This simulation environment will restore the configuration of the actual production environment as much as possible, including the dependencies between microservices 151, network topology, data traffic, etc., to provide an accurate simulation effect.

[0080] In some embodiments, the electronic device 120 performs at least one fault injection operation on the first microservice using a probe component based on the first fault orchestration solution to perform fault simulation on the first microservice.

[0081] In some embodiments, the first fault orchestration scheme may indicate key parameters such as the fault type of the fault simulation performed on the first microservice, the time point of the fault injection, the duration of the fault, and the scope of the fault impact.

[0082] As an example, according to the instructions of the first fault orchestration scheme, the electronic device 120 uses the probe component to perform a fault injection operation on the first microservice in a simulation environment. This process may involve simulating different types of faults such as service delays, data loss, and resource exhaustion. The use of the probe component enables the electronic device 120 to introduce a predetermined fault at a specific time point and under specific conditions, and to control the start, end, type, and severity of the fault injection, ensuring that the simulation is carried out according to the predetermined plan.

[0083] As an example, the electronic device 120 can implement the construction of various abnormal situations such as delay, abnormal return, resource limitation, etc. based on the chaos engineering experiment tool. Then, the electronic device 120 sends instructions to the probe component through the chaos engineering experiment tool, so that the probe component performs operations including starting or stopping fault injection, adjusting fault parameters, etc.

[0084] In some embodiments, if the user 140 confirms that the first fault orchestration scheme is valid, the electronic device 120 performs a fault simulation on the first microservice based on the first fault orchestration scheme. If the user 140 confirms that the first fault orchestration scheme is invalid, the electronic device 120 generates a second fault orchestration scheme for the first microservice based at least on the user 140's feedback on the first fault orchestration scheme. If the user 140 confirms that the second fault orchestration scheme is valid, the electronic device 120 performs a fault simulation on the first microservice based on the second fault orchestration scheme.

[0085] In some embodiments, after confirming a fault arrangement plan (e.g., the first fault arrangement plan), the electronic device 120 may present the current fault arrangement plan to the user 140 and request the user 140 to confirm the validity of the current fault arrangement plan. Then, in block 306, based on the feedback from the user 140, it is determined whether the current fault arrangement plan is valid.

[0086] Exemplarily, if the user 140 confirms that the current fault orchestration scheme is valid, this means that the current fault orchestration scheme can accurately reflect the fault conditions that the microservice 151 may encounter and is applicable to the current test requirements. In this case, in box 307, the electronic device 120 will directly perform a fault simulation on the first microservice based on the current fault orchestration scheme. However, if the user 140 confirms that the current fault orchestration scheme is invalid, this means that some settings in the current fault orchestration scheme do not meet the actual test requirements, or cannot effectively simulate the real fault conditions that the microservice 151 may encounter. In this case, in box 308, the electronic device 120 will regenerate the fault orchestration scheme based at least on the user 140's feedback on the current fault orchestration scheme. The feedback from the user 140 may include the type of fault that needs to be adjusted, the irrationality of the injection time point, the duration that is too long or too short, and other issues. The electronic device 120 will comprehensively consider these feedbacks, revise and optimize the original scheme, and generate a new fault orchestration scheme that better meets the test requirements.

[0087] Once the new fault arrangement scheme is generated and its validity is confirmed by the user 140, the electronic device 120 will perform fault simulation on the first microservice based on the new scheme. Similar to the original scheme, the electronic device 120 will perform fault injection based on the parameters set in the new fault arrangement scheme, thereby realizing the corresponding fault simulation.

[0088] In some embodiments, after performing at least one fault injection operation on the first microservice, the electronic device 120 determines a fault simulation result of the first microservice based at least on a service state after the fault is injected into the first microservice.

[0089] In some embodiments, during the fault simulation process, the electronic device 120 may monitor the performance indicators and error logs of the first microservice in real time, and determine the service status of the first microservice (e.g., service available or service unavailable, etc.) based on the information. In block 309, the electronic device may record the simulation results for subsequent analysis and evaluation.

[0090] In some embodiments, once the fault simulation is completed, the electronic device 120 analyzes the collected data to determine the performance of the first microservice in the fault simulation. This includes whether the first microservice has the expected fault response, the speed of fault recovery, and whether other unexpected problems have been caused. The electronic device 120 can incorporate these contents into the fault simulation results. These fault simulation results will provide valuable feedback to the user 140, helping the user 140 understand the robustness and fault tolerance of the first microservice, thereby guiding the user 140 to analyze the replay simulation results to help with subsequent optimization work.

[0091] According to the various embodiments described above, it can be clearly understood that, in response to the problem of complex usage processes of existing chaos engineering platforms, the embodiments of the present disclosure provide an intuitive way to automatically build a fault orchestration plan based on the topology information of microservices 151. Through the topology information of microservices 151, user 140 can understand the dependencies between microservices 151 from a global perspective, which provides a powerful reference for user 140 when designing simulation scenarios, enabling it to more accurately select simulation objects and design targeted test scenarios. The automatic generation of fault orchestration plans simplifies the operation process of user 140 and improves work efficiency. In this way, the embodiments of the present disclosure enable user 140 to test the microservice system more efficiently and accurately to ensure the stability and reliability of the microservice system.

[0092] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 4 A schematic structural block diagram of an apparatus 400 for model training according to some embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in the electronic device 120. Each module / component in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0093] Reference Figure 4 , the apparatus 400 includes a topology information determination module 410, a fault arrangement module 420 and a fault simulation module 430. In some embodiments,

[0094] The topology information determination module 410 is configured to determine topology information of a first microservice in a target microservice system, where the topology information of the first microservice at least indicates at least one second microservice associated with the first microservice in the target microservice system.

[0095] The fault orchestration module 420 is configured to determine, from a group of candidate faults, at least one candidate fault for the first microservice and involving the selected second microservice based on the user's selection result of the second microservice indicated by the topology information, to obtain a first fault orchestration solution.

[0096] The fault simulation module 430 is configured to perform a fault simulation on the first microservice based on the first fault orchestration solution to determine a fault simulation result of the first microservice.

[0097] In some embodiments, the topology information determination module 410 is further configured to: in the target microservice system, use the probe component to collect historical operation information related to the first microservice; and based on the historical operation information related to the first microservice, obtain at least one microservice having a calling relationship with the first microservice from the target microservice system to obtain at least one second microservice associated with the first microservice.

[0098] In some embodiments, the historical operation information is constructed by using the probe component to collect results of a target device configured with the first microservice, and the collection results include at least one of the following: traffic data of the target device; or event data of the target device.

[0099] In some embodiments, the fault simulation module 430 is further configured to: based on the first fault orchestration solution, use the probe component to perform at least one fault injection operation on the first microservice to perform fault simulation on the first microservice.

[0100] In some embodiments, the fault simulation module 430 is further configured to: after performing at least one fault injection operation on the first microservice, determine a fault simulation result of the first microservice based at least on a service state after the fault is injected into the first microservice.

[0101] In some embodiments, the fault simulation module 430 is further configured to: if the user confirms that the first fault orchestration solution is valid, perform a fault simulation on the first microservice based on the first fault orchestration solution; if the user confirms that the first fault orchestration solution is invalid, generate a second fault orchestration solution for the first microservice based at least on the user's feedback on the first fault orchestration solution; and if the user confirms that the second fault orchestration solution is valid, perform a fault simulation on the first microservice based on the second fault orchestration solution.

[0102] In some embodiments, the apparatus 400 further includes a presentation module. The presentation module is configured to: present a microservice topology map generated based on the topology information, wherein the microservice topology map at least displays the first microservice, each second microservice in the at least one second microservice, and the call relationship between the first microservice and each second microservice in the at least one second microservice; and determine the selection result of the user on the second microservice indicated by the topology information based at least on the user's operation on the second microservice displayed on the microservice topology map.

[0103] In some embodiments, the fault orchestration module 420 is further configured to: based on the user's selection result of the second microservice indicated by the topology information, use a machine learning model to determine at least one candidate fault for the first microservice and involving the selected second microservice from a group of candidate faults, so as to obtain a first fault orchestration solution.

[0104] In some embodiments, a set of candidate faults is constructed based on at least a first fault library, and the first fault library includes at least one fault pre-set for the target microservice system.

[0105] In some embodiments, a set of candidate faults is constructed based on a first fault library and a second fault library, wherein the second fault library includes at least one newly added fault recorded after the target microservice system is running.

[0106] Figure 5 1 is a block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. The electronic device 500 may be used to implement, for example, Figure 1 The electronic device 120 shown. It should be understood that Figure 5 The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

[0107] Reference Figure 5 , the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0108] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 500.

[0109] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0110] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0111] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0112] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0113] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0114] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0115] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0116] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0117] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The determination of the terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for fault simulation, comprising: Determine topology information of a first microservice in a target microservice system, where the topology information of the first microservice at least indicates at least one second microservice associated with the first microservice in the target microservice system; Based on a selection result of the second microservice indicated by the topology information by the user, determining at least one candidate fault for the first microservice and involving the selected second microservice from a group of candidate faults to obtain a first fault orchestration solution; as well as Based on the first fault orchestration solution, a fault simulation is performed on the first microservice to determine a fault simulation result of the first microservice.

2. The method according to claim 1, wherein determining the topology information of the first microservice in the target microservice system comprises: In the target microservice system, using a probe component to collect historical operation information related to the first microservice; as well as Based on the historical operation information related to the first microservice, at least one microservice having a calling relationship with the first microservice is acquired from the target microservice system to obtain at least one second microservice associated with the first microservice.

3. The method according to claim 2, wherein the historical operation information is constructed by using the probe component to collect results of the target device on which the first microservice is set, and wherein the collection results include at least one of the following: Traffic data of the target device; or The event data of the target device.

4. The method according to claim 1, wherein based on the first fault orchestration scheme, performing fault simulation on the first microservice comprises: Based on the first fault orchestration solution, at least one fault injection operation is performed on the first microservice using a probe component to perform the fault simulation on the first microservice.

5. The method according to claim 4, wherein based on the first fault orchestration solution, performing fault simulation on the first microservice further comprises: After performing at least one fault injection operation on the first microservice, the fault simulation result of the first microservice is determined based on at least a service state of the first microservice after the fault is injected into the first microservice.

6. The method according to claim 1, wherein based on the first fault orchestration solution, performing fault simulation on the first microservice comprises: If the user confirms that the first fault orchestration solution is valid, based on the first fault orchestration solution, perform the fault simulation on the first microservice; If the user confirms that the first fault orchestration solution is invalid, generating a second fault orchestration solution for the first microservice based at least on the user's feedback on the first fault orchestration solution; as well as If the user confirms that the second fault orchestration solution is effective, a fault simulation is performed on the first microservice based on the second fault orchestration solution.

7. The method according to claim 1, further comprising: Presenting a microservice topology map generated based on the topology information, wherein the microservice topology map at least shows the first microservice, each second microservice in the at least one second microservice, and a call relationship between the first microservice and each second microservice in the at least one second microservice; as well as A selection result of the user on the second microservice indicated by the topology information is determined based at least on an operation of the user on the second microservice displayed on the microservice topology map.

8. The method according to claim 1, wherein determining a first fault orchestration solution based on a selection result of a user for a second microservice indicated by the topology information comprises: Based on the user's selection result of the second microservice indicated by the topology information, a machine learning model is used to determine at least one candidate fault for the first microservice and involving the selected second microservice from the set of candidate faults to obtain the first fault orchestration plan. 9 . The method according to claim 1 , wherein the set of candidate faults is constructed based on at least a first fault library, and wherein the first fault library includes at least one fault pre-set for the target microservice system.

10. The method according to claim 9, wherein the set of candidate faults is constructed based on the first fault library and the second fault library, and wherein the second fault library includes at least one newly added fault recorded after the target microservice system is running.

11. A device for fault simulation, comprising: A topology information determination module is configured to determine topology information of a first microservice in a target microservice system, wherein the topology information of the first microservice at least indicates at least one second microservice associated with the first microservice in the target microservice system; A fault orchestration module is configured to determine, from a group of candidate faults, at least one candidate fault for the first microservice and involving the selected second microservice based on a selection result of the user for the second microservice indicated by the topology information, so as to obtain a first fault orchestration solution; as well as The fault simulation module is configured to perform a fault simulation on the first microservice based on the first fault orchestration solution to determine a fault simulation result of the first microservice.

12. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer microservice stored thereon, wherein the computer microservice can be executed by a processor to implement the method according to any one of claims 1 to 10.

14. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images