Fault injection method and device

Abstract

The embodiment of the invention provides a fault injection method and device, and relates to the technical field of computers. The method comprises the following steps: adding dyeing information to a target service request through an observation system; disseminating dyeing information among the micro-services through an observation system; under the condition that the target service request is transmitted to the target micro-service, obtaining dyeing information corresponding to the target service request from the observation system through the service grid system; under the condition that dyeing information corresponding to the target service request is obtained through the service grid system, matching processing is carried out on the target service request through the service grid system according to a preset routing rule; and under the condition that the target service request is successfully matched, executing a corresponding fault injection action in the target micro-service through the service grid system. According to the technical scheme provided by the embodiment of the invention, when fault injection is carried out, independent agent code logic does not need to be maintained for micro-services realized by different languages.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a fault injection method and apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the continuous development and widespread application of cloud-native technologies, the scale and complexity of microservice systems have entered a new stage, and the stability and availability of microservice systems have been constantly challenged.

[0003] In order to improve the stability and availability of the microservice system, during the system testing phase, we artificially inject faults into the system to detect how the system handles anomalies. We can then adjust and improve the system based on the detection results to reduce abnormal problems that may occur when the system is officially used.

[0004] However, the inventors have found that multiple microservices in an existing microservice system are usually developed using different development languages. Thus, when performing fault injection on such a microservice system, a set of fault injection agent codes needs to be maintained for different development languages, resulting in poor scalability and high complexity of such a fault injection method.

[0005] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0006] The embodiments of the present application provide a fault injection method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0007] One aspect of an embodiment of the present application provides a fault injection method for use in a microservice system, wherein an observability system and a service grid system are deployed in the microservice system, and the microservice system includes multiple microservices, and the method includes: Adding coloring information to the target service request through the observability system; Propagate the coloring information between microservices through the observability system; When the target service request is transmitted to the target microservice, obtaining coloring information corresponding to the target service request from the observability system through the service grid system; When the coloring information corresponding to the target service request is obtained through the service grid system, matching processing is performed on the target service request according to a preset routing rule through the service grid system; When the target service request is matched successfully, the corresponding fault injection action is performed in the target microservice through the service grid system.

[0008] Optionally, adding coloring information to the target service request through the observability system includes: Intercepting incoming service requests through the service grid system; Detecting, by the service grid system, whether the intercepted service request is the target service request; When the intercepted service request is the target service request, the observability system is called by the service grid system to add coloring information to the target service request.

[0009] Optionally, the observability system is OpenTelemetry, and the service mesh system is Istio.

[0010] Optionally, calling the observability system through the service grid system to add coloring information to the target service request includes: OpenTelemetry is called through Istio to add OpenTelemetry header information to the target service request, where the OpenTelemetry header information is used to transmit tracing information between microservices, and the tracing information includes the dyeing information.

[0011] Optionally, when the coloring information corresponding to the target service request is obtained through the service grid system, matching the target service request according to a preset routing rule through the service grid system includes: When the coloring information corresponding to the target service request is obtained through Istio, the routing rule issued by the control plane in the Istio is obtained through the service agent in the Istio; Detecting, by the service proxy, whether there is a request header matching rule in the routing rule; In a case where the routing rule includes the request header matching rule, matching processing is performed on the target service request based on the request header matching rule.

[0012] Optionally, the method further comprises: In the event that the target service request fails to match, the target service request is executed by the target microservice according to the routing rule.

[0013] Another aspect of an embodiment of the present application provides a fault injection device for use in a microservice system, wherein an observability system and a service grid system are deployed in the microservice system, and the microservice system includes multiple microservices, wherein the device includes: An adding module, used for adding coloring information to a target service request through the observability system; A propagation module, configured to propagate the dyeing information between microservices through the observability system; an acquisition module, configured to acquire, when the target service request is transmitted to a target microservice, coloring information corresponding to the target service request from the observability system through the service grid system; A matching module, configured to, when the coloring information corresponding to the target service request is obtained through the service grid system, match the target service request according to a preset routing rule through the service grid system; The execution module is used to execute a corresponding fault injection action in the target microservice through the service grid system when the target service request is matched successfully.

[0014] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0015] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0016] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0017] The above technical solution adopted in the embodiment of the present application may include the following advantages: by adding coloring information (implementing the coloring function) to the target service request through an observability system (for example, OpenTelemetry), service requests of different protocols can be colored, avoiding adding additional header information to service requests of different protocols. In addition, when the target service request is transmitted to the target microservice, the coloring information corresponding to the target service request is obtained from the observability system through the service grid system (for example, Istio), and when the coloring information corresponding to the target service request is obtained through the service grid system, the target service request is matched and processed according to the preset routing rules through the service grid system, and when the target service request is successfully matched, the corresponding fault injection action is executed in the target microservice through the service grid system, thereby realizing the fault injection operation of the microservice system in a non-embedded manner through the service grid system, without the need to maintain independent agent code logic for microservices implemented in different languages. Since the above method implements fault injection in a non-embedded manner, the above fault injection method has very good scalability and a simple injection method. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 The operating environment diagram of the fault injection method according to the first embodiment of the present application is schematically shown; Figure 2 The flowchart of the fault injection method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S200; Figure 4 A detailed flow chart schematically shows the steps of matching the target service request according to the preset routing rules by the service grid system when the coloring information corresponding to the target service request is obtained by the service grid system; Figure 5 A schematic diagram showing an application example of a fault injection method according to an embodiment of the present application; Figure 6 A block diagram of a fault injection device according to Embodiment 2 of the present application is schematically shown; and Figure 7The hardware architecture diagram of a computer device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0022] First, the following terms are explained: Non-intrusive: In the context of software systems or test environments, non-intrusive refers to a method that does not substantially modify the internal structure, core code, or normal operating logic of the target system. It avoids directly implanting a large number of code snippets specifically for testing into the object being tested (such as applications, services, etc.) as much as possible.

[0023] Request level: The request level focuses on each individual request unit in the software system. In a network-based software system (such as Web services, distributed systems, etc.), a request is an instruction issued by the client to the server to obtain information or perform an operation. Request-level operations mean processing each individual request. Consider the entire life cycle of a request, including the stages of sending, transmitting, receiving, processing, and returning a response.

[0024] Fault injection: Fault injection is a technique used to test the reliability, fault tolerance and robustness of software systems. It observes the system's response by intentionally introducing various types of faults (such as hardware failures, software errors, network problems, etc.) into the system's operating environment. These faults can be simulated hardware failures (such as memory failures, CPU failures, etc.), software failures (such as program crashes, data corruption, etc.) or network failures (such as delays, packet loss, incorrect routing, etc.).

[0025] Microservice architecture: Microservice architecture is an architectural style for building applications. It decomposes a large, complex application into a set of small, loosely coupled, independently deployed and run microservices. These microservices focus on completing specific business functions, such as user management, order processing, payment processing, etc.

[0026] Istio: Istio is an open source service mesh system for managing, securing, and monitoring inter-service communications in a microservice architecture.

[0027] OpenTelemetry: OpenTelemetry is an open source observability framework for collecting, processing, and exporting tracing, metrics, and logs data in distributed systems. It provides a set of standard APIs, SDKs, and tools that enable developers and operators to fully understand the performance and behavior of the system.

[0028] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: With the continuous development and widespread application of cloud-native technologies, the scale and complexity of microservice systems have entered a new stage, and the stability and availability of microservice systems have been constantly challenged.

[0029] In order to improve the stability and availability of the microservice system, during the system testing phase, we artificially inject faults into the system to detect how the system handles anomalies. We can then adjust and improve the system based on the detection results to reduce abnormal problems that may occur when the system is officially used.

[0030] However, multiple microservices in an existing microservice system are usually developed using different development languages. Therefore, when performing fault injection on such a microservice system, a set of fault injection agent codes needs to be maintained for different development languages, which results in poor scalability and high complexity of this fault injection method.

[0031] To this end, the embodiment of the present application provides a fault injection technical solution. In this technical solution, the following effects can be achieved: (1) by adding coloring information to the target service request through the observable system (implementing the coloring function), service requests of different protocols can be colored, avoiding adding additional header information to service requests of different protocols; (2) the fault injection operation of the microservice system is realized in a non-embedded manner through the service grid system, without the need to maintain independent agent code logic for microservices implemented in different languages; (3) the above fault injection method has very good scalability and the injection method is simple. See below for details.

[0032] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0033] like Figure 1 As shown, the operating environment diagram includes: a service platform 2, and clients (4A, 4B, ..., 4N).

[0034] The service platform 2 can be connected to the client terminals ( 4A, 4B, ..., 4N) via the network.

[0035] The service platform 2 may be a single server, a server cluster or a cloud computing service center.

[0036] The service platform 2 can provide fault injection services to the client.

[0037] The service platform 2 may be located in a data center such as a single location, or distributed in different geographical locations (e.g., in multiple locations). The service platform 2 may provide services via a network. The network includes various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network may include physical links such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, or wireless links such as cellular links, satellite links, Wi-Fi links, and the like. The service platform 2 may be deployed with a microservice system, an observability system, and a service mesh system to facilitate the implementation of fault injection services.

[0038] The client (4A, 4B, ..., 4N) may be configured to access the content and services of the service platform 2. The client (4A, 4B, ..., 4N) may include an electronic device carrying or externally connected to a display panel, such as a mobile device, a tablet device, a laptop computer, a workstation, a virtual reality device, a gaming device, a digital streaming device, a vehicle terminal, a smart TV, a set-top box, etc., and may also include a virtualized computing instance. The virtualized computing instance may include a virtual machine, such as a simulation of a computer system, an operating system, a server, etc. The computing device may load a virtual machine based on a virtual image and / or other data defining specific software (e.g., an operating system, a dedicated application, a server) for simulation. As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices.

[0039] The client (4A, 4B, ..., 4N) may be associated with one or more users. A single user may also use one or more of the clients (4A, 4B, ..., 4N) to access the service platform 2. The client (4A, 4B, ..., 4N) may travel to various locations and use different networks to access the service platform 2.

[0040] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs. It should be noted that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments described here.

[0041] The following uses the service platform 2 as the execution subject to introduce the technical solution of the present application through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here.

[0042] Embodiment 1 The method embodiment may be used in a microservice system in which an observability system and a service grid system are deployed, and the microservice system includes a plurality of microservices.

[0043] Among them, the observability system is a system that can implement the link tracking and propagation mechanism between services. Through this system, developers and operation and maintenance personnel can fully understand the performance and behavior of the system to be observed (for example, the microservice system).

[0044] The observability system can be an observability framework such as OpenTelemetry, OpenTracing, OpenCensus, Prometheus, ELK (Elastic Stack), Fluentd, Datadog, etc.

[0045] A service mesh system is an infrastructure layer for handling inter-service communication. It is responsible for delivering reliable network requests for building complex cloud-native applications. In practice, a service mesh system is usually implemented as a set of lightweight network proxies that are deployed with applications but are transparent to the applications.

[0046] The service mesh system can be Istio, Linkerd, Consul, etc.

[0047] A microservice system is a system built using a microservice architecture.

[0048] Figure 2 The flowchart of the fault injection method according to the first embodiment of the present application is schematically shown.

[0049] like Figure 2 As shown, the fault injection method may include steps S200 to S208, wherein: Step S200: adding coloring information to the target service request through the observability system.

[0050] Step S202: propagate the coloring information between microservices through the observability system.

[0051] Step S204, when the target service request is transmitted to the target microservice, the coloring information corresponding to the target service request is obtained from the observability system through the service grid system.

[0052] Step S206, when the coloring information corresponding to the target service request is obtained through the service grid system, the target service request is matched and processed according to a preset routing rule through the service grid system.

[0053] Step S208: When the target service request is successfully matched, a corresponding fault injection action is performed in the target microservice through the service grid system.

[0054] The fault injection method provided in this embodiment adds coloring information (implements the coloring function) to the target service request through an observability system (e.g., OpenTelemetry), so that service requests of different protocols can be colored, avoiding adding additional header information to service requests of different protocols. In addition, when the target service request is transmitted to the target microservice, the service grid system (e.g., Istio) obtains the coloring information corresponding to the target service request from the observability system, and when the coloring information corresponding to the target service request is obtained through the service grid system, the target service request is matched and processed according to the preset routing rules through the service grid system, and when the target service request is successfully matched, the corresponding fault injection action is executed in the target microservice through the service grid system, thereby realizing the fault injection operation of the microservice system in a non-embedded manner through the service grid system, without the need to maintain independent agent code logic for microservices implemented in different languages. Since the above method implements fault injection in a non-embedded manner, the above fault injection method has very good scalability and a simple injection method.

[0055] The following combination Figure 2 , each step in steps S200~S208 and other optional steps are explained in detail.

[0056] Step S200 , adding coloring information to the target service request through the observability system.

[0057] The target service request is a service request that needs to inject faults. The target service request may carry request header information. The request header information may include path information, user information, etc. For example, the path information (Path) is: " / index", and the user information (User) is: "Jason".

[0058] Color information can consist of numbers or other available character types.

[0059] When the observability system is deployed in a microservice system, the observability system can add a specific identifier (i.e., "coloring") to the service request at the entrance of the service request, and then transmit and trace these service requests carrying the identifier in the basic framework layer of the service request transmission link.

[0060] Different service requests can be distinguished by different dyeing information, thereby ensuring that the service requests can be accurately controlled and managed.

[0061] In this embodiment, the observability system is used to add coloring information to the target service request, so that there is no need to invasively modify the code of the microservice system. In addition, since the observability system adds coloring information to the service request in a non-invasive manner, the observability system can be used to add coloring information to service requests of any protocol (such as http, grpc, etc.).

[0062] In an exemplary embodiment, the observability system is OpenTelemetry.

[0063] In this embodiment, since OpenTelemetry provides a set of standardized interfaces and data formats, OpenTelemetry can be used to easily add coloring information in a standard data format to the target service request.

[0064] In an alternative embodiment, see Figure 3 , step S200 may include: Step S300: intercepting the incoming service request through the service grid system.

[0065] Step S302, detecting, by the service grid system, whether the intercepted service request is the target service request; Step S304: when the intercepted service request is the target service request, the observability system is called by the service grid system to add coloring information to the target service request.

[0066] In actual applications, since it is necessary to perform fault injection on all incoming service requests, when a service request is received, the incoming service request can be intercepted by the service grid system.

[0067] In one implementation, a request marker agent may be pre-created in the service grid system to intercept incoming service requests. After intercepting the service request, the request marker agent may further detect whether the service request is a target service request.

[0068] Among them, when the request marking agent detects whether the service request is the target service request, it can make a judgment based on the attribute information of the target service request obtained in advance. For example, the attribute information of the target service request includes "user:Jason". When the request marking agent detects whether the service request is the target service request, it can detect whether the service request contains the attribute information of "user:Jason". If the service request contains the attribute information, it can be determined that the service request is the target service request. When the request marking agent detects that the service request is the target service request, it will call the observability system to add coloring information to the target service request to achieve the coloring of the target service request, so that the processing link of the target service request can be tracked based on the coloring information. When the request marking agent detects that the service request is not the target service request, it will release the service request according to the normal processing logic so that the service request can flow through to the corresponding microservice for response.

[0069] It should be noted that, in order to enable the request marking proxy to obtain the attribute information of the target request, a control plane (controller) can be created in advance in the service grid system, through which the attribute information of the target request can be sent to the request marking proxy. In addition, the control plane can also send corresponding routing rules to multiple service proxies (proxy) created in the grid system.

[0070] It should be noted that the service request may be a request for requesting the target microservice application to complete a target business, and the target business may be a business that can be executed by the target microservice application.

[0071] In this embodiment, the incoming service request is intercepted and processed so as to achieve accurate fault injection for a specific target service request in the service request, thereby ensuring that the fault injection is accurate and controllable.

[0072] In an exemplary embodiment, the service mesh system is Istio.

[0073] In this embodiment, by using Istio as a service mesh system, microservices can be better managed.

[0074] In an optional embodiment, when the observability system is OpenTelemetry and the service mesh system is Istio, calling the observability system through the service mesh system to add coloring information to the target service request includes: calling OpenTelemetry through Istio to add OpenTelemetry header information to the target service request, the OpenTelemetry header information is used to transmit tracing information between microservices, and the tracing information includes the coloring information.

[0075] OpenTelemetry header information can include trace_id, span_id, and other related baggage information.

[0076] In an example, the OpenTelemetry header information is “Ts:T= / indexJason”.

[0077] In this embodiment, Istio calls OpenTelemetry to add OpenTelemetry header information to the target service request, thereby achieving non-invasive transmission of tracing information between microservices.

[0078] Step S202 , the coloring information is propagated between microservices through the observability system.

[0079] After the observability system adds coloring information to the target service request, it will propagate the coloring information among multiple microservices in parallel along with the target service request.

[0080] Step S204 , when the target service request is transmitted to the target microservice, the coloring information corresponding to the target service request is obtained from the observability system through the service grid system.

[0081] In actual applications, when the target service request is transmitted to the target microservice according to its transmission path, the service grid system will query the observability system to determine whether there is coloring information corresponding to the target service request in the observability system. If the coloring information corresponding to the target service request exists in the observability system, the coloring information will be obtained from the observability system so that the target service request can be matched and processed based on the coloring information later.

[0082] Step S206 When the coloring information corresponding to the target service request is obtained through the service grid system, the target service request is matched and processed according to a preset routing rule through the service grid system.

[0083] Among them, the routing rules may include matching rules for service requests and processing rules for service requests. The matching rules and processing rules can be configured by the user according to actual conditions. For example, the matching rule is a request header matching rule. The request header matching rule is used to match and process the request header of the target service request.

[0084] In an example, the routing rule may be: “Match: * (match all)”, “Action: Forward (action: forward)”.

[0085] In another example, the routing rule may be: "Match: Header (match request header)", and the specific matching conditions are "Ts:T= / indexJason", "Action: Abort (action: abort)".

[0086] In an alternative embodiment, see Figure 4 In the case where the coloring information corresponding to the target service request is obtained through the service grid system, matching processing is performed on the target service request according to a preset routing rule through the service grid system, including: Step S400, when the coloring information corresponding to the target service request is obtained through Istio, the routing rules issued by the control plane in the Istio are obtained through the service agent in the Istio.

[0087] Step S402: Detect, by the service proxy, whether there is a request header matching rule in the routing rule.

[0088] Step S404: When the routing rule includes the request header matching rule, matching processing is performed on the target service request based on the request header matching rule.

[0089] The service proxy and control plane in Istio are pre-created in Istio. The service proxy is used to coordinate the inbound and outbound traffic of all microservices in the service grid. The control plane is responsible for managing and configuring the service proxy in the data plane, and sending the pre-configured routing rules to the service proxy so that the service proxy can obtain the pre-configured routing rules.

[0090] In this embodiment, after obtaining the routing rules issued by the control plane in the Istio through the service proxy in the Istio, it is detected whether the routing rules contain request header matching rules. If the routing rules contain request header matching rules, the target service request will be matched based on the request header matching rules to determine whether the specific matching conditions in the request header matching rules contain the coloring information corresponding to the target service request.

[0091] It can be understood that when the specific matching conditions in the request header matching rules include the coloring information corresponding to the target service request, it can be determined that the target service request matches successfully; when the specific matching conditions in the request header matching rules do not include the coloring information corresponding to the target service request, it can be determined that the target service request matches failed.

[0092] In this embodiment, matching processing of the target service request is implemented based on the request header matching rule, so that the matching result can be obtained quickly and the matching efficiency is improved.

[0093] Step S208 , when the target service request is matched successfully, the corresponding fault injection action is performed in the target microservice through the service grid system.

[0094] In this embodiment, when the target service request is successfully matched, the service grid system can directly execute the corresponding fault injection action in the target microservice in a non-invasive manner without modifying the original code of the microservice system.

[0095] The fault injection action may include delaying the fault injection action, terminating the fault injection action, etc.

[0096] In an optional embodiment, the method further includes: In the event that the target service request fails to match, the target service request is executed by the target microservice according to the routing rule.

[0097] In this embodiment, when the target service request fails to match, the target service request is executed by the target microservice according to the routing rule to achieve response processing of the target service request. For example, the target service request can be forwarded to a downstream microservice so that the target service request can be processed by the downstream microservice.

[0098] In order to make this application easier to understand, the following Figure 5 An exemplary application is provided. An HTTP request with a path of " / index" and a user of "Jason" in the request header enters the system. The request first passes through the "Request Marker Agent" (a request marker agent created in the observability system), which processes the request and adds additional marker information (coloring information). The newly added markers include "Tp: v-t'-p'-f" and "Ts: T= / indexJason". The request processed by the Request Marker Agent reaches Proxy A, which processes the request according to the "Routing Rule" issued by the control plane of the observability system. Since the current routing rules are "Match: *" and "Action: Forward", Service Proxy A forwards the request to the next service, and the marker information in the request header is also propagated. When the request reaches Proxy B, since the routing rule here is "Match: Header", the specific conditions are "Ts: T= / indexJason", "Action: Abort (Action: Abort). Since the "Ts" field in the request header meets the matching condition, Agent B aborts the request according to the rule.

[0099] Embodiment 2 Figure 6 The block diagram of the fault injection device according to the second embodiment of the present application is schematically shown, which is used in a microservice system, in which an observability system and a service grid system are deployed, and the microservice system includes multiple microservices. The fault injection device 600 can be divided into one or more program modules, and one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can perform specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 6 As shown, the fault injection device 600 may include: an adding module 610, a propagation module 620, an acquisition module 630, a matching module 640, and an execution module 650, wherein: An adding module 610, configured to add coloring information to a target service request through the observability system; A propagation module 620, configured to propagate the coloring information between microservices through the observability system; An acquisition module 630 is configured to acquire, when the target service request is transmitted to a target microservice, coloring information corresponding to the target service request from the observability system through the service grid system; A matching module 640 is used to match the target service request according to a preset routing rule through the service grid system when the coloring information corresponding to the target service request is obtained through the service grid system; The execution module 650 is used to execute a corresponding fault injection action in the target microservice through the service grid system when the target service request is matched successfully.

[0100] As an optional embodiment, adding coloring information to the target service request through the observability system includes: Intercepting incoming service requests through the service grid system; Detecting, by the service grid system, whether the intercepted service request is the target service request; When the intercepted service request is the target service request, the observability system is called by the service grid system to add coloring information to the target service request.

[0101] As an optional embodiment, the observability system is OpenTelemetry, and the service mesh system is Istio.

[0102] As an optional embodiment, calling the observability system by the service grid system to add coloring information to the target service request includes: OpenTelemetry is called through Istio to add OpenTelemetry header information to the target service request, where the OpenTelemetry header information is used to transmit tracing information between microservices, and the tracing information includes the dyeing information.

[0103] As an optional embodiment, when the coloring information corresponding to the target service request is obtained through the service grid system, matching the target service request according to a preset routing rule through the service grid system includes: When the coloring information corresponding to the target service request is obtained through Istio, the routing rule issued by the control plane in the Istio is obtained through the service agent in the Istio; Detecting, by the service proxy, whether there is a request header matching rule in the routing rule; In a case where the routing rule includes the request header matching rule, matching processing is performed on the target service request based on the request header matching rule.

[0104] As an optional embodiment, the execution module 650 is further configured to: In the event that the target service request fails to match, the target service request is executed by the target microservice according to the routing rule.

[0105] Embodiment 3 Figure 7 The hardware architecture diagram of a computer device 700 suitable for implementing a fault injection method according to the fifth embodiment of the present application is schematically shown. In some embodiments, the computer device 700 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 700 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Figure 7 As shown, the computer device 700 includes but is not limited to: a memory 710, a processor 720, and a network interface 730 which can communicate with each other through a system bus. Among them: The memory 710 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 710 can be an internal storage module of the computer device 700, such as a hard disk or memory of the computer device 700. In other embodiments, the memory 710 can also be an external storage device of the computer device 700, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 700. Of course, the memory 710 can also include both the internal storage module of the computer device 700 and its external storage device. In this embodiment, the memory 710 is generally used to store the operating system and various application software installed on the computer device 700, such as the program code of the fault injection method, etc. In addition, the memory 710 can also be used to temporarily store various types of data that have been output or are to be output.

[0106] The processor 720 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips in some embodiments. The processor 720 is generally used to control the overall operation of the computer device 700, such as performing control and processing related to data interaction or communication with the computer device 700. In this embodiment, the processor 720 is used to run the program code stored in the memory 710 or process data.

[0107] The network interface 730 may include a wireless network interface or a wired network interface, and the network interface 730 is generally used to establish a communication link between the computer device 700 and other computer devices. For example, the network interface 730 is used to connect the computer device 700 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 700 and the external terminal. The network may be a wireless or wired network such as an Intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0108] It should be pointed out that Figure 7 Only a computer device having components 710 - 730 is shown, but it should be understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0109] In this embodiment, the fault injection method stored in the memory 710 may also be divided into one or more program modules and executed by one or more processors (such as the processor 720) to complete the embodiment of the present application.

[0110] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the fault injection method in the embodiment are implemented.

[0111] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the fault injection method in the embodiment, etc. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0112] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0113] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0114] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A fault injection method, characterized in that: Used in a microservice system, wherein an observability system and a service grid system are deployed in the microservice system, and the microservice system includes multiple microservices, and the method includes: Adding coloring information to the target service request through the observability system; Propagate the coloring information between microservices through the observability system; When the target service request is transmitted to the target microservice, obtaining coloring information corresponding to the target service request from the observability system through the service grid system; When the coloring information corresponding to the target service request is obtained through the service grid system, matching processing is performed on the target service request according to a preset routing rule through the service grid system; When the target service request is matched successfully, the corresponding fault injection action is performed in the target microservice through the service grid system.

2. The method according to claim 1, characterized in that Adding coloring information to the target service request through the observability system includes: Intercepting incoming service requests through the service grid system; Detecting, by the service grid system, whether the intercepted service request is the target service request; When the intercepted service request is the target service request, the observability system is called by the service grid system to add coloring information to the target service request.

3. The method according to claim 1, characterized in that The observability system is OpenTelemetry, and the service mesh system is Istio.

4. The method according to claim 3, characterized in that Calling the observability system through the service grid system to add coloring information to the target service request includes: OpenTelemetry is called through Istio to add OpenTelemetry header information to the target service request, where the OpenTelemetry header information is used to transmit tracing information between microservices, and the tracing information includes the dyeing information.

5. The method according to claim 3, characterized in that: In the case where the coloring information corresponding to the target service request is obtained through the service grid system, matching processing is performed on the target service request according to a preset routing rule through the service grid system, including: When the coloring information corresponding to the target service request is obtained through Istio, the routing rule issued by the control plane in the Istio is obtained through the service agent in the Istio; Detecting, by the service proxy, whether there is a request header matching rule in the routing rule; In a case where the routing rule includes the request header matching rule, matching processing is performed on the target service request based on the request header matching rule.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: In the event that the target service request fails to match, the target service request is executed by the target microservice according to the routing rule.

7. A fault injection device, characterized in that: Used in a microservice system, wherein an observability system and a service grid system are deployed in the microservice system, the microservice system includes multiple microservices, and the device includes: An adding module, used for adding coloring information to a target service request through the observability system; A propagation module, configured to propagate the dyeing information between microservices through the observability system; an acquisition module, configured to acquire, when the target service request is transmitted to a target microservice, coloring information corresponding to the target service request from the observability system through the service grid system; A matching module, configured to, when the coloring information corresponding to the target service request is obtained through the service grid system, perform matching processing on the target service request according to a preset routing rule through the service grid system; The execution module is used to execute a corresponding fault injection action in the target microservice through the service grid system when the target service request is matched successfully.

8. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images