Micro-service chaos test method and device based on Istio and link tracking

Abstract

The invention discloses a micro-service chaos test method and device based on Istio and link tracking, and relates to the technical field of cloud native, and the method comprises the steps: S1, submitting a test task; s2, initializing a test environment; s3, tracking a micro-service-level calling link of the request through a link tracking function, and generating a request tracking tree in combination with the request definition and the micro-service-level calling link; s4, implementing a bottom-up test strategy: starting to implement test cases from bottom to top from a leaf node of the request tracking tree, wherein the execution of a single test case comprises the steps of injecting a candidate fault, initiating a specified request, and triggering an Envoy filter to obtain a preset response; and S5, generating a final test result. Generation and execution of the test cases are automatically completed by the system, complexity and labor intensity of manual operation are greatly reduced, and efficient tracking data acquisition and processing are achieved.

Description

Technical Field

[0001] The present invention belongs to the field of cloud native technology, and specifically relates to a microservice chaos testing method and device based on Istio and link tracing. Background Art

[0002] With the development of cloud computing and container technology, microservice architecture has gradually become the mainstream of modern distributed system design. It splits a single application into multiple independent services, each of which is responsible for a specific business function and can be independently developed, deployed and expanded. Compared with the traditional monolithic architecture, the microservice architecture has advantages in scalability, development and deployment efficiency, maintainability and manageability. However, systems based on microservice architecture are usually more complex, and the occurrence of a fault may affect multiple aspects of the system. Therefore, for testers, identifying and resolving these complex faults is crucial to improving the resilience of microservice architecture systems.

[0003] Kubernetes (K8s for short) is an open source container orchestration platform designed to automate the deployment, expansion and management of applications. It makes running containerized applications in large-scale cluster environments more efficient and reliable by providing powerful features such as automated deployment, elastic scaling, service discovery, load balancing and self-healing capabilities. Kubernetes is widely used in modern cloud computing and microservice architecture, and has become a standard solution for managing containerized applications.

[0004] Istio is an open source service mesh solution designed for microservice architecture, providing traffic management, security, observability, policy control and other functions. By deploying an agent (envoy) next to each service instance, Istio can achieve communication, load balancing, authentication and authorization, monitoring and fault injection between services, thereby enhancing the elasticity, reliability and security of the system, making the management and operation of distributed microservices more efficient and controllable.

[0005] Tracing is a method for monitoring and debugging distributed systems. It generates a complete request path by recording and tracing the services and components that each request passes through in the system. It helps developers and operators understand the internal operation status of the system, identify performance bottlenecks, and locate problems, thereby improving the observability and reliability of the system.

[0006] Chaos engineering is used to simulate various failure scenarios that may occur in the actual production environment in the system to trigger problems that may occur in the system when facing these failures. Relevant personnel propose solutions based on the triggered problems, thereby improving the system's overall ability to cope with risks. However, manual chaos engineering experiments are inefficient and require a lot of manpower costs; random injection of faults has duplications and omissions, low execution efficiency, and cannot fully cover the test space. Testing in an isolated environment requires a lot of resources to cooperate, and for some organizations and enterprises with high information security requirements (such as finance, military, technology, electricity and other industries), it is almost impossible to deploy a system dedicated to testing in other environments. Existing microservice chaos testing tools are: Chaos Engineering Platform: Chaos Mesh is a typical chaos engineering platform suitable for fault injection in Kubernetes environments. However, each time a fault injection is performed, it is necessary to manually specify the injection target and configure the fault parameters. After the injection is completed, a test request must be manually issued and the effect of the fault injection must be manually checked to see if it meets expectations.

[0007] HTTP layer fault injection tool: Filibuster, as a representative of HTTP layer fault injection tools, can inject pre-configured faults into downstream requests and responses at the network layer after issuing a test request. It automatically detects the dependencies between microservices through request tracing, generates the required fault combinations, avoids unnecessary fault injection, and automatically checks whether the test results meet expectations. However, this method requires modifying the source code of the microservice under test to implement request tracing and fault injection.

[0008] Random fault injection tool: Chaos Monkey is a typical representative of random fault injection tools, which supports regular random injection of crash faults in production environments. Although it can effectively discover system problems, it will also have an impact on real users and it is difficult to quantify the progress of the test.

[0009] Microservice chaos testing tool based on automatic dependency discovery and proxy: This tool has a high dependence on resources, is not suitable for experiments in production environments, and is difficult to apply and promote in practice.

[0010] Therefore, the research on microservice chaos testing methods based on Istio and link tracing is of great significance. Summary of the invention

[0011] In order to solve the above technical problems, the present invention provides a microservice chaos testing method and device based on Istio and link tracing. By using Envoy to intercept requests to obtain request definitions and return preset responses, the dependent services do not need to actually process these requests. By directly returning preset responses, various service response situations can be quickly simulated to speed up the testing process; the microservice granularity links involved are obtained through link tracing, and a request tracing tree corresponding to the microservice and the request is generated in combination with the request definition intercepted by Envoy. Subsequently, a bottom-up testing strategy is implemented to fully cover the test space.

[0012] To achieve the above purpose, the technical solution adopted by the present invention is as follows: On the one hand, the present invention provides a microservice chaos testing method based on Istio and link tracing, the method comprising: Step S1, submitting a test task: the test task content includes the namespace where the current system under test is located, the request definition for selecting to trigger the link under test, and the fault candidate list of each microservice; Step S2: Initialize the test environment: Ensure that Istio has been installed, the Sidecar mode has been enabled in the system under test, and the link tracking function has been integrated; Step S3, generate a request tracking tree: initiate a request through the processed request definition that triggers the link under test, use Envoy to filter out the request definition that passes through each microservice, and use the link tracking function to track the microservice-level call link of this request, and generate a request tracking tree by combining the request definition and the microservice-level call link; Step S4, implement a bottom-up testing strategy: implement test cases from the leaf nodes of the request tracing tree from the bottom up. The execution of a single test case includes injecting candidate faults, initiating a specified request, and triggering the Envoy filter to obtain a preset response. Step S5, generating final test results: Generate test results in a specified format by collecting information related to requests, preset responses, microservice calls, and injected fault types.

[0013] On the other hand, the present invention provides a microservice chaos testing device based on Istio and link tracing, the device comprising: The Trace processor is used to mark and initiate requests to the link under test, generate a request tracing tree based on the link tracing data of this request and the request definition of each request filtered by the Envoy filter, and trigger the test case generator; The test case generator is used to generate test cases based on the leaf nodes of the request tracing tree generated by the Trace processor and the fault candidate list corresponding to each service submitted by the front end; Envoy filter, which is used to process the requests to downstream services triggered by the requests marked by the Trace processor into request definitions in a specified format, and to trigger the generation of response filters for the specified requests initiated by the test case executor or return preset responses based on the existing response filters; The test case executor is used to inject the fault specified by the test case, initiate the specified request, and trigger the Envoy filter. If the specified request has a corresponding response, it will get the preset response. If not, the result returned by the tested service and the information of this test case are submitted to the Envoy filter to generate a response filter. The test result processor is used to generate a test result in a specified format according to the test process data generated by the test case executor.

[0014] Compared with the prior art, the present invention has the following advantages: 1. Low labor cost: The present invention significantly reduces the need for manual intervention through automated testing processes. By using Istio's automatic sidecar injection and link tracking tools, automatic identification and tracking of test links are achieved. The generation and execution of test cases are automatically completed by the system, which greatly reduces the complexity and labor intensity of manual operations.

[0015] 2. High execution efficiency: The bottom-up test strategy is adopted to improve the execution efficiency of the test by executing test cases in parallel. The Envoy filter can effectively intercept and process requests and responses, reducing unnecessary network overhead. By dynamically injecting the OpenTelemetry Agent and configuring the OpenTelemetry Collector, efficient tracking data collection and processing are achieved.

[0016] 3. Wide application scenarios: The present invention supports testing in a production environment and is applicable to microservice systems of various scales. Whether it is a simple small-scale microservice architecture or a complex large-scale distributed system, the method and device of the present invention can be effectively applied.

[0017] 4. Low resource usage: Using Envoy to intercept and process requests and responses reduces the impact on system performance. The application of sampling strategies and preset response mechanisms further reduces the demand for computing and storage resources during the test process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a microservice chaos testing method based on Istio and link tracing of the present invention; Figure 2 Generate a request tracking tree phase flow chart for the present invention; Figure 3A flow chart of a bottom-up testing strategy for implementing the present invention; Figure 4 This is a structural block diagram of a microservice chaos testing device based on Istio and link tracing in the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only specific embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The present invention provides a microservice chaos testing method based on Istio and link tracing, the principle of which is: by identifying and executing the initial request of the test link selected by the user, the request_handler hook function (referred to as the request filter in the present invention) in the Lua filter of Envoy deployed by the Sidecar mechanism of Istio is triggered. The request filter extracts the request definition (including request path, request method, request header, and request body) of each microservice triggered by this request. At the same time, the application service through the dynamic injection of OpenTelemetry Agent has been monitored by the tracing tool, and the tracing tool will track the call link of the target request. Based on the request call link information and the request definition of each microservice, a request tracing tree of a specified format is generated. The data format of the tracing tree is a tree structure, and each node represents a request triggered on this link. Next, the test case generation operation is performed through the request tracing tree, and a coroutine is created for all nodes of the request tracing tree. In each coroutine, it is determined whether the current microservice under test has a downstream dependent microservice. If there is a downstream dependent microservice, it is necessary to determine whether the test case of the downstream dependent microservice has been executed; if it has been completed, the test case generation operation can be performed on the current microservice. Test case generation is divided into three steps: select the request of the current microservice under test, select the fault to be injected, and select the test case of the downstream dependent microservice. Each generated test case will be executed immediately by the test case executor. If the current microservice under test has no downstream dependent microservice, the test case generation operation is directly performed on the current microservice. During the execution of the test case, the fault object that can be injected by the Chaos Mesh client is generated through the specified candidate fault definition, and the fault injection operation is performed through the Chaos Mesh client. After the request is responded to, the fault is restored. Then, the selected request is attached with an execution status specific identifier and the request is executed again. The response filter intercepts specific requests and returns a preset response to the target service. Through the bottom-up execution strategy, until all test cases of the root node are executed, the test task is completed. Finally, combined with the test case and the response obtained, the test result processor is processed into data in a specified format and returned to the front end for display.

[0021] Embodiment 1 Combination Figure 1 The method of the present invention is described in detail.

[0022] Step S1, submit the test task: the test task content includes the namespace where the current system under test is located, the request definition selected to trigger the link under test, and the fault candidate list of each microservice.

[0023] Step S2, initialize the test environment: Make sure that Istio is installed and the Sidecar mode is enabled in the system under test and the link tracking function is integrated.

[0024] Step S3, generate a request tracking tree: initiate a request through the processed request definition that triggers the link under test, use Envoy to filter out the request definition that passes through each microservice, and use the link tracking tool to track the microservice-level call link of this request. Combine the request definition and the microservice-level call link to generate a request tracking tree.

[0025] Step S4, implement bottom-up testing strategy: start from the leaf node of the request tracking tree and implement the test from bottom to top. Initiate a request stored on the service node. If the service node does not initiate a request to the downstream service, the response of this service node is used as the preset response for the next same request to be intercepted by Envoy and return the preset response; if the service node initiates a request to the downstream service, the request to the downstream service will be intercepted by Envoy and return the preset response. The execution of a single test case includes four steps: injecting candidate faults, initiating a specified request, obtaining a preset response, and setting a response filter for this test case.

[0026] Step S5, generate final test results: Generate final test results by collecting requests, responses, microservices, injection fault types and downstream dependency fault types, and provide them to the front end for display.

[0027] In one embodiment, the above step S1 includes: Step S11, select the namespace where the system under test is located, where the namespace refers to the namespace in k8s; Step S12, select the request definition that triggers the link under test. The test link is a series of request links triggered by the request corresponding to the function point displayed by the front end. For example: when the function point displayed by the front end is the order function, the series of services that can be triggered can be simply described as the front-end service first requests the user service, obtains the user verification information, and then calls the payment service according to the verification information. If the payment is successful, it will trigger the shopping cart service to empty the shopping cart. At the same time, the inventory service will also reduce the corresponding number of items, and the transportation service will start to generate transportation information; this request link can be described as: front-end service->user service->payment service, front-end service->shopping cart service, front-end service->transportation service, front-end service->inventory service, where "A->B" indicates that A initiates a request to B service. The request definition is considered to be the definition of HTTP request in the present invention, including request path, request method, request header, and request body.

[0028] Step S13: configure the fault candidate list of each microservice, configure the chaotic faults that need to be injected into each microservice, and the chaotic faults act on the container of the application service, such as network failure, CPU pressure failure, memory pressure failure, I / O failure, etc.

[0029] In one embodiment, the above step S2, initializing the test environment: ensuring that Istio is installed and the Sidecar mode is enabled in the system under test. At the same time, dynamically inject the OpenTelemetry Agent into all applications in the system and configure the corresponding OpenTelemetry Collector. Specifically including: Step S21: Check whether Istio is installed and whether the Sidecar mode is enabled in the system under test: Istio and Sidecar mode are prerequisites for ensuring that each application service is installed with Envoy, and Envoy is a prerequisite for ensuring that the request filtering module and the response filtering module are effective. If Istio is not installed, you need to exit the current process and install Istio.

[0030] Step S22, check whether the system has integrated the link tracking function: If the link tracking function is not integrated, the following steps can be performed to implement the link tracking function in a non-invasive manner. First, it is necessary to dynamically inject the OpenTelemetry Agent into the runtime of the container of all applications in the system. The method used is to inject the prepared OpenTelemetry Agent file into the runtime of the application container through the jattach tool (the present invention currently only considers the JVM runtime); then configure the OpenTelemetry Collector, and configure the link tracking service in the Collector. In this invention, jaeger is taken as an example.

[0031] In one embodiment, in combination Figure 2 Step S3 is described below. The above step S3 specifically includes: Step S31, process the request definition and initiate the request: add unique identifier information to the request header of the request definition to distinguish the trace triggered by this request from a large number of traces, and then initiate this request. Record the request initiation time and response time as the time interval for obtaining the trace; Step S32, filter the request definition of microservices: All request definitions of each microservice can be filtered out through the configured Envoy Lua Filter script and the custom log handler; Step S33, obtain the request call link and generate a request call tree: track the microservice call link of this request through the link tracking tool and combine the request definition data with the request call link to generate a request call tree; In one embodiment, the above step S32 specifically includes: Step S321, configure the request filter: mainly configure the namespace where the test system is located, the envoy_on_request hook function, and the byte stream parsing function. The specific instructions are as follows: envoy_on_request hook function: Define the envoy_on_request hook function to handle related operations when a request is received; obtain the request method and request path; collect all request headers; output basic information such as request URL, method, path, and request header.

[0032] Get request body: Ensure that the request body is obtained, even if the request body is empty; when the request method is POST and the request body is not empty, read and convert the request body into a string; parse JSON data and record the parsing results or error log.

[0033] In this configuration, the key parts include using the envoy_on_request hook function to extract and record basic information of the request (such as request method, path, header, and body), converting the request body from a byte stream to a readable string through the byte stream parsing function (pcall), and trying to parse JSON data. If the parsing is successful, the parsed JSON data is recorded; if the parsing fails, the error log is recorded.

[0034] The above configuration ensures that the basic information of the request can be effectively extracted and processed in the envoy_on_request hook function, thereby providing necessary data support for subsequent chaos testing and tracing.

[0035] Step S322: Parse the Envoy log to obtain the definition data of each microservice request: obtain the service log according to the service name, and filter out the target log. Specifically: Get service logs based on service name and filter out target logs. Specifically: If this request triggers a request to the user service, the Envoy log of the user service will contain the output information of the request definition for this request. First, you need to increase the Envoy log level of the service to view the Info level log. For example, you can use the following command to increase the log output level: kubectl exec user-5d7b595bcd-w85vh -c istio-proxy -n sock-shop --curl -X POST http: / / localhost:15000 / logging?level=debug This command can be used to increase the log output level. Then, use the following command to filter out the output of each part of the request definition that belongs to this call link: kubectl logs -f user-5d7b595bcd-w85vh -c istio-proxy -n sock-shop | grep "info" | grep ${traceId} Finally, the corresponding request definition data is parsed through the custom regular logic.

[0036] In one embodiment, the above step S33 specifically includes: Step S331: Obtain all trace data within the specified time interval by recording the request initiation time and response time, as well as the service names mainly involved, and then filter out the target trace data according to the set specific identification symbol. The specific steps are as follows: In the present invention, the triggered requests are all issued by the front-end, so the front-end service name can be used as a parameter to query the trace. Use the API provided by the link tracking tool to obtain the trace. For example, obtain all traces within 5 minutes through the following URL: http: / / localhost:8083 / jaeger / api / traces?end=1686837600000000&limit=10&lookback=5m&service=front-end&start=1686837300000000 Then, the target trace, i.e. the request call link, is filtered out from the span of all front-end services according to the specific identifier.

[0037] S332. Generate a request call tree by combining the request definition data and the trace: Generate a request call tree structure based on the obtained trace. Specifically: Obtain the service names in all spans and perform deduplication to determine the nodes of the request call tree. Organize the relationship between each node through the parentId in the span to construct the edges of the request call tree. Combine the request definition data with each node to complete the generation of the request call tree.

[0038] In one embodiment, in combination Figure 3 Step S4 is described. Specifically, step S4 includes: Step S41, traverse all nodes of the request call tree and generate a coroutine for each node: traverse all nodes of the request call tree and generate a coroutine for each node. Each coroutine is responsible for processing the microservice corresponding to a node, ensuring parallel execution of test cases to improve test efficiency. This process utilizes the concurrent characteristics of the coroutine, allowing multiple nodes to be processed simultaneously, thereby speeding up the test process.

[0039] Step S42, monitor each node, and determine whether the test can be executed according to the state of the node; after generating the coroutine, determine whether the test can be executed according to the state of each node. The specific judgment criteria include: determining whether it is a leaf node, if it is a leaf node, immediately execute and generate test cases, if it is a non-leaf node, it is necessary to determine whether the test tasks of its downstream dependent services have been completed, if all the test tasks of the downstream services are in the completed state, then a test case can be generated for the current tested service.

[0040] Step S43, generate test cases for the microservices corresponding to the nodes: each test case should include the ID of the current service under test, request definition, a test case combination of downstream dependent services (select one test case for each downstream dependent service), and the fault injection type selected from the submitted test tasks. At the same time, each microservice test case includes a case where no fault is injected.

[0041] Step S44, execute test cases sequentially: according to the selected fault definition, use the Chaos Mesh client to inject faults, initiate the generated test request to the microservice, simulate the real request scenario, the response filter returns the preset response, records the test results, and sets the corresponding response filter configuration to set the preset response of the current request.

[0042] In one embodiment, the above step S43 specifically includes: Step S431, select request definition: According to the request definition list of the node, determine the specific request in the test case, including the request path, method, header information and request body, to ensure that the test case can accurately reflect the actual request behavior of the microservice.

[0043] Step S432, select a test case combination for downstream dependent services: select a corresponding test case combination for each downstream dependent service to ensure that when testing the current service, its call to the downstream dependent service can be fully considered, so as to ensure the integrity and consistency of the test. For example, when testing the order service, if the order service depends on the user service and the inventory service, it is necessary to select test cases for the user service and the inventory service respectively: User service test case: select a test case that simulates the response of the user service under normal and network interruption failure conditions, for example, the scenario of successful or failed user information acquisition. Normal situation: the user service returns user information normally. Network interruption failure condition: the user service is unavailable and returns an error message. Inventory service test case: select a test case that simulates the response of the inventory service under normal and service crash failure conditions, for example, the scenario of successful or failed inventory query. Normal situation: the inventory service returns inventory information normally. Service crash failure condition: the inventory service is unavailable and returns an error message. In the above manner, when testing the order service, its dependence on the user service and the inventory service can be fully covered.

[0044] Step S433, select the fault injection type: select the fault type to be injected from the submitted test tasks, clarify the fault scenario in the test case, and ensure that each test case contains a situation where no fault is injected, because the no fault situation is also a scenario.

[0045] In one embodiment, the above step S44 includes: Step S441: According to the selected fault definition, use the Chaos Mesh client to inject faults: the fault definition is the fault type data that the Chaos Mesh client can identify, for example: network delay: simulate network delay failure, by introducing delay on the specified network connection, test the performance and response of the microservice under network instability. For example, introduce a 500ms delay to the network connection between the Order Service and the User Service.

[0046] Step S442: Initiate a test request to the microservice according to the request definition selected by the test case: Before initiating the request, add the execution status specific identifier generated according to the microservice ID, fault injection type and request path to the request header, for example, "001-user-service-cpu-overload- / api / v1 / users". Step S443: The request to the service under test triggers the request of the downstream service, and the response filter returns the preset response: If there is a downstream dependent service for the service under test, then the request will inevitably trigger the response filter. The response filter will return the specified response content, which is the preset response under a specific request, a specific combination of downstream dependent service test cases, and a specific fault condition. The specific operations are as follows: The response filter is implemented based on the envoy_on_response hook function of Envoy's Lua filter. The response filter corresponds to the request one by one. In the envoy_on_response hook function, first determine whether the request path corresponding to the current response is the specified target request path and test case ID. If the request path is the target path and the test case ID is the target ID, the corresponding response data is obtained from the preset response content and returned to the client. The preset response content is generated based on the specified request, the specified downstream dependent service test case combination, and the specified fault condition to ensure the accuracy and consistency of the test.

[0047] Step S444, recovering the fault after receiving the response: After each test case is executed, it is necessary to ensure that the fault is recovered immediately. This is to prevent the fault state of the current test case from affecting the execution of the next test case. The steps of recovering the fault include canceling the injected fault, resetting the system state, and ensuring that all services are restored to normal operation.

[0048] Step S445, after receiving the response, a response filter for this test case needs to be generated: Based on the predefined response filter template, the filter is customized by string replacement. Replace ${namespace} with the namespace of the current system. Replace ${req_url} with the current request path. Replace ${testCase_ID} with the ID of the current test case. Replace ${respond_body} with the specific response content to be returned. Apply the customized response filter to the system to ensure that the preset content is returned in the response to the corresponding request. Finally, use the k8s client to deploy it to the current k8s environment.

[0049] Embodiment 2 like Figure 4 As shown, an embodiment of the present invention provides a microservice chaos testing device based on Istio and link tracing, and the device includes: A microservice chaos testing device based on Istio and link tracing, the device comprising: The Trace processor is used to mark and initiate requests to the link under test, generate a request tracing tree based on the link tracing data of this request and the request definition of each request filtered by the Envoy filter, and trigger the test case generator; The test case generator is used to generate test cases based on the leaf nodes of the request tracing tree generated by the Trace processor and the fault candidate list corresponding to each service submitted by the front end; Envoy filter, which is used to process the requests to downstream services triggered by the requests marked by the Trace processor into request definitions in a specified format, and to trigger the generation of response filters for the specified requests initiated by the test case executor or return preset responses based on the existing response filters; The test case executor is used to inject the fault specified by the test case, initiate the specified request, and trigger the Envoy filter. If the specified request has a corresponding response, it will get the preset response. If not, the result returned by the tested service and the information of this test case are submitted to the Envoy filter to generate a response filter. The test result processor is used to generate a test result in a specified format according to the test process data generated by the test case executor.

[0050] Wherein, the Trace processor includes a measured link processing module and a trace data processing module; A test case generator, wherein the test case generator includes a test case generation module based on a request tracking tree; Envoy filter, which includes request filter module and response filter module; A test case executor comprises a request processing module, a response filter processing module and a fault injection module.

[0051] Specifically, the link processing module under test attaches a tracking state specific identifier to the request definition selected by the front end, initiates a specific network request according to the type of request, and records the request initiation time and response time of the request.

[0052] Specifically, the tracing data processing module queries all traces within this time interval according to the request initiation time and response time recorded by the tested link processing module through the API provided by the link tracing tool. Since the present invention supports implementation in a production environment, it is necessary to distinguish the target trace. The module distinguishes the target trace through a specific identifier specified by the tested link processing module, and generates a request tracing tree through the obtained trace and the request definition of each microservice filtered out by the request filter.

[0053] Specifically, based on the test case generation module of the request tracking tree, it is determined whether the current microservice under test has downstream dependent microservices to operate. If there are downstream dependent microservices, it is necessary to determine whether the test cases of the downstream dependent microservices have been executed; if the execution is completed, the test case generation operation can be performed on this microservice. If the current microservice under test has no downstream dependent microservices, the test case generation operation can be directly performed on the current microservice under test. Test case generation is divided into three steps: selecting the request of the current microservice under test, selecting the fault to be injected, and selecting the test case of the downstream dependent microservice.

[0054] Specifically, the request filtering module filters out each request definition on the request call link through the TraceId obtained by the Trace processor, and parses out the request path, request method, request parameters and request header.

[0055] Specifically, the response filtering module intercepts the specified request and returns a preset response to the target service.

[0056] Specifically, the request processing module obtains the request definition of the current microservice under test from the request tracking tree, attaches the execution state specific identifier, and then initiates the request.

[0057] Specifically, after the request is responded to, the response filter processing module generates an execution status specific identifier according to the microservice ID, fault injection type and request path. A preset response for this test case is generated according to the response and the specific identifier and the response filter template resource. The corresponding preset response is replayed by specifying the execution status specific identifier of the request. If there is no preset response, the microservice itself returns the response.

[0058] Specifically, the fault injection module generates a fault object that can be fault-injected through the Chaos Mesh client through the specified candidate fault definition, and then performs the injection operation of this fault through the Chaos Mesh client, and recovers the fault after the request is responded to.

[0059] Specifically, the test result processor generates a test result in a specified format according to the test process data generated by the test case executor.

[0060] Through the collaborative work of the above components, the present invention provides an efficient and automated microservice chaos testing device that can effectively test the stability and robustness of the system in a production environment and provide detailed test results for analysis and optimization.

[0061] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A microservice chaos testing method based on Istio and link tracing, characterized in that: The method comprises: Step S1, submitting a test task: the test task content includes the namespace where the current system under test is located, the request definition for selecting to trigger the link under test, and the fault candidate list of each microservice; Step S2: Initialize the test environment: Ensure that Istio has been installed, the Sidecar mode has been enabled in the system under test, and the link tracking function has been integrated; Step S3, generate a request tracking tree: initiate a request through the processed request definition that triggers the link under test, use Envoy to filter out the request definition that passes through each microservice, and use the link tracking function to track the microservice-level call link of this request, and generate a request tracking tree by combining the request definition and the microservice-level call link; Step S4, implement a bottom-up testing strategy: implement test cases from the leaf nodes of the request tracing tree from the bottom up. The execution of a single test case includes injecting candidate faults, initiating a specified request, and triggering the Envoy filter to obtain a preset response. Step S5, generating final test results: Generate test results in a specified format by collecting information related to requests, preset responses, microservice calls, and injected fault types.

2. According to claim 1, a microservice chaos testing method based on Istio and link tracing is characterized in that: The step S1 comprises: Step S11, select the namespace where the system under test is located; Step S12: Select a request definition that triggers a link under test, where the link under test represents a series of request links triggered by a request corresponding to a function point displayed by the front end; Step S13: configure the fault candidate list of each microservice, including configuring the chaotic faults that need to be injected into each microservice.

3. According to a microservice chaos testing method based on Istio and link tracing according to claim 1, it is characterized in that: The step S2 comprises: Step S21: Check whether Istio is installed on the system under test and whether the Sidecar mode is enabled; Step S22: Check whether the system under test has integrated the link tracking function. If not, dynamically inject the OpenTelemetry Agent into the containers of all applications of the system under test, and configure the link tracking service in the OpenTelemetry Collector.

4. According to claim 1, a microservice chaos testing method based on Istio and link tracing is characterized in that: The step S3 comprises: Step S31, process the request definition and initiate the request: add unique identifier information to the request header of the request definition, initiate the request, and record the request initiation time and response time as the time interval for obtaining the Trace; Step S32, filter the request definition of microservices: filter out all request definitions of each microservice through the configured Envoy Lua Filter script and the custom log handler; Step S33, obtaining the request call link and generating a request call tree: using a link tracking tool to track the microservice call link of the request initiated, and combining the request definition data with the microservice call link of the request to generate a request call tree.

5. According to a microservice chaos testing method based on Istio and link tracing according to claim 1, it is characterized in that: The step S4 comprises: Step S41, traverse all nodes of the request call tree, generate a coroutine for each node, and each coroutine is responsible for processing the microservice corresponding to a node; Step S42: monitor each node, and determine whether the test can be executed according to the state of the node. If it is a leaf node, a test case can be generated. If it is a non-leaf node, determine whether the test task of its downstream dependent service has been completed. If it has been completed, a test case can be generated; Step S43: Generate test cases for the microservices corresponding to the nodes, each test case including the ID of the current service under test, request definition, a test case combination of downstream dependent services, and a fault injection type selected from the submitted test tasks; Step S44: According to the selected fault definition, the Chaos Mesh client is used to inject faults, and the generated test cases are initiated to the microservices to simulate the real request scenario, and the response filter returns the preset response.

6. A microservice chaos testing device based on Istio and link tracing, characterized in that: The device comprises: The Trace processor is used to mark and initiate requests to the link under test, generate a request tracing tree based on the link tracing data of this request and the request definition of each request filtered by the Envoy filter, and trigger the test case generator; The test case generator is used to generate test cases based on the leaf nodes of the request tracing tree generated by the Trace processor and the fault candidate list corresponding to each service submitted by the front end; Envoy filter, which is used to process the requests to downstream services triggered by the requests marked by the Trace processor into request definitions in a specified format, and to trigger the generation of response filters for the specified requests initiated by the test case executor or return preset responses based on the existing response filters; The test case executor is used to inject the fault specified by the test case, initiate the specified request, and trigger the Envoy filter. If the specified request has a corresponding response, it will get the preset response. If not, the result returned by the tested service and the information of this test case are submitted to the Envoy filter to generate a response filter. The test result processor is used to generate a test result in a specified format according to the test process data generated by the test case executor.

7. According to claim 6, a microservice chaos testing device based on Istio and link tracing is characterized in that: The Trace processor includes a tested link processing module and a Trace processor; The test case generator includes a test case generation module based on a request tracing tree; The Envoy filter includes a request filtering module and a response filtering module; The test case executor comprises a request processing module, a response filter processing module and a fault injection module.

8. According to claim 6, a microservice chaos testing device based on Istio and link tracing is characterized in that: The tested link processing module attaches a tracking status specific identifier to the request definition selected by the front end, then initiates a network request according to the request category, and records the request initiation time and response time of the request; the Trace processor, through the API provided by the link tracking tool, queries all traces within this time interval according to the request initiation time and response time recorded by the tested link processing module, and generates a request tracking tree.

9. According to claim 6, a microservice chaos testing device based on Istio and link tracing is characterized in that: The test case generation module based on the request tracking tree determines whether the current microservice under test has downstream dependent microservices for operation. If there are downstream dependent microservices, it is necessary to determine whether the test cases of the downstream dependent microservices have been executed; If the execution is completed, the test case generation operation is performed for this microservice; If the current microservice under test has no downstream dependent microservices, the test case generation operation is performed directly on the current microservice under test.

10. A microservice chaos testing device based on Istio and link tracing according to claim 6, characterized in that: The request filtering module filters out each request definition on the request call link through the Trace processor, and parses the request path, request method, request parameters and request header; The response filtering module intercepts the specified request and returns a preset response to the target service.

11. A microservice chaos testing device based on Istio and link tracing according to claim 6, characterized in that: The request processing module obtains the request definition of the current microservice under test from the request tracking tree, attaches the execution status specific identifier, and then initiates the request; The response filter processing module generates an execution status specific identifier according to the microservice ID, fault injection type and request path after the request is responded to, and generates a preset response for this test case according to the response and the specific identifier and the response filter template resource; The fault injection module generates a fault object for fault injection through a specified candidate fault definition, and then performs the injection operation of the fault object through the Chaos Mesh client, and recovers the fault after the request is responded to.

Images