Microservice key performance parameter identification method, system, device and storage medium

Abstract

The application discloses a micro-service key performance parameter identification method, system and device and a storage medium, wherein the method comprises the following steps: acquiring micro-service call chain data of a target micro-service system, judging whether the target micro-service system is abnormal in performance, and determining normal call chain data and abnormal call chain data; generating a first causal diagram according to the normal call chain data, and obtaining normal causal relationships of each node through causal learning; generating a second causal diagram according to the abnormal call chain data, determining the delay contribution degree of each node according to the second causal diagram and the normal causal relationships, and determining key micro-services according to the delay contribution degree; constructing a causal performance model; extracting a plurality of causal paths according to the causal performance model, determining the average causal effect of each configuration parameter on the target delay performance, and determining micro-service key performance parameters according to the average causal effect. The application improves the accuracy of micro-service performance abnormality positioning and can be widely applied to the technical field of micro-services.

Description

Technical Field

[0001] The present invention relates to the field of microservice technology, and in particular to a method, system, device and storage medium for identifying key performance parameters of a microservice. Background Art

[0002] Due to the unpredictability of external loads and the complex interactions between microservices, cloud providers often overprovision microservices with resources to ensure service reliability, often resulting in resource waste. While containerized microservices can easily scale resources to meet service-level agreements (SLAs), the varying loads borne by each microservice make it unrealistic to determine the resource specifications required for each microservice. This makes fine-grained resource management extremely difficult.

[0003] Currently, much work aims to guarantee SLAs in microservices scenarios, primarily through algorithms that detect critical bottleneck services along a request path and increase their CPU, memory, and other resources to ensure service reliability. However, this approach faces the following major challenges:

[0004] 1) Different requests pass through different microservices, and the complex call relationships make it difficult to locate the root cause of performance failures.

[0005] 2) Microservices are sensitive to resource allocation. Once improper resources are allocated, their request processing performance may be reduced or even cause the system to crash. Therefore, it is very difficult to avoid unsafe configuration ranges in the resource configuration space.

[0006] 3) A complete microservice system typically includes services responsible for processing logic, database services (such as MongoDB), and caching services (such as Memcached). Although extensive work has been conducted on database performance tuning, much of the focus has been on the impact of software parameters such as the number of threads and network connections on performance, while ignoring the underlying resource configuration. For example, even though sufficient memory resources were allocated to MongoDB, the improper design of its cacheSizeGB parameter still limited its request processing performance.

[0007] Therefore, optimizing microservice performance requires not only adjusting resource allocation but also exploring the parameters of the software providing the service itself to more efficiently guarantee SLAs. Based on research on microservice performance diagnosis and optimization scenarios, related research work needs to address the following issues: First, how to promptly and accurately detect microservice SLA violations when performance anomalies occur; second, how to identify the bottleneck services that play a key role in performance anomalies under the load of microservice dependencies; and third, how to select the key parameters that have the greatest impact on performance from a large variety of resource parameters and software parameters.

[0008] Due to the unique nature of microservice architectures, numerous microservices are invoked during request execution. Therefore, performance issues can propagate through the interactions between microservices. A common approach is to collect latency data from microservices during runtime and then use machine learning methods such as SVM to train a classifier to determine if an anomaly is occurring. However, this approach cannot identify the root cause of the performance issue and can only superficially identify the microservice as one of the key microservices experiencing the problem, limiting the accuracy of pinpointing microservice performance anomalies.

[0009] Because microservices are composed of numerous foundational software components, including databases like MongoDB and MySQL, locating key microservices presents the challenge of identifying these critical components, which themselves have numerous adjustable parameters. Due to the sheer size of microservice systems, many studies fail to address the configuration of the software itself, focusing solely on adjusting resource parameters such as the number of CPU cores and memory allocation during deployment. This approach fails to identify key microservice performance parameters, further impacting the accuracy of locating microservice performance anomalies.

[0010] Explanation of terms:

[0011] Microservices: A software development architecture that addresses the shortcomings of monolithic architectures, where all functionality is implemented in a single program (high coupling and difficulty updating or improving applications). Instead, it splits an application into multiple independent processes, each running as a service. Services communicate through well-defined interfaces. Because each service runs independently, updates, deployments, and expansions can be performed individually, reducing coupling between services. Microservices offer advantages such as flexible scalability, easy deployment, and high resilience.

[0012] REST: Web applications are divided into two parts: the front-end and the back-end. Due to the wide variety of front-end devices, a unified mechanism is necessary to facilitate communication between the front-end and the back-end. RESTful API is a relatively mature API design theory for internet applications. REST (Representational State Transfer) refers to a set of architectural constraints and principles. If an architecture conforms to the constraints and principles of REST, it is called a RESTful architecture. The GET method can be used to obtain data, and the POST method can be used to submit data. Following the design concept of microservices, RESTful uses the standard Json format. Currently, HTTP is the only instance related to REST. REST is also the communication protocol between microservices.

[0013] HTTP (Hypertext Transfer Protocol) is the most widely used network transmission protocol on the Internet, used to transmit data (HTML files, image files, query results, etc.). The HTTP communication process consists of a request from the client to the server and a response from the server to the client. The response message contains an HTTP response code, which indicates whether the request was fulfilled and the response was normal. Users can check the response code to determine whether the request was received and responded to normally by the server.

[0014] Microservice call chain trace: Microservice call chain trace refers to the recording and tracking of the operations performed and the time consumed by each microservice in the entire process of a request or transaction in a microservice architecture, from the client initiating the request, through the invocation of multiple microservices, and ultimately returning it to the client. Trace records the entire request call chain, including the response time, error messages, request and response data, and more for each microservice. Trace can help developers and operations personnel better understand the service call relationships and performance bottlenecks in the microservice architecture, thereby better troubleshooting and optimizing performance. Trace can provide the following information:

[0015] 1) The starting and ending points of the request, and each microservice it passes through.

[0016] 2) The response time of each microservice, as well as the request and response data.

[0017] 3) Error information for each microservice, as well as the time and location of the error.

[0018] 4) Resource usage of each microservice, including CPU, memory, network, etc.

[0019] Tracing typically requires integrating a tracing system, such as Zipkin or Jaeger, into a microservice architecture. These tracing systems can implement trace recording and tracing by adding a trace SDK to each microservice. Within each microservice, the trace SDK records the start and end times of requests, as well as request and response data, and sends this information to the tracing system for aggregation and analysis.

[0020] Causal Learning: Causal learning is a machine learning method that aims to make predictions and decisions by discovering causal relationships between inputs and outputs. Unlike traditional correlation learning methods, causal learning attempts to understand and model the causal relationships between variables, rather than just the correlations between them. Causal learning typically involves using causal inference methods to determine the causal relationships between variables, which includes using causal graphs and latent causal models to infer causal effects. The goal of this approach is to discover the causal factors that truly affect the output, rather than just the variables that are superficially correlated with the output. Some common methods for causal learning include:

[0021] 1) Causal diagram: A causal diagram is a graphical representation method used to show the causal relationship between variables. It can help people intuitively understand the causal relationship between variables and guide the process of causal inference.

[0022] 2) Latent causal model: A latent causal model is a statistical model used to estimate the causal effects between variables. It can help people infer the causal relationship between variables from observed data without the need for experiments.

[0023] 3) Causal inference: Causal inference is a statistical inference method used to determine the causal relationship between variables. It helps people infer causal effects from observed data, rather than just correlation.

[0024] Microservice performance violations: This refers to situations in which the performance of one or more microservices in a microservice architecture does not meet expectations or specifications. This may manifest as excessive service response times, decreased service throughput, or reduced service availability. Microservice performance violations may be caused by a variety of factors, including but not limited to the following:

[0025] 1) Network latency: Services in a microservice architecture typically communicate over the network. Network latency may slow down communication between services, thus affecting overall performance.

[0026] 2) Resource competition: Services in a microservice architecture typically share resources, such as database connections and caches. When multiple services compete for the same resources at the same time, it may cause resource bottlenecks, thereby affecting overall performance.

[0027] 3) Error handling: Services in a microservice architecture usually call each other. When an error occurs in a service, it may cause the error to propagate, thereby affecting the overall performance.

[0028] 4) Service topology: Services in a microservice architecture typically form a complex topology. When the calling relationships between services are complex, it may make performance issues difficult to locate and debug.

[0029] An SLA (Service Level Agreement) is a written agreement between a service provider and a customer regarding service levels. An SLA typically includes performance metrics, service level objectives, and support and maintenance requirements. It aims to clarify both parties' expectations for service quality and the service provider's operational responsibilities. An SLA typically covers the following areas:

[0030] 1) Service Scope: Clearly describe the scope of services that the service provider should provide, including requirements on the content, scope, time, etc. of the services.

[0031] 2) Service Level Indicators: These specifically describe the performance indicators of the service, such as response time, availability, throughput, etc., so that customers and service providers can measure and monitor the quality of service.

[0032] 3) Service level objectives: Clearly define the service level objectives that the service provider should achieve, such as system availability requirements, fault handling time, etc.

[0033] 4) Service support and maintenance: Describes the support and maintenance services that the service provider should provide during the service operation process, including troubleshooting, technical support, update maintenance and other aspects.

[0034] 5) Compensation and liability for breach of contract: stipulate the compensation mechanism and liability for breach of contract of both parties when the service level does not meet the requirements, as well as the methods and procedures for resolving disputes.

[0035] SLO (Service Level Objective): A specific goal or standard set by a service provider or operations team to ensure service quality. SLOs are part of an SLA and are used to quantify and measure service performance to ensure the service meets user needs and expectations. SLOs typically include the following aspects:

[0036] 1) Service Level Indicators: These clearly describe the performance indicators of the service, such as response time, availability, and throughput, so that customers and service providers can measure and monitor the quality of the service.

[0037] 2) Target value: specifies the numerical target that the service level indicator should achieve, for example, the system availability target is 99.9%, the average response time does not exceed 100 milliseconds, etc.

[0038] 3) Measurement methods: clearly describe how service level indicators are measured and monitored, and how relevant data are collected and recorded.

[0039] 4) Time frame: Specify the time frame for service level objectives, such as monthly, quarterly or annually. Summary of the Invention

[0040] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0041] To this end, an object of an embodiment of the present invention is to provide a method for identifying key performance parameters of a microservice, which improves the accuracy of locating microservice performance anomalies.

[0042] Another object of an embodiment of the present invention is to provide a microservice key performance parameter identification system.

[0043] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0044] In a first aspect, an embodiment of the present invention provides a method for identifying key performance parameters of a microservice, comprising the following steps:

[0045] Obtain microservice call chain data of the target microservice system, determine whether the target microservice system has performance abnormalities based on the microservice call chain data, and if the target microservice system has performance abnormalities, determine normal call chain data and abnormal call chain data based on the microservice call chain data;

[0046] Generate a first causal graph based on the normal call chain data, and obtain a normal causal relationship between nodes in the first causal graph through causal learning;

[0047] Generate a second causal graph based on the abnormal call chain data, and determine the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and then determine the key microservice based on the delay contribution;

[0048] Determining configuration parameters, intermediate performance indicators, and target delay performance of the key microservice, and constructing a causal performance model based on the configuration parameters, the intermediate performance indicators, and the target delay performance;

[0049] Extracting multiple causal paths based on the causal performance model, determining an average causal effect of each of the configuration parameters on the target delay performance based on the causal paths, and then determining several of the configuration parameters as microservice key performance parameters based on the average causal effect.

[0050] Furthermore, in one embodiment of the present invention, the step of obtaining microservice call chain data of a target microservice system, determining whether the target microservice system has abnormal performance based on the microservice call chain data, and when the target microservice system has abnormal performance, determining normal call chain data and abnormal call chain data based on the microservice call chain data specifically includes:

[0051] Obtain the call information of each request on each microservice of the target microservice system through the Docker-based Jaeger tracing framework, and use the call information as a node to generate the microservice call chain data, wherein the call information includes the execution interface and the residence time;

[0052] Determine the P95 delay value of the target microservice system within a target period according to the residence time of each node;

[0053] When the P95 delay value is less than or equal to a preset first threshold, it is determined that the performance of the target microservice system is normal;

[0054] When the P95 delay value is greater than the first threshold, the target microservice system performance is determined to be abnormal, and the normal nodes in the microservice call chain data whose residence time is less than or equal to the P95 delay value and the abnormal nodes whose residence time is greater than the P95 delay value are determined, and then normal call chain data is generated according to the normal nodes, and abnormal call chain data is generated according to the abnormal nodes.

[0055] Furthermore, in one embodiment of the present invention, the step of generating a first causal graph based on the normal call chain data and obtaining the normal causal relationship of each node in the first causal graph through causal learning specifically includes:

[0056] Reversing the directional relationship of each node of the normal call chain data to obtain the first causal graph;

[0057] Determine the causal mechanism of each node of the first causal graph to obtain an FCM causal model;

[0058] Causal learning is performed according to the FCM causal model to obtain the normal causal relationship of each node in the first causal graph.

[0059] Furthermore, in one embodiment of the present invention, the step of generating a second causal graph based on the abnormal call chain data, determining the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and then determining the key microservice based on the delay contribution specifically includes:

[0060] Reversing the directional relationship of each node of the abnormal call chain data to obtain the second causal graph;

[0061] Applying the normal causal relationship layer by layer to each node of the second causal graph, and determining the delay contribution of each node according to the degree of influence of each node on the end-to-end delay after applying the normal causal relationship;

[0062] The microservice corresponding to the node with the largest delay contribution is determined as the key microservice.

[0063] Furthermore, in one embodiment of the present invention, the step of determining the configuration parameters, intermediate performance indicators, and target delay performance of the key microservice, and constructing a causal performance model based on the configuration parameters, the intermediate performance indicators, and the target delay performance, specifically includes:

[0064] Determine a configuration file of the key microservice, use read and write latency as an optimization target, and optimize the configuration file using a Bayesian optimization algorithm to obtain the target latency performance, the intermediate performance indicator, and the corresponding configuration parameters, where the configuration parameters include software parameters and resource parameters;

[0065] Determine a first connection relationship between the configuration parameter and the intermediate performance indicator, and determine a second connection relationship between the intermediate performance indicator and the target delay performance, and determine causal pointing information of the first connection relationship and the second connection relationship through information entropy to obtain the causal performance model.

[0066] Furthermore, in one embodiment of the present invention, the step of extracting multiple causal paths based on the causal performance model and determining the average causal effect of each of the configuration parameters on the target delay performance based on the causal paths specifically includes:

[0067] Extracting multiple causal paths according to the causal performance model, wherein the intermediate nodes of the causal paths are the configuration parameters or the intermediate performance indicators, and the end nodes of the causal paths are the target delay performance;

[0068] Determining the average impact of each of the causal paths on the delay, and selecting a number of key causal paths based on the average impact;

[0069] The causal effect of each intermediate node of the critical causal path on the end node is determined, and the average causal effect of each configuration parameter on the target delay performance is determined based on the causal effect.

[0070] Furthermore, in one embodiment of the present invention, the step of determining the plurality of configuration parameters as key performance parameters of the microservice based on the average causal effect is specifically as follows:

[0071] Determining a plurality of the configuration parameters whose average causal effect is greater than a preset second threshold as the key performance parameters of the microservice;

[0072] or,

[0073] The configuration parameters are sorted from large to small according to the average causal effect, and several configuration parameters whose sorting values ​​are less than or equal to a preset third threshold are selected as the microservice key performance parameters.

[0074] In a second aspect, an embodiment of the present invention provides a microservice key performance parameter identification system, including:

[0075] A microservice system performance judgment module is used to obtain microservice call chain data of a target microservice system, and judge whether the target microservice system has performance abnormalities based on the microservice call chain data. If the target microservice system has performance abnormalities, normal call chain data and abnormal call chain data are determined based on the microservice call chain data.

[0076] a normal causal relationship learning module, configured to generate a first causal graph based on the normal call chain data, and obtain the normal causal relationship of each node in the first causal graph through causal learning;

[0077] a key microservice determination module, configured to generate a second causal graph based on the abnormal call chain data, determine the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and further determine the key microservice based on the delay contribution;

[0078] A causal performance model construction module is used to determine the configuration parameters, intermediate performance indicators and target delay performance of the key microservice, and to construct a causal performance model based on the configuration parameters, the intermediate performance indicators and the target delay performance;

[0079] A key performance parameter determination module is used to extract multiple causal paths based on the causal performance model, and determine the average causal effect of each of the configuration parameters on the target delay performance based on the causal paths, and then determine several of the configuration parameters as microservice key performance parameters based on the average causal effect.

[0080] In a third aspect, an embodiment of the present invention provides a device for identifying key performance parameters of a microservice, including:

[0081] at least one processor;

[0082] at least one memory for storing at least one program;

[0083] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned method for identifying key performance parameters of a microservice.

[0084] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a program executable by a processor, wherein the program executable by the processor is used to execute the above-mentioned method for identifying key performance parameters of a microservice when executed by the processor.

[0085] The advantages and benefits of the present invention will be described in part in the following description and will become apparent from the following description or learned through practice of the present invention:

[0086] An embodiment of the present invention obtains microservice call chain data of a target microservice system, determines whether the target microservice system has abnormal performance based on the microservice call chain data, and when the target microservice system has abnormal performance, determines normal call chain data and abnormal call chain data based on the microservice call chain data, then generates a first causal graph based on the normal call chain data, and obtains the normal causal relationship of each node in the first causal graph through causal learning, then generates a second causal graph based on the abnormal call chain data, and determines the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and then determines the key microservices based on the delay contribution, and then determines the configuration parameters, intermediate performance indicators and target delay performance of the key microservices, and constructs a causal performance model based on the configuration parameters, intermediate performance indicators and target delay performance, and then extracts multiple causal paths based on the causal performance model, and determines the average causal effect of each configuration parameter on the target delay performance based on the causal path, and then determines several configuration parameters as key performance parameters of the microservice based on the average causal effect. The embodiments of the present invention can mine the causal relationships hidden behind the data through causal learning and causal performance models, thereby more accurately locating the key microservices and key performance parameters that cause performance anomalies in the microservice system, thereby improving the accuracy of locating microservice performance anomalies; in addition, through the coordinated tuning of software parameters and resource parameters, the performance of the microservice system can be guaranteed to be normal without increasing resource parameters, thereby more efficiently managing system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0088] Figure 1 A flowchart of the steps of the method for identifying key performance parameters of microservices provided by an embodiment of the present invention;

[0089] Figure 2 A structural block diagram of a microservice key performance parameter identification system provided by an embodiment of the present invention;

[0090] Figure 3 This is a structural block diagram of a device for identifying key performance parameters of microservices provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0092] In the description of the present invention, "a plurality" means two or more. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly indicating the number of the indicated technical features, or as implicitly indicating the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art.

[0093] Reference Figure 1 , an embodiment of the present invention provides a method for identifying key performance parameters of microservices, which specifically includes the following steps:

[0094] S101. Obtain microservice call chain data of a target microservice system, and determine whether the target microservice system has performance abnormalities based on the microservice call chain data. When the target microservice system has performance abnormalities, determine normal call chain data and abnormal call chain data based on the microservice call chain data.

[0095] Specifically, a complete online system consists of multiple microservices, and executing a single request requires the collaborative work of multiple microservices. Therefore, end-to-end latency is actually the sum of the time a request spends processing across multiple microservices. While microservices facilitate development and maintenance by separating functionality, the collaborative work between microservices can cause performance issues to propagate along their interactions. Therefore, it's crucial to identify the root cause of increased latency.

[0096] As an optional implementation method, microservice call chain data of the target microservice system is obtained, and whether the target microservice system has abnormal performance is determined based on the microservice call chain data. When the target microservice system has abnormal performance, the step of determining normal call chain data and abnormal call chain data based on the microservice call chain data specifically includes:

[0097] S1011. Obtain call information of each request on each microservice of the target microservice system through the Docker-based Jaeger tracing framework, and use the call information as a node to generate microservice call chain data. The call information includes the execution interface and the residence time.

[0098] S1012. Determine the P95 latency value of the target microservice system within the target period based on the residence time of each node;

[0099] S1013. When the P95 delay value is less than or equal to a preset first threshold, it is determined that the performance of the target microservice system is normal;

[0100] S1014. When the P95 delay value is greater than the first threshold, determine that the performance of the target microservice system is abnormal, and determine the normal nodes in the microservice call chain data whose residence time is less than or equal to the P95 delay value and the abnormal nodes whose residence time is greater than the P95 delay value, and then generate normal call chain data based on the normal nodes and generate abnormal call chain data based on the abnormal nodes.

[0101] Specifically, an embodiment of the present invention deploys a Docker-based Jaeger tracing framework in the system. This framework can track the time a request stays on each microservice after entering the system from the front end, as well as the interface information for executing the request. The embodiment of the present invention collects the execution status of each request on a microservice over a period of time, and this data is used as a data set for locating key services. Here, the information about a request executed on a microservice is usually referred to as trace data, and the call information executed on each microservice is referred to as span information. Therefore, a trace is actually composed of multiple spans, and the entire trace forms a directed acyclic graph, in which a node represents each call of a microservice.

[0102] The embodiment of the present invention uses the P95 latency value to evaluate whether there are performance issues with the microservice system within one minute. The P95 latency value indicates that 95% of the request delays within this minute are lower than the current value. As an indicator of the current system performance, as long as the P95 latency value exceeds the set SLO (Service Level Objective) threshold, it means that the system currently has performance issues. The embodiment of the present invention uses trace data with a latency higher than the P95 latency value as abnormal call chain data for root cause location; and uses the remaining trace data as normal call chain data for learning the normal causal relationship of microservice calls.

[0103] S102: Generate a first causal graph based on normal call chain data, and obtain the normal causal relationship of each node in the first causal graph through causal learning.

[0104] As an optional implementation, generating a first causal graph based on normal call chain data and obtaining the normal causal relationship of each node in the first causal graph through causal learning specifically includes:

[0105] S1021. Reverse the directional relationship of each node of the normal call chain data to obtain a first causal graph;

[0106] S1022. Determine the causal mechanism of each node in the first causal graph to obtain an FCM causal model;

[0107] S1023. Perform causal learning based on the FCM causal model to obtain the normal causal relationship of each node in the first causal graph.

[0108] Specifically, the end-to-end delay is jointly determined by the execution time of all spans on the trace. On the trace path, the delay of the child node will significantly affect the delay of the parent node, because the execution time of the parent span includes the execution time of the child span. Therefore, the embodiment of the present invention converts the trace, a natural directed acyclic graph, into a causal graph for locating the root cause of the delay. Specifically, the pointing relationship on the trace path is based on the call relationship, and the parent span points to the child span, but the impact of the delay is the opposite, that is, the delay of the child span will affect the delay of the parent span. Therefore, the embodiment of the present invention reverses the pointing relationship on the trace to form a causal graph based on the impact of delay, in which there is only one child node, namely the front-end node.

[0109] The embodiment of the present invention needs to use causal relationships to find the nodes that contribute most to the final delay increase, so it is necessary to set up causal mechanisms between the nodes of the causal graph. The embodiment of the present invention assigns an appropriate causal mechanism to each node of the causal graph based on the Functional Causal Model. j Represented as the variable of node j, f j represents the causal mechanism on node j, PA j Variable representing the parent node, N j represents the noise, which explains how the delay of the child node is generated from the parent node.

[0110] X j :=f j (PA j ,N1)

[0111] Using the FCM causal model, embodiments of the present invention abstract a node's latency as the latency inherited from its parent node, plus the noise generated by that node. This noise is considered the node's local latency. The relationships between nodes indicate that when a node experiences performance issues, it's likely due to a problem with its noise data. Because noise can't be inherited from a parent node but can be passed down from child nodes, the root cause can be found layer by layer, where a node is truncated.

[0112] This embodiment of the present invention performs causal learning based on the FCM causal model to obtain the normal causal relationships between the nodes in the first causal graph, that is, the normal causal relationships between the microservice call information. It should be noted that the specific process of causal learning is conventional in the art and will not be described in detail in this embodiment of the present invention.

[0113] S103: Generate a second causal graph based on the abnormal call chain data, and determine the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and then determine the key microservices based on the delay contribution.

[0114] As a further optional implementation, a second causal graph is generated based on the abnormal call chain data, and the delay contribution of each node in the second causal graph is determined based on the second causal graph and the normal causal relationship, and then the key microservice is determined based on the delay contribution. This step specifically includes:

[0115] S1031. Reverse the directional relationship of each node in the abnormal call chain data to obtain a second causal graph;

[0116] S1032. Apply the normal causal relationship layer by layer to each node of the second causal graph, and determine the delay contribution of each node based on the degree of influence of each node on the end-to-end delay after applying the normal causal relationship;

[0117] S1033. Determine the microservice corresponding to the node with the largest delay contribution as the key microservice.

[0118] Specifically, for abnormal call chain data, a corresponding second causal graph is also generated. The learned normal causal relationships are then applied layer by layer to each node in the second causal graph. The delay contribution of each node is determined based on the degree of impact of each node on end-to-end delay after the normal causal relationship is applied. This allows an assessment of whether each node has a significant impact on end-to-end delay after the causal relationship is changed. In this way, embodiments of the present invention can determine that the microservice corresponding to the node with the largest delay contribution is a critical microservice.

[0119] S104: Determine configuration parameters, intermediate performance indicators, and target delay performance of key microservices, and build a causal performance model based on the configuration parameters, intermediate performance indicators, and target delay performance.

[0120] Specifically, because microservices involve many data storage software (such as MongoDB and MySQL) and front-end proxy software (such as Nginx), these software have many configurable parameters. However, not all parameters have an impact on performance. Therefore, embodiments of the present invention construct a causal relationship diagram (i.e., a causal performance model) between performance goals, intermediate performance indicators, and configuration parameters to discover a small number of parameters that have the greatest impact on performance.

[0121] As an optional implementation, the step of determining configuration parameters, intermediate performance indicators, and target latency performance of key microservices and constructing a causal performance model based on the configuration parameters, intermediate performance indicators, and target latency performance specifically includes:

[0122] S1041. Determine the configuration file of the key microservice, set read and write latency as the optimization target, and optimize the configuration file using the Bayesian optimization algorithm to obtain the target latency performance, intermediate performance indicators, and corresponding configuration parameters, which include software parameters and resource parameters.

[0123] S1042. Determine a first connection relationship between the configuration parameter and the intermediate performance indicator, and determine a second connection relationship between the intermediate performance indicator and the target delay performance, and determine causal pointing information of the first connection relationship and the second connection relationship through information entropy to obtain a causal performance model.

[0124] Specifically, the embodiment of the present application deploys each software with multiple configurable parameters, such as MongoDB, runs it on a server using a Docker container, and uses YCSB to perform load stress testing, and collects its performance delay as the final goal. In addition, the embodiment of the present application uses the Prometheus monitoring platform to record the intermediate performance indicators of MongoDB during each run, such as mongodb_ss_mem_virtual, which represent the running state of MongoDB.

[0125] The embodiment of the present application uses the Bayesian optimization algorithm (BO) to take read-write delay as the optimization goal, and generates 1000 configuration files for these software. During the running process, the embodiment of the present application records the delay results and intermediate performance indicators. These data are integrated into a dataset containing configuration parameters, intermediate performance indicators and target performance, which is used to build a causal relationship diagram.

[0126] In order to learn the causal performance model for different software, the embodiment of the present application uses the Fast Causal Inference algorithm to model the relationship between configuration parameters, intermediate performance indicators and target performance. The Fast Causal Inference algorithm can still perform well in the presence of some ambiguous confounding factors, which is very helpful for this scenario. The configuration parameters of the software are the parameters that can be changed by the configuration file or command line during the deployment of the software itself, such as the cacheSizeGB, journalCompressor and other parameters of MongoDB. The intermediate performance indicators are some indicators that record the running state of the software during the running process of the software, such as cpu utilization, memory usage, file open number, etc. The target performance is the running performance that we need to adjust finally, such as the delay of the request. The embodiment of the present application uses causal relationship to find the pointing relationship between the three, and can find the key parameters that affect the final target performance.

[0127] In building the causal performance model, first, according to the pre-set constraints, create the skeleton of the causal diagram, for example, the embodiment of the present application stipulates that there is no direct pointing relationship between the software parameters, so in the initial construction process, the connection relationship only exists between the software parameters and the intermediate performance indicators, and between the intermediate performance indicators and the target performance; Then, the embodiment of the present application prunes the connection relationship in the graph, and removes the edges between nodes that have no influence relationship. Here, Fisher z-test is used for continuous variables and mutual information is used for discrete variables to determine whether there is a correlation between two nodes, and if not, the established edge is removed; Finally, although the graph that two nodes have a relationship is obtained, the pointing relationship of the edge is not clear, so the embodiment of the present application uses information entropy based on information theory to determine the specific causal pointing relationship between two nodes.

[0128] S105, extract a plurality of causal paths according to the causal performance model, and determine the average causal effect of each configuration parameter on the target delay performance according to the causal paths, and further determine a plurality of configuration parameters as the key performance parameters of the microservice according to the average causal effect.

[0129] Further, as an optional implementation, the step of extracting a plurality of causal paths according to the causal performance model and determining the average causal effect of each configuration parameter on the target delay performance according to the causal paths specifically includes:

[0130] S1051, extract a plurality of causal paths according to the causal performance model, wherein each intermediate node in the causal path is a configuration parameter or an intermediate performance indicator, and the end node of the causal path is the target delay performance;

[0131] S1052, determine the average influence degree of each causal path on the delay, and select a plurality of key causal paths according to the average influence degree;

[0132] S1053, determine the causal effect of each intermediate node of the key causal path on the end node, and determine the average causal effect of each configuration parameter on the target delay performance according to the causal effect.

[0133] Specifically, after obtaining the causal performance model, the embodiment of the present application extracts paths (i.e. causal paths) from the causal graph, and sorts them from high to low according to their average causal influence on the delay, wherein each intermediate node in the causal path is a configuration parameter or an intermediate performance indicator, and the last node of the path is the target delay performance. Through path extraction and sorting, the embodiment of the present application simplifies the complex causal graph into a few useful causal paths, and further calculates the average causal effect of each node on the final performance indicator, so as to determine the most critical few parameters as the parameters that need to be optimized and adjusted.

[0134] Further, as an optional implementation, the step of determining a plurality of configuration parameters as the key performance parameters of the microservice according to the average causal effect specifically includes:

[0135] S1054, determine a plurality of configuration parameters with an average causal effect greater than a preset second threshold as the key performance parameters of the microservice;

[0136] Or,

[0137] S1055, sort each configuration parameter according to the average causal effect from large to small, and select a plurality of configuration parameters with a sorting value less than or equal to a preset third threshold as the key performance parameters of the microservice.

[0138] Specifically, an embodiment of the present invention selects a configuration parameter whose average causal effect is greater than a second threshold as a key performance parameter, or selects several configuration parameters with a high average causal effect value as the key performance parameter. Subsequent optimization and adjustment of the key performance parameter can improve the performance of the target microservice system.

[0139] The above describes in detail the steps of the embodiment of the present invention. It can be understood that the embodiment of the present invention can mine the causal relationship hidden behind the data through causal learning and causal performance models, thereby more accurately locating the key microservices and key performance parameters that cause performance anomalies in the microservice system, thereby improving the accuracy of locating microservice performance anomalies; in addition, through the coordinated tuning of software parameters and resource parameters, the performance of the microservice system can be guaranteed to be normal without increasing resource parameters, thereby more efficiently managing system resources.

[0140] Compared with the prior art, the embodiments of the present invention also have the following advantages:

[0141] 1) The location of key microservices does not require the prior training of a machine learning model. Given the dynamically changeable call relationships among microservices, the embodiments of the present invention only need to collect the latest trace data within the near term of detecting a performance failure to discover the root cause service through causal relationships.

[0142] 2) The embodiment of the present invention screens key parameters for each software used in the microservice, not only for the parameters of the software itself, but also in combination with resource parameters. By obtaining intermediate performance indicators as intermediate variables of causal relationships, more accurate key parameters can be obtained.

[0143] 3) Wide range of applications. The embodiments of the present invention are applicable to the key bottleneck location of any microservice architecture, can minimize the huge configuration space, and improve the exploration efficiency of the algorithm.

[0144] Reference Figure 2 , an embodiment of the present invention provides a microservice key performance parameter identification system, including:

[0145] The microservice system performance judgment module is used to obtain the microservice call chain data of the target microservice system and judge whether the target microservice system has performance abnormalities based on the microservice call chain data. When the target microservice system has performance abnormalities, the normal call chain data and abnormal call chain data are determined based on the microservice call chain data.

[0146] A normal causal relationship learning module is used to generate a first causal graph based on normal call chain data, and obtain the normal causal relationship of each node in the first causal graph through causal learning;

[0147] The key microservice determination module is configured to generate a second causal graph according to the abnormal call chain data, determine a delay contribution degree of each node of the second causal graph according to the second causal graph and the normal causal relationship, and further determine a key microservice according to the delay contribution degree.

[0148] The causal performance model construction module is configured to determine a configuration parameter of the key microservice, an intermediate performance indicator, and a target delay performance, and construct a causal performance model according to the configuration parameter, the intermediate performance indicator, and the target delay performance.

[0149] The key performance parameter determination module is configured to extract a plurality of causal paths according to the causal performance model, determine an average causal effect of each configuration parameter on the target delay performance according to the causal paths, and further determine a plurality of configuration parameters as microservice key performance parameters according to the average causal effect.

[0150] The content in the method embodiments is applicable to the system embodiments, the system embodiments specifically implement the same functions as the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0151] With reference to Figure 3 The embodiment of the present application provides a microservice key performance parameter identification device, which comprises:

[0152] At least one processor;

[0153] At least one memory for storing at least one program;

[0154] When the at least one program is executed by the at least one processor, the at least one processor implements the microservice key performance parameter identification method.

[0155] The content in the method embodiments is applicable to the device embodiments, the device embodiments specifically implement the same functions as the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0156] The embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a program executable by a processor, and the program executable by the processor is used to execute the microservice key performance parameter identification method.

[0157] The computer readable storage medium of the embodiment of the present application can execute the microservice key performance parameter identification method provided by the method embodiment of the present application, execute the steps of any combination of the method embodiment, has the corresponding functions and beneficial effects of the method.

[0158] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0159] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0160] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0161] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0162] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0163] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0164] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0165] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0166] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0167] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for identifying key performance parameters of microservices, characterized in that: The following steps are involved: Obtain microservice call chain data of the target microservice system, determine whether the target microservice system has performance abnormalities based on the microservice call chain data, and if the target microservice system has performance abnormalities, determine normal call chain data and abnormal call chain data based on the microservice call chain data; Generate a first causal graph based on the normal call chain data, and obtain a normal causal relationship between nodes in the first causal graph through causal learning; Generate a second causal graph based on the abnormal call chain data, and determine the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and then determine the key microservice based on the delay contribution; Determining configuration parameters, intermediate performance indicators, and target delay performance of the key microservice, and constructing a causal performance model based on the configuration parameters, the intermediate performance indicators, and the target delay performance; Extracting multiple causal paths based on the causal performance model, determining an average causal effect of each of the configuration parameters on the target delay performance based on the causal paths, and then determining several of the configuration parameters as microservice key performance parameters based on the average causal effect.

2. A microservice key performance parameter identification method according to claim 1, characterized in that: The step of obtaining microservice call chain data of the target microservice system, determining whether the target microservice system has performance abnormalities based on the microservice call chain data, and determining normal call chain data and abnormal call chain data based on the microservice call chain data when the target microservice system has performance abnormalities specifically includes: Obtain the call information of each request on each microservice of the target microservice system through the Docker-based Jaeger tracing framework, and use the call information as a node to generate the microservice call chain data, wherein the call information includes the execution interface and the residence time; Determine the P95 delay value of the target microservice system within a target period according to the residence time of each node; When the P95 delay value is less than or equal to a preset first threshold, it is determined that the performance of the target microservice system is normal; When the P95 delay value is greater than the first threshold, the target microservice system performance is determined to be abnormal, and the normal nodes in the microservice call chain data whose residence time is less than or equal to the P95 delay value and the abnormal nodes whose residence time is greater than the P95 delay value are determined, and then normal call chain data is generated according to the normal nodes, and abnormal call chain data is generated according to the abnormal nodes.

3. A microservice key performance parameter identification method according to claim 1, characterized in that: The step of generating a first causal graph based on the normal call chain data and obtaining the normal causal relationship of each node in the first causal graph through causal learning specifically includes: Reversing the directional relationship of each node of the normal call chain data to obtain the first causal graph; Determine the causal mechanism of each node of the first causal graph to obtain an FCM causal model; Causal learning is performed according to the FCM causal model to obtain the normal causal relationship of each node in the first causal graph.

4. A microservice key performance parameter identification method according to claim 1, characterized in that: The step of generating a second causal graph based on the abnormal call chain data, determining the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and then determining the key microservice based on the delay contribution specifically includes: Reversing the directional relationship of each node of the abnormal call chain data to obtain the second causal graph; Applying the normal causal relationship layer by layer to each node of the second causal graph, and determining the delay contribution of each node according to the degree of influence of each node on the end-to-end delay after applying the normal causal relationship; The microservice corresponding to the node with the largest delay contribution is determined as the key microservice.

5. A microservice key performance parameter identification method according to claim 1, characterized in that: The step of determining the configuration parameters, intermediate performance indicators, and target delay performance of the key microservice, and constructing a causal performance model based on the configuration parameters, the intermediate performance indicators, and the target delay performance specifically includes: Determine a configuration file of the key microservice, use read and write latency as an optimization target, and optimize the configuration file using a Bayesian optimization algorithm to obtain the target latency performance, the intermediate performance indicator, and the corresponding configuration parameters, where the configuration parameters include software parameters and resource parameters; Determine a first connection relationship between the configuration parameter and the intermediate performance indicator, and determine a second connection relationship between the intermediate performance indicator and the target delay performance, and determine causal pointing information of the first connection relationship and the second connection relationship through information entropy to obtain the causal performance model.

6. A microservice key performance parameter identification method according to claim 1, characterized in that: The step of extracting multiple causal paths according to the causal performance model and determining the average causal effect of each of the configuration parameters on the target delay performance according to the causal paths specifically includes: Extracting multiple causal paths according to the causal performance model, wherein the intermediate nodes of the causal paths are the configuration parameters or the intermediate performance indicators, and the end nodes of the causal paths are the target delay performance; Determining the average impact of each of the causal paths on the delay, and selecting a number of key causal paths based on the average impact; The causal effect of each intermediate node of the critical causal path on the end node is determined, and the average causal effect of each configuration parameter on the target delay performance is determined based on the causal effect.

7. A microservice key performance parameter identification method according to any one of claims 1 to 6, characterized in that: The step of determining the plurality of configuration parameters as key performance parameters of microservices based on the average causal effect is specifically as follows: Determining a plurality of the configuration parameters whose average causal effect is greater than a preset second threshold as the key performance parameters of the microservice; or, The configuration parameters are sorted from large to small according to the average causal effect, and several configuration parameters whose sorting values ​​are less than or equal to a preset third threshold are selected as the microservice key performance parameters.

8. A microservice key performance parameter identification system, characterized in that: include: A microservice system performance judgment module is used to obtain microservice call chain data of a target microservice system, and judge whether the target microservice system has performance abnormalities based on the microservice call chain data. If the target microservice system has performance abnormalities, normal call chain data and abnormal call chain data are determined based on the microservice call chain data. a normal causal relationship learning module, configured to generate a first causal graph based on the normal call chain data, and obtain the normal causal relationship of each node in the first causal graph through causal learning; a key microservice determination module, configured to generate a second causal graph based on the abnormal call chain data, determine the delay contribution of each node in the second causal graph based on the second causal graph and the normal causal relationship, and further determine the key microservice based on the delay contribution; A causal performance model construction module is used to determine the configuration parameters, intermediate performance indicators and target delay performance of the key microservice, and to construct a causal performance model based on the configuration parameters, the intermediate performance indicators and the target delay performance; A key performance parameter determination module is used to extract multiple causal paths based on the causal performance model, and determine the average causal effect of each of the configuration parameters on the target delay performance based on the causal paths, and then determine several of the configuration parameters as microservice key performance parameters based on the average causal effect.

9. A device for identifying key performance parameters of microservices, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a microservice key performance parameter identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to execute a microservice key performance parameter identification method according to any one of claims 1 to 7 when executed by the processor.

Images