A microservice fault self-adaptive positioning method in a cloud-native environment

Abstract

The application discloses a microservice fault self-adaptive positioning method in a cloud native environment and belongs to the technical field of cloud native fault diagnosis. The method comprises the following steps: acquiring real-time call chain data of each microservice instance in a cloud native environment, and analyzing response time sequences and state code sequences; constructing a basic health feature vector of each microservice instance according to the sequences; taking the vector as node input and taking the call relationship between microservices as edge weight to construct a full-service call graph; in each round of fault positioning iteration, extracting a current abnormal propagation path from the full-service call graph, and dynamically adjusting the node search direction of the next round of fault positioning iteration according to the health feature difference of adjacent nodes in the current abnormal propagation path. The method uses multi-dimensional health features and a call relationship graph to adaptively approach a fault root cause node.

Description

Technical Field

[0001] This invention relates to the field of cloud-native fault diagnosis technology, specifically to an adaptive fault localization method for microservices in a cloud-native environment. Background Technology

[0002] In cloud-native environments, microservice architectures are large-scale and have complex call relationships. A single request often spans dozens of service instances. When one instance fails, the anomaly propagates downstream along the call chain, causing numerous cascading alarms. Existing fault localization solutions typically rely on call chain data collected by a distributed tracing system, using fixed thresholds to determine if response time or error rate is abnormal, and then tracing back upstream along the call topology to find the first node with the anomaly, which is then used as the root cause. This approach depends on a single health metric, focusing only on average response time or success rate, without utilizing multi-dimensional information such as response time fluctuations and status code distribution to characterize the true health status of service instances. When the fault manifests as intermittent slow response or abnormal status codes, a single metric is prone to missing or lagging, causing upstream healthy nodes to be mistakenly identified as the root cause.

[0003] Furthermore, existing call chain-based localization methods, after acquiring an anomaly node, often search in a fixed direction (such as only traversing upstream in reverse or only traversing downstream in forward), failing to fully utilize the structured difference information obtained in each iteration. In real-world fault scenarios, the health status changes between adjacent nodes along the anomaly propagation path are not linearly decreasing; typically, significant jumps in health indicators occur at the actual fault location. Fixed search strategies cannot identify these jump characteristics, nor can they dynamically adjust the search starting point for the next round, causing the localization process to repeatedly traverse long call chains or terminate prematurely, making it difficult to achieve rapid convergence while maintaining accuracy.

[0004] To address the aforementioned issues, a method is needed that can extract multi-dimensional health features from call chain data and adaptively adjust the search direction on the full service call graph to achieve efficient and accurate localization of the root causes of microservice failures. Summary of the Invention

[0005] This invention provides an adaptive fault location method for microservices in a cloud-native environment, aiming to solve the problems of low location accuracy and slow convergence speed caused by the single health feature and fixed search strategy in existing microservice fault location technologies.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a microservice fault adaptive localization method in a cloud-native environment. By constructing the health characteristics of microservice instances and the service call relationship graph, the fault search direction is dynamically adjusted during the iteration process to achieve accurate localization of the root cause of the fault.

[0007] In this invention, call chain records for each microservice instance within the most recent time window are collected from the link tracing component in the cloud-native environment, and response time sequences and status code sequences are parsed from them. Preferably, the length of the time window is dynamically adjusted according to the average request processing latency of the microservice instance; the greater the latency, the shorter the time window, thus ensuring that a sufficient number of sample points can be collected under different loads, guaranteeing the stability of feature extraction.

[0008] Based on the parsed response time and status code sequences, the average response time, response time fluctuation, and percentage of abnormal status codes for each microservice instance are calculated and concatenated in a fixed order to form a three-dimensional basic health feature vector. This feature vector comprehensively reflects the real-time health status of the microservice instance from three dimensions: response latency, latency stability, and request success rate, providing a quantitative basis for subsequent anomaly detection.

[0009] Each microservice instance in the cloud-native environment is mapped to a node in a graph, and the node stores its basic health feature vector. The number of direct calls between any two microservice instances within a preset historical time period is counted, and after normalization, it is used as the weight of the directed edge connecting the corresponding node, thus constructing a full service call graph. As a technical solution of the present invention, the normalization process adopts a maximum-minimum value normalization method, mapping the number of direct calls to the [0,1] interval, so that the edge weight can intuitively represent the tightness of the call relationship.

[0010] During fault localization, multiple rounds of iterative search are performed. In the first iteration, nodes whose basic health feature vector magnitude exceeds a preset threshold in the full service call graph are marked as candidate nodes for the root cause of the anomaly. Starting from each candidate node, the search proceeds backward along the incoming edge direction to nodes without incoming edges, obtaining the initial anomaly propagation path. The preset threshold is dynamically calculated based on the sum of the mean and standard deviation of the vector magnitudes of all nodes in the full service call graph, allowing the anomaly detection threshold to adapt to fluctuations in the overall service health status. In subsequent iterations, each node in the suspected fault node set output from the previous iteration is used as a starting point, and the search proceeds forward along the outgoing edge direction to nodes without outgoing edges, forming the anomaly propagation path for the current round. This path extraction method, from reverse tracing to forward tracking, can first capture potential fault sources and then trace the scope of influence downstream along the call chain, allowing the search scope to gradually approach the actual fault point with each iteration.

[0011] For the abnormal propagation path extracted in the current round, the Manhattan distance between the health feature vectors of each pair of adjacent upstream and downstream nodes in the path is calculated to obtain the health feature difference value. The difference values ​​on the same path are arranged in upstream and downstream order to form a difference sequence, and the position where the difference value shows a step drop is identified. The downstream node at that position is marked as a suspected fault node. Preferably, the step drop is identified by calculating the ratio of two adjacent difference values ​​in the difference sequence. When the ratio of the latter difference value to the former difference value is greater than a preset step threshold, it is determined that a step drop has occurred at the position of the adjacent node pair, and the downstream node is recorded as a suspected fault node of the path. The suspected fault nodes of all abnormal propagation paths are collected and deduplicated to form the starting point set of the next round of iteration, thereby dynamically adjusting the node search direction. Through the above processing, the fault localization process can automatically focus on the specific service node causing the abnormal propagation based on the drastic changes in health features between the upstream and downstream of the call chain, reducing the interference caused by abnormal propagation and improving the root cause localization accuracy.

[0012] In a preferred embodiment of the present invention, after each iteration, it is determined whether the current set of suspected faulty nodes is completely identical to the set of suspected faulty nodes from the previous iteration. If they are completely identical, the iteration terminates, and the current set is output as the final set of root cause nodes. Otherwise, the next iteration continues until a preset maximum number of iterations is reached, at which point the set of suspected faulty nodes obtained in the last iteration is output. This convergence determination mechanism avoids unnecessary computational overhead and ends the search promptly when the location result is stable, ensuring the real-time performance and efficiency of fault location.

[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows: The response time sequence and status code sequence are parsed from the microservice call chain data. Then, the average response time, response time fluctuation, and percentage of abnormal status codes for each microservice instance are calculated, and these three are concatenated in a fixed order to form a three-dimensional basic health feature vector. This vector simultaneously reflects the central tendency, dispersion, and business success status of service responses, providing a more complete characterization of the health level of microservice instances in terms of latency and availability compared to a single indicator. When constructing the full service call graph, this three-dimensional basic health feature vector is used as node input, and the normalized value of the number of direct calls between any two microservice instances is used as the directed edge weight, resulting in a weighted directed graph that includes both the health status of each instance and the intensity of dependent calls. When a service instance experiences performance fluctuations or intermittently returns error status codes, its response time fluctuation and percentage of abnormal status codes increase, causing a significant change in the magnitude of the basic health feature vector. This multi-dimensional feature can reflect potential anomalies before the average response time deteriorates, thus providing earlier and more comprehensive triggering conditions for extracting anomaly propagation paths. Meanwhile, edge weights are generated by normalization based on the actual number of calls, reflecting the real traffic dependencies between microservices. This ensures that the abnormal propagation paths extracted along directed edges match the actual call intensity of the business, avoiding the introduction of erroneous paths due to occasional weak call relationships and improving the extraction quality of abnormal propagation paths.

[0014] During the fault localization iteration, for each extracted abnormal propagation path, the health feature differences between adjacent nodes are calculated pairwise. The Manhattan distance is used to quantify the deviation between the basic health feature vectors of downstream and upstream nodes, resulting in a difference sequence. From this difference sequence, positions where the difference value shows a step-down are detected. Downstream nodes at these positions are marked as suspected faulty nodes, and these serve as the starting point set for the next round of iteration. Through this difference-driven approach, the search direction is no longer fixed to a single upstream backtracking or a single downstream expansion. Instead, each round can adaptively adjust based on the jump positions of the health feature differences between adjacent nodes, and the search starting point spontaneously aggregates towards the vicinity of the truly faulty node. When there are multiple intermediate services between the faulty node and the apparent node, the step-down position can directly shift the focus to the node whose health suddenly deteriorates, skipping a series of passively propagating nodes that are also affected. This significantly reduces invalid traversal and diagnostic steps, enabling rapid convergence of the search range and accurate identification of the root cause node in complex call topologies. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 This is a flowchart of a microservice fault adaptive localization method in a cloud-native environment; Figure 2 It is a flowchart for constructing the full service call graph; Figure 3 This is a flowchart of the anomaly propagation path extraction process in the fault location iteration; Figure 4 This is a flowchart for determining the termination condition of fault location iteration and outputting the final root cause node of the fault. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 This invention provides a microservice fault adaptive localization method in a cloud-native environment, comprising the following steps: acquiring real-time call chain data for each microservice instance in the cloud-native environment; parsing the response time sequence and status code sequence of each microservice instance from the real-time call chain data; constructing a basic health feature vector for each microservice instance based on the response time sequence and status code sequence; constructing a full-service call graph using the basic health feature vector of each microservice instance as node input and the call relationship between microservices as edge weights; and in each fault localization iteration, extracting the current anomaly propagation path from the full-service call graph and dynamically adjusting the node search direction for the next fault localization iteration based on the health feature differences between adjacent nodes in the current anomaly propagation path.

[0019] When acquiring real-time call chain data for each microservice instance in a cloud-native environment and parsing the response time sequence and status code sequence of each microservice instance from the real-time call chain data, the specific implementation process is as follows: Using a tracing component deployed in the cloud-native environment, call chain records corresponding to all external requests received by each microservice instance within the most recent time window are collected. The length of the time window is dynamically adjusted based on the average request processing latency of the microservice instance; the higher the average request processing latency of the microservice instance, the shorter the time window length. The method for determining the time window length is as follows:

[0020] in, Indicates the length of the time window, in milliseconds; This is a preset window adjustment constant, in milliseconds (ms). This represents the average request processing latency for a microservice instance, in milliseconds. The value is set to 1000 milliseconds, based on the following: In a typical microservice scenario, when... When it is 10 milliseconds, A 100-millisecond window is sufficient to collect a adequate number of request samples, ensuring the validity of subsequent statistics. Average request processing latency for a microservice instance. This is achieved by calculating the average processing time of the microservice instance across all call chain records over a previous historical period. For each collected call chain record, the actual processing time consumed by the microservice instance in processing the request is extracted, and this extracted processing time is used as the microservice instance's response time for that request. Simultaneously, the HTTP or gRPC status code returned by the microservice instance is extracted from the call chain record, and this extracted status code is used as the microservice instance's status code for that request. For each microservice instance, the response times corresponding to all requests received within the time window are arranged sequentially according to the time of the requests, forming the microservice instance's response time sequence. The status codes corresponding to all requests received within the time window are arranged sequentially according to the same time sequence as the response time sequence, forming the microservice instance's status code sequence.

[0021] The specific implementation process for constructing the basic health feature vector for each microservice instance based on its response time sequence and status code sequence is as follows: For each microservice instance, the arithmetic mean of all response times in the microservice instance's response time sequence is calculated, and the result is taken as the average response time of the microservice instance. The standard deviation of the microservice instance's response time sequence is calculated, and the result is taken as the response time fluctuation value of the microservice instance. The number of occurrences of non-successful status codes in the status code sequence of the microservice instance is counted, and the ratio of the number of occurrences of non-successful status codes to the total length of the status code sequence is calculated; the ratio is taken as the proportion of abnormal status codes for the microservice instance. Non-successful status codes include HTTP status codes other than those in the 2xx range, and non-OK status codes in gRPC status codes.

[0022] After obtaining the three metrics—average response time, response time fluctuation, and percentage of abnormal status codes—for a microservice instance, these three metrics are concatenated in a fixed order to form a three-dimensional basic health feature vector for the microservice instance. The concatenation order is set as follows: average response time is located in the first dimension of the three-dimensional vector, response time fluctuation is located in the second dimension, and the percentage of abnormal status codes is located in the third dimension. For each microservice instance, the average response time is entered into the first dimension of the three-dimensional vector, the response time fluctuation is entered into the second dimension, and the percentage of abnormal status codes is entered into the third dimension. The resulting three-dimensional real-valued vector is the basic health feature vector of the microservice instance.

[0023] When constructing the full service call graph using the basic health feature vector of each microservice instance as node input and the call relationship between microservices as edge weights, the specific implementation process is as follows: See Figure 2 This approach maps each microservice instance running in the cloud-native environment to a separate node in the full service call graph, establishing a one-to-one correspondence between nodes and microservice instances. Within each node, a basic health feature vector of the microservice instance corresponding to that node is stored. This basic health feature vector adopts the three-dimensional real-number vector representation described in the previous embodiment.

[0024] Collect all call chain records generated within a preset historical time period in the cloud-native environment. The length of the preset historical time period is set to a fixed value based on the system throughput characteristics, for example, 3600 seconds. Scan all collected call chain records one by one, and for any two microservice instances, count the total number of direct call relationships between the two microservice instances. A direct call relationship means that in the call chain record, one microservice instance initiates a request to another microservice instance as the caller, and there are no other intermediate microservice instances between the two microservice instances.

[0025] The number of direct calls between any two microservice instances is normalized, and the normalized value is used as the weight of the directed edge connecting the corresponding nodes of the two microservice instances in the full service call graph. When there is no direct call relationship between two microservice instances, no directed edge is established between the two corresponding nodes in the full service call graph.

[0026] The normalization process uses a maximum-minimum normalization method, mapping the number of direct calls to the [0,1] interval. The normalization process is performed according to the following formula:

[0027] in, This represents the weight of the directed edge in the full service call graph, pointing from the node corresponding to the i-th microservice instance to the node corresponding to the j-th microservice instance. The value range is [0,1]; This represents the number of direct calls made by the i-th microservice instance to the j-th microservice instance. This represents the maximum number of direct calls between all pairs of microservice instances. It is determined by traversing all pairs of microservice instances with direct call relationships in the full service call graph and comparing the number of direct calls between them. This represents the minimum number of direct calls between all pairs of microservice instances. It is determined by traversing all pairs of microservice instances with direct call relationships in the full service call graph and comparing the number of direct calls between them.

[0028] In practice, and All values ​​are determined once during the full service call graph construction phase and are not dynamically updated as the system runs. The directed edge pointing from the node corresponding to the i-th microservice instance to the node corresponding to the j-th microservice instance has the same direction as the direction in which the i-th microservice instance calls the j-th microservice instance in the call chain record. Through the above normalization process, the weight values ​​of all directed edges in the full service call graph fall within the range of [0,1]. The directed edge corresponding to the microservice instance pair with the highest number of direct calls has a weight value of 1, and the directed edge corresponding to the microservice instance pair with the lowest number of direct calls has a weight value of 0.

[0029] See Figure 3 In each iteration of fault location, the specific implementation process of extracting the current anomaly propagation path from the full service call graph distinguishes between the first and subsequent fault location iterations based on the iteration number, employing different path extraction methods for each. When calculating the preset threshold, the sum of the mean and standard deviation of the vector magnitudes of the basic health feature vectors of all nodes in the full service call graph is used as the preset threshold, specifically determined according to the following formula:

[0030] in, Indicates a preset threshold; This represents the arithmetic mean of the vector magnitudes of the basic health feature vectors of all nodes in the full service call graph; This represents the standard deviation of the vector magnitude of the basic health feature vectors of all nodes in the full service call graph. (Calculation) The method is as follows: traverse each node in the full service call graph, extract the basic health feature vector stored in the node, and the basic health feature vector is a three-dimensional real number vector. Let the three components of the basic health feature vector of node X be... , , The vector magnitude of the basic health feature vector of node X is calculated as follows: For each node in the full service call graph, calculate the vector magnitude of the corresponding basic health feature vector. Then, perform an arithmetic mean calculation on all vector magnitudes, and use the result as the basis for the calculation. .calculate The method is as follows: Utilize the numerical sequence of the vector magnitudes of the basic health feature vectors of all nodes in the full service call graph, calculate the standard deviation of the vector magnitude sequence, and use the result as... .

[0031] In the first iteration of fault localization, the process of extracting the anomaly propagation path is as follows: traverse each node in the full service call graph, calculate the vector magnitude of the basic health feature vector corresponding to the node, and compare the vector magnitude with a preset threshold. Compare the vectors whose magnitude exceeds a preset threshold. The nodes are marked as candidate nodes for the root cause of the exception. For each candidate node, a reverse traversal is performed along the incoming edges of the entire service call graph, starting from the candidate node. The specific method of reverse traversal is as follows: obtain all directed edges in the entire service call graph that end at the candidate node, i.e., all incoming edges of the candidate node; start from the candidate node and move backward along the incoming edges to the source node of each incoming edge; for each source node, continue to obtain all incoming edges in the entire service call graph that end at that source node, and move backward along the incoming edges to the source nodes of these incoming edges; repeat the reverse movement operation until the node reached has no incoming edges in the entire service call graph. Arrange all the nodes traversed during the reverse traversal in reverse order from back to front, i.e., from the starting node to the candidate node in the entire service call graph, forming a node sequence. Each different backtracking path corresponds to a node sequence, and each node sequence is recorded as an initial exception propagation path. The candidate node for the root cause of the anomaly may have multiple incoming edges, thus generating multiple different reverse traversal paths. Each reverse traversal path forms an initial anomaly propagation path.

[0032] In the subsequent iterations of fault location, the process of extracting the anomaly propagation path is as follows: Obtain the set of suspected fault nodes output by the previous iteration, and use each node in the set as the starting point for the current anomaly propagation path extraction. For each node serving as the starting point, perform a forward traversal along the outgoing edges of the entire service call graph. The specific method of the forward traversal is as follows: Obtain all directed edges in the entire service call graph that originate from the node serving as the starting point (i.e., all outgoing edges of the node serving as the starting point), and move forward along the outgoing edges from the starting point to the endpoint node of each outgoing edge; for each endpoint node, continue to obtain all outgoing edges in the entire service call graph that originate from this endpoint node, and move forward along the outgoing edges to the endpoint node of the outgoing edge; repeat the forward movement operation until the node reached has no outgoing edges in the entire service call graph. All nodes traversed sequentially from the starting point during the forward traversal are arranged in the order in which they are visited to form a node sequence, and each node sequence is recorded as a current anomaly propagation path. A starting node may have multiple outgoing edges, resulting in multiple different forward traversal paths. Each forward traversal path forms a current anomaly propagation path.

[0033] The specific implementation process for dynamically adjusting the node search direction for the next round of fault location iteration based on the differences in health characteristics of adjacent nodes in the current anomaly propagation path is as follows: For the current anomaly propagation path extracted in each round of fault location iteration, traverse each pair of adjacent nodes in the current anomaly propagation path. A pair of adjacent nodes consists of an upstream node and a downstream node in the path, where the upstream node is the party that initiates the call in the call chain in the current anomaly propagation path, and the downstream node is the party that receives the call in the call chain in the current anomaly propagation path.

[0034] For each pair of adjacent nodes, the Manhattan distance between the basic health feature vector of the downstream node and the basic health feature vector of the upstream node is calculated. The calculated Manhattan distance value is used as the health feature difference value of this pair of adjacent nodes. The Manhattan distance is calculated according to the following formula:

[0035] in, This represents the difference in health characteristics between the p-th node and the q-th node, which are adjacent nodes in the current abnormal propagation path. This refers to the number of the upstream node in the adjacent node pair. The numbering of the downstream nodes for adjacent nodes; This represents the first component value of the basic health feature vector of the upstream node. This represents the second component value of the basic health feature vector of the upstream node. This represents the third-dimensional component value of the basic health feature vector of the upstream node; This represents the first component value of the basic health feature vector of the downstream node. This represents the second component value of the basic health feature vector of the downstream node. This represents the third-dimensional component value of the basic health feature vector of the downstream node.

[0036] In practice, for each current anomaly propagation path, the health characteristic difference values ​​of all adjacent nodes on that path are arranged sequentially from upstream to downstream to form a difference sequence. The first element in the difference sequence is the health characteristic difference value between the first and second adjacent nodes in the current anomaly propagation path, the second element is the health characteristic difference value between the second and third adjacent nodes in the current anomaly propagation path, and so on.

[0037] The specific implementation method for finding the position where the difference value shows a step decrease in the difference sequence is as follows: Calculate the ratio of two adjacent difference values ​​in the difference sequence. Obtain the k-th difference value and the (k+1)-th difference value in the difference sequence, calculate the quotient obtained by dividing the (k+1)-th difference value by the k-th difference value, and use the quotient as the ratio of two adjacent difference values. Compare the ratio of two adjacent difference values ​​with a preset step threshold. When the ratio of two adjacent difference values ​​is greater than the preset step threshold, it is determined that a step decrease has occurred at the position of the adjacent node corresponding to the (k+1)-th difference value. Here, k starts from 1 and traverses to the length of the difference sequence minus 1.

[0038] The preset step threshold is set to a fixed value of 2. The basis for this setting is that when the difference in health characteristics between adjacent node pairs decreases by more than 50%, the ratio is greater than 2, indicating that the health characteristics of the downstream node have significantly improved compared with the health characteristics of the upstream node. At this position, the downstream node is more likely to be the direct bearer of the fault rather than the source of the fault. Therefore, the step drop position is used as the boundary for fault source localization.

[0039] Iterate through all possible values ​​of k in the difference sequence, performing the ratio calculation and step descent judgment operation for each pair of adjacent difference values. For each position determined to have a step descent, record the corresponding pair of adjacent nodes, extract the identifier of the downstream node from this pair, and mark the extracted downstream node as a suspected fault node in the current anomaly propagation path.

[0040] After identifying all step descent positions and extracting downstream nodes in a current anomaly propagation path, a set of suspected fault nodes corresponding to this current anomaly propagation path is obtained. The above operation is performed on all anomaly propagation paths extracted in the same fault localization iteration to collect the suspected fault nodes corresponding to each anomaly propagation path.

[0041] All collected suspected fault nodes are deduplicated, removing duplicate node identifiers. The remaining nodes constitute a set of suspected fault nodes. This set of suspected fault nodes serves as the starting point set for extracting anomaly propagation paths in the next round of fault location iterations. During the anomaly propagation path extraction process in the next round of fault location iterations, each node in the set of suspected fault nodes is used as the starting point for a forward traversal operation.

[0042] See Figure 4 After dynamically adjusting the node search direction for the next round of fault location iteration, the termination condition judgment of the fault location iteration and the output operation of the final set of fault root cause nodes are also performed. The specific implementation process is as follows: After extracting a set of suspected faulty nodes in each fault location iteration and dynamically adjusting the node search direction for the next iteration, the set of suspected faulty nodes extracted in the current iteration and the set extracted in the previous iteration are obtained. The current set of suspected faulty nodes is compared with the previous set to determine if they are identical. The specific method for determining if the two sets are identical is as follows: compare the total number of node identifiers in the current set with the total number of node identifiers in the previous set. If the total number of node identifiers is equal, iterate through each node identifier in the current set and check if each node identifier has a corresponding entry in the previous set. If the total number of node identifiers is equal and every node identifier in the current set can be found in the previous set, then the two sets are considered identical. If the total number of node identifiers is unequal, or if at least one node identifier in the current set cannot be found in the previous set, then the two sets are considered not identical.

[0043] If the current set of suspected faulty nodes is completely identical to the previous set, the fault location iteration process terminates, and the next iteration is not executed. The current set of suspected faulty nodes is output as the final root cause node set, where each node identifier in the final root cause node set corresponds to a microservice instance in the cloud-native environment that is identified as the root cause of the fault.

[0044] In cases where the current set of suspected faulty nodes differs from the previous set, it checks whether the number of fault location iterations already executed has reached the preset maximum number of iterations. The preset maximum number of iterations is set to a fixed value of 10. This setting is based on the following: In cloud-native environments where the depth of microservice call chains typically does not exceed ten layers, ten iterations are sufficient to cover the complete traversal process from the first candidate node for the root cause of the anomaly along the call chain forward to the node at the end of the chain. If convergence is not achieved after more than ten iterations, it indicates that the fault location process may be stuck in a loop or oscillation.

[0045] If the current fault location iteration round has not reached the preset maximum iteration round number of 10, then continue to execute the next round of fault location iteration. In the next round of fault location iteration, the set of suspected fault nodes output in the current round is used as the starting point set for extracting the abnormal propagation path in the next round, and the operations of abnormal propagation path extraction and dynamic adjustment of node search direction described in some embodiments are repeatedly executed.

[0046] If the current fault location iteration round has reached the preset maximum iteration round number of 10, the fault location iteration process terminates and no further iterations are performed. The set of suspected fault nodes output in the last fault location iteration is output as the final root cause node set. Each node identifier in the final root cause node set corresponds to a microservice instance in the cloud-native environment that is located as the root cause of the fault.

[0047] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A microservice fault adaptive localization method in a cloud-native environment, characterized in that, Perform the following steps: Obtain real-time call chain data for each microservice instance in the cloud-native environment, and parse the response time sequence and status code sequence of each microservice instance from the real-time call chain data; Based on the response time sequence and status code sequence of each microservice instance, construct the basic health feature vector of each microservice instance; Using the basic health feature vector of each microservice instance as node input and the call relationship between microservices as edge weights, a full service call graph is constructed. In each round of fault location iteration, the current anomaly propagation path is extracted from the full service call graph, and the node search direction for the next round of fault location iteration is dynamically adjusted based on the differences in health characteristics of adjacent nodes in the current anomaly propagation path.

2. The microservice fault adaptive localization method in a cloud-native environment according to claim 1, characterized in that, The specific method for obtaining real-time call chain data for each microservice instance in the cloud-native environment and parsing the response time sequence and status code sequence of each microservice instance from the real-time call chain data is as follows: Collect call chain records of all external requests received by each microservice instance within the most recent time window using the link tracing component in the cloud-native environment; Extract the actual processing time consumed by the current microservice instance when processing the request from each call chain record as the response time, and extract the HTTP status code or gRPC status code returned by the current microservice instance as the status code. The response time sequence is formed by arranging all response times of the same microservice instance in chronological order within a time window, and the status codes are formed by arranging all status codes in the same chronological order.

3. The microservice fault adaptive localization method in a cloud-native environment according to claim 2, characterized in that, The length of the time window is dynamically adjusted based on the average request processing latency of the microservice instance; the greater the latency, the shorter the time window.

4. The microservice fault adaptive localization method in a cloud-native environment according to claim 2, characterized in that, The specific method for constructing the basic health feature vector of each microservice instance based on the response time sequence and status code sequence of each microservice instance is as follows: Calculate the arithmetic mean of the response time series for each microservice instance to obtain the average response time of that microservice instance; Calculate the standard deviation of the response time series for each microservice instance to obtain the response time fluctuation value of that microservice instance; For each microservice instance, the ratio of the number of occurrences of unsuccessful status codes to the total length of the status code sequence is calculated to obtain the percentage of abnormal status codes for that microservice instance. The average response time, response time fluctuation value, and percentage of abnormal status codes of the microservice instance are concatenated in a fixed order to form a three-dimensional basic health feature vector of the microservice instance.

5. The microservice fault adaptive localization method in a cloud-native environment according to claim 1, characterized in that, The specific method for constructing the full service call graph using the basic health feature vector of each microservice instance as node input and the call relationship between microservices as edge weights is as follows: Each microservice instance in the cloud-native environment is mapped to an independent node in the full service call graph, and each node stores the basic health feature vector of the corresponding microservice instance. Collect all call chain records within a preset historical time period in the cloud-native environment, and count the number of times direct call relationships occur between any two microservice instances; The number of direct calls between any two microservice instances is normalized and used as the weight of the directed edge connecting these two nodes in the full service call graph. If there is no direct call relationship between two microservice instances, no directed edge is established.

6. The microservice fault adaptive localization method in a cloud-native environment according to claim 4, characterized in that, The normalization process uses a maximum-minimum value normalization method to map the number of direct calls to the [0,1] interval.

7. The microservice fault adaptive localization method in a cloud-native environment according to claim 1, characterized in that, The specific method for extracting the current anomaly propagation path from the full service call graph in each round of fault location iteration is as follows: In the first round of fault location iteration, nodes whose vector magnitude of the basic health feature vector in the full service call graph exceeds a preset threshold are marked as candidate nodes for the root cause of the anomaly. Starting from each candidate node for the root cause of the anomaly, the graph is traversed backward along the direction of the incoming edges to nodes without incoming edges, and the node sequence on each traversal path is recorded as the initial anomaly propagation path. In non-first-round fault location iterations, obtain the set of suspected fault nodes output by the previous fault location iteration. Starting from each node in the set of suspected fault nodes, traverse forward along the outgoing edge direction of the full service call graph to the node with no outgoing edge, and record the node sequence on each traversal path as the current anomaly propagation path.

8. The microservice fault adaptive localization method in a cloud-native environment according to claim 7, characterized in that, The preset threshold is dynamically calculated based on the sum of the mean and standard deviation of the vector magnitudes of the basic health feature vectors of all nodes in the full service call graph.

9. The microservice fault adaptive localization method in a cloud-native environment according to claim 7, characterized in that, The specific method for dynamically adjusting the node search direction for the next round of fault location iteration based on the differences in health characteristics of adjacent nodes in the current anomaly propagation path is as follows: For each pair of adjacent nodes in the current abnormal propagation path, calculate the Manhattan distance between the basic health feature vector of the downstream node and the basic health feature vector of the upstream node to obtain the health feature difference value of the adjacent nodes. The health characteristic differences of all adjacent nodes on the same abnormal propagation path are arranged into a difference sequence from upstream to downstream. The position where the difference value drops sharply is found from the difference sequence, and the node downstream of the position is marked as a suspected fault node in the current abnormal propagation path. Collect suspected fault nodes along all abnormal propagation paths, deduplicate them to form a set of suspected fault nodes, and use this set of suspected fault nodes as the starting point set for extracting abnormal propagation paths in the next round of fault location iteration.

10. The microservice fault adaptive localization method in a cloud-native environment according to claim 9, characterized in that, The specific method for finding the location where the difference value shows a step decrease from the difference sequence is as follows: The ratio of two adjacent difference values ​​is calculated for the difference sequence. If the ratio of two adjacent difference values ​​is greater than a preset step threshold, it is determined that a step drop has occurred at the position of the adjacent node pair corresponding to the next difference value. Record the positions of all node pairs that exhibit a step drop, extract the identifier of the downstream node from each node pair position, and designate the downstream node as a suspected fault node in the current abnormal propagation path.

Images