An unsupervised anomaly detection method based on multi-source data of a micro-service system

Abstract

The application provides a kind of microservice system multi-source data based on unsupervised anomaly detection method, and the specific steps are four parts of microservice selection, multi-source feature sequence construction, anomaly detection and interpretation.Microservice selection, extract tracking data to build call tree, analyze fault propagation mode, and define the leaf node in call tree as fault sensitive instance corresponding to microservice.Multi-source feature sequence construction, analyze the correlation between indicators and tracking and different faults, correlate in time and realize serialization of features.Anomaly detection, use sliding window mechanism to divide time series features into continuous data blocks, judge data block state by encoding and reconstruction of data block.Anomaly interpretation, extract data block reconstruction distance, obtain microservice instance and corresponding features with larger anomaly probability.The application can be used for enterprise, cloud service provider to evaluate the running state of key business, application service and system in internal cloud environment, and provide protection for improving application service quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of micro-service system anomaly detection, and particularly relates to a micro-service system multi-source data-based unsupervised anomaly detection method. BACKGROUND

[0002] With the rapid development of cloud native technology, the continuous innovation of containerization, micro-service, container orchestration tools and other technologies makes the deployment, expansion and management of large-scale public applications on the cloud more flexible, and the migration of domestic and foreign enterprise applications to the cloud has become a trend. However, the service architecture of large-scale applications is more complex, and the number of service components is more massive. The failure of cross-layer components can seriously affect the quality of application services. It is crucial to implement unified, accurate and interpretable anomaly detection for micro-service systems, which provides data support for further fault localization and protection.

[0003] However, for complex micro-service architecture, it is challenging to implement unified, accurate and interpretable anomaly detection. For the various modal data generated by the system, existing methods mostly use single modal data for detection. This method lacks correlation analysis of multi-source features for different types of faults, and the quantification of the representation ability of different modal features for faults is lacking. Secondly, many existing micro-service anomaly detection methods are mainly aimed at micro-service instances, and need to construct independent detection models for the data generated by each instance, which will inevitably occupy a large amount of resources. In addition, this method regards each instance as a relatively independent individual, ignoring the mutual influence between micro-service instances when a fault occurs. Finally, the existing anomaly detection methods have low ability to dynamically capture the mutual influence between micro-service instances, lack analysis of the anomaly data structure of the micro-service system, and cause normal data that cannot be ignored to be misjudged as abnormal. Therefore, a method is needed that starts from the overall perspective of the system, uses multi-source data with high correlation with faults to achieve accurate anomaly detection and interpretable anomaly analysis.

[0004] In general, the current methods for feature analysis of microservice systems, microservice instance call correlation, and anomaly detection have the following three problems: (1) single-modal data feature analysis: using single-modal data such as indicators and traces to represent faults, the correlation analysis of multi-source features for different types of faults is lacking, and the contribution of different modal data features to multiple fault detection cannot be quantitatively measured; (2) instance-based model construction method: the instance-based model construction method needs to construct an independent model for each instance of the microservice system. This will result in a large amount of resource consumption, and cannot provide the overall running state of the system, and ignores the mutual influence relationship between microservice instances. (3) anomaly detection method: the existing anomaly detection method has low ability to dynamically capture the mutual influence between microservice instances, which will cause the normal data to be misjudged as abnormal, further reducing the performance of the downstream fault localization method. SUMMARY

[0005] To solve the above problems, the present application provides an unsupervised anomaly detection method based on multi-source data of a microservice system. The method selects multi-source data of fault-sensitive microservice instances in the system, extracts multi-source data features that contribute greatly to fault representation, and constructs a unified monitoring model for the microservice system based on the Transformer encoder, to achieve accurate anomaly detection and provide detailed anomaly analysis reports. The specific steps of the method are divided into four parts: fault-sensitive microservice instance selection, multi-source data feature sequence construction, anomaly event detection based on reconstruction, and anomaly explanation. In the microservice instance selection, the call tree structure is constructed by extracting the Trace data, the fault propagation pattern of different position anomalies is analyzed, and the microservice instances corresponding to the leaf nodes in the call tree are defined as fault-sensitive instances. In the multi-source data feature sequence construction, the correlation of indicators and trace data in representing different types of faults is analyzed, the feature set with high contribution to fault detection is selected, and the two modal features are associated in the time dimension to finally realize the serialization of the features. In the anomaly event detection, the sliding window mechanism is used to divide the time series features into continuous data blocks, and the encoding and reconstruction of the data blocks are used to judge whether the data blocks are abnormal. In the anomaly explanation, the reconstruction distance of the data block is extracted, the microservice instances with high anomaly probability and the corresponding features are extracted, and more detailed anomaly information is provided. The present application can be used by enterprises and cloud service providers to evaluate the running state of key business, application services and systems in the internal cloud environment, and to provide protection for improving the quality of application services.

[0006] To achieve the purpose of the present application, the specific technical steps of the present application are as follows:

[0007] Step (1) builds a hierarchical call tree structure based on the trace data collected from the microservice system. Analyze the propagation pattern of the fault in the call tree and determine the corresponding topology position of the microservice instance in the tree structure. The microservice corresponding to the leaf node in the call tree is regarded as the fault sensitive instance, and the specific sensitive microservice instance set is analyzed for different scale microservice systems;

[0008] Step (2) analyzes the correlation differences between different types of faults and system-level indicators and application-level indicators based on the metric data collected from the microservice system, and determines the indicator set that contributes to the characterization of the fault. Based on the trace data, the features that characterize the duration of each span and the features that characterize the call relationship between multiple spans are extracted;

[0009] Step (3) performs time alignment and clock synchronization operations on the multi-source data features of the fault sensitive microservice instance, i.e. the selected indicator features and the two types of trace features, and processes the event stream into continuous time feature sequences. Use the sliding window mechanism to divide the time feature sequence into continuous data blocks, and encode the data blocks into embedding vectors;

[0010] Step (4) builds an anomaly detection model based on the Transformer encoder. By minimizing the distance between the data block reconstruction and the data block itself, the feature form of the multi-source data during the normal operation of the microservice system is learned. The anomaly detection model determines the anomaly by the distance between the specific data block and the reconstruction, and provides detailed explainability analysis for the anomaly.

[0011] Further, the step (1) specifically comprises the following sub-steps:

[0012] (1.1) Collect 2-5 hours of trace data from the normally operating microservice system, extract the TraceID, SpanID, ParentID and corresponding microservice instance features of each trace data at each time. Aggregate all spans with the same TraceID into the same trace, and determine the call relationship between different spans through ParentID to identify the common calls between instances;

[0013] (1.2) Based on multiple call records, a hierarchical call tree structure is built. Each node in the call tree represents a microservice, and the edge represents the call between microservices. The hierarchical design maximizes the representation of the call priority between microservices;

[0014] (1.3) Based on the constructed call tree, further analyze the propagation mode of the fault. The propagation mode of the fault is different according to the location of the fault occurrence. The fault occurrence location is divided into leaf nodes and intermediate nodes, and the differences in fault propagation mode caused by different fault locations are discussed;

[0015] (1.4) The fault propagation mode shows that the leaf nodes of the call tree are usually the endpoints of business logic and are severely dependent on the normal operation of other microservices. The characteristic changes caused by faults are more obvious on the leaf nodes. The microservice instances represented by the leaf nodes in the call tree are determined as fault-sensitive microservice instances.

[0016] Further, the step (2) specifically comprises the following sub-steps:

[0017] (2.1) Based on commonly used index characteristics and Kubernetes (K8s) platform structure, 11 kinds of system-level and application-level indicators are analyzed, and mutual information technology (MI) is used to measure the differences in the characterization ability of different indicators for different types of faults. From the 11 indicators, 3 indicators are selected to mainly represent the resource type fault;

[0018] (2.2) Considering that the index characteristics are insufficient to represent network type faults and program code faults, further Span duration characteristics and Span offset time characteristics are proposed from the Trace data, and the characterization ability of the two Trace characteristics for network delay, error return and abnormal code faults is studied;

[0019] Further, the detailed information of the multi-source data characteristics in (2.1) to (2.2) is shown in Table 2:

[0020] Table 2 Multi-source characteristics

[0021]

[0022] The step (3) specifically comprises the following sub-steps:

[0023] (3.1) The collected fault-sensitive instance indicators have a serialized structure, and the Metric data is standardized;

[0024] (3.2) The tracking of the failure-sensitive instance collected by multiple spans consists of several steps. First, according to the TraceID, the multiple spans belonging to the same trace at each time are aggregated. Then, the corresponding call relationship between the parent span and the child span in each trace is determined using the ParentID. Considering the serious dependence of trace data on user behavior, it may occur that the failure-sensitive microservice neither calls other microservices nor is called by other microservices. In this case, the trace-related features of the instance may be missing. Therefore, a missing value filling operation needs to be performed and similar standardization processing needs to be performed on the Trace data;

[0025] (3.3) Time alignment and time synchronization are implemented. The timestamps of the serialized indicators and trace data are aligned, the multi-source data features with the same Unix timestamp are retained, and the correspondence between the two kinds of data is established at these timestamps, ensuring that the two kinds of monitoring data clocks are synchronized. In addition, when collecting the original trace, the processed traces need to be aggregated every minute to obtain the indicator and trace data features of the microservice system every minute;

[0026] (3.4) The obtained multi-source features are divided into continuous data blocks in the time dimension using a sliding window mechanism. The features of the failure-sensitive instance at each time are a token as input to the model, and the embedding vector is obtained by adding position encoding and flattening processing.

[0027] The step (4) specifically comprises the following sub-steps:

[0028] (4.1) The normal operation mode is learned using a reconstruction-based model learning system, and an optimized Transformer model is used as an encoder to encode the normal time series into a latent space to obtain the reconstruction error corresponding to the original data;

[0029] (4.2) The embedding vector and the reconstruction vector of each data block are obtained through embedding encoding and reconstruction. Whether an anomaly occurs is determined according to the distance change between the original data and the reconstructed data;

[0030] (4.3) An interpreter is designed to analyze the anomaly. By calculating the distance between the embedding vector and the reconstruction vector of the original data block, the microservice instance and the corresponding feature that contribute more to the anomaly judgment are determined to help further locate the failure.

[0031] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the unsupervised anomaly detection method based on multi-source data of a microservice system when executing the program.

[0032] A computer readable storage medium, having stored thereon computer instructions, which, when executed by a processor, implement the unsupervised anomaly detection method based on multi-source data of a micro-service system.

[0033] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0034] (1) The present application provides an unsupervised anomaly detection method based on multi-source data of a micro-service system, which can associate the obtained multi-source data, accurately and quickly analyze the overall running state of the system at a specific time. When detecting system anomalies, it has the ability to provide abnormal micro-service instances and significant change feature information, realizing the monitoring and security maintenance of the micro-service system on the cloud.

[0035] (2) The present application includes correlation analysis of multi-source data features and different types of faults, constructs a hierarchical micro-service call tree structure based on tracking data, encodes and reconstructs system features using a Transformer encoder, and obtains more accurate anomaly detection results, providing detailed anomaly analysis reports.

[0036] (3) The unsupervised anomaly detection method based on multi-source data of a micro-service system can be applied to enterprises and cloud service providers to evaluate the running state of key business, application services and systems in the internal cloud environment, and has high application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The framework of the unsupervised anomaly detection method based on multi-source data of a micro-service system;

[0038] Figure 2 The span duration and span offset feature map in tracking;

[0039] Figure 3 The fault propagation mode diagram for different fault locations. DETAILED DESCRIPTION

[0040] The technical solutions provided by the present application will be described in detail below with specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0041] Example 1: The unsupervised anomaly detection method based on multi-source data of a micro-service system provided by the present application has a structure framework as shown in Figure 1 The steps include:

[0042] (1) Based on the trace data collected from the microservice system, a hierarchical call tree structure is constructed. The propagation pattern of the fault in the call tree is analyzed, and the topological position of the microservice instance in the tree structure is determined. The microservice corresponding to the leaf node in the call tree is regarded as the fault-sensitive instance, and the specific sensitive microservice instance set is analyzed for microservice systems of different scales;

[0043] The specific process of this step is as follows:

[0044] (1.1) Collect the Trace data within 3 hours from the normally running microservice system, extract the TraceID, SpanID, ParentID and corresponding microservice instance characteristics of each Trace data at each time. All spans with the same TraceID are aggregated into the same trace, and the calling relationship between different spans is determined through the ParentID to identify the common calling between instances;

[0045] (1.2) Based on multiple calling records, a hierarchical call tree structure is constructed, as shown in Figure 3 Each node in the call tree represents a microservice, and the edge represents the calling between microservices. The microservice that is not called by any other microservice is the root node of the tree; a microservice that does not call any other microservice is the leaf node of the tree; other microservices are intermediate nodes. When there is an edge between the high-level node and the low-level node in the call tree, it means that the high-level microservice calls the low-level microservice. The hierarchical design maximizes the representation of the calling priority between microservices;

[0046] (1.3) Based on the constructed call tree, the propagation pattern of the fault is further analyzed. The propagation pattern of the fault is different according to the location of the fault. The fault location is divided into leaf nodes and intermediate nodes, and the differences in fault propagation patterns caused by different fault locations are discussed. If the abnormal leaf node is not called by other nodes during this period, the fault is limited to this node, as shown in Figure 3 (a); if the abnormal leaf node is called by the parent node during this period, the error may affect some ancestors of the node and their descendants, as shown in Figure 3 When the intermediate node fails, if the abnormal intermediate node is not called during the fault, the fault may propagate along the calling chain and affect some child nodes of the node, as shown in Figure 3 (c). If the abnormal intermediate node is called, the fault may exhibit the pattern in (c), and also affect some ancestor nodes and their descendant nodes of the node, similar to the pattern in (b);

[0047] (1.4) The failure propagation pattern shows that the feature changes caused by the failure are more obvious at the leaf nodes of the call tree, because the leaf nodes of the call tree are usually the endpoints of business logic and are heavily dependent on the normal operation of other microservices. The microservice instances represented by the leaf nodes in the call tree are determined as failure-sensitive microservice instances.

[0048] (2) Based on the collected metric data from the microservice system, the correlation differences between different types of failures and system-level metrics and application-level metrics are analyzed to determine the set of metrics that contribute to the characterization of the failure. Based on the Trace data, the features that characterize the duration of each span and the features that characterize the call relationship between multiple spans are extracted;

[0049] The specific process of this step is as follows:

[0050] (2.1) Based on the commonly used indicator features and the Kubernetes (K8s) platform structure, 11 system-level and application-level indicators are analyzed, and the mutual information technology (MI) is used to measure the differences in the characterization ability of different indicators for different types of failures. From the 11 indicators, 3 indicators are selected to mainly characterize the resource type failure;

[0051] (2.2) Considering the insufficient ability of the indicator features to characterize network type failures and program code failures, further Span duration features and Span offset time features are proposed from the Trace data. In the microservice system, the Trace data represents the hierarchical relationship in the service call chain by using parent spans and child spans. As shown in Figure 2 , there are three types of call relationships between parent spans and child spans, and the same three relationships exist between multiple child spans that share the same parent span. According to the relationship between the end times of the parent span and the child span, the Span Offset is divided into positive offset and negative offset. The characterization ability of the two Trace features for network delay, error return, and abnormal code failures is studied;

[0052] Further, the detailed information of the multi-source data features in (2.1) to (2.2) is shown in Table 3.

[0053] Table 3 Multi-source features

[0054]

[0055] Step (3) performs time alignment and clock synchronization operations on the multi-source data features of the failure-sensitive microservice instances, i.e., the selected indicator features and the two Trace features, to process the event stream into a continuous time feature sequence. The sliding window mechanism is used to divide the time feature sequence into continuous data blocks, and the data blocks are encoded into embedding vectors;

[0056] The specific process of this step is as follows:

[0057] (3.1) The collected fault-sensitive instance indicators have a serialized structure, and the Metric data is standardized. Considering that the training phase uses system-generated normal data, while the test phase uses mixed data composed of a large number of normal data interspersed with a small amount of abnormal data. To avoid the standardization process causing the training model to predict all normal data as abnormal, the training set and the test set need to be standardized respectively, and the specific process is shown in formula 1:

[0058] M' d = M d -mean(M train ) / std(M train ) (1)

[0059] Where M d represents the original Metric data, M' d represents the standardized data, mean(M train ) and std(M train ) represent the mean and standard deviation of the training set, respectively;

[0060] (3.2) The collected fault-sensitive instance traces are composed of multiple spans. First, aggregate multiple spans belonging to the same trace at each time according to TraceID. Then, use ParentID to determine the corresponding call relationship between the parent span and the child span in each trace. Considering that trace data is heavily dependent on user behavior, it may occur that the fault-sensitive microservice neither calls other microservices nor is called by other microservices. In this case, the trace-related features of this instance may be missing. Therefore, a missing value filling operation needs to be performed and similar standardization processing is performed on the Trace data; considering the continuity of the microservice state, the missing values are filled with the average value of the features in the range [t-ε, t+ε], and the present scheme uses ε=2. Then, apply a similar standardization process to the two trace-related features that change over time, similar to that used for metric serialization;

[0061] (3.3) Implement time alignment and clock synchronization. Align the timestamps of the serialized indicators and the trace data, retain the multi-source data features with the same Unix timestamp, and establish a correspondence between the two kinds of data at these timestamps to ensure that the two kinds of monitoring data clocks are synchronized. In addition, when collecting the original trace, it is necessary to aggregate the processed traces every minute to obtain the indicator and trace data features of the microservice system every minute;

[0062] (3.4) Using a sliding window mechanism to divide the acquired multi-source features into consecutive data blocks in the time dimension, where each data block contains three-dimensional features corresponding to W time points. The features of the fault-sensitive instances at each time point are used as a token input to the model, as shown in Figure 1 t , without losing the abnormal information carried in the original data D t , the data D t is flattened in the feature and micro-service instance dimensions to obtain 2-dimensional time series features Finally, the embedding vector is obtained by adding the position encoding, and the embedding process is shown in equation 2:

[0063]

[0064] Where t represents a certain time, E pos represents the position encoding, W, M and F are the sliding window size, the number of microservices and the number of features, respectively.

[0065] (4) Construct an anomaly detection model based on the Transformer encoder. By minimizing the distance between the data block reconstruction and the data block itself, learn the feature form of multi-source data during the normal operation of the micro-service system. The anomaly detection model determines the anomaly by the distance between the specific data block and the reconstruction, and provides detailed explainability analysis for the anomaly;

[0066] The specific process of this step is as follows:

[0067] (4.1) Use the reconstruction-based model to learn the normal operation mode of the system, use the optimized Transformer model as the encoder, and encode the normal time series into the latent space to obtain the reconstruction error corresponding to the original data; The structure of the encoder is shown in Figure 1 , including alternating multi-head self-attention (MSA) blocks, MLP blocks and output blocks;

[0068] (4.2) Through embedding encoding and reconstruction, obtain the embedding vector Max and the reconstruction vector Rec of each data block. According to the distance change between the original data and the reconstructed data, it is judged whether an anomaly occurs, and if it significantly increases, it is an anomaly; Use the reconstruction error between the original data and the reconstructed data to measure the distance between them and as the loss function of model training, the calculation process is shown in equation 3:

[0069]

[0070] ​Wherein, M is the number of microservices, F is the number of features. During training, normal data is divided by 8:2, and during testing, the reconstruction error is used to judge the anomaly. Each sliding window outputs a detection result, and if any time is abnormal, the entire data block is marked as abnormal. Finally, the prediction result is compared with the label to evaluate the detection performance;

[0071] (4.3) Design an interpreter to analyze the anomaly. By calculating the distance between the embedding vector M of the original data block and the reconstruction vector R, the microservice instance and the corresponding feature that contributes more to the anomaly judgment are determined to help further locate the fault. The specific calculation process is shown in formula 4:

[0072]

[0073] Wherein, M and F represent the number of microservices and the number of features, respectively.

[0074] The technical means disclosed in the present application scheme is not limited to the technical means disclosed in the above-mentioned embodiments, but also includes the technical solutions composed of any combination of the above technical features. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can also be made, which are also considered within the scope of protection of the present application.

Claims

1. A method for unsupervised anomaly detection based on multi-source data of a micro-service system, characterized in that, The method comprises the following steps: Step (1) based on the trace data collected from the micro-service system, a hierarchical call tree structure is constructed, the propagation mode of the fault in the call tree is analyzed, and the corresponding topological position of the micro-service instance in the tree structure is determined, the micro-service corresponding to the leaf node in the call tree is taken as the fault sensitive instance, and the sensitive micro-service instance set is analyzed for different scale micro-service systems; Step (2) based on the metric data collected from the micro-service system, the correlation difference between different types of faults and system level indicators and application level indicators is analyzed, the indicator set which contributes to the characterization of the fault is determined, and based on the trace data, the features for characterizing the duration of each span and the features for characterizing the call relationship of multiple spans are extracted; Step (3) for the multi-source data features of the fault sensitive micro-service instance, that is, the selected indicator features and the two kinds of trace features, time alignment and clock synchronization operations are performed, the event stream is processed into a continuous time feature sequence, the sliding window mechanism is used to divide the time feature sequence into continuous data blocks, and the data blocks are encoded into embedding vectors; Step (4) an abnormality detection model based on a transformer encoder is constructed, the feature form of the multi-source data during the normal operation of the micro-service system is learned by minimizing the distance between the data block reconstruction and the data block itself, and the abnormality detection model determines the abnormality by the distance between the original data block and the reconstruction, and provides detailed explainability analysis for the abnormality; Wherein, the step (3) specifically comprises the following sub-steps: (3.1) the collected fault sensitive instance indicators have a serialized structure, and the metric data is standardized; The training set and the test set need to be standardized respectively, and the specific process is shown in formula 1: M' d = M d - mean(M train ) / std(M train ) (1) where M d represents the original Metric data, M' d represents the standardized data, mean(M train ) and std(M train ) are the mean and standard deviation of the training set, respectively. (3.2) the collected fault sensitive instance trace is composed of multiple spans, first, the multiple spans belonging to the same trace at each time are aggregated according to the trace ID, then the corresponding call relationship between the parent span and the child span in each trace is determined using the parent ID, (3.3) time alignment and clock synchronization are realized, the timestamps of the serialized indicators and the trace data are aligned, the multi-source data features with the same Unix timestamp are retained, and the corresponding relationship between the two kinds of data is established at these timestamps, ensuring that the clocks of the two kinds of monitoring data are synchronized, in addition, the processed traces collected every minute are needed to collect the original traces to obtain the metric and trace data features of the micro-service system every minute; (3.4) Using a sliding window mechanism to divide the acquired multi-source features into continuous data blocks in the time dimension, where each data block contains three-dimensional features corresponding to W time points, and the features of fault-sensitive instances at each time point are taken as a token of model input. In order to process three-dimensional data D t , without losing the abnormal information carried in the original data D t , D t is flattened in the feature and micro-service instance dimensions to obtain 2-dimensional time sequence features Finally, the embedding vector is obtained by adding position encoding, and the embedding process is shown in formula 2: where t represents a certain time, E pos represents position encoding, W, M and F are the size of sliding window, the number of microservices and the number of features, respectively. 2.The method of claim 1, wherein, The step (1) specifically comprises the following sub-steps: (1.1) collect 2-5 hours of trace data from the normally operating micro-service system, extract the trace ID, span ID, parent ID and corresponding micro-service instance features of each trace data at each time, aggregate all spans with the same trace ID into the same trace, and determine the call relationship between different spans through the parent ID to identify the common call between instances; (1.2) Based on multiple call records, a hierarchical call tree structure is constructed, each node in the call tree represents a microservice, and the edge represents the call between microservices. The hierarchical design maximizes the representation of the call priority between microservices; (1.3) Based on the constructed call tree, the propagation mode of the fault is analyzed. The propagation mode of the fault is different according to the location of the fault. The fault location is divided into leaf nodes and intermediate nodes. The difference in fault propagation mode caused by different fault locations is discussed; (1.4) The fault propagation mode shows that the leaf nodes of the call tree are usually the endpoints of business logic and are severely dependent on the normal operation of other microservices. The characteristic changes caused by faults are more obvious on the leaf nodes. The microservice instances represented by the leaf nodes in the call tree are determined as fault-sensitive microservice instances. 3.The method of claim 2, wherein, The step (2) specifically comprises the following sub-steps: (2.1) Based on commonly used index characteristics and Kubernetes (K8s) platform structure, 11 kinds of system level and application level indicators are analyzed. The mutual information technology (MI) is used to measure the difference in the representation ability of different indicators for different types of faults. From the 11 indicators, 3 indicators are selected to represent the fault of the resource type; (2.2) Span duration feature and Span offset time feature are proposed from Trace data, and the representation ability of the two Trace features for network delay, error return and abnormal code three faults is studied.

4. The method of claim 3, wherein, The detailed information of the multi-source data features in steps (2.1) to (2.2) is as follows: Metric includes CPU usage of microservice, memory usage of microservice and CPU utilization of host node; Trace includes duration of Span and time offset of sub-Span to parent Span.

5. The method of claim 3, wherein, The step (4) specifically comprises the following sub-steps: (4.1) Use the model learning system based on reconstruction to learn the normal operation mode. An optimized Transformer model is used as an encoder to encode the normal time series into a latent space to obtain the reconstruction error corresponding to the original data; (4.2) Through embedding coding and reconstruction, the embedding vector Max and the reconstruction vector Rec of each data block are obtained. According to the distance change between the original data and the reconstructed data, it is judged whether an anomaly occurs. If it significantly increases, it is an anomaly. The reconstruction error between the original data and the reconstructed data is used to measure the distance between them and as the loss function for model training. The calculation process is shown in formula 3: Where M is the number of microservices, F is the number of features, and during training, the normal data is divided by 8:

2. During testing, the reconstruction error is used to judge the anomaly. Each sliding window outputs a detection result. If there is an anomaly at any time, the entire data block is marked as abnormal. Finally, the prediction result is compared with the label to evaluate the detection performance; (4.3) Design an interpreter to analyze the anomaly. The distance between the embedding vector M and the reconstruction vector R of the original data block is calculated to locate the fault. The specific calculation process is shown in formula 4: Where M and F represent the number of microservices and the number of features, respectively.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: The processor implements the micro-service system multi-source data-based unsupervised anomaly detection method in any one of claims 1-5 when executing the program.

7. A computer readable storage medium having stored thereon computer instructions, characterized in that: The computer instructions implement the micro-service system multi-source data-based unsupervised anomaly detection method in any one of claims 1-5 when executed by the processor.

Images