Learning from Distributed Traces for Anomaly Detection and Root Cause Analysis

Abstract

A technique for facilitating anomaly detection and root cause analysis using distributed trace data. In one example, a system can include a processor executing computer-executable components stored in a memory. The computer-executable components can include a pre-processing component and a monitor component. The pre-processing component can generate trace frames including vectorized representations of textual trace data produced by microservices of a microservice application. The monitor component can use the trace frames to identify a state of the microservice application.

Description

[Technical Field]

[0001] The subject disclosure relates to computing devices and, more particularly, to techniques that use distributed trace data to facilitate anomaly detection and / or root cause analysis. Summary of the Invention

[0002] SUMMARY OF THE INVENTION The following is a summary intended to provide a basic understanding of one or more embodiments of the present invention. This summary is not intended to identify key or critical elements or to delineate the scope of particular embodiments or the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. One or more embodiments described herein describe a system, device, computer-implemented method, or computer program product, or combination thereof, that uses distributed trace data to facilitate anomaly detection and / or root cause analysis.

[0003] According to one embodiment, a system can include a processor executing computer-executable components stored in a memory. The computer-executable components can include a pre-processing component and a monitor component. The pre-processing component can generate trace frames including vectorized representations of textual trace data produced by microservices of a microservice application. The monitor component can use the trace frames to identify a state of the microservice application.

[0004] According to another embodiment, a computer-implemented method can include generating, by a system operatively coupled to a processor, trace frames including vectorized representations of textual trace data produced by microservices of a microservice application. The computer-implemented method can further include identifying, by the system, a state of the microservice application using the trace frames.

[0005] According to another embodiment, a computer program product may include a computer-readable storage medium having program instructions embodied thereon. The program instructions are executable by a processor to cause the processor to perform operations. The operations may include generating, by the processor, trace frames including vectorized representations of textual trace data produced by microservices of the microservice application. The operations may further include, by the processor, identifying a state of the microservice application using the trace frames. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram of an example, non-limiting system that can facilitate anomaly detection and / or root cause analysis using distributed trace data, according to one or more embodiments described herein. [Figure 2] FIG. 1 is an example, non-limiting dependency graph depicting dependencies between microservices in an e-commerce microservices application, according to one or more embodiments described herein. [Figure 3] 1 is an exemplary, non-limiting Gantt chart depicting a trace according to one or more embodiments described herein. [Figure 4] FIG. 1 is an illustration of exemplary, non-limiting textual trace data according to one or more embodiments described herein. [Figure 5] 1 is an exemplary, non-limiting call graph depicting a trace according to one or more embodiments described herein. [Figure 6] FIG. 1 is a diagram of an example, non-limiting list of span names, according to one or more embodiments described herein. [Figure 7] 1 is an exemplary, non-limiting directed acyclic graph (DAG) depicting a trace of a microservices application, according to one or more embodiments described herein. [Figure 8] 1 is an example, non-limiting DAG depicting a transformation trace of a microservices application, according to one or more embodiments described herein. [Figure 9] 1 is an example, non-limiting DAG depicting another variant trace of a microservices application, according to one or more embodiments described herein. [Figure 10] FIG. 1 is a diagram of an example, non-limiting architecture for implementing a framework for detecting anomalous behavior by microservices applications, according to one or more embodiments described herein. [Figure 11] FIG. 2 is a schematic diagram of an example, non-limiting, high-level concept of finding a d-dimensional vector that represents a span of a trace, according to one or more embodiments described herein. [Figure 12] FIG. 10 is a diagram of an example, non-limiting application of reverse depth-first search (DFS) and one-hop breadth-first search (BFS) graph traversal algorithms applied to traces of a microservices application, according to one or more embodiments described herein. [Figure 13] FIG. 1 illustrates an example, non-limiting application of multi-path DFS (MPDFS) and one-hop BFS graph traversal algorithms to traces of a microservices application, according to one or more embodiments described herein. [Figure 14]FIG. 10 is a diagram of another example, non-limiting application of MPDFS and one-hop BFS graph traversal algorithms to traces of a microservices application, according to one or more embodiments described herein. [Figure 15] FIG. 1 is a diagram of an example, non-limiting trace frame, according to one or more embodiments described herein. [Figure 16] FIG. 1 is a diagram of an example, non-limiting, convolutional long short-term memory (ConvLSTM) unit, according to one or more embodiments described herein. [Figure 17] FIG. 1 is a diagram of an example, non-limiting ConvLSTM-based autoencoder model, according to one or more embodiments described herein. [Figure 18] FIG. 10 is a diagram of an example, non-limiting sequence of λ trace frames, according to one or more embodiments described herein. [Figure 19] FIG. 1 is a diagram of an example, non-limiting ConvLSTM-based autoencoder model implemented by stacking multiple pairs of encoding and decoding ConvLSTM units, according to one or more embodiments described herein. [Figure 20] 1 is an example, non-limiting table depicting the results of an example demonstration according to one or more embodiments described herein. [Figure 21] 1 is an exemplary, non-limiting graph depicting reconstruction loss over time in accordance with one or more embodiments described herein. [Figure 22] 10 is an example, non-limiting graph depicting the effect of varying embedding dimension (d) and context window size (w) on IA-3 input anomalies, in accordance with one or more embodiments described herein. [Figure 23] 10 is an example, non-limiting graph depicting the effect of varying embedding dimension d and context window size w on IA-5 input anomalies, in accordance with one or more embodiments described herein. [Figure 24]10 is an example, non-limiting graph depicting the effect of varying time step length λ on IA-3 injection anomalies, in accordance with one or more embodiments described herein. [Figure 25] 10 is an example, non-limiting graph depicting the effect of varying time step length λ on IA-5 injection anomalies, in accordance with one or more embodiments described herein. [Figure 26] 1 is an example, non-limiting table depicting diagnostic performance of an example demonstration according to one or more embodiments described herein. [Figure 27] 1 is an example, non-limiting graph depicting relative diagnostic performance of example demonstrations according to one or more embodiments described herein. [Figure 28] FIG. 1 is a flow diagram of an example, non-limiting, computer-implemented method for facilitating anomaly detection and root cause analysis using distributed trace data, according to one or more embodiments described herein. [Figure 29] FIG. 1 is a block diagram of an exemplary, non-limiting operating environment capable of facilitating one or more embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION

[0007] The following detailed description is merely illustrative and is not intended to limit the embodiments or the application and / or uses of the embodiments, nor is it intended to be bound by any stated or implied information presented in the preceding "Background" or "Summary" sections or in the "Detailed Description" section.

[0008] One or more embodiments will now be described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more embodiments. It will be apparent, however, that in various instances, one or more embodiments may be practiced without these specific details.

[0009] Microservices architectures are becoming increasingly popular when designing software applications because they enable developers to work on independent schedules and deliver at high velocity, thereby reducing time to market. For applications deployed to hybrid cloud environments, microservices architectures are particularly attractive because the loosely coupled components of microservices applications improve scalability, flexibility, and maintainability, accelerating developer productivity. As a result, many companies are switching from traditional monolithic designs to microservices architectures. Maintaining the health status of microservices applications has become challenging for information technology (IT) professionals, such as DevOps and site reliability engineers (SREs). However, existing anomaly detection and root cause analysis (RCA) techniques designed for monolithic systems may not be effective for use with microservices applications.

[0010] To effectively handle a large number of microservices, including microapplications, DevOps and SREs often focus on measuring and monitoring application or system metrics (e.g., central processing unit (CPU) utilization, remote procedure call (RPC) latency, etc.) believed to accurately reflect the status of the microapplications. However, metrics-monitoring approaches typically do not scale well as the complexity of an application's constituent services increases. Furthermore, modern cloud-native services can be deployed on virtual machine (VM) or container-based clusters, where multiple virtualization layers and hardware heterogeneity can make it difficult to identify key metrics for a healthy service. Also, the number of RPCs for a single microservice can range from one or a few RPCs to hundreds of RPCs. Because these microservices form a complex service mesh for an application, monitoring results from either aggregated or separate metrics that are heavily dependent on the workload can make such metrics unsuitable for RCA against detected anomalies.

[0011] In addition to metrics, some monitoring methods leverage logs to detect anomalous status and have shown good performance for a moderate variety of anomalies. Such log-based methods first parse unstructured logs into a structured format and then group them by distinct tasks or threads. They can then build statistical or machine learning models on well-behaved logs and detect anomalies for new logs by calculating deviations from the good patterns. However, such log-based methods have difficulty dealing with complex concurrency patterns introduced by microservices, which can lead to reduced accuracy of the trained models. Therefore, the problem of anomaly detection and diagnosis for microservice applications remains largely unsolved.

[0012] Unlike metrics and logs around a single service or component, traces (one of the three pillars of IT observability) provide a holistic view of a distributed system. Distributed traces can be assembled from causally related events created at tracing points across process boundaries. As such, distributed traces can help developers troubleshoot subtle software bugs, find performance bottlenecks, and understand resource utilization in large-scale distributed systems. However, utilizing distributed traces for anomaly detection and RCA is challenging for several reasons. For example, trace events contain heterogeneous content in both text and numeric formats, so simply applying word embeddings to raw textual data cannot capture the complete set of relationships between these events in the context of a trace. As another example, learning the features of healthy traces involves not only modeling dependencies in the sequence of events within a single trace, but also more complex dependencies across traces.

[0013] 1 is a block diagram of an example, non-limiting system 100 that can facilitate anomaly detection and / or root cause analysis using distributed trace data in accordance with one or more embodiments described herein. System 100 includes a memory 110 for storing computer-executable components and one or more processors 120 operably coupled to memory 110 via one or more communication buses 130 for executing the computer-executable components stored in memory 110. As shown in FIG. 1, the computer-executable components include a pre-processing component 140 and a monitor component 150.

[0014] The preprocessing component 140 can generate trace frames that include vectorized representations of textual trace data produced by the microservices of the microservice application. In embodiments, the preprocessing component can generate the trace frames using a machine learning model trained using a graph traversal algorithm, a word embedding technique, or a combination thereof.

[0015] The monitor component 150 can use the trace frames to identify the state of the microservice application. In embodiments, the monitor component can identify the state of the microservice application by comparing the trace frames with reconstructed trace frames generated by a machine learning model. In embodiments, the machine learning model can capture spatial and temporal dependencies between microservices in the microservice application when operating under normal conditions.

[0016] In embodiments, the monitor component can identify a state of the microservice application by comparing a sequence of trace frames to a sequence of reconstructed trace frames generated by a machine learning model. The sequence of trace frames can include a trace frame. In embodiments, the machine learning model can be a convolutional long short-term memory (ConvLSTM)-based autoencoder model.

[0017] In an embodiment, the computer-executable components stored in memory 110 may further include a diagnostic component 160. The diagnostic component 160 may perform RCA by evaluating the contribution of each microservice to a given reconstruction loss. The functionality of the computer-executable components utilized by this embodiment is described in further detail below.

[0018] Figure 2 is a diagram of an example, non-limiting dependency graph 200 depicting dependencies between microservices in an e-commerce microservice application, according to one or more embodiments described herein. As illustrated by Figure 2, the e-commerce microservice application is structured as a loosely coupled collection of small, well-defined, stateless services known as microservices. In Figure 2, the loosely coupled collection of microservices includes a front-end microservice 210, a checkout microservice 220, a product catalog microservice 230, a shipping microservice 240, and an advertising microservice 250. Each microservice includes a well-defined application program interface (API) through which the loosely coupled collection of microservices communicates and generates responses to requests that users trigger from the front-end microservice 210 of the e-commerce microservice application.

[0019] FIG. 3 is an exemplary, non-limiting Gantt chart 300 depicting a trace according to one or more embodiments described herein. As used herein with respect to a microservice application, a “trace” refers to the end-to-end execution flow of a single user-triggered request from a front-end microservice (e.g., a web UI) to a back-end microservice (e.g., a database), including all intermediate microservices. In FIG. 3, the trace depicted by Gantt chart 300 corresponds to a single checkout request triggered by a user from a front-end microservice. By definition, the end-to-end execution flow of a trace is composed of spans and their causal ordering. A “span” refers to a named operation with a set of attributes (e.g., duration, host / process identifier (ID)) that describes a single unit of end-to-end execution flow. As illustrated by FIG. 3, a trace can be composed of operations on multiple microservices. For example, the trace depicted by Gantt chart 300 is composed of operations on a front-end microservice, a checkout microservice, a product catalog microservice, and a shipping microservice. A span often references other spans, forming a happened-before causal order. For example, a span that references another span that precedes it in the end-to-end flow of execution can be called a child span. In this example, the other span that precedes the child span in the end-to-end flow of execution can be called a parent span. Spans 320 and 330 in Gantt chart 300 represent the parent and child spans, respectively. A span that does not have a parent span can be called a root span. Span 310 in Gantt chart 300 represents the root span.

[0020] FIG. 4 is a diagram of example, non-limiting textual trace data 400 in accordance with one or more embodiments described herein. The textual trace data 400 generally represents a textual view of a trace depicted by the Gantt chart 300 corresponding to a checkout request. As illustrated by FIG. 4 , the textual trace data 400 includes a trace ID 410 and a start time 420 of an earlier span in time. The textual trace data 400 further includes an array of spans, such as span 430 and span 440, that include the trace depicted by the Gantt chart 300. In FIG. 4 , the array of spans is sorted by start time, and adjacent spans in the array of spans are separated by double vertical bar symbols "||." Each span in the textual trace data 400 includes a microservice name and an operation name separated by double semicolon symbols "::." The operation name refers to the operation being performed, and the microservice name refers to the microservice of the e-commerce microservice application that is performing the operation. For example, span 430 includes a microservice name of "frontend" and an action name of "Sent.hipstershop.CheckoutService.PlaceOrder." As another example, span 440 includes a microservice name of "productcatalogservice" and an action name of "Recv.hipstershop.ProductCatalogService.GetProduct."

[0021] FIG. 5 is an exemplary, non-limiting call graph 500 depicting a trace according to one or more embodiments described herein. In particular, call graph 500 represents the trace depicted by Gantt chart 300 corresponding to a checkout request. Each node in call graph 500 represents an operation performed by a microservice of an e-commerce microservice application. For example, node 510 represents the "Sent.hipstershop.CheckoutService.PlaceOrder" operation performed by the front-end microservice of the e-commerce microservice application corresponding to span 430 in textual trace data 400. As illustrated by FIG. 5, each node in call graph 500 includes several attributes of the operation represented by that node, such as duration, CPU utilization, memory utilization, etc. For example, node 530 indicates that the duration of the "Sent.hipstershop.CartService.GetCart" operation performed by the checkout microservice of the e-commerce microservice application was 4.8 milliseconds (ms). Figure 5 further illustrates that, within the end-to-end execution flow, several microservices of the e-commerce microservice application may perform multiple operations when processing a checkout request. For example, the "Sent.hipstershop.CheckoutService.PlaceOrder" and "Sent.hipstershop.CurrencyService.GetSupportedCurrencies" operations, represented by nodes 510 and 520, respectively, are performed by front-end microservices of the e-commerce microservice application.

[0022] 7 is an exemplary, non-limiting, directed acyclic graph (DAG) depicting a trace, according to one or more embodiments described herein. In FIG. 7, a trace is depicted as a DAG formed by named spans and following the causal ordering of the named spans. Specifically, the DAG in FIG. 7 depicts a trace of requests to add items to a shopping cart (T 1 ) and the trace of the checkout request (T 2 ) as shown in Figure 7. 1 DAG 700 includes three named spans from the list of span names 600 illustrated by FIG. 6. These three named spans include: (i) frontend-additem span A, (ii) catalog-getproduct span B, and (iii) cart-additem span C. DAG 700 includes trace T 1 Figure 7 depicts the causal ordering of three named spans, including the frontend-additem span A, followed by the catalog-getproduct span B, followed by the cart-additem span C. 2 DAG 750 further shows that trace T includes four named spans from list of span names 600. These four named spans include: (i) frontend-checkout span E, (ii) checkout-placeorder span F, (iii) catalog-getproduct span B, and (iv) payment-charge span G. DAG 750 also shows that trace T 2as frontend-checkout span E, followed by checkout-placeorder span F, followed by catalog-getproduct span B, and payment-charge span G. Collectively, the combination of named spans and their causal order can have strong implications for the program logic as designed by the developer. If a system administrator observes that the trace deviates from its normal shape, the microservice application may be in an abnormal status. Therefore, learning trace patterns can be used to determine the application status.

[0023] In Fig. 7, we consider the overview of the trace at time t and trace T 1 and T 2 Suppose that two variations of the traces 800 and 900 in Figures 8 and 9 can be observed, respectively. At first glance, the traces 800 and 900 in Figure 8 appear to be

[0024]

number

[0025] is the trace T drawn on DAG700 in Figure 7. 1 Compared to the original, the extra currency-convert span D makes it questionable. However, the trace

[0026]

number

[0027] Adding a currency-convert span D to the trace

[0028]

number

[0029] In the execution flow of a trace, the latency of the preceding spans (i.e., frontend-additem span A and catalog-getproduct span B) can be correlated. For example, high latency in frontend-additem span A can be caused by a rare imported product being added to the shopping cart. As shown by comparing Figure 7 and Figure 8, the latency of frontend-additem span A is higher in trace T than in the preceding spans (i.e., frontend-additem span A and catalog-getproduct span B). 1 Trace from 4ms in

[0030]

number

[0031] The time increased to 60 ms in the trace.

[0032]

number

[0033] This relatively high latency of the frontend-additem span A in the trace may involve converting the price of a rare imported item into the domestic currency.

[0034]

number

[0035] This factor can be called intra-requirement dependency.

[0036] Figure 9 illustrates another factor that can affect the execution flow of traces. With a model trained on a sequence of individual traces, the training data used to train the model is the traces depicted in DAG 900 of Figure 9.

[0037]

number

[0038] Trace T depicted in Figure 7 for sample DAG750 2 If it contains, the model is a trace

[0039]

number

[0040] is reported as an anomalous trace.

[0041]

number

[0042] is normal, and deviations occur due to previous requests for adding items to the shopping cart. For example, it is normal to fetch product details (e.g., price, color, size) twice from the catalog microservice to fulfill a checkout request if the previous request was to add two items to the shopping cart. In other words, the trace

[0043]

number

[0044] Catalog-GetProduct Span B Number of Traces

[0045]

number

[0046] It depends on the number of additional item requests that occur before the checkout request (e.g., trace T 1 This is due to inter-requirement dependencies.

[0047] As discussed above, microservices architectures are attractive for designing applications to be deployed in hybrid cloud environments because loosely coupled components (e.g., microservices) offer a variety of advantages over monolithic service architectures. Example advantages include better scalability, flexibility, maintainability, and accelerated developer productivity. Anomaly detection and diagnosis can be a key aspect of building reliable, dependable microservices applications. For example, avoiding service-level agreement (SLA) violations and their potential corresponding impact involves effectively and efficiently detecting anomalies so that IT professionals (e.g., DevOps and / or SREs) can take further action to resolve the underlying issue in a timely manner.

[0048] As a core component of modern cloud-native services, distributed tracing provides an excellent source for recognizing anomalous behavior by learning the characteristics of healthy microservice applications. Learning from the vast amount of trace data holistically associated with a microservice application can be challenging due to the heterogeneous mix of text and numeric values. Further challenges can arise due to complex dependencies within a single trace and across multiple traces. Existing techniques typically do not consider such spatial and temporal dependencies between microservices, which can lead to an increase in false positives.

[0049] Embodiments of the present disclosure address the aforementioned challenges by providing a framework for characterizing intra- and inter-request dependencies in trace data from healthy microservice applications in order to perform anomaly detection and identify a set of root causes. To that end, the following disclosure provides techniques that facilitate anomaly detection and / or RCA in microservice applications based on request contextual data using machine learning models, such as neural networks. To that end, embodiments of the present disclosure build a machine learning model that can jointly learn intra- and inter-request dependencies from healthy trace data to effectively detect anomalous behavior by microservice applications.

[0050] One aspect of the disclosed framework involves an embedding algorithm that encodes textual trace events in a low-dimensional vector space using a neural language model applied to a graph traversal of the trace. Word embedding techniques generally convert text into low-dimensional vectors to create representations of words in the low-dimensional space. Such representations can capture the meaning of a word by considering its relationship to other words in the context (e.g., a sentence), based on the hypothesis that words that are close in context are statistically more dependent on each other. Natural language processing (NLP) embedding models (e.g., word2vec, GloVe, and BERT) can learn syntactic and semantic relationships between words in various corpora. Spans in microservice application traces can convey meaning like words in natural language text. Therefore, embodiments of the disclosed framework can implement an embedding mechanism for named spans in textual trace data.

[0051] Another aspect of the disclosed framework involves a deep learning model for anomaly detection and diagnosis that incorporates intra- and inter-request dependencies via a convolutional long short-term memory (ConvLSTM) encoder. An embodiment of the deep learning model produces a reconstructed input via a ConvLSTM decoder. In an embodiment, the reconstruction loss between the deep learning model's input and the reconstructed input output by the deep learning model encompasses characteristics in healthy application trace data, leading to a level of interpretability for anomaly detection and diagnosis.

[0052] ConvLSTM networks facilitate time series analysis and sequence modeling tasks, such as precipitation nowcasting and video analysis. ConvLSTM combines the strengths of two neural networks: convolutional neural networks (CNNs) and long short-term memory (LSTMs). As a result, ConvLSTMs not only learn temporal correlations within input time series, but also spatial correlations across multiple time series. This spatio-temporal analysis enabled by ConvLSTMs is applicable to correlations across spans within a single trace (e.g., intra-request dependencies) as well as correlations across multiple traces (e.g., inter-request dependencies).

[0053] FIG. 10 is a diagram of an example, non-limiting architecture 1000 implementing a framework for detecting anomalous behavior by a microservices application, according to one or more embodiments described herein. The architecture 1000 includes a model training stage 1010 and an anomaly detection and diagnosis stage 1050. The model training stage 1010 can build two machine learning models to facilitate capturing intra- and inter-request dependencies in distributed trace data of a healthy microservices application 1015. As shown by FIG. 10 , the two machine learning models built by the model training stage 1010 include an embedding model 1020 and a status model 1030. In the model training stage 1010, an embedding algorithm 1025 (span2vec) can learn representations of textual span names in a low-dimensional space using a neural language model. Using the vectorized span names and other span attributes, a ConvLSTM-based autoencoder status model 1030 can be trained in a model training stage 1010 using a ConvLSTM-based autoencoder algorithm 1035. In an anomaly detection and diagnosis stage 1050, the embedding model 1020 and the status model 1030 can be used in conjunction with a diagnostic process 1040 to detect anomalies and / or provide descriptive diagnostic results.

[0054] u∈U represents the span name,

[0055]

number

[0056] Let denote the numeric attribute of a span, where U denotes the complete set of span names for the application, and

[0057]

number

[0058] is an f-dimensional vector. The trace T of n spans is

[0059]

number

[0060] and the text portion of the trace T can be expressed using the function defined by Equation 1:

[0061]

number

[0062] The problem is that historical data of m traces collected from healthy applications, i.e.

[0063]

number

[0064] and a set of span names, U, the objective can be defined as: (i) T m (ii) detecting subsequent anomalous traces, and (ii) identifying spans in the anomalous traces that point to the root cause.

[0065] Embodiments of the embedding model output by the span2vec embedding algorithm (span2vec embedding model) include:

[0066]

number

[0067] It involves learning a vectorized representation of the span names of text in a sequence, and in NLP terminology, we can think of a span sequence as a "sentence" and a span within a sequence as a "word." More specifically,

[0068]

number

[0069] Given a set of text representations (e.g., Equation 1), the target of the span2vec embedding model is a d-dimensional vector such that for a span u∈U, related spans are nearby in the vector space.

[0070]

number

[0071] An example, non-limiting, high-level conceptual overview of this target of the span2vec embedding model is illustrated in Figure 11. As shown in Figure 11, an embodiment of the span2vec embedding model 1110 can receive as input a DAG 1120 depicting a trace of a microservice application and output a d-dimensional vector that embeds each span of the trace.

[0072] Building a language model generally involves two inputs: a corpus and a vocabulary V. While it is obvious that V = U, the span sequence defined by Equation 1 cannot be used literally as a corpus. The span sequence of a trace depicted by DAG 1200 in Figure 12 can be ordered by the individual creation times of each span comprising the trace. In this example, the span sequence of a trace depicted by DAG 1200 can be (A, B, C, D, E, F). However, this span sequence provides an incomplete representation of the span relationships in DAG 1200. For example, the parent-child connection between span D and span F is missing in this span sequence. Furthermore, because the two branches of the trace that fork at B can continue in parallel, the ordering may be indeterminate, which can lead to nondeterministic results.

[0073] An embodiment of the span2vec embedding model can solve this problem using a graph traversal algorithm that considers both sequential and parallel relationships between spans in the trace DAG. By way of example, the span2vec embedding model can utilize a combination of improved depth-first search (DFS) and breadth-first search (BFS) graph traversal algorithms. In this example, the reverse DFS graph traversal algorithm can initialize the span sequence of the trace described by DAG 1200 in FIG. 12 with leaf nodes (e.g., leaf nodes E and D of DAG 1200) and recursively add their parent nodes to the span sequence until the reverse DFS graph traversal algorithm encounters a root node (e.g., root node A). The operation of the reverse DFS graph traversal algorithm is represented by dashed arrows in FIG. 12. FIG. 12 illustrates an example, non-limiting output 1210 of the reverse DFS graph traversal algorithm applied to the trace described by DAG 1200. Next, for each node in the DAG 1200 that has three or more child nodes, the one-hop BFS graph traversal algorithm can initialize a new span sequence with that node as its first element and concatenate all of that node's child nodes into the new span sequence. The nodes in the DAG 1200 involved in this application of the one-hop BFS graph traversal algorithm are represented by the shaded vertices in Figure 12. Figure 12 further illustrates an example, non-limiting output 1220 of the one-hop BFS graph traversal algorithm applied to the trace traced by the DAG 1200.

[0074] 13 is a diagram of an example, non-limiting application of a multi-path DFS (MPDFS) and one-hop BFS graph traversal algorithm to traces of a microservice application, according to one or more embodiments described herein. FIG. 14 is a diagram of another example, non-limiting application of a MPDFS and one-hop BFS graph traversal algorithm to traces of a microservice application, according to one or more embodiments described herein. An embedding algorithm (e.g., embedding algorithm 1025 of FIG. 10) may be implemented using an input trace set

[0075]

number

[0076] We build a span vocabulary from the input trace set, as illustrated in Figures 13 and 14.

[0077]

number

[0078] A sequence can be generated for each trace containing the span context. The embedding algorithm can then construct a DAG (or call graph) to create a span context sequence as input, creating a textual embedding of the trace data. Each node in the DAG can represent a span (or an operation on a microservice with one or more characteristics, such as duration). An edge between two nodes in the DAG indicates a call relationship between the two operations represented by the two nodes. Based on the DAG, the embedding algorithm can generate a sequence of spans for each node in the DAG, which can be analogous to a word in NLP. To maintain logical relationships (e.g., semantics in NLP), an exemplary embedding algorithm can traverse the DAG using two strategies, the MPDFS and one-hop BFS graph traversal algorithms shown in Figures 13-14. The embedding algorithm can feed the span context sequence into a skip-gram network and extract hidden layers as embedding weights.

[0079] In an embodiment, an embedding matrix for a vocabulary V for building a language model can be generated using Algorithm 1:

[0080] [Table 1]

[0081] As shown above, Algorithm 1 computes the embedding matrix for a vocabulary V as a set of traces

[0082]

number

[0083] The output can be based on the input containing , a vocabulary of span names V, and two embedding parameters d and w. In one loop, Algorithm 1 generates a trace set

[0084]

number

[0085] It can iterate over all traces in T. For each trace T, Algorithm 1 can first construct a DAG, where vertices are span names and edges are parent-child references between spans (e.g., line 2 of Algorithm 1). Algorithm 1 can then perform inverse DFS and one-hop BFS graph traversal algorithms on the DAG and add sequences to corpus X (e.g., lines 3-4 of Algorithm 1). In this embodiment, Algorithm 1 can generate an embedding model for span vocabulary V using a neural linguistic toolkit (e.g., genism) (e.g., line 5 of Algorithm 1). The effects of the embedding dimension (d) and the context window size (w) input parameters are evaluated in more detail below with reference to Figures 22-23.

[0086] Having learned the vectorized span representation, the following disclosure is based on the historical trace data.

[0087]

number

[0088] To learn dependencies in traces of a healthy microservice application using the historical trace data, we first describe the input data preparation and application status model (e.g., status model 1030 in Figure 10).

[0089]

number

[0090] can be divided into rows of trace frames, such as the example, non-limiting trace frame 1500 of FIG. 15. FIG. 15 shows that trace frame 1500 can be formed by l traces, and all traces can be padded or truncated to have n spans. After embedding (e.g., with the span2vec embedding model), the text span names can be represented by d-dimensional vectors and concatenated with other numeric attributes. Thus, a representation of a trace with n spans would look like this:

[0091]

number

[0092] In Equation 2, f is the dimension of the numeric attribute of the span. Given Equation 2, the trace frame x can be written as defined by Equation 3:

[0093]

number

[0094] According to Equation 3, the trace frame 1500 is the historical trace data that can characterize the status of the microservice application.

[0095]

number

[0096] Each row of trace frame 1500 represents a trace of a microservice application, and each column of trace frame 1500 represents a span of the trace. Each element of the matrix formed by trace frame 1500 is a d-dimensional vector embedding of a given span. To the extent that trace frame 1500 contains l traces, the historical trace data

[0097]

number

[0098] From, in total

[0099]

number

[0100] A trace frame can be formed.

[0101] Embodiments of the disclosed framework utilize historical trace data to learn intra- and inter-request dependencies.

[0102]

number

[0103] The ConvLSTM-based autoencoder model further involves constructing a ConvLSTM-based autoencoder model using training data including a sequence of λ-trace frames, X=(x1, x2, , x λ ) as input. The ConvLSTM-based autoencoder model encodes a sequence of λ trace frames into a reduced dimension and reconstructs this sequence with an objective function that minimizes the reconstruction loss between X and X'.

[0104]

number

[0105] The building block of a ConvLSTM-based autoencoder model can be a ConvLSTM unit 1600, as shown in Figure 16. The ConvLSTM unit 1600 includes a CNN layer 1610 and an LSTM layer 1620. The input to the ConvLSTM unit 1600 can be a trace frame x and a hidden state from a previous trace frame in a sequence of λ trace frames. A ConvLSTM-based autoencoder model 1700 can be constructed by chaining λ ConvLSTM units together in the encoding and decoding stages, as illustrated in Figure 17. At each time step (i.e., at each ConvLSTM unit of the ConvLSTM-based autoencoder model 1700), the convolutional layer can capture both intra- and inter-request dependencies by preserving the spatial relationship between a span and its neighbors (these are spans of the same trace and preceding or subsequent traces). The LSTM layer of the ConvLSTM-based autoencoder model 1700 is able to learn temporal dependencies in a sequence of λ trace frames. The effect of the time step length λ is discussed in more detail below with respect to Figures 24-25.

[0106] Although FIG. 17 illustrates the ConvLSTM-based autoencoder model 1700 as including a single pair of encoding and decoding ConvLSTM units, multiple pairs of encoding and decoding ConvLSTM units can be stacked to implement a ConvLSTM-based autoencoder model according to embodiments of the present disclosure. FIG. 19 is a diagram of an exemplary, non-limiting ConvLSTM-based autoencoder model 1900 implemented by stacking multiple pairs of encoding and decoding ConvLSTM units according to one or more embodiments described herein. In FIG. 19, the ConvLSTM-based autoencoder model 1900 includes a time step length λ of 10 trace frames. As discussed above, each trace frame providing input to the ConvLSTM-based autoencoder model includes l traces, each containing n spans. In FIG. 19, l and n are each set to 256. The embedding size d of the d-dimensional vector filling each span is 30.

[0107] As shown in Figure 19, the ConvLSTM-based autoencoder model 1900 includes an input layer 1910, a spatial encoder 1920, a temporal encoder 1930, a bottleneck state layer 1940, a temporal decoder 1950, a spatial decoder 1960, and an output layer 1970. The spatial encoder 1920 can capture spatial dependencies between microservices in a microservice application, creating textual trace data embedded in the trace frame provided to the input layer 1910. The temporal encoder 1930 can capture temporal dependencies between microservices in a microservice application. The dimensionality of the trace frame provided to the input layer 1910 can be reduced by the bottleneck state layer 1940. The temporal decoder 1950 can reconstruct the temporal dependencies between microservices in a microservice application. The spatial decoder 1960 can reconstruct the spatial dependencies between microservices in a microservice application. The reconstructed trace frame, with the same dimensions as the trace frame provided to the input layer 1910, is provided to the output layer 1970.

[0108] Equation 4 expresses the input sequence X at time t. t and the output sequence for training a ConvLSTM-based autoencoder model.

[0109]

number

[0110] We define a loss function to measure the reconstruction error between

[0111]

number

[0112] According to Equation 4, e(·) can be calculated as the error between two trace frames. Given Equation 2 and Equation 3, e(·) can be written as:

[0113]

number

[0114] According to Equation 5, l is the number of traces in one frame, n is the number of spans in one trace, and ||·||2 denotes the L2 norm.

[0115] After training, the span2vec embedding and status model are then used to measure the historical trace data.

[0116]

number

[0117] In test data containing λ, we can detect anomalies and / or diagnose root causes through reconstruction loss, as defined by Equation 5. First, the span2vec embedding model can convert the text span names in the test data into vectors, and prepare the test data in the format of trace frames, as defined by Equations 2 and 3. In practice, the trace frames can be applied to a sliding window of λ frames with stride S = 1 (i.e., the number of frames skipped as the sliding window advances). The λ × l traces in the sliding window can form a batch of inputs to the status model, as defined in Equation 5, to generate the reconstruction loss. The reconstruction loss value for traces classified as "normal" is [H u ,H l ]. For traces with reconstruction loss values outside this range, the reconstruction loss value should be classified as "anomalous". For anomaly detection, H u and Hl There are various ways to define H. u and H l are H u =95 th Percentile {L training} and H l =5 th Percentile {L training}, where L training is the loss on the training data.

[0118] Based on the reconstruction loss calculated for the ConvLSTM-based autoencoder model, a diagnostic process (e.g., diagnostic process 1040 in Figure 10) identifies the trace frame with the largest contribution to the total loss of the sequence of trace frames.

[0119]

number

[0120] can be chosen greedily, where X t is the sequence of trace frames that are classified as anomalous (i.e., L t >H u ∨L t <H l ). Then, the selected trace frame e * In, the top-k traces are recursively selected, and spans can be recursively selected in the order they were added, as defined in Equation 5. In this way, the search space for root causes can be guided and narrowed.

[0121] The following disclosure describes an example demonstration setup for an implementation of the disclosed framework for anomaly detection and / or RCA in microservices applications, hereinafter referred to as DeepTrace. The example demonstration utilized three microservices applications deployed on a five-node Kubernetes cluster, including a ride-sharing application, an e-commerce web application, and a social networking web application.

[0122] The rideshare application contains a web user interface (UI) that serves user requests and makes RPC calls to five backend microservices. The rideshare application allows two types of user requests: (i) browsing the UI and (ii) requesting a ride. To request a ride, a user of the rideshare application selects a destination address, and the rideshare application dispatches a driver. Traces are generated for all microservices that comprise the rideshare application.

[0123] The e-commerce web application has a more complex architecture than the ride-share application, involving 12 microservices and seven programming languages. The e-commerce web application allows six types of user requests, such as adding an item to a cart, checking out, and setting a currency. Each request type involves a subset of microservices and results in a different configuration of traces. Three microservices in the e-commerce web application do not produce traces because they are not equipped with tracing libraries.

[0124] The social network web application contains 36 microservices. Unlike the ride-sharing and e-commerce web applications, most of the microservices in the social network web application have their own dedicated data stores, including memcached for caching or MongoDB for persistent storage. In the example demonstration, seven types of requests were considered, including registering a new user, authenticating login, composing a message, and browsing the timeline. Traces are generated for all microservices in the social network web application.

[0125] In the example demonstration, we used an open-loop workload generator (e.g., LOCUST2019) to emulate user behavior to drive requests to three microservice applications. Training data for the open-loop workload generator was collected with varying numbers of emulated users, assuming the microservice applications were healthy. This training data included 50K, 57K, and 76K traces from a ride-sharing application, an e-commerce web application, and a social networking web application, respectively.

[0126] The test data for the example demonstration was generated using injected anomalies and contains 25K traces. The injected anomalies utilized in the example demonstration are listed in Table 1 below. Table 1 lists how the injected anomalies were customized for each microservice application to induce a 60-second anomaly.

[0127] [Table 2]

[0128] In an IA-0 launch anomaly, the microservice application is healthy and receives a burst of user requests, but the SLA is not violated. Therefore, the IA-0 launch anomaly is used to test the robustness of the ConvLSTM-based autoencoder model. In an IA-1 launch anomaly, the microservices in the microservice application experience increased latency and respond slowly to requests. In an IA-2 launch anomaly, the microservices in the microservice application are disabled, causing failures for some user requests. Both IA-1 and IA-2 launch anomalies affect the performance of the microservice application in that some user requests return with errors or respond slowly due to the weakened microservices.

[0129] In an IA-3 injection anomaly, communication between two microservices in a microservice application is affected by some network incident, resulting in high packet loss rates. An IA-3 injection anomaly has the combined effect of performance and execution anomalies. An IA-4 injection anomaly occurs when some caching services or replicas in a microservice application are corrupted, resulting in atypical traces. In an IA-5 injection anomaly, a microservice application is subjected to a surge of repeated requests (e.g., checkouts using counterfeit credit cards), a form of DDoS attack, disrupting normal user request patterns. Both IA-4 and IA-5 injection anomalies result in atypical trace patterns caused by abnormal request patterns or execution paths.

[0130] In the example demonstration, DeepTrace (i.e., an implementation of the disclosed framework) in a microservices application was compared with several other methods. All machine learning models in the example demonstration were implemented in Tensorflow, and the graph traversal with span2vec embedding algorithm was implemented in Go. The example demonstration compared DeepTrace with a first set of methods, including usage-based methods and tr-PCA. The usage-based method monitors the CPU and memory utilization of cluster nodes. The usage-based method signals an alarm when utilization exceeds a certain threshold (e.g., mean ± 3 deviations). Another method in the first set, tr-PCA, logs the durations of spans in a trace as an ordered list and applies principal component analysis to find correlations between spans. The tr-PCA method finds correlations between spans through clustering of normal data using Mahalanobis distance. If the Mahalanobis distance from the test data to the normal class exceeds a threshold, the tr-PCA method identifies the trace as an anomaly.

[0131] The example demonstration also compared DeepTrace to a set of deep learning-based methods, including tr-ED and tr-LED. Both methods in this set use the same input data as tr-PCA. The tr-ED method trains an autoencoder stacked with multiple layers of dense networks, while the tr-LED method's autoencoder uses an LSTM network to capture intra-request dependencies.

[0132] The example demonstration also compares DeepTrace with two variants of DeepTrace to justify the effectiveness of the span2vec embedding algorithm and the ConvLSTM-based autoencoder model. The first variant of DeepTrace, DT-no-s2v, uses an embedding layer in Tensorflow as a substitute for the span2vec embedding algorithm. The second variant of DeepTrace, DT-LSTM, replaces the ConvLSTM unit in the ConvLSTM-based autoencoder model with an LSTM unit.

[0133] The following disclosure describes the results of the example demonstration discussed above. The example demonstration uses precision, recall, and F-score to measure the performance of anomaly detection. Because anomalies are introduced one at a time, the example demonstration calculates metrics for each anomaly separately and the average F-score for these anomalies.

[0134] Depending on the input / output shape, anomaly detection methods label trace data at various granularities. DeepTrace and the first variant of DeepTrace (DT-no-s2v) obtain a sequence of λ trace frames, each formed by l traces, so λ × l traces are considered together (i.e., all anomalous or none anomalous). Two baselines that utilize LSTMs, namely tr-LED and the second variant of DeepTrace (DT-LSTM), label sequences of traces with the same status. The status is reported separately by the remaining methods tested by example demonstration: the usage-based method, the tr-PCA method, and the tr-ED method. Regardless of the granularity, metrics are calculated consistently across these methods for all traces.

[0135] The example demonstration presents results for an implementation of the disclosed framework and two of its variants for d = 64, w = 5, λ = 10, and l = 16. For other baselines, the example demonstration explores their parameter space and reports the best results. FIG. 20 illustrates Table 2000, which presents the results of the example demonstration below, with the highest (lowest) score in each column highlighted by a shaded background (underlined). Because the IA-0 input anomaly has no anomalous data, the example demonstration reports accuracy (last column in Table 2000). This is because precision, recall, and F-score are meaningless in this case (the number of true positives is 0). The results presented in Table 2000 are emulated results the example demonstration obtained from randomly generated data. Those skilled in the art will appreciate that aspects of the results presented in table 2000 may be tuned to different data sets in non-emulated implementations.

[0136] From Table 2000, several observations can be made. For example, two non-deep learning baselines (i.e., utilization-based and tr-PCA methods) cannot effectively handle the anomaly detection task. Table 2000 shows that compared to the set of deep learning-based methods, traditional methods capture less complex relationships in the traces. Intuitively, when resources are underutilized, utilization-based methods may have difficulty detecting anomalies.

[0137] Another observation that can be made from Table 2000 is that a set of deep learning-based methods that incorporate temporal relationships via LSTM networks perform better than dense networks. This observation suggests that intra-request dependencies within a single trace should be considered. Specifically, although trained on the same input data, the tr-LED method outperforms the tr-ED method in most columns. This may be especially true for IA-3, IA-4, and IA-5 injection anomalies, where the anomalies are due to irregular execution paths within a single trace.

[0138] Another observation that can be made from Table 2000 is that DeepTrace, an implementation of the disclosed framework, has the overall best performance among all evaluated methods, with all F-scores >0.8. Two improved versions of DeepTrace also achieve good performance for ride-sharing and e-commerce web applications. However, the average F-scores of the two variants decrease for the social network web application, which has a more complex architecture. For example, for IA-5 injection anomalies, the two variants miss many anomaly traces for the social network web application (recall = 0.37 and 0.59), because intra-request dependencies are hardly affected by repeated adversarial requests. In contrast, DeepTrace is able to report most of the anomalies (recall = 0.7), confirming the importance of capturing both inter- and intra-request dependencies and the effectiveness of the span2vec embedding algorithm.

[0139] To further understand how anomalous traces are labeled, FIG. 21 illustrates an example, non-limiting graph 2100 depicting DeepTrace's reconstruction loss calculated for a social network web application over time. In graph 2100, the horizontal line indicates the upper bound (H u ) and lower bound (Hl ) defines the normal range of reconstruction loss between. Additionally, the grey bands in graph 2100 indicate the individual durations of injection anomalies.

[0140] Graph 2100 shows that DeepTrace successfully identifies the social network web application as healthy during workload fluctuations (IA-0) because the loss is within the normal range. Graph 2100 also shows four spikes and one dip that deviate from the normal range for each input anomaly (i.e., input anomalies IA-1 through IA-5). It is noteworthy that graph 2100 shows that the reconstruction loss for the IA-5 input anomaly is below the lower bound, which is different from the other input anomalies. Because the user registration trace is shorter than most of the other requests, a large proportion of the zero-padded vectors for the registration trace may have a reconstruction loss below the threshold. This confirms that it may be beneficial to set a lower bound on the normal range for reconstruction loss in anomaly detection methods. Another observation is that for each input anomaly, the spikes and dips are skewed to the left on the time axis. Quantitatively, these deviations can lead to false positives and false negatives. In some cases, this discrepancy may be due to DeepTrace classifying status at the granularity of the sequence of trace frames, so that normal traces in abnormal trace frames at the onset of a spike may be incorrectly labeled as abnormal. However, most of the traces are correctly labeled.

[0141] The example demonstration investigated how changes to the parameters of DeepTrace, an implementation of the disclosed framework, affect its performance in anomaly detection. To this end, the example demonstration experimented on IA-3 and IA-5 injected anomalies, varying their reconstruction losses differently in graph 2100 of FIG. 21. Graph 2100 depicts results for a social network web application, although results are similar for other applications used in the example demonstration.

[0142] The output of the span2vec embedding algorithm may depend on two parameters: the embedding dimension (d) and the size of the context window (w). Figures 22-23 illustrate example, non-limiting graphs depicting the effect of varying d and w on the F-score. Graph 2200 in Figure 22 shows that increasing the embedding dimension (d) or window size (w) slightly improves the F-score for the IA-3 launch anomaly, indicating that the results are not sensitive to the two parameters. In contrast, graph 2300 in Figure 23 shows that increasing d = 4 to d = 32 for the IA-5 launch anomaly improves performance by more than 10%, but the gain is less noticeable or disappears after d = 32 due to overfitting. Additionally, note that increasing the window size (w) may degrade performance for the IA-5 launch anomaly. Recall that in the IA-5 launch anomaly, the anomaly is caused by a surge of repeated requests, which breaks inter-request dependencies. A larger context window size takes into account more of the relevance of the spans, and therefore considers further additions to the loss to bring the loss above the lower threshold, resulting in a decrease in recall and F-score. Therefore, in some cases, it may be beneficial to choose an embedding dimension (d) larger than the vocabulary size, and a smaller window size (w), such as 5.

[0143] 24-25 illustrate example, non-limiting graphs depicting the effect of increasing the time step length, λ, that embodiments of the disclosed framework use as the number of trace frames input to a ConvLSTM-based autoencoder model. In particular, graphs 2400 and 2500 depict the effect of varying the time step length, λ, for IA-3 and IA-5 input anomalies, respectively. As shown by graphs 2400 and 2500, a longer time step length, λ, can increase recall but decrease precision. The trend is similar for both input anomalies. Graphs 2400 and 2500 further demonstrate that a median time step (λ=10) can produce overall improved performance in terms of F-score.

[0144] Figure 26 illustrates table 2600 depicting the diagnostic results for a social network web application obtained by an example demonstration. As discussed above, once a sequence of trace frames in a sliding window is classified as an anomaly, the diagnostic process can first select the frame with the largest contribution to the reconstruction loss. The diagnostic process can then recursively examine that trace and its spans to find the faulty microservice. The top-2 selected traces presented in column 2 of table 2600 are strongly related to the anomaly type. For certain anomalies, such as the IA-1 injection anomaly, after recognizing the suspect trace, it may be simple to identify the faulty microservice from the visualized Gantt chart of these traces. For other anomalies, spans from the selected traces can be examined in the order in which the spans were added to the reconstruction loss, allowing for sifting through the microservices involved in these spans. Column 3 of column 2600 presents the first two or three microservices. The microservices identified as faulty are highlighted in bold in column 3 of table 2600. In an example demonstration, it was observed that an implementation of the disclosed framework can include the failing microservice in the list for every anomaly.

[0145] FIG. 27 illustrates an exemplary, non-limiting graph 2700 depicting the relative diagnostic performance for all three microservice applications utilized in the example demonstration. Graph 2700 depicts the relative diagnostic performance in terms of Acc@k for all anomalies. As used herein, Acc@k refers to the probability that the actual faulty microservice is included in the top-k reported microservices. Some baseline methods perform poorly in anomaly detection in the example demonstration, which impacts their diagnostic results. Therefore, graph 2700 presents only the diagnostic performance of the tr-LED method (i.e., the best baseline method, excluding the two variants of DeepTrace), and this also applies to the diagnostic methods. As illustrated by FIG. 27, graph 2700 depicts diagnostic performance results for k = 1, 3. Comparing DeepTrace(@3) with DeepTrace(@1) in graph 2700, it can be observed that the diagnostic accuracy of DeepTrace improves as k increases. From graph 2700, we can also observe that DeepTrace outperforms the tr-LED method for all three microservice applications.

[0146] FIG. 28 is a flow diagram of an example, non-limiting, computer-implemented method 2800 for facilitating anomaly detection and root cause analysis using distributed trace data, according to one or more embodiments described herein. Repeated descriptions of similar elements employed in other embodiments described herein are omitted for brevity. At 2802, the computer-implemented method 2800 may include generating, by a system operatively coupled to a processor (e.g., using the pre-processing component 140), trace frames including vectorized representations of textual trace data produced by microservices of a microservice application. In an embodiment, the system may generate the trace frames using a machine learning model trained using a graph traversal algorithm, a word embedding technique, or a combination thereof.

[0147] At 2804, the computer-implemented method 2800 may include, by the system (e.g., using the monitor component 150), identifying a state of the microservice application using the trace frames. In an embodiment, the system may identify the state of the microservice application by comparing the trace frames to reconstructed trace frames generated by a machine learning model. In an embodiment, the machine learning model may capture spatial and temporal dependencies between microservices of the microservice application when operating under normal conditions. In an embodiment, the system may identify the state of the microservice application by comparing a sequence of trace frames to a sequence of reconstructed trace frames generated by the machine learning model. The sequence of trace frames may include the trace frames. In an embodiment, the machine learning model may be a ConvLSTM-based autoencoder model. In an embodiment, the computer-implemented method 2800 may further include, by the system, performing RCA by evaluating the contribution of each microservice to a given reconstruction loss.

[0148] To provide context for various aspects of the disclosed subject matter, Figure 29 and the following discussion are intended to provide a general description of a suitable environment in which various aspects of the disclosed subject matter may be implemented. Figure 29 illustrates a suitable operating environment 2900 for implementing various aspects of the present disclosure, which may also include a computer 2912. The computer 2912 may also include a processing unit 2914, a system memory 2916, and a system bus 2918. The system bus 2918 couples system components, including but not limited to the system memory 2916, to the processing unit 2914. The processing unit 2914 may be any of a variety of available processors. Dual microprocessors and other multiprocessor architectures may also be employed as the processing unit 2914. The system bus 2918 can be any of several types of bus structures, including a memory bus or memory controller, a peripheral or external bus, or a local bus using any of a variety of available bus architectures, including, but not limited to, Industrial Standard Architecture (ISA), MicroChannel Architecture (MSA), Enhanced ISA (EISA), Intelligent Electronic Device (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus (USB), Advanced Graphics Port (AGP), Firewire (IEEE 1094), and Small Computer System Interface (SCSI), or a combination thereof. The system memory 2916 can also include volatile memory 2920 and nonvolatile memory 2922. The basic input / output system (BIOS), containing the basic routines for transferring information between elements within the computer 2912, such as during startup, is stored in the nonvolatile memory 2922.By way of example, and not limitation, non-volatile memory 2922 may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) such as ferroelectric RAM (FeRAM). Volatile memory 2920 may also include random access memory (RAM) acting as external cache memory. By way of example, and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM.

[0149] The computer 2912 may also include removable / non-removable, volatile / non-volatile computer storage media. For example, FIG. 29 illustrates disk storage 2924. Disk storage 2924 may also include devices such as, but not limited to, a magnetic disk drive, a floppy disk drive, a tape drive, a Jaz drive, a Zip drive, an LS-100 drive, a flash memory card, or a memory stick. Disk storage 2924 may also include a storage medium separately or in combination with other storage media, including, but not limited to, an optical disk drive such as a compact disc read-only memory (CD-ROM), a CD-writeable drive (CD-R drive), a CD-rewriteable drive (CD-RW drive), or a digital versatile disc read-only memory (DVD-ROM). A removable or non-removable interface, such as interface 2926, is typically used to facilitate connection of the disk storage 2924 to the system bus 2918. FIG. 29 also illustrates software that acts as an intermediary between a user and the basic computer resources described within the preferred operating environment 2900. Such software may include, for example, an operating system 2928. The operating system 2928, which may be stored on disk storage 2924, acts to control and allocate resources of the computer 2912. System applications 2930 take advantage of the management of resources by the operating system 2928 through program modules 2932 and program data 2934, which are stored, for example, in either system memory 2916 or on disk storage 2924. It is to be understood that the present disclosure may be implemented with various operating systems or combinations of operating systems. A user enters commands or information into the computer 2912 through input devices 2936.Input devices 2936 include, but are not limited to, pointing devices such as mice, trackballs, styluses, touch pads, keyboards, microphones, joysticks, game pads, satellite dishes, scanners, TV tuner cards, digital cameras, digital video cameras, web cameras, etc. These and other input devices connect to the processing unit 2914 through the system bus 2918 via interface ports 2938. Interface ports 2938 include, for example, serial ports, parallel ports, game ports, and universal serial buses (USB). Output devices 2940 use some of the same types of ports as the input devices 2936. Thus, for example, a USB port can be used to provide input to the computer 2912 and to output information from the computer 2912 to the output device 2940. Output adapter 2942 is provided to illustrate that there are some output devices 2940, such as monitors, speakers, and printers, among other output devices 2940, that require special adapters. Output adapters 2942 include, by way of example and not limitation, video cards and sound cards that provide a means of connection between output device(s) 2940 and system bus 2918. It should be noted that other devices and / or systems of devices, such as remote computer(s) 2944, provide both input and output capabilities.

[0150] The computer 2912 can operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 2944. The remote computer 2944 may be a computer, server, router, network PC, workstation, microprocessor-based appliance, peer device, or other common network node, and may include many of the elements typically described for the computer 2912. For simplicity, only a memory storage device 2946 is illustrated with the remote computer 2944. The remote computer 2944 is logically connected to the computer 2912 through a network interface 2948, which is then physically connected via a communication connection 2950. The network interface 2948 encompasses wired and / or wireless communication networks such as a local area network (LAN), a wide area network (WAN), a cellular network, and the like. LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ring, and the like. WAN technologies include, but are not limited to, point-to-point links, circuit-switched networks such as Integrated Services Digital Networks (ISDN) and their variations, packet-switched networks, and Digital Subscriber Lines (DSL). Communications connection 2950 refers to the hardware / software employed to connect network interface 2948 to system bus 2918. While shown internal to computer 2912 for illustrative clarity, communications connection 2950 may be external to computer 2912. Hardware / software for connecting to network interface 2948 may also include, by way of example only, internal and external technologies such as modems, including telephone-grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.

[0151] The present invention may be a system, method, apparatus, or computer program product, or combinations thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the present invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction-execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media further includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or grooved structures having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being ephemeral signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over electrical wires.

[0152] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to an individual computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the individual computing / processing device. The computer readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state configuration data, configuration data for integrated circuits, or procedural or similar programming languages, such as Smalltalk, object-oriented programming languages such as C++, and the "C" programming language. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer as a stand-alone software package, partly on the user's computer and partly on a remote computer, or all on a remote computer or server.In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to individualize the electronic circuitry by utilizing state information of the computer-readable program instructions to perform aspects of the present invention.

[0153] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to make a machine, such that the instructions, executed by a processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart illustrations and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium and implemented by a computer, programmable data processing apparatus, or other device or their respective processors, such that the computer-readable storage medium having the instructions stored thereon comprises an article of manufacture including instructions for implementing aspects of the functions / acts specified in one or more blocks of the flowchart illustrations and / or block diagrams. The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to create a computer-implemented process, causing the computer, other programmable apparatus, or other device to perform a series of operable functions, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0154] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified functions or actions or executes a combination of special-purpose hardware and computer instructions.

[0155] While the subject matter has been described above in the general context of computer-executable instructions for a computer program product executing on one or more computers, those skilled in the art will appreciate that the present disclosure can also be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks and / or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the computer-implemented methods of the present invention can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputing devices, mainframe computers, as well as computers, handheld computing devices (e.g., PDAs, telephones), microprocessor-based or programmable home or business electronic devices, etc. The illustrated aspects can also be practiced in distributed computing environments where tasks are performed by remote processing devices linked through a communications network. However, some, if not all, aspects of the present disclosure can be practiced on stand-alone computers. In a distributed computing environment, program modules can be located in both local and remote memory storage devices. For example, in one or more embodiments, the computer-executable components may execute from a memory that may include or be comprised of one or more distributed memory units. As used herein, the terms "memory" and "memory unit" are interchangeable. Furthermore, one or more embodiments described herein may execute the code of the computer-executable components in a distributed manner, e.g., multiple processors are combined or act in cooperation to execute code from one or more distributed memory units.As used herein, the term "memory" can encompass a single memory or memory unit in one location or multiple memories or memory units in one or more locations.

[0156] As used in this application, the terms “component,” “system,” “platform,” “interface,” etc. can refer to or include a computer-related entity or an entity related to a machine operable with one or more specialized functionalities, or both. The entities disclosed herein can be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, or a computer, or combinations thereof. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process or a thread of execution, or both, and a component can be localized on one computer or distributed between two or more computers, or both. In another example, individual components can execute from various computer-readable media having various data structures stored thereon. Components may communicate through local and / or remote processes, such as through signals carrying one or more data packets (e.g., data from one component may interact with another component within a local system, within a distributed system, or across networks, or a combination thereof, such as the Internet, which interacts with other systems via signals). As another example, a component may be a device with specialized functionality provided by mechanical parts operated by electrical or electronic circuits, operated by a software or firmware application executed by a processor. In such cases, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application.As yet another example, a component can be a device that provides specialized functionality through electronic components without mechanical parts, where the electronic components can include a processor or other means for executing software or firmware that provides at least some of the functionality of the electronic components. In one aspect, a component can emulate an electronic component via a virtual machine, for example, within a cloud computing system.

[0157] Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise stated or clear from the context, "X utilizes A or B" is intended to mean any of the natural inclusive permutations. That is, if X utilizes A, X utilizes B, or X utilizes both A and B, then "X utilizes A or B" is satisfied under any of the foregoing cases. Moreover, as used within the specification of the present subject matter and the accompanying drawings, the articles "a" and "an" should be construed generally to mean "one or more" unless otherwise stated to cover the singular or clear from the context. As used herein, the terms "example," "illustrative," and / or "exemplary" are used to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. Additionally, any aspect or design described herein as "example" and / or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs, and is not intended to exclude equivalent exemplary structures and techniques known to those skilled in the art.

[0158] As employed in the subject specification, the term "processor" can refer to substantially any computing processing unit or device, including, but not limited to, a single-core processor, a single processor with software multithreaded execution capabilities, a multi-core processor, a multi-core processor with software multithreaded execution capabilities, a multi-core processor with hardware multithreading technology, a parallel platform, and a parallel platform with distributed shared memory. Additionally, a processor can refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, a processor can utilize nanoscale architectures, such as, but not limited to, molecular and quantum dot-based transistors, switches, and gates, to optimize space usage or improve the performance of user equipment. A processor can also be implemented as a combination of computing processing units. In this disclosure, terms such as "store," "storage," "data store," "database," and substantially any other information storage component associated with the operation and functionality of a component are utilized to refer to a "memory component" entity embodied in a "memory" or a component that includes a memory. It should be understood that the memory and / or memory components described herein can be either volatile memory or non-volatile memory, or can include both volatile and non-volatile memory.By way of example, and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memory may include, for example, RAM that can act as external cache memory. By way of example, and not limitation, RAM is available in many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the disclosed memory components of the systems or computer-implemented methods herein are intended to include these and any other suitable types of memory, but are not intended to be limited to including such.

[0159] What has been described above includes merely exemplary systems and computer-implemented methods. Of course, for purposes of describing this disclosure, it is not possible to describe every conceivable combination of components or computer-implemented methods, but one of ordinary skill in the art will recognize that many further combinations and permutations of the present disclosure are possible. Furthermore, to the extent that the terms "includes," "has," "possesses," and the like are used in the detailed description, claims, appendices, and drawings, such terms are intended to be inclusive in a manner similar to how the term "comprising" is interpreted when employed as a transitional phrase in a claim.

[0160] While descriptions of various embodiments have been presented for illustrative purposes, they are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. a processor executing the following computer-executable components stored in memory: a preprocessing component that generates trace frames containing vectorized representations of textual trace data produced by microservices of the microservice application; and a monitor component that uses the trace frames to identify a state of the microservice application. Equipped with the pre-processing component generates the trace frames using a machine learning model trained using a graph traversal algorithm, a word embedding technique, or a combination thereof; system.

2. A processor executing the following computer-executable components stored in memory: a preprocessing component that generates trace frames containing vectorized representations of textual trace data produced by the microservices of the microservice application; a monitor component that uses the trace frames to identify a state of the microservice application; and A diagnostic component that performs root cause analysis by assessing the contribution of each microservice to a given rebuild loss. A system comprising:

3. 3. The system of claim 1, wherein the monitor component identifies the state of the microservice application by comparing the trace frames with reconstructed trace frames produced by a machine learning model.

4. 4. The system of claim 1, wherein the machine learning model captures spatial and temporal dependencies between the microservices of the microservice application when operating under normal conditions.

5. 5. The system of claim 1, wherein the monitor component identifies the state of the microservice application by comparing a sequence of trace frames with a sequence of reconstructed trace frames generated by a machine learning model, the sequence of trace frames including the trace frame.

6. The system of claim 5 , wherein the machine learning model encodes the sequence of trace frames.

7. 1. A computer-implemented method executed by a computer system, the computer system comprising: generating, by a system operatively coupled to a processor, trace frames including vectorized representations of textual trace data produced by microservices of a microservice application, the trace frames being generated using a machine learning model trained using a graph traversal algorithm, a word embedding technique, or a combination thereof; identifying, by the system, a state of the microservice application using the trace frames; 11. A computer-implemented method comprising:

8. A computer-implemented method executed by a computer system, the computer system comprising: generating, by a system operatively coupled to the processor, trace frames that include vectorized representations of textual trace data produced by microservices of the microservices application; Identifying, by the system, a state of the microservice application using the trace frames; performing a root cause analysis by assessing the contribution of each microservice to a given rebuild loss, 11. A computer-implemented method comprising:

9. 9. The computer-implemented method of claim 7 or 8, wherein the system identifies the state of the microservice application by comparing the trace frames with reconstructed trace frames produced by a machine learning model.

10. 10. The computer-implemented method of claim 7 or 9, wherein the machine learning model captures spatial and temporal dependencies between the microservices of the microservices application when operating under normal conditions.

11. 11. The computer-implemented method of claim 7, wherein the system identifies the state of the microservice application by comparing a sequence of trace frames to a sequence of reconstructed trace frames generated by a machine learning model, the sequence of trace frames including the trace frame.

12. 12. The computer-implemented method of claim 11, wherein the machine learning model is a convolutional long short-term memory (ConvLSTM) based autoencoder model.

13. A computer program for causing a computer to execute the computer-implemented method according to any one of claims 7 to 12.

14. A computer-readable storage medium having the computer program according to claim 13 recorded thereon.

Images