Methods, systems, and media for encoding stack trace information

Abstract

Methods, systems, and media for encoding stack trace information are disclosed. Embodiments provide a thread categorization method that uses a categorization signature to represent a stack trace in a compact form. Some embodiments can receive a stack trace comprising a sequence of stack frames. Some embodiments can generate, based on the sequence of stack frames, a trace signature representing a set. Some embodiments can receive one or more subsequent stack traces. For each of the one or more subsequent stack traces, some embodiments can determine whether a subsequent trace signature representing a sequence of stack frames included within the subsequent stack trace has been generated. If not, some embodiments can generate, based on the trace signature and other subsequent trace signatures generated based on the trace signature, a subsequent trace signature representing the subsequent sequence of stack frames.

Description

[0001] This application is a divisional application of the invention patent application 201780036501.7, filed on May 8, 2017, entitled "METHOD, SYSTEM, AND MEDIUM FOR ENCODING STACK TRACE INFORMATION".

[0002] Cross Reference to Related Applications

[0003] This application is a continuation-in-part of U.S. Non-Provisional Application No. 15 / 588,523, filed May 5, 2017, entitled "Compression Techniques for Encoding Stack Trace Information," U.S. Provisional Application No. 62 / 333,786, filed May 9, 2016, entitled "Correlation of Thread Intensity and Heap Usage to Identify Heap-Hoarding Stack Traces," U.S. Provisional Application No. 62 / 333,798, filed May 9, 2016, entitled "Memory Usage Determination Techniques," U.S. Provisional Application No. 62 / 333,804, filed May 9, 2016, entitled "Compression Techniques for Encoding Stack Traces Information," U.S. Provisional Application No. 62 / 333,811, filed May 9, 2016, entitled "Correlation of Stack Segment Intensity in Emergent Relationships," U.S. Provisional Application No. 62 / 333,809, filed May 9, 2016, entitled "Systems and Methods of Stack Trace Analysis," and U.S. Provisional Application No. 62 / 340,256, filed May 23, 2016, entitled "Characterization of Segments of Time-Series," U.S. Non-Provisional Application No. 15 / 588,531, filed May 5, 2017, entitled "Correlation of Thread Intensity And Heap Usage To Identify Heap-Hoarding Stack Traces," U.S. Non-Provisional Application No. 15 / 588,526, filed May 5, 2017, entitled "Memory Usage Determination Techniques," and U.S. Non-Provisional Application No. 15 / 588,521, filed May 5, 2017, entitled "Correlation of Stack Segment Intensity in Emergent Relationships," under 35 U.S.C. § 121.119(e) of the International Application, which is hereby incorporated by reference in its entirety for all purposes.

[0004] This application is related to the following concurrently filed applications, the entire contents of which are incorporated herein by reference for all purposes:

[0005] (1) PCT Non-Provisional Application No. / ,, entitled "CORRELATION OF THREAD INTENSITY AND HEAP USAGE TO IDENTIFY HEAP-HOARDING STACK TRACES," filed May 8, 2017 (Attorney Docket No. 088325-1047566 (175200PC)).

[0006] (2) PCT Non-Provisional Application No. / ,, entitled "MEMORY USAGE DETERMINATION TECHNIQUES," filed May 8, 2017 (Attorney Docket No. 088325-1047576 (175210PC)).

[0007] (3) PCT Non-Provisional Application No. / ,, entitled "CORRELATION OF STACK SEGMENT INTENSITY IN EMERGENT RELATIONSHIPS," filed May 8, 2017 (Attorney Docket No. 088325-1047582 (175230PC)). BACKGROUND

[0008] Generally, cloud service providers maintain operational resources to meet service level agreements (SLAs) with customers. Providers constantly monitor performance metrics of the cloud services they provide to ensure that the services meet the SLAs. However, operational resources can not be able to avoid SLA violations because available tools can lack the ability to predict or detect impending SLA violations. Moreover, when such violations do occur, operations can take a long time to resolve such violations because the tools can lack the ability to diagnose the root cause of the SLA violations. As a result, customer experience can be adversely affected.

[0009] Additionally, such SLAs can require systematic analysis of data and proactive action on actionable information in the data to avoid SLA violations and also determine whether the agreement is met. Compliance with service level agreements and other requirements can be very burdensome and become more burdensome over time.

[0010] To obtain the above capabilities, what is needed is a technique that represents a system using a high-level state model that is easily updated based on low-level events of the system and system measurements. As for obtaining metrics about low-level events, the application program underlying the system can be instrumented to collect the exact measurements of events. However, in this approach, the instrumentation itself can affect the measurements. This problem can be more pronounced when the execution time of the instrumentation code around a method dominates the execution time of the method itself (e.g., if the method is called a high number of times). SUMMARY

[0011] Certain techniques are disclosed for categorizing threads based on stack traces obtained from a series of thread dumps. Some embodiments can utilize tuples of labeled binary trees to categorize stack traces through a synthesis and analysis process.

[0012] One embodiment is directed to a method. The method can include receiving, by a computer, a stack trace comprising a sequence of stack frames; generating, based at least in part on the sequence of stack frames, a trace signature representing the sequence of stack frames; receiving, by the computer, one or more subsequent stack traces; and for each of the one or more subsequent stack traces: determining whether a subsequent trace signature representing a subsequent sequence of stack frames comprised by the subsequent stack trace has been generated; and if the subsequent trace signature has not been generated, generating, based at least in part on the trace signature and other subsequent trace signatures generated based on the trace signature, a subsequent trace signature representing the subsequent sequence of stack frames. BRIEF DESCRIPTION OF DRAWINGS

[0013] The illustrative embodiments are described below with reference to the following drawings.

[0014] Figure 1 An exemplary run-time profile of a single thread over a period of time is depicted at a relatively high frequency of sampling.

[0015] Figure 2 An exemplary call context tree is depicted.

[0016] Figure 3 An exemplary thread dump of a virtual machine over a period of time is depicted in accordance with some embodiments.

[0017] Figures 4-6 An exemplary thread categorization signature is depicted in accordance with some embodiments.

[0018] Figure 7 A simplified flow diagram is shown that depicts generating and / or modifying one or more thread categorization signatures in response to a thread dump in accordance with some embodiments.

[0019] Figure 8A simplified flow diagram is shown that depicts generation or modification of thread categorization signatures in response to detecting a branch point.

[0020] Figure 9 A simplified flow diagram is shown that depicts identification of code corresponding to high heap usage in accordance with some embodiments.

[0021] Figure 10 A simplified flow diagram is shown that depicts calculation of a degree of correlation between various classes of threads and high heap usage in accordance with some embodiments.

[0022] Figure 11 An example plot is depicted in which weights assigned to sample measurements are plotted across a time range of an example data set relative to sampling time intervals associated with the sample measurements.

[0023] Figure 12 An example chart is depicted showing trend plots derived for heap usage in a production environment by different linear regression techniques.

[0024] Figure 13 An example chart is depicted showing additional trend plots that illustrate incorrect results given by a standard robust regression technique.

[0025] Figure 14 A simplified flow diagram is shown that depicts generation of a signal prediction in accordance with some embodiments.

[0026] Figure 15 A simplified diagram of a distributed system for implementing certain embodiments is depicted.

[0027] Figure 16 A simplified block diagram of one or more components of a system environment in which services can be provisioned as cloud services in accordance with some embodiments is depicted.

[0028] Figure 17 An example computer system that can be used for implementing certain embodiments is depicted.

[0029] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. DETAILED DESCRIPTION

[0030] I. SUMMARY

[0031] In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. However, it will be apparent that various embodiments can be practiced without these specific details. The drawings and description are not intended to be restrictive.

[0032] The present disclosure generally relates to using heap usage statistics and thread intensity statistics to identify code blocks within a multi-threaded process (e.g., an application) for potential optimization and to predict future heap usage and / or thread intensity. Thread intensity statistics can be used to track a process's responsiveness, load, and resource usage without tooling detection of the underlying code of the process or using code injection. In particular, the intensity of a thread's type or a stack segment's type can indicate a statistical measure of the "hotness" of a code block being executed by the thread or referenced by the stack segment. The hotness of a code block can be quantified by the amount of execution (e.g., the number of calls to the code block multiplied by the execution time of the code block). Hotter code blocks have higher call counts and / or longer response times.

[0033] By analyzing a series of thread dumps taken from a process at regular or irregular time intervals, some embodiments can provide a statistical sampling solution that is (1) low overhead, (2) non-intrusive, (3) provides always-on monitoring, and (4) avoids the problem of tooling detection code dominating the execution time of the code being tooling detected (i.e., the Heisenberg problem).

[0034] Some embodiments can categorize threads and stack segments based on intensity statistics. By monitoring stack traces of individual threads included in thread dumps received from a software execution environment (e.g., a virtual machine), a monitoring process can categorize threads into one or more thread classes based on the contents of their stack traces. As more stack traces are analyzed, some embodiments can observe branching of thread classes into sub-classes and eventually establish a hierarchy of thread classes. For example, if it is observed that stack segment (A) is a component of stack segment (A, B, D), it can be said that thread type (A, B, D) is a sub-class of thread type (A). It can also be said that thread type (A, C) is a sub-class of thread type (A). Thread type (A) includes sub-classes (A, B, D) and (A, C) in the sense that the aggregation of intensity statistics corresponding to (A, B, D) and (A, C) can be represented by the intensity statistics corresponding to (A). Further, some embodiments can travel along a thread class hierarchy (e.g., traverse a tree or graph) to observe how the intensity of a particular thread class can be proportionally attributed to the intensity of one or more sub-classes of the thread class. For example, the thread intensity of (A) can be proportionally attributed to the thread intensity of (A, B, D) and (A, C). In other embodiments, each stack trace can be represented as a binary tree.

[0035] Some embodiments can provide one or more sequential filters to estimate metrics, rates of change, accelerations, seasonality factors, and residuals. Techniques to represent separate seasonality indices for multiple time periods (e.g., weekday periods and weekend periods) and to normalize seasonality factors for the multiple time periods can be performed by such embodiments. In particular, some embodiments can represent separate seasonality index sequences for each of multiple time periods. For example, the multiple time periods can include weekday periods, weekend periods, end-of-quarter periods, or various holiday periods. In estimating seasonality indices for multiple time periods, some embodiments can also (1) renormalize the seasonality indices to provide a common scale and a common reference level across all time periods, and (2) fit a smoothing spline across adjacent time periods to provide a smooth transition between the cycle of a time period or between the cycles of two adjacent time periods. With renormalization, seasonality factors across multiple time periods can have a common scale.

[0036] Some embodiments can correlate trends between strength statistics and heap usage statistics for various thread classes to identify thread classes whose strength statistics are highly correlated with high heap usage. Among thread classes whose strength statistics are highly correlated with high heap usage in a software execution environment, it is likely that inefficient heap memory usage will be found. Once these thread classes are identified, the code associated with these thread classes can be investigated and / or optimized.

[0037] Some embodiments can construct and maintain models (e.g., univariate, multivariate) of the multi-threaded environment (e.g., virtual machine) of an executing process, where the models include seasonal trends, linear trends, and first-order non-linear trends of the strength of each thread class. These models can be used to obtain long-term predictions about seasonal adjustments of system performance trends.

[0038] By (1) dynamically categorizing threads and observing how the strengths of subclasses of thread classes contribute to the aggregate strength of a thread class, and (2) observing how closely various thread classes correlate with detected periods of high heap usage, some embodiments can facilitate detecting and observing performance faults within a cloud service provisioning system. Because even slight performance faults often reveal problems in a process that can lead to SLA violations, enabling service providers to detect and resolve performance faults can greatly reduce the risk of such violations.

[0039] II. Runtime Profiling of Threads

[0040] Figures 1-2 Techniques are depicted for profiling a running thread to determine how long various stack segments exist on the call stack of the thread relative to each other. Figure 1An exemplary runtime profile of a single thread 100 over a period of time at a relatively high frequency of sampling rate is depicted. In some cases, certain techniques can utilize a runtime profiler to obtain multiple stack trace samples of a thread to construct a call context tree 200 as shown in FIG. 2. If the sampling interval employed by the runtime profiler is relatively short compared to the execution time of the thread, the observed count (i.e., call count) statistics of each call context of the thread can be used to accurately estimate and / or represent the execution time of the call context with respect to the sampling interval. Figure 2

[0041] For example, as shown in FIG. 1, the total execution time of the thread 100 can be between 100 milliseconds and 1 second, while the sampling interval is between 10 milliseconds and 100 milliseconds. During the execution of the thread, there can be different call contexts in the stack of the thread depending on the methods invoked by the thread. The thread can begin execution of the thread by invoking a set of methods corresponding to stack segment A. Figure 1

[0042] It should be noted that a stack segment corresponds to a linearly linked set of one or more stack frames. Linearly linked stack frames are always observed together within a stack trace and thus have the same strength statistics. Thus, stack segment A can correspond to multiple stack frames, such as stack frames al, a2, and a3. Sampling of the thread can result in a stack trace in the stack frame list that describes the entire call context of the thread being sampled. If some of the listed stack frames are linearly linked, these stack frames can be conceptually grouped into stack segments. Thus, a stack trace can include one or more stack segments, each stack segment including one or more stack frames.

[0043] As the thread continues its execution, the code associated with stack segment A can cause the thread to invoke a set of methods corresponding to stack segment B. Next, the code associated with stack segment B can cause the thread to invoke another set of methods corresponding to stack segment D. After a short period of time, the runtime profiler can obtain sample 1 of the thread 100, resulting in a first stack trace. From the first stack trace, the runtime profiler can determine that stack segments A, B, and D are on the stack at the time of sampling. After the sampling interval, the runtime profiler can obtain another sample 2 of the thread, resulting in a second stack trace. From the second stack trace, the runtime profiler can determine that stack segments A, B, and D are on the stack. As the thread continues to execute, the methods associated with stack segment D can return, causing the stack frames corresponding to stack segment D to pop off the stack. Next, the runtime profiler can obtain another sample 3 of the thread, resulting in a third stack trace. From the third stack trace, the runtime profiler can determine that stack segments A and B are on the stack.

[0044] ​​As the thread executes, stack segment B calls stack segment E, and stack segment E calls stack segment F. Next, sample 4 is fetched, producing a fourth stack trace indicating that stack segments A, B, E, and F are on the stack. Stack segments F, E, and B return one after another. Next, sample 5 is fetched, producing a fifth stack trace indicating that only stack segment A is on the stack. Stack segment A causes stack segment C to be pushed onto the stack. Before stack segment C returns, samples 6 and 7 are fetched, producing a sixth and seventh stack trace, both indicating that stack segments A and C are on the stack. Finally, stack segment C returns, leaving only stack segment A on the stack. The thread completes execution when the method associated with stack segment A returns.

[0045] like Figure 2 As shown, the call context tree 200 depicts the execution time of stack segment AF relative to the sampling interval. Node 202 indicates that stack segment A was observed in all seven samples. Node 204 indicates that stack segment B was observed in four of the seven samples. Node 206 indicates that stack segment C was observed in two of the seven samples. Node 208 indicates that stack segment D was observed in two of the seven samples. Node 210 indicates that stack segment E was observed in one of the seven samples. Node 212 indicates that stack segment F was observed in one of the seven samples. Because the total execution time of thread 100 is approximately ten times the duration of the sampling interval, the observation count for each stack segment can be closely correlated with the execution time of the stack segment. For example, since stack segment B was observed four times, it can be inferred that the relative execution time of stack segment B is at least four times that of the sampling interval.

[0046] In some cases, the environment in which thread 100 executes (i.e., the software execution environment) can correspond to a virtual machine (e.g., a hotspot Java Virtual Machine (JVM)), where a thread dump is performed once per sampling interval. Before the JVM performs a thread dump, it can signal all executing threads (e.g., thread 100) to pause at a safepoint. This safepoint mechanism can be analogous to the mechanism used by a garbage collector to pause threads before performing a full garbage collection. Note that threads running in kernel mode (e.g., running / blocking on I / O operations) may not pause at a safepoint until the thread exits kernel mode (e.g., returns to JVM mode).

[0047] However, it should be noted that frequently invoking the safetypoint mechanism can lead to significant overhead. Therefore, runtime profiling techniques that rely on high sampling rates may be more suitable for development or testing environments than for production environments.

[0048] To reduce overhead, some embodiments employ a system model to compensate for the reduced sampling rate. For example, some embodiments can track the intensity of threads of a multithreaded process and sample only threads whose intensity exceeds a threshold, which determine the latency. One advantage of embodiments that employ a reduced sampling rate or an adaptive sampling rate is that threads running in kernel mode are less likely to pause at a safe point. Other methods of reducing overhead can involve lengthening the sampling interval to be commensurate with the intensity of the threads being sampled. For example, while a one-minute sampling interval can result in negligible overhead within a production environment, a one-minute sampling interval can be short enough to derive the relative execution times of threads in a production environment and their constituent stack segments. Accordingly, some embodiments can provide an always-on performance monitoring solution for production systems that exhibit smooth or cyclically smooth average traversals to satisfy the assumptions of Little's formula. In such embodiments, the always-on performance monitoring solution can be implemented in a monitoring process (i.e., a control system) that periodically samples threads executing within one or more virtual machines of the production system.

[0049] III. Categorizing Threads

[0050] Various embodiments provide techniques for sequentially analyzing a series of thread dump samples taken from one or more virtual machines (e.g., JVMs) to identify thread classes and track intensity statistics pertaining to the thread classes. For example, during execution of one or more multithreaded processes within a virtual machine, a control system can periodically take a thread dump of the virtual machine. The thread dump can produce a stack trace for each thread executing in the virtual machine. For each stack trace received, the control system can analyze text contained in the stack trace to categorize the associated thread and update intensity statistics tracked for all thread classes based on the stack trace.

[0051] In addition to categorizing threads, embodiments can categorize new stack segments each time a new stack segment appears along a previously categorized stack segment at a branching point. When the control system observes a first stack trace before discovering any thread classes, the control system can consider the entire sequence of stack frames within the stack trace to be linearly connected because the entire sequence of stack frames has only appeared together so far. In response, the control system can initialize a thread class to categorize the entire stack trace (i.e., the entire sequence of stack frames). When the control system observes a subsequent stack trace that includes a different sequence of stack frames, the control system can initialize an additional thread class to categorize each unique permutation of stack frames. In some cases, the control system can observe a stack trace that does not share any stack frames (i.e., have any stack frames in common) with previously observed stack traces. In response, the control system can initialize a separate thread class to categorize the new stack trace as a whole.

[0052] However, more commonly, the control system can observe a stack trace that shares one or more stack frames with a previously observed stack trace. Returning to Figure 1 For example, assume that the first stack trace observed by the control system is {(A, B, D)} (i.e., the stack trace in Sample 1 or Sample 2), where the stack trace contains stack frames contained in stack segments A, B, and D. The control system can initialize a thread class {(A, B, D)} to categorize all threads that include stack frames contained in stack segments A, B, and D. Next, assume that the second stack trace observed by the control system is {(A, C)} (i.e., the stack trace in Sample 6 or Sample 7). At this point, the control system can determine that while the first and second stack traces are different, the first and second stack traces share all stack frames contained in stack segment A, which leads to a branching point at stack segment A. In response, the control system can initialize a thread class {(A, C}) to categorize all threads that include stack segments A and C on their call stacks.

[0053] It should be noted that because the stack frames in stack segment A have been observed to be separated from the stack frames in stack segment (B, D), stack segments A and (B, D) are no longer considered by the controller system to be linearly connected. However, the control system still considers the stack frames in stack segment A to be linearly connected and the stack frames in stack segment (B, D) to be linearly connected. At this point, the control system can initialize several thread segment components of thread class {(A, B, D)} and thread class {(A, C)} to categorize new stack segments formed by the newly discovered branching point. Specifically, the control system can initialize thread segment (A), thread segment (B, D), and thread segment (C), where thread segment (A) and (B, D) are components of thread class {(A, B, D}) and thread segment (A) and (C) are components of thread class {(A, C)}.

[0054] Some embodiments can use a categorized signature to represent stack traces and stack segments. Specifically, a trace signature can be used to represent a stack trace of a particular thread class and a segment signature can be used to represent a stack segment of a particular thread segment. Each trace signature can correspond to a tuple of a labeled binary tree established via a synthesis and analysis process. Meanwhile, each segment signature of a thread segment can correspond to a node in the tuple that corresponds to a thread class of which the thread segment is a component. Later in the analysis process, the tuples can be used like parse trees (e.g., as part of a production grammar) to recognize incoming stack traces.

[0055] Returning to the example above, the thread class {(A, B, D)} can correspond to a single tuple of the binary tree after observing the first stack trace but before observing the second stack trace. Because the entire sequence of frames within the first stack trace is considered a single stack segment, the single binary tree can include a single root node representing the stack segment (A, B, D). After observing the second stack trace, the tuple can still include only a single binary tree. However, the binary tree can now include three separate nodes: a root node representing the stack segment (A, B, D), a first child node of the root node representing the stack segment (A), and a second child node of the root node representing the stack segment (B, D). Reference is made to the following figures for further details. Figures 4-6 The process of synthesizing trace signatures and segment signatures is discussed in further detail.

[0056] Each node in the binary tree can be uniquely identified by a label or identifier, which can be referred to as a compact code. In some embodiments, the threads of a particular thread class can be represented by one or more compact codes that identify each of the top-ranked tuple nodes corresponding to the thread class. In a manner similar to Huffman coding or other entropy coding schemes, some embodiments can associate shorter tuples with more popular (i.e., having higher thread intensity) and / or first-discovered thread classes. Thus, more common types of threads can be compactly represented by shorter sequences of compact codes. In some embodiments, this can be ensured by first analyzing the probability distribution of stack traces in an offline analysis (i.e., offline processing) and feeding the stack traces to the control system in descending order of frequency.

[0057] In embodiments that do not rely on offline analysis, the control system can receive stack traces in the order in which thread dumps are periodically acquired from one or more virtual machines (i.e., online processing).

[0058] The order in which different types of stack traces are observed can be influenced by the intensity of each type of stack trace. In other words, stack traces with higher intensity are statistically more likely to be observed earlier in the sequence. Thus, such embodiments can assume that (1) the thread intensity of a particular thread class is representative of the probability of occurrence of the associated stack traces, and (2) stack traces associated with higher intensity thread classes are often observed before stack traces associated with lower intensity thread classes. In this regard, the control system will naturally derive the most compact representation for the highest intensity threads. Thus, by relying on thread intensity statistics rather than offline processing, some embodiments can provide the best compression algorithm for stack traces observed in response to a series of thread dumps.

[0059] A. Seasonality of thread intensity

[0060] For each thread class identified, some embodiments can estimate seasonal trends in the intensity of the thread class. As described above, the intensity of a thread class or thread segment can refer to a statistical measure of the "hotness" of the code blocks referenced by the associated stack traces or stack segments. The hotness of a code block can be quantified by the number of calls to the code block multiplied by the execution time of the code block. The individual raw thread intensity measure of a thread class can be a count of the number of threads of that thread class in a particular thread dump. The average thread intensity measure per thread dump can correspond to the traffic intensity, supplied load, or queue length of the thread type. For average traversable processes, the Little's formula can relate the expected intensity to the expected response time to the expected number of arrivals during the corresponding sampling interval and the arrival rate λ, as follows:

[0061]

[0062] In some embodiments, the seasonal trend analysis process can use variable filter parameters to account for irregular sampling intervals (e.g., sampling heap usage and / or taking thread dumps) and overcome the Cauchy distribution problem. The process can also support filtering multiple types of time periods (e.g., weekday periods, weekend periods, and holiday periods) sequentially at different lengths (e.g., 1 day, 2 days). Furthermore, the process can adjust the rate of thread dumps according to seasonal adjustments to reduce overhead while maintaining a certain level of confidence in the thread intensity statistics determined based on the thread dumps. In some cases, adjusting the thread dump rate can also minimize the volume of thread dump data that needs to be transmitted over a network (e.g., LAN, Internet) to other machines (e.g., big data repositories) for offline processing.

[0063] In some embodiments, the seasonal trend analysis process can divide the weekday periods (i.e., 24-hour periods) into 96 fifteen-minute intervals, which yields 96 seasonal indices (i.e., seasons) for each weekday period. The process can divide the weekend periods (i.e., 48-hour periods) into 192 fifteen-minute intervals, which yields 192 seasonal indices for each weekend period. Upon receiving a dataset of a particular length (e.g., a thread dump or time series of heap usage over 10 days that includes one or two weekends), the process can apply the multi-period trend analysis filter separately to the weekday periods and the weekend periods in order to isolate seasonal patterns observed over individual weekdays and seasonal patterns observed over entire weekends, thereby yielding a set of 96 seasonal factors for the 96 seasonal indices of each weekday and a set of 192 seasonal factors for the 192 seasonal indices of each weekend. The process can then renormalize the weekday seasonal factors and the weekend seasonal factors such that the seasonal factor "1" represents a common reference level for the weekday periods and the weekend periods.

[0064] It should be noted that if a seasonal factor greater than 1 is assigned to a seasonal index, then that seasonal index has a value higher than the average compared to the rest of the period. On the other hand, if a seasonal factor less than 1 is assigned to a seasonal index, then that seasonal index has a value lower than the average compared to the rest of the period. For example, if the seasonal factor for the thread intensity of a particular thread class for the seasonal index corresponding to the 9 AM - 9:15 interval is 1.3, then the average thread intensity interval for that particular thread class during the 9 AM - 9:15 AM interval is 30% higher than the average thread intensity for that particular thread class over the entire weekday.

[0065] In some embodiments, the seasonal trend analysis process can separate out holidays (e.g., Labor Day, Christmas) as separate periods that repeat at a frequency of once every 12 months, while the weekday periods repeat every 24 hours and the weekend periods repeat every 5 days or 7 days. A set of seasonal factors for such holiday periods can be renormalized with those for the weekday periods and the weekend periods such that the seasonal factor 1 represents a common reference level for all periods. Other frequencies for each period can also be appropriate as desired. As an example, holidays can be separated out at a frequency of every 6 months (etc.) while weekdays are periods that repeat every 12 hours (etc.).

[0066] In some embodiments, determining and tracking the intensity statistics can also include predicting future values and rates of change. However, the sampling interval can be irregular, and can even become arbitrarily close to zero. In the case where the sampling interval becomes arbitrarily close to zero, the rate of change can become a random variable of a Cauchy distribution whose mean and standard deviation are indeterminate. To overcome the Cauchy distribution problem with respect to determining seasonal trends with adaptive sampling intervals, some embodiments can employ various adaptations such as Holt's double exponential filter, Winter's triple exponential filter, Wright's extension for irregular time intervals, Hanzak's adjustment factor for time proximate intervals, outlier detection, and clipping with adaptive scaling with an outlier cutoff. These five sets of exponential filters can be applied sequentially to a dataset to estimate a set of seasonal factors for weekday periods and weekend periods.

[0067] B. Categorized Signatures and Compression Schemes

[0068] Certain embodiments can assign to a stack trace of a thread a variable length compact code sequence, where the length of the sequence depends on the intensity of the thread. An exemplary stack trace is represented as follows:

[0069] oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:991)

[0070] oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1285) ...

[0072] oracle.mds.core.MetadataObject.getBaseMO(MetadataObject.java:1048)

[0073] oracle.mds.core.MDSSession.getBaseMO(MDSSession.java:2769)

[0074] oracle.mds.core.MDSSession.getMetadataObject(MDSSession.java:1188) ...

[0076] oracle.adf.model.servlet.ADFBindingFilter.doFilter(ADFBindingFilter.java: 150) ...

[0078] oracle.apps.setup.taskListManager.ui.customization.CustomizationFilter.doFilter(CustomizationFilter.java: 46) ...

[0080] weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java: 2209)

[0081] weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java: 1457) ...

[0083] weblogic.work.ExecuteThread.execute(ExecuteThread.java: 250)

[0084] weblogic.work.ExecuteThread.run(ExecuteThread.java: 213)

[0085] In the example stack trace, the stack frame "oracle mds core MetadataObject getBaseMO" below the Java Database Connectivity (JDBC) driver stack segment (i.e., the two stack frames each including "oracle.jdbc.driver...") indicates that a Metadata Services (MDS) library calls a JDBC operation corresponding to the JDBC stack segment. The stack frame "oracle adf model servlet ADFBindingFilter doFilter" below the MDS library stack segment (i.e., the three stack frames each including "oracle.mds...") indicates that an application development framework (ADF) operation calls the MDS operation. As shown by the WebLogic stack segment at the bottom of the stack trace (i.e., the four stack frames each including "weblogic..."), the ADF operation is invoked by a Hypertext Transfer Protocol (HTTP) Servlet request.

[0086] As an example, the above stack trace can be encoded and compressed using a two-level Huffman encoding scheme, resulting in a compact code sequence representing the example stack trace. At the first level, a compression tool (e.g., gzip) can detect substrings in the stack trace, such as "ServletRequestImpl.java" and "weblogic.servlet.internal.ServletRequestImpl.run," and derive Huffman codes for these substrings based on the frequency with which these substrings appear in the stack trace. To increase the compression ratio, shorter Huffman codes can be assigned to substrings that appear more frequently. After the first level of compression, the compressed stack trace can include an encoding dictionary as metadata, which can be used to recover the substrings from the Huffman codes.

[0087] The second level can involve applying another level of compression to the compressed stack trace by replacing stack segments of the stack trace with segment signatures. The following is discussed with respect to Figures 4-6 The steps of applying the second level of compression are discussed in more detail.

[0088] C. Example Data Structures

[0089] A classification signature can be represented in memory via one or more object types. Specifically, some embodiments can use a ThreadClassificationlnfo object to represent a classification signature of a thread class (i.e., a trace signature), a Segmentlnfo object to represent a classification signature of a thread segment (i.e., a segment signature), a StackFramelnfo object to represent each element in a linearly connected stack frame within a stack segment, and a SeasonalTrendlnfo object to encapsulate and track intensity statistics of a thread class or a thread segment.

[0090] Exemplary class / interface definitions defining ThreadClassificationInfo objects, SegmentInfo objects, StackFrameInfo objects, and SeasonalTrendlnfo objects are provided below:

[0091]

[0092]

[0093]

[0094]

[0095] As can be seen from the above definitions, each ThreadClassificationlnfo object, Segmentlnfo object, and StackFramelnfo object includes a unique identifier (i.e., id), a name, a counter (i.e., numOfOccur) tracking the number of times the same type of object (e.g., the same thread class, the same thread segment, the same type of stack frame) was observed in the most recent thread dump, and another counter tracking the number of times the same type of object was observed in all thread dumps.

[0096] ThreadClassificationlnfo objects can include a list of Segmentlnfo objects and a SeasonalTrendlnfo object. In this regard, ThreadClassificationlnfo can correspond to a tuple of a binary tree, while the list of Segmentlnfo objects correspond to the nodes making up the binary tree. The SeasonalTrendlnfo object can record intensity statistics (e.g., filter states) related to the thread class represented by the ThreadClassificationlnfo object.

[0097] A SegmentInfo object can include a list of StackFrameInfo objects, a first child SegmentInfo object (i.e., firstSegment), a second child SegmentInfo object (i.e., secondSegment), a coalescing (i.e., parent) SegmentInfo object (i.e., coalescingSegment), a list of predecessor sibling SegmentInfo objects (i.e., predecessors), a list of successor sibling SegmentInfo objects (i.e., successors), and a SeasonalTrendInfo object. In this regard, a SegmentInfo object can correspond to a stack segment. If a SegmentInfo object corresponds to a leaf node, the list of StackFrameInfo objects can correspond to linearly connected stack frames included in the stack segment. If a SegmentInfo object is adjacent to a branch point, the sibling SegmentInfo objects can correspond to stack segments on opposite sides of the branch point, and the coalescing SegmentInfo object can correspond to a parent stack segment that includes the stack segment and the sibling stack segment. If a SegmentInfo object does not correspond to a leaf node, the child SegmentInfo objects can correspond to child segments of the stack segment created when a branch point is discovered in the stack segment. The SeasonalTrendInfo object can record intensity statistics related to the thread segment represented by the SegmentInfo object.

[0098] Some embodiments can categorize stack segments of a stack trace by associating a list of StackFrameInfo objects observed with a single SegmentInfo node. In other words, a SegmentInfo node is a coalescing node for each StackFrameInfo object of a stack segment. Each StackFrameInfo object can have a single coalescing SegmentInfo node. When a branch point is detected somewhere along the linearly connected StackFrameInfo objects of a SegmentInfo node, some embodiments can create two new SegmentInfo nodes and split the linearly connected StackFrameInfo objects into two groups of linearly connected StackFrameInfo objects among the new SegmentInfo nodes. It can then reconnect the two StackFrameInfo objects through the branch point.

[0099] Each of the new SegmentInfo nodes becomes a coalescing node for the StackFrameInfo objects in the segment that is part of that node. Certain embodiments can update the coalescingSegment of the StackFrameInfo objects accordingly, so that each StackFrameInfo object references the correct coalescing SegmentInfo node. The two new SegmentInfo nodes are denoted as the left sibling node and the right sibling node. The two new SegmentInfo nodes also become children of the original SegmentInfo node, which in turn becomes their parent. The parent SegmentInfo node can become a coalescing node for the two new SegmentInfo nodes.

[0100] The process of splitting stack segments in response to discovered branch points can result in a binary tree structure composed of SegmentInfo nodes. This splitting process can be viewed as a thread class (i.e., a stack trace class) forking into thread children. As the strengths of the individual stack frames in a stack segment diverge over time, certain embodiments can continuously split the stack segment into smaller stack segments, enabling the thread class hierarchy to be dug down to observe how the strength of the thread class can be proportioned to the strengths of the thread children.

[0101] In some embodiments, the SegmentInfo nodes inside the binary tree are parent nodes whose StackFrameInfo objects are not all linearly connected, as some stack frames are connected by branch points. In contrast, the StackFrameInfo objects of leaf SegmentInfo nodes can be linearly connected. Within a SegmentInfo node, the StackFrameInfo objects can be oriented as a stack with a bottom StackFrameInfo and a top StackFrameInfo, whether linearly connected or connected by branch points. By convention, the top StackFrameInfo object in a left sibling SegmentInfo node can be connected by a branch point to the bottom StackFrameInfo object of a right sibling SegmentInfo node.

[0102] Each Segmentlnfo node can include a SeasonalTrendlnfo object to track intensity statistics for the thread (sub) class represented by that Segmentlnfo node. When a Segmentlnfo node is split into two new sub-Segmentlnfo nodes, some embodiments can clone the SeasonalTrendlnfo object of the Segmentlnfo node into two new SeasonalTrendlnfo objects and set one SeasonalTrendlnfo object in each sub-Segmentlnfo node.

[0103] Some embodiments provide the ability to copy the filter state of a parent Segmentlnfo node to the new child Segmentlnfo nodes by splitting the process. In doing so, some embodiments can continuously track the ratio of intensity statistics between the parent node and sibling Segmentlnfo nodes. Specifically, the intensity statistics of the child Segmentlnfo nodes are initially each the same as the intensity statistics of the parent Segmentlnfo node. However, as new samples are obtained, the intensity statistics of the child Segmentlnfo nodes can begin to diverge from the intensity statistics of the parent node and from each other. As the filter state of the new stack segments are updated separately, the filter state of the new stack segments begin to diverge from each other and from the filter state of the original stack segment.

[0104] In some cases, the intensity statistics between parent Segmentlnfo nodes and sibling Segmentlnfo nodes can converge to a certain ratio over time. Some embodiments can apply the parent-child relationships and sibling relationships between Segmentlnfo nodes to define a correlation model for a multivariate state estimation technique. Specifically, if a process is stationary, then the ratio of intensity statistics between related Segmentlnfo nodes can converge to a stationary state. Specifically, if a process is strictly or broadly stationary, then the joint probability distribution of intensity statistics at first and second time instances (which can include the means, variances, autocovariances, and cross covariances of related Segmentlnfo nodes) can not vary with respect to time. Thus, it can be expected that the ratio of intensity statistics between parent Segmentlnfo nodes and sibling Segmentlnfo nodes converges over time. Thus, by continuously tracking the intensity statistics of sibling Segmentlnfo nodes via branch points and determining that the ratio of intensity statistics between parent Segmentlnfo nodes and sibling Segmentlnfo nodes converges over time, some embodiments can use these ratios to define a correlation model for a multivariate state estimation technique. The resulting model can be used for anomaly detection and generating predictions.

[0105] A StackFramelnfo object can include one or more predecessor StackFramelnfo objects and / or one or more successor StackFramelnfo objects (i.e., predecessors and successors), a coalescing Segmentlnfo object (i.e., coalescingSegment), and information identifying the code referenced by the StackFramelnfo object (i.e., classMethodLineNumber). If a StackFramelnfo object is not adjacent to a branch point, then the StackFramelnfo object can be linearly connected to a single predecessor stack frame and a single successor stack frame. A StackFramelnfo object can reference a contained Segmentlnfo object through the member variable coalescingSegment.

[0106] When the process latest thread dump, the member variable numOfOccur of each ThreadClassificationlnfo object, Segmentlnfo object, and StackFramelnfo object can be reset to 0. Each stack trace obtained from the thread dump can be parsed from the bottom to the top of the stack trace. After applying the first level of the Huffman encoding scheme to compress the stack trace, each line of the stack trace can be parsed into a StackFramelnfo object. After parsing the list of StackFramelnfo objects into a list of Segmentlnfo objects, some embodiments can attempt to match the list of Segmentlnfo objects to a ThreadClassificationlnfo object that contains a matching list of Segmentlnfo objects. If no such ThreadClassificationlnfo object exists, some embodiments can register a new ThreadClassificationlnfo object to represent the list of Segmentlnfo objects. Thereafter, some embodiments can then update the matching / new ThreadClassificationlnfo object and the numOfOccur and totalNumOfOccur member variables of each Segmentlnfo object and StackFramelnfo object in the matching / new ThreadClassificationlnfo object. Note that if a Segmentlnfo node is a leaf level node, the numOfOccur member variable of the node will be equal to the numOfOccur member variable of each StackFramelnfo element in the Segmentlnfo node.

[0107] Next, some embodiments can update the intensity statistics metrics encapsulated in the associated SeasonalTrendlnfo objects. Specifically, some embodiments can update the rawMeasure member variable in each SeasonalTrendlnfo object by setting it to the numofOccur member variable of the containing ThreadClassificationlnfo object or Segmentlnfo object. Note that in some embodiments, rawMeasure can be updated only at every N thread dump, in which case the rawMeasure of the SeasonalTrendlnfo object is set to the corresponding numofOccur divided by N. In some embodiments, such embodiments can update the rawMeasure member variable of the SeasonalTrendlnfo object only if the numofOccur member variable of the associated ThreadClassificationlnfo object or the associated Segmentlnfo object is non-zero. If the numofOccur member variable is non-zero, the rawMeasure of the SeasonalTrendlnfo object is set to the value of numofOccur divided by N, where N is the number of thread dumps since the last update to rawMeasure. In such embodiments, the method treats the case where numofOccur is zero as if there were no measurements available. In this regard, rawMeasure is not updated when no measurements are available. In other words, these embodiments keep track of the number 'N' of thread dumps since the last update to rawMeasure. Thread intensity measurements can correspond to an irregular time series. It should be noted that exponential filters of irregular time intervals (e.g., Holt's double exponential filter and Winter's triple exponential filter disclosed above) can effectively filter rawMeasure to obtain deseasonalized metrics and seasonal factors from a set of measurements taken at irregular time intervals.

[0108] It should be noted that each SeasonalTrendlnfo object can include time series data generated by five sets of exponential filters applied to each of the following statistical measurements: raw measure of thread intensity, rate at which thread intensity is increasing or decreasing, acceleration or deceleration of that rate, seasonal factor of thread intensity, and residual component. Within the SeasonalTrendlnfo object, the state of the five sets of exponential filters for the variables, filter constants, filter parameter adjustment weight factors (for adjusting irregular time intervals between samples), and filter parameters can be represented by this time series data.

[0109] D. Generation of Exemplary Categorized Signatures

[0110] Figure 3 An exemplary thread dump of a virtual machine 300 over a period of time is depicted in accordance with some embodiments. As with the runtime profiling of Figure 1 , the sampling interval of 100 ms to 1 second, the sampling interval employed by the control system in Figure 3 may be longer (e.g., between 20 seconds and 1 minute) to reduce the sampling overhead. As shown in Figure 3 , over two to three sampling intervals, processes executing within the virtual machine 300 produce threads 302, 304, 306, 308, 310, and 312. Each of the threads 302-312 is associated with a separate call stack at the time of execution, and thus can produce a stack trace at the time a thread dump is taken. Figure 3 A total of three thread dumps are depicted as being taken: thread dump N, thread dump N+1, and thread dump N+2.

[0111] Figure 3 Three different types of stack traces are shown as being observed in the three consecutive thread dumps in the order (A, B, D), (A, B, D), (A, C), and (A, B, E). The stack trace (A, B, D) is observed twice. Prior to thread dump N being taken, thread 302 is spawned and begins execution. When thread dump N is taken, the stack trace (A, B, D) of thread 302 is observed. It should be noted that even though stack segment A, stack segment B, and stack segment D have not yet been identified, for the sake of convenience, the names of these stack segments will be used throughout the example depicted in Figure 3 . As one sampling interval elapses after thread dump N is taken, thread 302 ends, thread 304 is spawned, and thread 304 ends without ever being sampled while threads 306 and 308 are spawned. When thread dump N+1 is taken, thread 308 produces the stack trace (A, B, D), while thread 310 produces the stack trace (A, C). As another sampling interval elapses after thread dump N+1 is taken, threads 306 and 308 end, thread 310 is spawned and ends without ever being sampled, and thread 312 is spawned. When thread dump N+2 is taken, thread 312 produces the stack trace (A, B, E). As can be seen from Figure 3 , the (A, B, D) thread type is the first thread type to be observed, and the (A, B, D) thread type has a higher intensity than the (A, C) or (A, B, E) thread types.

[0112] After thread dump N, the control system can register the single Segmentlnfo(A, B, D) node as the canonical signature for the stack trace (A, B, D). The control system can then associate the SeasonalTrendlnfo(A, B, D) object with the Segmentlnfo(A, B, D) node and update the state encapsulated by the node:

[0113] Segmentlnfo(A, B, D).numOfOccur = 1.

[0114] Segmentlnfo(A, B, D).totalNumOfOccur = 1.

[0115] Figure 4 A set of canonical signatures 400 is depicted, including a single canonical signature 450 that has been registered in response to the stack trace (A, B, D). As can be seen in Figure 4 the canonical signature 450 includes a single node 402 corresponding to Segmentlnfo(A, B, D), where Segmentlnfo(A, B, D) is shown to be a merged node for all stack frames al-d3 of the stack trace.

[0116] When the stack trace (A, B, D) is observed again in thread dump N+1, the control system can update the Segmentlnfo(A, B, D) node as follows:

[0117] Segmentlnfo(A, B, D).numOfOccur = 1.

[0118] Segmentlnfo(A, B, D).totalNumOfOccur = 2.

[0119] When the stack trace (A, C) is first observed in thread dump N+1, the control system determines that the entire set of stack frames within stack segment (A, B, D) is no longer linearly connected. Now, there is a branching point between the last stack frame of the set of stack frames, denoted by 'A' (e.g., from the top to the bottom of the stack trace), and the first stack frame of the set of stack frames, denoted by 'B, D', because, in any given stack trace, the next stack frame after the last stack frame can be either (1) the first stack frame of (B, D), or (2) the first stack frame of the set of stack frames, denoted by 'C'. Thus, the control system can split stack segment (A, B, D) into stack segment (A) and stack segment (B, D) by creating nodes Segmentlnfo(A) and Segmentlnfo(B, D) and assigning these two nodes as children of Segmentlnfo(A, B, D). For stack trace (A, C), the control system can initialize stack segment (C) by creating node Segmentlnfo(C) and register the ordered tuple comprising Segmentlnfo(A) and Segmentlnfo(C) as the categorization signature of stack trace (A, C).

[0120] In some embodiments, the control system can clone the SeasonalTrendlnfo(A, B, D) object into SeasonalTrendlnfo(A) and SeasonalTrendlnfo(B, D) objects of nodes Segmentlnfo(A) and Segmentlnfo(B, D), respectively, and create a new SeasonalTrendlnfo(C) for Segmentlnfo(C), as follows:

[0121] SeasonalTrendlnfo(A) <- SeasonalTrendlnfo(A, B, D)

[0122] SeasonalTrendlnfo(B, D) <- SeasonalTrendlnfo(A, B, D)

[0123] SeasonalTrendlnfo(C) <- new SeasonalTrendlnfo

[0124] The control system can also update the above Segmentlnfo nodes as follows:

[0125] Segmentlnfo(A).numOfOccur = 2

[0126] Segmentlnfo(A).totalNumOfOccur = 3

[0127] Segmentlnfo(C).numOfOccur = 1

[0128] Segmentlnfo(C).totalNumOfOccur = 1

[0129] Figure 5 A set of categorized signatures 500 is depicted, including the categorized signature 450 and a new categorized signature 550 generated in response to the first observation of the stack trace (A, C). As can be seen from Figure 5 the categorized signature 450 now includes three nodes: node 402, node 502, and node 504. Node 402 corresponds to Segmentlnfo(A, B, D), which is a merged node of node 502 and node 504. Node 502 corresponds to Segmentlnfo(A), which merges stack frames al-a3. Node 504 corresponds to Segmentlnfo(B, D), which merges stack frames bl-d3. The categorized signature 550 includes two nodes: node 506, which corresponds to Segmentlnfo(A) shown merging stack frames al-a3, and node 508, which corresponds to Segmentlnfo(C) shown merging stack frames cl-c3.

[0130] When the stack trace (A, B, E) is first observed in thread dump N+2, the control system determines that the entire set of stack frames within stack segment (B, D) is no longer linearly connected. There is now a branching point between the last stack frame of the set of stack frames denoted by 'B' and the first stack frame of the set of stack frames denoted by 'D', because, in any given stack trace, the next stack frame after the last stack frame can be either (1) the first stack frame of (D), or (2) the first stack frame of the set of stack frames denoted by 'E'. Accordingly, the control system can split stack segment (B, D) into stack segment (B) and stack segment (D) by creating nodes Segmentlnfo(B) and Segmentlnfo(D) and assigning these two nodes as child nodes of Segmentlnfo(B, D). For stack trace (A, B, E), the control system can initialize stack segment 'E' by creating node Segmentlnfo(E) and register the ordered tuple including Segmentlnfo(A), Segmentlnfo(B), and Segmentlnfo(E) as the categorized signature for stack trace (A, B, E).

[0131] In some embodiments, the control system can clone the SeasonalTrendlnfo(B, D) object into the SeasonalTrendlnfo(B) and SeasonalTrendlnfo(D) objects of the Segmentlnfo(B) and Segmentlnfo(D) nodes, respectively, and create a new SeasonalTrendlnfo(E) for Segmentlnfo(E), as follows:

[0132] SeasonalTrendlnfo(B) <- SeasonalTrendlnfo(B, D)

[0133] SeasonalTrendlnfo(D) <- SeasonalTrendlnfo(B, D)

[0134] SeasonalTrendlnfo(E) <- new SeasonalTrendlnfo

[0135] The control system can also update the above Segmentlnfo nodes as follows:

[0136] Segmentlnfo(A).numOfOccur = 1

[0137] Segmentlnfo(A).totalNumOfOccur = 4

[0138] Segmentlnfo(B).numOfOccur = 1

[0139] Segmentlnfo(B).totalNumOfOccur = 3

[0140] Segmentlnfo(E).numOfOccur = 1

[0141] Segmentlnfo(E).totalNumOfOccur = 1

[0142] Figure 6 A set of categorized signatures 600 is depicted, including categorized signatures 450 and 550 and a new categorized signature 650 generated in response to the stack trace (A, B, E). As Figure 6As can be seen, the classified signature 450 now includes five nodes: node 402, node 502, node 504, node 602, and node 604. Node 504 corresponds to Segmentlnfo(B, D), which is a merged node of node 602 and node 604. Node 602 corresponds to Segmentlnfo(B), which merges stack frames bl - b3. Node 604 corresponds to Segmentlnfo(D), which is a merged node of stack frames dl - d3. The classified signature 550 has not changed. The classified signature 650 includes three nodes: node 606 corresponding to Segmentlnfo(A) shown merging stack frames al - a3, node 608 corresponding to Segmentlnfo(B) shown merging stack frames bl - b3, and node 610 corresponding to Segmentlnfo(E) shown merging stack frames el - e3.

[0143] As Figure 6 shown, the classified signature of stack trace (A, B, D) can consist of a single Segmentlnfo node at the root of the classified signature 450. In other words, the stack trace (A, B, D), being the highest strength stack trace, has the most compact representation. Meanwhile, the stack trace (A, C) is assigned the second shortest classified signature with two ordered nodes (A) and (C). The last detected stack trace (A, B, E) is assigned the third shortest classified signature with three ordered nodes (A), (B), and (E). As Figures 4-6 shown, the ThreadClassificationlnfo object can correspond to a tuple of Segmentlnfo nodes, and the Segmentlnfo nodes can reference a binary tree (or binary subtrees) of other Segmentlnfo nodes and / or collections of StackFramelnfo objects. The ThreadClassificationlnfo object, Segmentlnfo nodes, and StackFramelnfo objects can together form a production grammar:

[0144] Threadl -> (A, B, D)

[0145] Thread2 -> (A) (C)

[0146] Thread3 -> (A) (B) (E)

[0147] (A, B, D) -> (A) (B, D)

[0148] (B, D) -> (B) (D)

[0149] A -> al, a2, a3

[0150] B -> b1, b2, b3

[0151] C -> c1, c2, c3

[0152] D -> d1, d2, d3

[0153] E -> e1, e2, e3

[0154] As described above, each stack frame ai, bi, ci, di, ei is a terminal, and the SegmentInfo node is a non-terminal of the grammar. Some embodiments can parse the stack frames of a stack trace from the bottom of the stack trace to the top of the stack trace (oriented from left to right in the following representation).

[0155] a1, a2, a3, b1, b2, b3, d1, d2, d3

[0156] (A), b1, b2, b3, d1, d2, d3 use production (A) -> a1, a2, a3

[0157] (A), (B), d1, d2, d3 use production (B) -> b1, b2, b3

[0158] (A), (B), (D) use production (D) -> d1, d2, d3

[0159] (A), (B, D) use production (B, D) -> (B) (D)

[0160] (A, B, D) use production (A, B, D) -> (A), (B, D)

[0161] Thread1 use production Thread1 -> (A, B, D)

[0162] As can be seen above, some embodiments can analyze stack frames via bottom-up syntax analysis, which can be similar to shift-reduce parsing or left-to-right "LR" parsing. This analysis can involve shifting and reducing stack frames and SegmentInfo nodes to construct a parse tree for a stack trace by operating from the leaves to the root of the tree. Some embodiments can synthesize a parse tree for an earlier occurrence of a stack trace of a thread, and analyze another occurrence of a stack trace of the thread by reducing (i.e., shift-reduce parsing, left-to-right "LR" parsing) against the same parse tree. Each node of the parse tree can be a compact tag for a class of stack traces, and the root of the parse tree can be a compact tag for a class of threads.

[0163] Figure 7 FIG. 7 illustrates a flowchart 700 of a process for generating and / or modifying one or more thread categorization signatures in response to a thread dump, in accordance with some embodiments. In some embodiments, the process depicted in flowchart 700 can be implemented by a computer system having one or more processors (e.g., computer system 1700 of FIG. 6), where the one or more processors can perform the steps based on computer code stored in a computer-readable medium. Figure 17 The steps described in FIG. 6 can be performed in any order, and with or without any other steps. Figure 7 The steps described in FIG. 6 can be performed in any order, and with or without any other steps.

[0164] Flowchart 700 begins at step 702, where embodiments perform a thread dump during execution of a multithreaded program. In particular, some embodiments can correspond to one or more monitoring processes that monitor a software execution environment in which a multithreaded program executes. The software execution environment can support a plurality of multithreaded processes, including the multithreaded program. In some cases, the software execution environment can be a virtual machine that supports the taking of thread dumps. In some embodiments, the one or more monitoring processes can execute within the virtual machine alongside the multithreaded program. In some embodiments, the one or more monitoring processes can execute on the same set of machines as the virtual machine or separately from the virtual machine on a different set of machines. The one or more monitoring processes can periodically initiate a thread dump of the virtual machine. For a particular thread dump, a stack trace can be obtained for each thread that was executing (e.g., that was spawned by the multithreaded program) at the time the particular thread dump was taken.

[0165] At step 704, embodiments receive a stack trace for each thread that was executing during the thread dump. The stack trace for a particular thread can correspond to one or more lines of text that describe the call stack of the thread. Each line within the stack trace corresponds to a particular stack frame on the call stack of the thread, and can describe a code block associated with the stack frame. In some embodiments, the stack frame can include a source code file and a line number pointing to the code block, as well as a class name and / or a method name associated with the code block.

[0166] At decision 706, embodiments determine whether another stack trace needs to be analyzed. If not, the flowchart ends at step 716. In particular, once all stack traces of a thread dump have been analyzed by one or more monitoring processes, some embodiments can update the intensity statistics encapsulated by one or more objects in memory. For example, member variables of one or more SeasonalTrendlnfo objects (e.g., rawMeasure, rawDeseasonalizedMeasure, smoothedWeekdaySeasonalFactor, and / or smoothedWeekendSeasonalFactor) can be updated based on which stack traces were obtained from the thread dump.

[0167] Otherwise, at step 708, embodiments determine whether an existing trace signature represents the sequence of stack frames included by the stack trace. In particular, some embodiments can use an existing set of categorized signatures that have been established based on stack frames received from previous thread dumps as a production grammar to determine whether the sequence of stack frames can be represented by one of the existing signatures. This can involve one or more shift-reduce operations in which portions of the stack trace are collapsed into leaf Segmentlnfo nodes and Segmentlnfo nodes themselves are collapsed into merged nodes. If the shift-reduce operation results in an ordered tuple that is registered as a categorized signature, then the categorized signature represents the sequence of stack frames included by the stack trace.

[0168] At decision 710, if such a trace (i.e., categorized) signature exists, the flowchart proceeds to step 714. Otherwise, at step 712, embodiments generate a new trace signature that represents the sequence of stack frames included by the stack trace. In other words, a branching point within a set of stack frames that were previously thought to be linearly connected has been discovered. Some embodiments can then generate one or more Segmentlnfo nodes, modify one or more binary trees, and / or modify one or more ordered tuples to generate a new categorized signature that represents the set of (previously) linearly connected stack frames included by the stack trace. The techniques for generating a new categorized signature are described in more detail below with respect to Figure 8 The techniques for generating a new categorized signature are described in more detail below.

[0169] At step 714, embodiments increment a counter associated with the trace signature before returning to decision 706. In particular, certain counters (e.g., numOfOccur and / or totalNumOfOccur) that are members of ThreadClassificationlnfo objects, Segmentlnfo objects, and / or StackFramelnfo objects can be incremented to track the number of stack traces, stack segments, and stack frames by type as they are received and discovered.

[0170] Figure 8 FIG. 8 illustrates a flowchart 800 of a process for generating or modifying thread categorization signatures in response to detecting a branch point, in accordance with some embodiments. In some embodiments, the process depicted in flowchart 800 can be implemented by a computer system having one or more processors (e.g., computer system 1700 of FIG. 7), where the one or more processors can perform the steps based on computer code stored in a computer-readable medium. Figure 17 The steps described in connection with FIG. 8 can be performed in any order, and can be performed with or without any of the other steps described in connection with FIG. 8. Figure 8 The steps described in connection with FIG. 8 can be performed in any order, and can be performed with or without any of the other steps described in connection with FIG. 8.

[0171] Flowchart 800 begins at step 802, where an embodiment determines whether one or more Segmentlnfo nodes have been previously generated. If so, the flowchart proceeds to step 804. Otherwise, the flowchart proceeds to step 814. Unless the stack trace being analyzed is the first stack trace received for a data set, the set of categorization signatures will likely contain one or more categorization signatures previously generated for previous stack traces, where these categorization signatures include Segmentlnfo nodes. Since stack traces of the same type received from the same process can share stack segments with each other, any stack trace of a new type received for the first time will likely result in the discovery of a branch point.

[0172] At step 804, an embodiment determines one or more subsequences of stack frames included in the sequence of stack frames included by the stack trace that are not represented by any previously generated nodes. In particular, some embodiments can consult existing categorization signatures and Segmentlnfo nodes when attempting to compress the sequence of stack frames contained by the stack trace through a series of shift-reduce operations. Any subsequence of stack frames of a sequence that cannot be reduced can be determined to be a new type of stack segment. In this case, some embodiments can determine that a Segmentlnfo node representing the new type of stack segment needs to be generated.

[0173] At step 806, an embodiment generates one or more additional nodes to represent the one or more subsequences of stack frames. In particular, a new StackFramelnfo object can be generated for each stack frame included in the new type of stack segment. A new Segmentlnfo node corresponding to the new type of stack segment can be generated, where the new Segmentlnfo node references each of the new StackFramelnfo objects.

[0174] At step 808, embodiments incorporate at least one of the one or more additional nodes into one or more previously generated binary trees of one or more previously generated tuples. One or more binary trees of one or more existing categorization signatures can be modified and / or extended to account for newly discovered branch points. In the case where a stack segment represented by a leaf Segmentlnfo node of an existing binary tree is split by a new branch point, that leaf node can become a merge node of two new leaf Segmentlnfo nodes.

[0175] At step 810, embodiments generate one or more additional binary trees, where at least one or more of the one or more binary trees includes at least one of the one or more additional nodes. In many cases, the one or more additional binary trees can be single-level trees with a single node. One of the newly generated binary trees can include the new Segmentlnfo node generated in step 806.

[0176] At step 812, embodiments generate an additional tuple including one or more additional binary trees to represent the stack trace. The additional tuple can correspond to a categorization signature representing a newly discovered type of stack trace. Some tuples can be ordered sets of single-level binary trees, each containing a single node, and can look similar to a list of nodes. Other tuples can correspond to a single multi-level binary tree. Still other tuples can include a combination of single-level binary trees and multi-level binary trees. In general, as more and more types of stack traces are discovered, each subsequent categorization signature generated can correspond to longer and longer ordered tuples. However, since it is more likely that common types of stack traces will be encountered first, longer categorization signatures are more likely to represent less frequently occurring stack traces. This can ensure that a higher percentage of stack traces are compressed into shorter categorization signatures. After step 812, the flowchart ends at step 820.

[0177] At step 814, embodiments generate a tuple including a single binary tree including a single node representing the stack trace. Since no Segmentlnfo nodes were found, the stack trace currently being analyzed can be the first. Accordingly, some embodiments can generate a categorization signature corresponding to a single binary tree with only one Segmentlnfo node. After step 814, the flowchart ends at step 820. In the future, upon encountering a different type of stack trace, the binary tree can be extended with a new Segmentlnfo node to represent the newly encountered branch point.

[0178] IV. Heap usage measurements at irregular intervals

[0179] Some embodiments can cause a control system to monitor time series data of heap allocations (i.e., heap usage) to estimate trends and predict future memory usage within a virtual machine. By detecting seasonal trends and predicting memory capacity requirements, some embodiments can dynamically reallocate shared system memory among virtual machines, thereby enabling elasticity of resource allocation. Prediction of capacity requirements can involve estimation of heap growth rates. To ensure sample accuracy, heap allocation measurements can be taken during a full garbage collection (GC) cycle, which occurs at irregular time intervals. Estimation of heap growth rates can involve division by random time intervals, which is complicated by the irregular time intervals intermittently arbitrarily close to zero. Noise in the growth rate measurements is a ratio of two Gaussian distributions that produces a Cauchy distribution, which can be difficult to filter. The mean and standard deviation of the Cauchy distribution are uncertain, in the sense that a large number of data points does not produce a more accurate estimate of the mean and standard deviation than a single data point. Increasing the sample pool can increase the likelihood of encountering sample points with large absolute values corresponding to intervals close in time to the division.

[0180] It should be noted that, unlike heap size measurements, which are sampled at irregular intervals due to the irregularity of full GC cycles, thread intensity measurements can be sampled at regular intervals to avoid time-close intervals. Even so, the same techniques described herein for trend analysis of heap allocations can be applied to seasonal trend analysis and prediction of thread and stack segment intensity measurements. In some embodiments, the techniques can be adjusted for variable delays due to CPU scheduling of threads and interference by full GC cycles. The techniques can also be adjusted for variable sampling intervals due to variable computation time required for categorization of stack segments. In cases where a particular thread or stack segment has not been observed in a thread dump, some embodiments can leave the numOfOccur member variable of the associated ThreadClassificationlnfo object or the associated Segmentlnfo object at zero, which can indicate that no measurements are available for that particular thread or stack segment. Such embodiments can not update the rawMeasure variable of the SeasonalTrendlnfo object. Such embodiments can update the rawMeasure member variable of the SeasonalTrendlnfo object only when the numOfOccur member variable of the associated ThreadClassificationlnfo object or the associated Segmentlnfo object is not zero. These embodiments can track the number 'N' of thread dumps since the last update of rawMeasure. Thread intensity measurements can correspond to a series with irregular time intervals.

[0181] The Holt-Winter triple exponential filter published in 1957 and 1960 can be used for seasonal trend analysis and prediction. "Forecasting Trends and Seasonal by Exponentially Weighted Averages" by C. C. Holt, published in Office of Naval Research Memorandum, no. 52 (1957) is incorporated herein by reference. "Forecasting Sales by Exponentially Weighted Moving Averages" by P. R. Wright, published in Management Science, vol. 6, no. 3, p. 324-342 (1960) is incorporated herein by reference. Wright extended the Holt-Winter formula in 1986 to support irregular time intervals. "Forecasting data published at irregular time intervals using an extension of Holt's method" by D. J. Wright, published in Management Science, vol. 32, no. 4, pp. 499-510 (1986) is incorporated herein by reference. In 2008, Hanzak proposed an adjustment factor for time proximate intervals. "Improved Holt Method for Irregular Time Series" by T. Hanzak, published in WDS'08 Proceedings Part I, pp. 62-67 (2008) is incorporated herein by reference.

[0182] If the time intervals decrease monotonically during congestion caused by memory leaks or deadlocks, then the adjustment factor for time proximate intervals, which is intended to compensate for higher relative noise strength due to random time proximate intervals in the rate estimate, can inadvertently suppress the change rate estimate. The non-linear or polynomial time complexity of the full GC algorithm can cause the thread run time interval to decrease as congestion worsens. In the case of memory leaks, as the time interval decreases, the run time can decrease, but the measurement time can increase because the virtual machine can be frozen for longer periods of time due to more frequent full GCs. If the virtual machine is frozen during a full GC, then new requests can queue outside the virtual machine. This backlog can accelerate the rate of change of heap usage during subsequent run times. In some embodiments, the Hanzak adjustment for time proximate intervals is used for trend analysis and prediction of heap allocation and tracking of accelerated heap growth rates.

[0183] In embodiments of the invention, the Holt-Winter triple exponential filter can be applied to seasonal trend analysis and prediction of heap usage to effectively implement elasticity of memory allocation. The standard Holt-Winter triple exponential filter that can be applied to demand prediction of regular time series can be specially adapted to work with random time intervals having irregular time proximate intervals. Embodiments of the invention can apply the Wright formula for irregular time intervals and the Hanzak adjustment for time proximate intervals to perform trend analysis and prediction of heap allocation. A non-trivial selection of filter structure appropriate for irregular time intervals generated by full GCs can be performed. The structure of the Holt-Winter-Wright-Hanzak filter can be derived from first principles to systematically design adaptability to match time series generated by full GC cycles.

[0184] In some embodiments, a formula applying exponential moving averages is used to smooth time series data, local linear trends, seasonal trends, forecasted error residuals, and forecasted absolute deviations in order to monitor and predict resource utilization metrics, such as heap memory usage and thread intensity. In some embodiments, the formula can be based on the Brown exponential filter proposed in 1956, the Holt double exponential filter proposed in 1957, the Winters triple exponential filter proposed in 1960, the Wright extension for irregular time intervals proposed in 1986, the Hanzak adjustment factor for time proximate intervals proposed in 2008, and outlier detection and clipping.The following publications are included herein by reference: R. G. Brown, "Exponential Smoothing for Predicting Demand," Cambridge, Arthur D. Little Inc. (1956), p. 15; C. C. Holt, "Forecasting Trends and Seasonal by Exponentially Weighted Averages," Office of Naval Research Memorandum, no. 52, (1957); P. R. Winters, "Forecasting Sales by Exponentially Weighted Moving Averages," Management Science, vol. 6, no. 3, p. 324-342, (1960); D. J. Wright, "Forecasting data published at irregular time intervals using an extension of Holt's method," Management Science, vol. 32, no. 4, pp. 499-510 (1986); T. Hanzak, "Improved Holt Method for Irregular Time Series," WDS'08 Proceedings Part I, pp. 62-67 (2008); and S. Maung, S. W. Butler and S. A. Henck, "Method and Apparatus for process Endpoint Prediction based on Actual Thickness Measurements," United States Patent 5503707 (1996).

[0185] V. Correlating thread intensity and heap usage

[0186] Various embodiments provide techniques for identifying stack traces (i.e., classes of threads) that are responsible for heap accumulation within a multi-threaded application by correlating trends between intensity statistics for various classes of threads produced by the application and heap usage statistics. In doing so, some embodiments can identify a high heap usage trend that is high during a time period in which one or more multi-threaded applications are executing within a software execution environment based on the heap usage statistics. As described above, some embodiments can then identify and collect intensity statistics for a plurality of classes of threads by analyzing thread dumps obtained from the software execution environment during the same time period as the high heap usage trend. Some embodiments can then identify "heap-accumulating" classes of threads (i.e., heap-accumulating stack traces) from the identified classes of threads by ranking the classes of threads by the degree of correlation between their intensity statistics and the high heap usage trend.

[0187] Some embodiments can refer to such classes of threads as being heap-accumulating because the code executed by such threads is likely to be inefficient in terms of heap memory usage. In other words, poorly written code and / or unoptimized code executed by these threads can cause the threads to accumulate large amounts of heap memory, thereby significantly contributing to the high heap usage trend.

[0188] It should be noted that such memory hotspots are important from the perspective of long-term operation of cloud-based services in production environments. Thus, by enabling such hotspots to be continuously detected and mitigated, some embodiments can directly impact the operational efficiency of cloud services. It should also be noted that such embodiments can be superior to profiling such applications using memory profiler tools because such tools can add too much overhead to the applications. Thus, memory profiler tools can be impractical for continuously profiling applications that are executing in production environments.

[0189] A. Inefficient heap usage in code

[0190] One common cause of inefficient memory usage is due to local variables defined in a thread's stack frame. Typically, when a running thread instantiates an object, the object occupies heap memory until the number of stack frames that have (directly or indirectly) referenced the object drops to zero, at which point the heap memory is released at the next garbage collection. Thus, local variables that hold large objects from stack frames that remain active for a long time can inadvertently have a large impact on heap memory usage because they do not allow the objects to be garbage collected.

[0191] Some embodiments assume that a portion 'p' of the total heap usage 'G' bytes can be attributed to a class of threads 'C'. Additionally, some embodiments can also assume that the average heap usage (i.e., heap usage per thread) in this class of threads 'C' is 'M' bytes. In this instance, let 'T' be the total number of threads in the class 'C'. Then, the total heap usage 'G' can be expressed as:C represents the expected number of threads of thread class 'C'. The following relationship gives 'T C ', which is defined as the thread intensity in the statistical model:

[0192]

[0193] In response to identifying the thread classes that are causing heap accumulation, certain embodiments can report (e.g., via a notification or warning) these thread classes to developers, performance engineers, and other relevant personnel. Thus, the code associated with this type of thread can require a detailed code review and code profiling. In some cases, certain associated stack frames can be examined. For example, the investigation can involve taking a heap dump during a time when heap usage is close to a seasonal peak to examine the stack frames contained in the stack traces of the thread classes that are causing heap accumulation. These stack frames can contain local variables that reference objects that contribute to the high heap usage (e.g., objects that consume a large amount of heap memory). This code examination and optimization can be done through visual code review, automated code review, profiling of the identified threads, just-in-time compiler optimization, dynamic bytecode injection, or a combination of these techniques. In some embodiments, the thread classes that are causing heap accumulation can be reported to other automated code optimization tools to leverage their code optimization capabilities.

[0194] Some embodiments can automatically redesign or rewrite the application code to make its use of memory more efficient. For example, some embodiments can automatically rewrite the code so that local variables release large objects as soon as possible without changing the behavior or correctness of the application. In some cases, this can involve a deep analysis of the code paths involved in the thread classes that are causing heap accumulation.

[0195] For example, consider the following code:

[0196] fileOS.write(buffer.toString().getBytes());

[0197] Some embodiments can determine that the above code is inefficient in terms of memory usage because three objects: buffer, buffer.toString(), and buffer.toString().getBytes() are held by local variables in the stack frame of the thread class that is causing heap accumulation. Specifically, these local variables prevent these three objects from being garbage collected while the thread is blocking in the file system call.

[0198] Some embodiments can modify the code as shown below so that at least two objects: buffer and buffer.toString() can be garbage collected while the thread is blocking in the file system call:

[0199] String temp1 = buffer.toString();

[0200] buffer = new StringBuffer(); / / allow garbage collection of the old buffer

[0201] byte[] temp2 = temp1.getBytes();

[0202] temp1 = null; / / allow garbage collection of the string

[0203] fileOS.write(temp2); / / this is a blocking call

[0204] temp2 = null; / / allow garbage collection of the bytes array

[0205] Some embodiments can use a non-intrusive way to examine stack frames of a heap-hung stack trace.

[0206] B. Initializing seasonal factors for weekday and weekend periods

[0207] To identify a heap-hung stack trace, some embodiments can (1) identify a high heap usage season by estimating seasonal trends of heap usage statistics of an execution environment, and (2) estimate seasonal trends of thread intensity statistics for each of one or more thread classes. Some techniques for determining seasonal trends of heap usage statistics and seasonal trends of thread intensity statistics for regular or irregular time intervals are disclosed in patent applications 14 / 109,578, 14 / 109,546, and 14 / 705,304, which are incorporated by reference herein for all purposes.

[0208] To determine the seasonal trends of statistics, time periods and intervals can be defined to which the seasonal trends are mapped. Specifically, a time period can be divided into a plurality of non-overlapping intervals. Each interval of the time period can be associated with a season index. For example, if the time period is one day and the interval is one hour, there should be 24 season indices to cover the time period. As another example, if the time period is one year and the interval is one month, there should be 12 season indices.

[0209] Some embodiments can model weekdays, weekends, and holidays as separate periods. If weekday and weekend periods are separate, there can be 5 cycles of weekday periods interleaved with 1 cycle of weekend periods, such that after processing 5 consecutive weekday periods, a single weekend period is processed. Thus, the frequency of consecutive weekday periods would be one weekday period per 24 hours, while the frequency of weekend periods would be one weekend period per 7 days. In embodiments where various holidays (e.g., Christmas and New Year's holidays) are modeled as separate periods, the frequency of a particular holiday period is once per year.

[0210] A season index can be a multiplicative season factor or an additive season term applied to an interval associated with the season index. For example, in embodiments where a multiplicative season factor is used to represent a season index, if an interval '9-10 AM' is associated with a season factor of 1.3, any measurement value sampled during the 9-10 AM interval can be adjusted up by 30% (i.e., multiplied by 1.3). In embodiments where a season index is represented by an additive season term, the additive season term is added to a measurement value.

[0211] A season categorizes a set of intervals according to some criteria. For example, given a time period of a year, the 12 periods of January, February, March, April, May, June, July, August, September, October, November, and December can be categorized into the following four northern meteorological seasons:

[0212] December, January, and February are categorized as a winter season

[0213] March, April, and May are categorized as a spring season

[0214] June, July, and August are categorized as a summer season

[0215] September, October, and November are categorized as an autumn season

[0216] Some embodiments can divide a weekday period into 96 15-minute intervals. In this regard, there are 96 season indices, where each of the 96 weekday season indices (i.e., weekday factors) maps to a different one of the 96 weekday intervals. Similarly, some embodiments can divide a weekend period into 192 15-minute intervals, resulting in 192 season indices, where each of the 192 weekend season indices (i.e., weekend factors) maps to a different one of the 192 weekend intervals.

[0217] To separate the seasonal pattern for the weekday period and the seasonal pattern for the weekend period, certain embodiments can apply the multi-period trend analysis filter to the weekday period separately from applying the multi-period trend analysis filter to the weekend period. Then, some embodiments can renormalize the weekday factor and the weekend factor such that the seasonal factor 1 represents a common reference level for both the weekday period and the weekend period. Thus, a seasonal factor greater than 1 can represent a heap usage that is higher than average during the interval to which the seasonal factor applies. Meanwhile, another seasonal factor less than 1 can represent a heap usage that is lower than average during the other interval to which the other seasonal factor applies.

[0218] In some embodiments, the techniques for multi-period trend analysis can be extended to separate holidays (e.g., Labor Day, Christmas, New Year's Day, etc.) as a separate period, where the holiday period repeats with a frequency of once every 12 months. Meanwhile, the weekday period repeats with a frequency of once every 24 hours, and the weekend period repeats with a frequency of once every 7 days. In such embodiments, the seasonal factor for the holiday period, the seasonal factor for the weekday period, and the seasonal factor for the weekend period can all be renormalized together such that the seasonal factor 1 represents a common reference level for the weekday period, the weekend period, and the holiday period.

[0219] Given a period (e.g., a weekday period, a weekend period, or a holiday / year period, etc.), let P represent the number of cycles of the period covered by a given set of measurement data (e.g., a time series of heap usage measurements across a particular time period) and let K represent the number of intervals within the period covered by the given set of data. If L represents the number of seasonal indices in the period, then K = P * L. For example, if there is at least 3 years of data within the set of data, one period corresponds to one year, and one interval corresponds to one month, then the number of available cycles P for the period is 3, and the number of available monthly intervals is 36.

[0220] Some embodiments can compute the average heap usage for each interval of a period based on data across multiple cycles of the period. Specifically, some embodiments can enumerate the intervals from 0 to (K - 1) and compute the average heap usage for each enumerated interval using the following formula

[0221]

[0222] k = 0, 1,..., K - 1; N k is the number of samples in interval k; and t i is the time of sample number i in interval k

[0223] Some embodiments can also compute the average heap usage for each period of the time period based on data across multiple cycles of the time period. Specifically, some embodiments can enumerate the periods of the time period from 0 to (P-1) and compute the average heap usage D p :

[0224]

[0225] N p is the number of samples in period p of the time period; and t i is the time of sample number i in period p of the time period

[0226] To initialize the seasonality factors for the time period, some embodiments can compute the seasonality factor for each season index in the time period using the following formula:

[0227]

[0228] Specifically, the seasonality factor for a particular interval can be equal to the ratio of the average heap usage for that interval across the entire dataset (computed by averaging the average heap usage for all identical intervals (e.g., all 9-10 AM intervals) in the entire dataset (e.g., the dataset spanning the entire week)) and the average heap usage for that time period across the entire dataset.

[0229] C. Re-normalization

[0230] As described above, some embodiments can re-normalize the weekday seasonality factors and the weekend seasonality factors such that the seasonality factor “1” represents a common reference level for the weekday time periods and the weekend time periods.

[0231] In general, certain embodiments can perform re-normalization by computing a weighted average of the seasonality factors across all time periods and dividing each seasonality factor by the weighted average. Consider the following example involving seasonality indices for multiple time periods of different lengths, where each time period is divided into 15-minute intervals:

[0232] Seasonality indices for weekdays: D i , i = 1, 2,... 96

[0233] Seasonality indices for weekends: E i , i = 1, 2,..., 192

[0234] Seasonality indices for individual holidays: H k,i , i = 1, 2,..., 96; k = 1, 2,... 10

[0235] Assume that in a particular year, there are 253 workdays (excluding holidays), 50.5 weekends, and 10 holidays, for a total of 253 + 50.5*2 + 10 = 364 days. In this example, some embodiments can use the following formula to calculate a weighted average of the seasonality factors, “A”, where the weights are proportional to the number of periods in each segment of the year (e.g., workday segments, weekend segments, and 10 individual holiday segments).

[0236]

[0237] Some embodiments can re-normalize the seasonality factors by dividing each seasonality factor D i ,E i and H k,i by A to obtain a new re-normalized seasonality factor for each segment.

[0238] Returning to the step of identifying stack traces for a heap hoard, after initializing the seasonality indices using the above formula, some embodiments can re-normalize the workday factors and weekend factors by dividing each weekend factor and each workday factor by the normalization factor, as follows:

[0239]

[0240] After re-normalizing the workday factors and weekend seasonality factors, the seasonality factor 1 should represent a common reference level for both the workday factors and the weekend factors.

[0241] D. Smoothing Spline Fitting

[0242] As mentioned above, some embodiments can fit a smoothing spline across multiple segments to provide a smooth transition between the periods of a segment (e.g., between two workday segments) or between the periods of two adjacent segments (e.g., between a workday segment and a weekend). In particular, fitting the spline can involve connecting the seasonality indices of one or more segments to smooth the transition between those segments.

[0243] In general, when certain embodiments (e.g., filters) reach the end of a period of segment A i and start a new period of segment A i , such as when repeating the workday period at the transitions from Monday to Tuesday, Tuesday to Wednesday, Wednesday to Thursday, and Thursday to Friday, such embodiments can connect the three sequences of seasonality indices A i and fit a smoothing spline across the entire sequence. Some embodiments can then take the middle segment of the smoothed sequence to represent a new smoothed seasonality index A i .

[0244] When certain embodiments (e.g., filters) reach the end of a period A i and start a new cycle of an adjacent period B i , such as when transitioning from Friday to Saturday, some embodiments can concatenate a sequence of season indices A i , a sequence of season indices B i , and a sequence of season indices C i for the period after period B i , and fit a smooth spline across the entire sequence. Some embodiments can then employ the middle segment of the smoothed sequence to represent a new smoothed season index B i . Some embodiments can also employ the first segment of the smoothed sequence to represent a smoothed season index A i .

[0245] When certain embodiments (e.g., filters) reach the end of a period B i and start a new cycle of an adjacent period C i , such as when transitioning from Sunday to Monday, some embodiments can concatenate a sequence of season indices A i for the period before period B i , a sequence of season indices B i , and a sequence of season indices C i , and fit a smooth spline across the entire sequence. Some embodiments can then employ the middle segment of the smoothed sequence to represent a new smoothed season index B i . Some embodiments can also employ the third segment of the smoothed sequence to represent a new smoothed season index C i .

[0246] With respect to cloud services, the load cycles during weekends and holidays are often different from the load cycles during weekdays. Conventional seasonal trend analysis solutions can typically only represent a season index for one period. To separate the season index for weekends from the season index for regular weekdays, such conventional solutions can rely on extending the range of a period to an entire week or an entire month. Furthermore, such conventional solutions can treat holidays separately.

[0247] Returning to the steps for identifying a stack trace for a heap dump, to smooth the weekday season factors, some embodiments can compose a season factor array by concatenating three sequences of weekday factors. For example, some embodiments can generate an array by executing the following code in the R programming language:

[0248] factors <- c(smoothedWeekdaySeasonalFactor, smoothedWeekdaySeasonalFactor, smoothedWeekdaySeasonalFactor)

[0249] Next, some embodiments can apply a spline to smooth the array of weekday factors. For example, some embodiments can call the smooth.spline function of R with a smoothing parameter of 0.3 to smooth the factors:

[0250] extendedWeekdayIndices <- 1 : (3 * 96)

[0251] f <- smooth.spline(extendedWeekdayIndices, factors, spar = 0.3)

[0252] Then, some embodiments can designate the middle sequence within the array (i.e., the middle 96 weekday factors) as the smoothed weekday factors. For example, some embodiments can obtain the smoothed weekday factors by executing the following code in the R programming language:

[0253] sandwichWeekdayIndices <- (96 + 1) : (96 * 2)

[0254] smoothedWeekdaySeasonalFactor <- predict(f, sandwichWeekdayIndices)$y

[0255] In a similar manner to smoothing the weekday factors, some embodiments can apply a spline to smooth the weekend factors. In particular, some embodiments can compose an array of seasonal factors by concatenating the sequence of weekend factors between two sequences of weekday factors. For example, some embodiments can generate the array by executing the following code in the R programming language:

[0256] factors <- c(smoothedWeekdaySeasonalFactor, smoothedWeekendSeasonalFactor, smoothedWeekdaySeasonalFactor)

[0257] Next, some embodiments can apply a spline to smooth the array of weekday and weekend factors. For example, some embodiments can call the smooth.spline function of R with a smoothing parameter of 0.3 to smooth the factors:

[0258] extendedWeekendIndices <- 1 : (2 * 96 + 192)

[0259] f <- smooth.spline(extendedWeekendIndices, factors, spar = 0.3)

[0260] Then, some embodiments can designate the middle sequence within the array (i.e., the middle 192 seasonal factors within the array, which are the weekend factors) as smoothed weekend factors. For example, some embodiments can obtain the smoothed weekend factors by executing the following code in the R programming language:

[0261] sandwichWeekendIndices <- (96 + 1) : (96 + 192)

[0262] smoothedWeekendSeasonalFactor <- predict(f, sandwichWeekendIndices)$y

[0263] It should be noted that some embodiments can represent the 96 weekday seasonal indices and the 192 weekend seasonal indices separately in order to separate the seasonal patterns observed during weekdays from the seasonal patterns observed during weekends. In some embodiments, the time series of sequentially filtered heap usage statistics can involve five sets of exponential filters, including a set for the heap usage measurements, a set for the seasonal factors, a set for the linear trend, a set for the accelerating trend, and a set for the residuals.

[0264] As mentioned above, in order to ensure sample accuracy, heap allocation measurements can be taken during a full garbage collection (GC) cycle that occurs at irregular time intervals. In cases where the heap usage is particularly high, the sampling interval can become arbitrarily close to zero due to the continuous garbage collection. Since the prediction involves the estimation of the rate of change, if the irregular time intervals become arbitrarily close to zero, the rate of change can become a random variable of the Cauchy distribution whose mean and standard deviation are indeterminate. Therefore, some embodiments can employ Holt’s double exponential filter, Winters’ triple exponential filter, Wright’s extension for irregular time intervals, Hanzak’s adjustment factor for time-close intervals, and outlier detection and clipping with adaptive scaling of the outlier cutoff, to overcome the Cauchy distribution problem for determining the seasonal trend of the statistics determined in association with the full GC. In some embodiments, the five sets of exponential filters can be applied sequentially to the time series to estimate the weekday factors and the weekend factors.

[0265] When certain embodiments (e.g., filters) reach the end of a processing cycle for a weekday and weekend period, such embodiments can divide each weekend factor and weekday factor by a normalization factor

[0266]

[0267] After the end of each period, some embodiments can apply a spline to smooth the seasonal factors. For example, when reaching the end of a weekday period before another weekday period (i.e., when transitioning from Monday to Tuesday, from Tuesday to Wednesday, from Wednesday to Thursday, or from Thursday to Friday), some embodiments can compose an array of seasonal factors by concatenating the three sequences of weekday factors. For example, some embodiments can generate the array by executing the following code in the R programming language:

[0268] factors <- c(smoothedWeekdaySeasonalFactor, smoothedWeekdaySeasonalFactor, smoothedWeekdaySeasonalFactor)

[0269] Next, some embodiments can apply a spline to smooth the array of weekday factors. For example, some embodiments can call the smooth.spline function of R with a smoothing parameter of 0.3 to smooth the factors:

[0270] extendedWeekdayIndices <- 1 : (3 * 96)

[0271] f <- smooth.spline(extendedWeekdayIndices, factors, spar = 0.3)

[0272] Then, some embodiments can designate the middle sequence within the array (i.e., the middle 96 weekday factors) as the smoothed weekday factors. For example, some embodiments can obtain the smoothed weekday factors by executing the following code in the R programming language:

[0273] sandwichWeekdayIndices <- (96 + 1) : (96 * 2)

[0274] smoothedWeekdaySeasonalFactor <- predict(f, sandwichWeekdayIndices)$y

[0275] In different instances, when reaching the end of the weekday period before the weekend period (i.e., transitioning from Friday to Saturday), some embodiments can compose the array of seasonal factors by concatenating the sequence of weekend seasonal factors between the two sequences of weekday seasonal factors. For example, some embodiments can generate the array by executing the following code in the R programming language:

[0276] factors <- c(smoothedWeekdaySeasonalFactor, smoothedWeekendSeasonalFactor, smoothedWeekdaySeasonalFactor)

[0277] Next, some embodiments can apply a spline to smooth the array of weekday and weekend factors. For example, some embodiments can call the smooth.spline function of R with a smoothing parameter of 0.3 to smooth the factors:

[0278] extendedWeekendIndices <- 1 : (2 * 96 + 192)

[0279] f <- smooth.spline(extendedWeekendIndices, factors, spar = 0.3)

[0280] Then, some embodiments can designate the left sequence within the array (i.e., the first 96 seasonal factors within the array, which are weekday factors) as the smoothed weekday factors. For example, some embodiments can obtain the smoothed weekday factors by executing the following code in the R programming language:

[0281] leftsideWeekendIndices <- 1 : 96

[0282] smoothedWeekdaySeasonalFactor <- predict(f, leftsideWeekendIndices)$y

[0283] In different instances, when reaching the end of the weekend period (i.e., transitioning from Sunday to Monday), some embodiments can compose the array of seasonal factors by concatenating the sequence of weekend seasonal factors between the two sequences of weekday seasonal factors. For example, some embodiments can generate the array by executing the following code in the R programming language:

[0284] factors <- c(smoothedWeekdaySeasonalFactor, smoothedWeekendSeasonalFactor, smoothedWeekdaySeasonalFactor)

[0285] Next, some embodiments can apply a spline to smooth the array of weekday and weekend factors. For example, some embodiments can call the smooth.spline function of R with a smoothing parameter of 0.3 to smooth the factors:

[0286] extendedWeekendIndices <- 1 : (2 * 96 + 192)

[0287] f <- smooth.spline(extendedWeekendIndices, factors, spar = 0.3)

[0288] Then, some embodiments can designate the middle sequence within the array (i.e., the middle 192 seasonal factors within the array, which are the weekend factors) as the smoothed weekend factors. For example, some embodiments can obtain the smoothed weekend factors by executing the following code in the R programming language:

[0289] sandwichWeekendIndices <- (96 + 1) : (96 + 192)

[0290] smoothedWeekendSeasonalFactor <- predict(f, sandwichWeekendIndices)$y

[0291] Some embodiments can also designate the right sequence within the array (i.e., the last 96 seasonal factors within the array, which are the weekday factors) as the smoothed weekday factors. For example, some embodiments can obtain the smoothed weekday factors by executing the following code in the R programming language:

[0292] rightsideWeekendIndices <- (96 + 192 + 1) : (2 * 96 + 192)

[0293] smoothedWeekdaySeasonalFactor <- predict(f, rightsideWeekendIndices)$y

[0294] It should be noted that some embodiments can perform the above-described renormalization and smoothing spline fitting each time a sequential filter reaches the end of a period of a time segment and starts a new period of the same time segment (e.g., the sequential filter reaches the end of a Monday) or the end of a period of a time segment and starts a new period of an adjacent time segment (e.g., the sequential filter reaches the end of a Friday).

[0295] E. Testing for Seasonal Periodicity

[0296] Some embodiments can test for the presence of seasonal periodicity for one or more candidate time segments of a data set to determine whether a separate seasonal index sequence should be represented for the time segment. In general, to determine whether a data set exhibits seasonal periodicity for a particular time segment, some embodiments can perform the following steps.

[0297] Let Q represent the number of seasonal indices in a time segment, P represent the number of available periods of the time segment, and K represent the number of available intervals across the periods of the time segment, where K = P*Q.

[0298] Some embodiments can compute the average metric in each interval of the periods of the time segment. To do so, some embodiments can enumerate the intervals from 0 to (K-1) and compute the average metric for each interval of the time segment using the following formula:

[0299]

[0300] k = 0, 1,..., K-1; N k is the number of samples in interval k; and t i is the time of sample number i in interval k

[0301] Then, some embodiments can compute the average metric for each period of the time segment. To do so, some embodiments can enumerate the periods of the time segment from 0 to (P-1) and compute the average metric for each period of the time segment using the following formula:

[0302]

[0303] N p is the number of samples in period p of the time segment;

[0304] and t i is the time of sample number i in period p of the time segment

[0305] Then, some embodiments can compute the sequence of seasonal indices for each period p of the time segment using the following formula:

[0306]

[0307] p = 0, 1,..., P - 1

[0308] Then, some embodiments can apply a null hypothesis test to detect whether a seasonal periodicity exists in the period. In this regard, the null hypothesis to be tested can correspond to the correlation coefficient r u,v between the seasonal index of the most recent period 'u' and the seasonal index of the previous period 'v' being zero. Specifically, some embodiments can determine the correlation coefficient r u,v :

[0309]

[0310]

[0311]

[0312] Some embodiments can employ various techniques to determine whether the correlation coefficient r u,v is large enough to indicate that a common seasonal periodicity exists between periods 'u' and 'v' above a significance level. For example, some embodiments can employ a Student-t test, a permutation test, or a Fisher transformation.

[0313] To test this hypothesis, some embodiments can define one or more test statistics, which can be a function of the parameters. In this case, what is being tested is the correlation coefficient r u,v . The following test statistic t has a Student's t-distribution with degrees of freedom 'n - 2' and is a function of r u,v . Some embodiments define the null hypothesis r u,v = 0, which assumes that the seasonal indices are not correlated between periods of the period. Some embodiments can search for evidence to reject the null hypothesis (i.e., r u,v = 0) by accepting the alternative hypothesis.

[0314]

[0315] Let F(t) denote the probability distribution. Given a significance level of 0.1, let t 0.9,(n-5) denote the value of the random variable t such that F(t) = 0.9. The alternative hypothesis is the one-sided condition:

[0316]

[0317] If the condition is true, the alternative hypothesis is accepted, which indicates that there is a common seasonal periodicity between the cycle years 'u' and 'v'. If there is a common seasonal periodicity between the most recent cycle and the previous cycle, some embodiments can proceed to compute the seasonal factors for each seasonal index of the cycle. Some embodiments apply the above formula to detect the presence of an annual seasonal periodicity in the heap usage of the software execution environment, as described below.

[0318] F. Detecting Annual Seasonal Periodicity in Heap Usage

[0319] When analyzing multi-year heap usage statistics of a software execution environment, some embodiments can detect more than one seasonal trend at different time scales. For example, such embodiments can detect a multi-year time series of heap usage statistics, an annual seasonal trend, and a daily seasonal trend, all of which are superimposed on a multi-seasonal trend. Accordingly, some embodiments can employ an appropriate time scale to analyze the annual seasonal trend, where the time scale has a period corresponding to 1 year and an interval corresponding to 1 month. Thus, a one-year long period can be divided into 12 one-month long intervals.

[0320] To determine whether a dataset exhibits an annual seasonal periodicity, some embodiments can first determine the multiplicative factor for the monthly indices in the dataset.

[0321] In a particular example, let P represent the number of years available in the dataset (i.e., the number of cycles of a one-year period). Furthermore, let Q represent the number of months available in the dataset (i.e., the number of intervals within a cycle). Thus, Q = 12 * P. Let K represent the number of weekdays or weekends available in the dataset. Let the index k range from 0 to (K - 1) to represent the enumeration of the available weekdays or weekends. Let N k represent the number of samples in the kth weekday or weekend. Using the following formula, some embodiments can apply the following formula to compute the average heap usage for each weekday or weekend in the dataset:

[0322] and N k is the number of samples in day k

[0323] Some embodiments can define a function H, H : (Year x Integer) -> Index, which maps an ordered pair comprising an index of a year and an integer corresponding to an index of a weekday or weekend within that year. Using the following formula, some embodiments can then compute the average heap usage per year from the average heap usage of the weekdays or weekends within that year:

[0324]

[0325] N pis the number of weekdays in a year period p;

[0326] and H(p, i) is the index of the i-th weekday in a year period p.

[0327] Some embodiments define a function G, G : (Month x Integer) → Index, which maps ordered pairs comprising an index of a month and an integer corresponding to an index of a weekday or weekend day within that month. Using the following formula, some embodiments can calculate an average heap usage for each monthly interval of the period from an average heap usage of weekdays or weekend days within that month:

[0328]

[0329] M m is the number of weekdays in a month m;

[0330] G(m, i) is the index of the i-th weekday in a month m;

[0331] and p is the index of a period of a year period.

[0332] In particular, the above formula yields which corresponds to a 12-month average for each period of a year-long period p. The average heap usage for a month can be divided by the annual average heap usage to obtain a multiplicative factor for the monthly index corresponding to that month. In such embodiments, if it is determined that the multiplicative factor for a particular month is greater than 1, then the heap usage in that month is above average. On the other hand, if it is determined that the multiplicative factor for a particular month is less than 1, then the heap usage in that month is below average.

[0333] After determining the multiplicative factor for the monthly index, some embodiments can apply a null hypothesis test to detect whether there is an annual seasonal cycle. In this regard, the null hypothesis to be tested can correspond to the hypothesis that the correlation coefficient r u,v between the monthly indices of the most recent year 'u' and the monthly indices of the previous year 'v' is zero. In particular, some embodiments can determine the correlation coefficient r u,v :

[0334]

[0335]

[0336]

[0337] Some embodiments can employ various techniques to determine the correlation coefficient r u,vwhether it is large enough to indicate that there is a common seasonal cycle between years 'u' and 'v' above a significance level. For example, some embodiments can employ a Student's t-test, a permutation test, or Fisher's transform.

[0338] If the null hypothesis is true (i.e., r u,v = 0), then the following test statistic t has a Student's t-distribution with 'n - 2' degrees of freedom

[0339]

[0340] Let F(t) denote the probability distribution. Given a significance level of 0.1, let t 0.9,(n-2) denote the value of the random variable t such that F(t) = 0.9. The alternative hypothesis is the one-sided condition:

[0341]

[0342] The condition of accepting the alternative hypothesis indicates that there is a common seasonal cycle between years 'u' and 'v'.

[0343] G. Determining the Season of the Year with the Highest Heap Usage

[0344] If it is determined that there is a common seasonal cycle between the most recent year and the previous years, some embodiments can calculate a seasonal factor for each month enumerated by monthly seasonal index 0 through 11 by employing the following formula:

[0345]

[0346] In an alternative embodiment, the monthly index of the most recent year (i.e., the cycle) can be used as the monthly seasonal index, as indicated by the following formula:

[0347]

[0348] To categorize the season of the year with the highest heap usage, some embodiments can identify the seasonal index N that corresponds to the month with the largest seasonal factor over a one-year long period. These embodiments can then use index N as a seed. Starting from N, such embodiments can scan the seasonal indices less than or greater than N (i.e., seasonal indices 0, 1, 2... N-1, N+1, N+2) that have a seasonal factor greater than a threshold T. In some embodiments, T is greater than 1. If there is more than one N such that then some embodiments can categorize more than one disjoint season of the highest heap usage. The function selects the s-th element of the sequence of indices N, the parameter s is used to select the s-th element of the sequence of indices N, Some embodiments can use the following recursive method to categorize the set of monthly season indices for a high heap usage season:

[0349]

[0350]

[0351]

[0352] It should be noted that the above recursion involves an unconstrained variable s that can be used to break ties. In some embodiments, s = 1 by default.

[0353] In certain embodiments, the closure V of season indices categorizes annual high heap usage seasons. A threshold T can be set as a percentage, such as 85% of the range of season factors. For example, suppose the season factors of 12 monthly season indices in a year-long period are given by the following table.

[0354]

[0355]

[0356] The range of multiplicative season factors is (1.34 - 0.76), i.e., 0.58. Thus, 85% of the range of season factors is (0.76 + 0.85 * 0.58), i.e., 1.253. Given the 85% threshold T, T = 1.25. Thus, such embodiments can categorize May, June, and July as an annual high heap usage season.

[0357] Some embodiments can select a segment of the most recent cycle of the dataset spanning a year-long period. For example, among the cycles of 2013, 2014, 2015, and 2016, these embodiments can select a segment covering data from 2015 to 2016. The selected data segment can span 2 or more weeks within an annual high heap usage season in the heap usage statistics. For example, if the season factors are given by the following table, then the data segment can be selected from November 2015, December 2015, and January 2016.

[0358] January February March April May June July August September October November December 1.26 1.12 1.01 0.99 0.95 0.9 0.76 0.82 1.0 1.2 1.29 1.34

[0359] H. Regression of filter constants and time zone offsets

[0360] Some embodiments can include an estimate of the time zone offset. If the time zone offset is not available, some embodiments can perform a non-linear regression on the segment of the dataset to estimate the time zone offset and use it to filter the data. By providing an estimate of the time zone offset, some embodiments can improve the estimation of the seasonal index in the transition between time periods.

[0361] In particular, some embodiments can perform a non-linear regression with the following filter constants (i.e., regression parameters as independent variables): measureFilterConstant a, rateFilterConstant b, accelerationFilterConstant k, seasonalFactorFilterConstant g, errorResidualFilterConstant d, and timeZoneOffset tz to minimize the mean squared error (MSE) and / or the mean absolute deviation (MAD) of the 1-step prediction residuals. In some embodiments, the timestamps can be shifted by the time zone offset tz in the regression. Some embodiments can use an optimization routine (e.g., an optimization routine provided by the R programming language) to apply the non-linear multiple regression. Some embodiments can derive the weekday and weekend seasonal factors using the optimal values of a, b, k, g, d, and tz as indicated in the following equations as employed by these embodiments:

[0362]

[0363]

[0364] Some embodiments include the time zone offset as a regression parameter so that the transition between the periods of a time period or between two adjacent time periods can be as accurate as possible.

[0365] I. Rank thread classes by degree of correlation

[0366] Once the annual high heap usage season is determined, some embodiments can compute and / or obtain the weekday / weekend factors representing the daily / weekly seasonal cycles covered by the recent (e.g., most recent) annual high heap usage season. It should be noted that the daily / weekly seasonal cycles in this segment of the dataset (i.e., during the annual high heap usage season) can be more pronounced than at other times (i.e., outside of the annual high heap usage season). Thus, determining the degree of correlation between the seasonal trends in the heap usage and the seasonal trends in the intensity statistics of one or more thread classes can be based on this segment of the dataset. In other words, for the correlation analysis, some embodiments can use the same time interval for deriving the seasonal trends of the individual thread classes as the time interval covered by the most recent annual high heap usage season.

[0367] It should be noted that to determine the seasonal trend of the strength statistics of a particular thread class, some embodiments can employ the techniques described above for determining the seasonal trend of heap usage. In other words, the seasonal trend of both thread strength statistics and heap usage statistics can involve using the same number of seasonal indices for weekday and weekend time periods (e.g., 96 seasonal indices for weekday time periods and 192 seasonal indices for weekend time periods).

[0368] After determining the seasonal trend of heap usage and the seasonal trend of the strength statistics of one or more thread classes, some embodiments can then compute a degree of correlation between the seasonal trend of heap usage and the seasonal trend of the strength statistics of the thread class for each of the one or more thread classes. In particular, the degree of correlation can be computed for the sequence of 96 seasonal factors or 192 seasonal factors. It should be noted that computing the degree of correlation between the seasonal trends can be more efficient than computing the degree of correlation between the sequences of heap usage metrics and thread strength metrics because the sequences of metrics can be much longer.

[0369] Let H represent a sequence of N seasonal factors for heap usage. Let T represent a sequence of N seasonal factors for the thread strength of a class of threads. The correlation coefficient of the two sequences of seasonal factors is given by CorrelationCoefficient(H, T) as defined below:

[0370]

[0371]

[0372]

[0373]

[0374]

[0375] Some embodiments can derive weekday and weekend seasonal factors for heap usage by regressing heap usage statistics included in a recent year high heap usage season. Let us denote the time interval for this segment of the dataset as (t1, t2). To analyze the correlation between the seasonal factors for the intensity statistics of a class of threads and the seasonal factors for heap usage, some embodiments can employ the seasonal factors from the same time interval (t1, t2) in the seasonal factor time series in the SeasonalTrendlnfo associated with the thread class. Specifically, this seasonal factor time series can be stored in the smoothedWeekdaySeasonalFactor member variable and the smoothedWeekendSeasonalFactor member variable in the associated SeasonalTrendinglnfo object.

[0376] Some embodiments can traverse all ThreadClassificationlnfo objects for thread classes and recursively traverse the Segmentlnfo objects in each ThreadClassificationlnfo object to collect the SeasonalTrendlnfo objects contained in the ThreadClassificationlnfo objects and the Segmentlnfo objects. In calculating the CorrelationCoefficient(H, T) between heap usage and each thread class using the formula identified above, some embodiments can retrieve the weekday factor or the weekend factor in each SeasonalTrendlnfo object. Once the correlation has been calculated for each thread class, some embodiments can rank the thread classes by the degree of correlation of the thread class to the seasonal trend of heap usage. The high ranking thread classes can then be classified as heap hoarding thread classes. Some embodiments can then analyze the stack traces and code associated with the heap hoarding thread classes to identify inefficient memory usage that can be manually or automatically corrected and / or improved.

[0377] It should be noted that some embodiments can be extended to determine the correlation coefficient based on time periods other than weekday and weekend periods, such as end-of-quarter periods.

[0378] Figure 9 FIG. 9 illustrates a flowchart 900 of a process for identifying code that can contribute to high heap usage within a software execution environment, in accordance with some embodiments. In some embodiments, the process depicted in flowchart 900 can be implemented by a computer system having one or more processors (e.g., computer system 1700 of FIG. 8), where the one or more processors can perform the steps based on computer code stored in a computer-readable medium. Figure 17 In some embodiments, the process depicted in flowchart 900 can be implemented by a computer system having one or more processors (e.g., computer system 1700 of FIG. 8), where the one or more processors can perform the steps based on computer code stored in a computer-readable medium.Figure 9 The steps described in the middle can be performed in any order, and with or without any other steps.

[0379] Flowchart 900 begins at step 902, where an embodiment determines a length of time that heap usage of one or more processes exceeds a threshold. The length of time can correspond to a season of annual high heap usage, and the threshold can correspond to a percentage of a range of seasonal factors assigned to intervals (e.g., 15-minute intervals) across one or more periods (e.g., weekday periods and weekend periods). In some embodiments, the threshold can be set by selecting a percentage. Once the percentage is selected, the threshold can be given by a product of the range of seasonal factors and the percentage and a sum of minimum seasonal factors. For example, if the selected percentage is 85%, the minimum seasonal factor is 0.76, and the maximum seasonal factor is 1.34, then the threshold can be given as (0.76 + 0.85 * (1.34 - 0.76)), which is 1.253. Thus, any interval having a multiplicative seasonal factor that exceeds 1.25 can be determined to be part of the length of time that heap usage exceeds the threshold.

[0380] At step 904, an embodiment determines heap information for the one or more processes during the length of time. The heap information can correspond to amounts of heap memory used by the one or more processes within the software execution environment at different points during the length of time. For example, the heap information can be based on heap usage metrics obtained from the software execution environment at irregular intervals (e.g., during full GCs). Further, the software execution environment can correspond to a production environment that includes one or more virtual machines (e.g., a JVM), and the one or more processes can support one or more cloud services.

[0381] At step 906, an embodiment determines thread information for the one or more processes during the length of time. In some embodiments, for each of one or more thread classes determined from the analyzed thread dumps, the thread information can include a thread intensity seasonal factor for each of a plurality of intervals.

[0382] In some embodiments, the heap information can include a heap usage seasonal factor for each of a plurality of intervals. In particular, the length of time can span one or more cycles of a first period (e.g., a weekday period) having a first length and one or more cycles of a second period (e.g., a weekend period) having a second length. Each period can be partitioned into a plurality of intervals. For example, a weekday period can be partitioned into 96 15-minute intervals, and a weekend period can be partitioned into 192 15-minute intervals.

[0383] It should be noted that each of the plurality of intervals can be mapped to a particular season (i.e., season index) of one of the time periods. For each season index, some embodiments can determine a heap usage season factor, and for each thread class determined, a thread intensity season factor, which can result in each interval being associated with a heap usage season factor and a plurality of thread intensity season factors (one thread intensity season factor per thread class). For example, assuming three different thread classes are discovered, then a weekday time period can have 96 heap usage season factors and 288 thread intensity season factors (96 thread intensity season factors per thread class of the three thread classes), while a weekend time period can have 192 heap usage season factors and 576 thread intensity season factors.

[0384] At step 908, embodiments correlate the heap information with the thread information to identify one or more lines of code of the one or more processes that correspond to heap usage that exceeds a threshold. The correlation of the heap information with the thread information is discussed in more detail below with respect to Figure 10 The step of correlating the heap information with the thread information is discussed in more detail.

[0385] At step 910, in response to identifying the one or more lines of code, embodiments initiate one or more actions associated with the one or more lines of code. For example, embodiments can generate an alert associated with the one or more lines of code that is sent to a relevant personnel or a code optimization tool. In response, the identified lines of code can be investigated and / or optimized. Alternatively, some embodiments can optimize the one or more lines of code to use heap memory in a more efficient manner.

[0386] Figure 10 FIG. 10 illustrates a flowchart 1000 of a process for computing a degree of correlation between thread classes and high heap usage, in accordance with some embodiments. In some embodiments, the process depicted in flowchart 1000 can be implemented by a computer system having one or more processors (e.g., computer system 1700 of FIG. 9), where the one or more processors can perform the steps based on computer code stored in a computer-readable medium. Figure 17 The steps described in FIG. 9 can be performed in any order, and with or without any of the other steps. Figure 10 The steps described in FIG. 9 can be performed in any order, and with or without any of the other steps.

[0387] Flowchart 1000 begins at step 1002, where embodiments obtain one or more thread dumps of one or more processes. As described above, the control system can periodically cause the software execution environment to perform thread dumps, where each thread dump includes one or more stack traces of threads produced by one or more processes executing within the software execution environment.

[0388] At step 1004, embodiments obtain one or more thread classes by receiving one or more thread dumps from one or more threads and categorizing each received thread based on a stack trace corresponding to the received thread. Once all thread dumps have been received and processed, embodiments can analyze each of the one or more thread classes to determine a degree of correlation between each thread class and high heap usage in steps 1006-1016.

[0389] At decision 1006, embodiments determine whether there is another thread class of the one or more thread classes for determining a degree of correlation with high heap usage. If so, embodiments can proceed to step 1008. Otherwise, embodiments can proceed to step 1018.

[0390] At optional step 1008, embodiments compute an average of the plurality of interval heap usage seasonality factors. At step 1010, embodiments compute an average of the plurality of interval thread intensity seasonality factors for the thread class. At optional step 1012, embodiments compute a variance of the plurality of interval heap usage seasonality factors. At step 1014, embodiments compute a variance of the plurality of interval thread intensity seasonality factors for the thread class. At step 1016, embodiments compute a degree of correlation between the thread class and heap usage exceeding a threshold.

[0391] At step 1018, embodiments select a given thread class from the one or more thread classes having a highest degree of correlation with heap usage exceeding a threshold. In particular, once the degree of correlation is computed for each thread class, some embodiments can rank the thread classes by the degree of correlation of the thread class with the heap usage seasonality trend. The high ranking thread class can then be selected as the given thread class.

[0392] At step 1020, embodiments identify one or more lines of code that can have contributed significantly to the high heap usage based on the given thread class. In particular, some embodiments can then analyze the file names and lines specified by the stack traces to locate the lines of code associated with the thread class of the heap dump. It should be noted that each thread of the one or more processes belonging to the given thread class executes the one or more lines of code.

[0393] VI. Overcoming Weak Exogeneity and Heteroskedasticity in Prediction

[0394] As described above, to ensure sample accuracy, heap allocation measurements can be taken during full garbage collection (GC) cycles that occur at irregular time intervals. In cases where heap usage is particularly high, the sampling interval can become arbitrarily close to zero due to the persistent garbage collection. Thus, time series data based on heap allocation measurements can exhibit weak exogeneity (where the process that generates the residuals depends to some extent on the process that generates the time intervals of full GC samples), as well as heteroscedasticity (where the variance of the residuals is not constant over time).

[0395] Conventionally, ordinary least squares regression that generates a linear trend assumes that the predictor variable and the response variable are generated by a process that is both exogenous and homoscedastic. However, with respect to a dataset based on measurements taken during full GC, the predictor variable (i.e., the irregular time intervals) and the response variable (i.e., the heap usage measurements taken during full GC) are not independent, as the frequency of taking full GC can increase when heap usage increases. Some embodiments can use robust and resistant regression methods to overcome the weak exogeneity and heteroscedasticity of the dataset.

[0396] Certain embodiments can utilize robust least squares regression to overcome the weak exogeneity and heteroscedasticity exhibited in these datasets. In particular, some embodiments can (1) decompose the time series of measurements into a de-seasonalized metric component (i.e., a de-seasonalized component) and a seasonal factor component (i.e., a seasonal effect factor), (2) apply a robust linear regression to the de-seasonalized metric component, (3) apply a smoothing spline filter to the seasonal factor component, and (4) reconstruct the linear regression line and the smoothed seasonal factor into a seasonal and linear trend model.

[0397] The least trimmed squares (LTS) estimator is a robust regression technique that is resistant to the influence of outliers. Given a set of N samples, the LTS estimator minimizes the sum of the smallest 50% squared residuals by trimming 50% of the samples corresponding to the largest squared residuals as outliers. The LTS estimator runs one iteration of ordinary least squares regression on all N samples to sort the residuals to select the smallest N / 2 residuals (i.e., the trimmed samples). Then, the LTS estimator iteratively re-runs the regression by updating the trimmed samples to reduce the mean of the squared residuals. However, the time complexity of the LTS algorithm can be relatively high compared to certain embodiments described below.

[0398] The generalized weighted least squares (WLS) estimator is a robust regression technique that multiplies the squared error residual of each sample by a weight that is inversely proportional to the variance of the sample. The weights employed by the WLS estimator can depend on prior knowledge of the data. For example, the prior knowledge can specify (1) the accuracy of different tools used to measure different sampling points, (2) the variance between redundant measurements corresponding to the same time instance, or (3) the variance between nearest neighbor groups of measurements. If the weights cannot be determined by prior knowledge, then the WLS estimator can run one iteration of ordinary least squares regression to estimate the residuals and iteratively re-run the regression using the inverse of the residuals as weights to produce a stable estimate of the linear model. However, the WLS algorithm has relatively high time complexity compared to certain embodiments described below.

[0399] In patent application 14 / 109,546 (which is incorporated herein by reference for all purposes), a set of equations for filtering the rate of change of a metric is disclosed. The filter monitors the trend of the metric:

[0400]

[0401]

[0402] Since the rate of change involves dividing by the length of the time interval (t n -t' n-1 ), some embodiments can adjust the filter parameters to give relatively small weight to samples when the length of the time interval (t n -t' n-1 ) is relatively short.

[0403] The filter parameters are adjusted by a scaling factor in the following equations:

[0404]

[0405]

[0406]

[0407] The rate filter parameters are used to filter the smoothed rate of change as follows. If the seasonal trend is not employed, then some embodiments can update the average value using the value as shown in the following formula:

[0408]

[0409] On the other hand, if seasonal trends are employed, then some embodiments can use one of the following formulas depending on whether the time falls in a weekend or weekday period, where and are the seasonal factors for weekend and weekday periods, respectively.

[0410]

[0411] Next, some embodiments can determine the de-seasonalized raw growth rate using the following formula:

[0412]

[0413] Then, some embodiments can update the moving average using the following formula:

[0414]

[0415] In particular, the rate filter parameter generated by the above equation represents a weight that is based on the length of the time interval that occurs between a particular sample and another sample that immediately precedes the particular sample. There is a one-to-one correspondence between the rate filter parameter and the measured data in the time series. Figure 11 depicts a plot of the filter parameter across the entire time range for an example data set. While the time range is divided into 6 overlapping sub-ranges, the plot in each sub-range shows a linear relationship between the sample time interval and the filter parameter. As can be seen from the plot, when the sample time interval is small, the filter parameter (i.e., the weight of the sample) is small. This adjustment dynamically reduces the weight of samples in the filter depending on the density of the sampling points around the current sampling point.

[0416] Some embodiments use the rate filter parameter to prune data points. Pruning data points can help to even the density of sampling points across the entire time range, thereby improving the robustness of the linear regression algorithm. With respect to data points representing measurements of heap usage in a software execution environment during a full GC cycle, data points that are close together can correspond to periods of higher heap usage (e.g., during load spikes) in which full GC is performed more often.

[0417] Some embodiments compare the rate filter parameter to a threshold value and exclude (i.e., prune) the corresponding data point from the robust linear regression if the rate filter parameter is less than the threshold value. Some embodiments can use the median or mean of the rate filter parameters as the threshold value. In particular, some embodiments can prune data points that are close together because such data points can represent a load surge or an outlier. Thus, some embodiments can mitigate weak exogeneity conditions by evening the density of data points along the time axis, which reduces the correlation between irregular time intervals and the residuals.

[0418] The time series for the prediction error residuals generated by the following equation in patent application 14 / 109,546 and the time series for the predicted measure This patent application is incorporated by reference herein for all purposes.

[0419]

[0420] Some embodiments can generate the error residuals for the predicted measure using the following formula:

[0421]

[0422]

[0423]

[0424] Because there is a correlation between the smoothed absolute error residuals generated by the filter and the variance of the residuals of the least squares regression, some embodiments can use the inverse of the smoothed absolute error residuals as the weights for the generalized weighted least squares regression. In doing so, some embodiments can mitigate heteroscedasticity conditions by assigning relatively small weights to sample values that have a relatively large deviation from the expected value. The expected value can represent the convolution of the near-neighbor group samples.

[0425] The following example code (written in the R programming language) illustrates how the pruned sample subset and the weights for the samples can be computed. As shown in the example code below, some embodiments can use the R function "rlm" that enables certain embodiments to specify the pruned sample subset and the weights for the samples for generating the weighted least squares regression. It should be noted that the rateFilterParameter, seasonalFactor, absoluteErrorResidual, measure, and time vectors in the example code are time series that have the same time range.

[0426]

[0427]

[0428] For each timestamp t corresponding to a timestamp of a data point n The rate filter parameter is given as a time series of values denoted by If where z is a threshold value, the corresponding data point Typically, some embodiments can use any value at the Nth percentile of the rate filter parameter (e.g., as the median of the 50th percentile) as the threshold value z.

[0429] In some embodiments, for each timestamp t corresponding to a timestamp of a data point n The absolute error residual is given as a time series of weights of samples at timestamp t n The absolute error residual is given as a time series of weights of samples at timestamp t may be inversely proportional to Some embodiments can compensate for variance changes between data points that represent short-term load surges or outliers.

[0430] To reduce the impact of outliers and short-term surges in heap usage on the linear regression, some embodiments can combine the technique of uniform density of data points with the technique of assigning smaller weights to samples that deviate from the sample values. In doing so, some embodiments can increase the robustness of the linear regression, which can help to capture long-term trends (e.g., in heap usage). It should be noted that using both techniques together can provide a better fit of the linear regression line to the data and can be more efficient than using a conventional LTS estimator or WLS estimator, which typically involves several iterations of regression.

[0431] To further improve the robustness of the regression, some embodiments can additionally identify transient states and remove sampling points that fall into the transient states as well as remove run-to-run segments that are outliers (e.g., segments of data corresponding to software execution environments that experience memory leaks, out-of-memory events, or very high growth rates).

[0432] Figure 12 Three trend graphs are shown, each derived by a different linear regression technique for heap usage in a production environment. The blue trend line 1205 can be derived by a standard linear regression algorithm that assigns equal weights to each sampling point. The brown trend line 1210 can be derived by a conventional robust regression algorithm. The red line 1215 represents a regression provided by the present embodiments described above, which is close to the brown trend line.

[0433] Figure 13 ​An additional graph is shown that illustrates how conventional regression techniques can provide incorrect results. As shown in the graph, a brown trend line 1305 representing conventional regression techniques closely fits two clusters of high density sample points. In contrast, a red line 1215 correctly tracks the trend in the sample points to provide a long-term prediction of heap usage in a software execution environment.

[0434] Figure 14 A flowchart 1400 of a process for generating a prediction of a signal is illustrated in accordance with some embodiments. In some embodiments, the process depicted in flowchart 1400 can be implemented by a computer system having one or more processors (e.g., computer system 1700 of FIG. 13), where the one or more processors can perform the steps based on computer code stored in a computer-readable medium. Figure 17 The steps described in FIG. 13 can be performed in any order, and with or without any other steps. Figure 14 The steps described in FIG. 13 can be performed in any order, and with or without any other steps.

[0435] Flowchart 1400 begins at step 1402, where an embodiment receives a signal comprising a plurality of metrics sampled over a time span from an environment in which one or more processes are being executed. In some embodiments, the plurality of metrics can be heap usage measurements taken by a control system that is monitoring heap usage within a software execution environment (e.g., a production environment), where the software execution environment comprises one or more executing processes.

[0436] At step 1404, an embodiment extracts a seasonal effect factor and a de-seasonalized component from the signal 1404. In some embodiments, the seasonal effect factor can correspond to a seasonality factor determined for each interval assigned to a time period of the dataset. In some embodiments, the de-seasonalized component can be obtained by applying the seasonality factor to the signal.

[0437] At step 1406, an embodiment applies one or more spline functions to the seasonal effect factor to generate a first model. In this regard, some embodiments can assign relatively small weights to sample values that sharply deviate from an expected value, where the expected value represents a convolution of a group of near neighbor samples.

[0438] At step 1408, an embodiment applies a linear regression technique to the de-seasonalized component to generate a second model. In particular, to compensate for relatively short time intervals experienced during high heap usage, some embodiments can adjust filter parameters to assign relatively small weights to samples taken during short intervals. Some embodiments can use a rate filter parameter to prune data points included in the dataset. Pruning data points can help to uniformly sample the density of points across the entire time range, thereby improving the robustness of the linear regression algorithm.

[0439] At step 1410, embodiments generate a prediction of the signal based on the first model and the second model. In some embodiments, the prediction of the signal can correspond to the regression line generated using the techniques described in steps 1406 and 1408. In particular, the generated prediction can better fit the signal.

[0440] At step 1412, embodiments initiate one or more actions associated with the environment based at least in part on the prediction. For example, if the prediction indicates that the heap usage will increase in the future, some embodiments can allocate additional resources (e.g., memory, RAM) to the software execution environment.

[0441] Figure 15 A simplified diagram depicting a distributed system 1500 for implementing embodiments is depicted. In the illustrated embodiment, the distributed system 1500 includes one or more client computing devices 1502, 1504, 1506, and 1508, which are configured to execute and operate a client application, such as a web browser, a proprietary client application, or the like, in conjunction with one or more services provided over the network 1510. The server 1512 can be communicatively coupled with the remote client computing devices 1502, 1504, 1506, and 1508 via the network 1510.

[0442] In various embodiments, the server 1512 can be adapted to execute one or more services or software applications. In certain embodiments, the server 1512 can also provide other services or software applications, which can include non-virtual and virtual environments. In some embodiments, these services can be provided as web-based or cloud services or under a Software as a Service (SaaS) model to users of the client computing devices 1502, 1504, 1506, and / or 1508. Users operating the client computing devices 1502, 1504, 1506, and / or 1508 can utilize one or more client applications to interact with the server 1512 to utilize the services provided by these components.

[0443] In Figure 15 In the configuration depicted in FIG. 15, the software components 1518, 1520, and 1522 of the system 1500 are shown as being implemented on the server 1512. In other embodiments, one or more of the components of the system 1500 and / or the services provided by these components can also be implemented by one or more of the client computing devices 1502, 1504, 1506, and / or 1508. Then, users operating the client computing devices can utilize one or more client applications to use the services provided by these components. These components can be implemented in hardware, firmware, software, or combinations thereof. It should be appreciated that a wide variety of system configurations are possible having different combinations of the features described herein. Therefore, it will be understood that the foregoing is a description of exemplary embodiments only, and that the system described herein is not limited to these embodiments. Consequently, within the scope of the appended claims, many variations of the system described can be Figure 15The embodiment shown in FIG. 15 is one example of a distributed system for implementing an embodiment system, and is not intended to be limiting.

[0444] Client computing devices 1502, 1504, 1506, and / or 1508 can include various types of computing systems. For example, a client computing device can include a portable handheld device (e.g., an iPhone®, cellular phone, an iPad®, computing tablet, a personal digital assistant (PDA)) or wearable device (e.g., a Google Glass® head-mounted display) that has software such as the Microsoft Windows operating system and / or various mobile operating systems. For example, a client computing device can include a smart phone, such as one running the iOS operating system; or a BlackBerry; one running a Android; one running a Windows Phone operating system; one running the Palm OS operating system; or a laptop computer running one of various Microsoft Windows operating systems; Apple Macintosh OS X operating system; or a Linux distribution. Client computing devices can also include general purpose personal computers including, by way of example, a personal computer and / or laptop computer running one of various versions of Microsoft Windows operating systems, Apple Macintosh OS X operating system, and / or a Linux operating system. Client computing devices can also include workstation computers running one of various versions of Microsoft Windows operating systems, Apple Macintosh OS X operating system, and / or a Linux operating system. Client computing devices can also include thin clients that are designed to provide access to a server-based computing environment using any type of terminal device, including by way of example IBM's Information Appliance, an Intel NetPC, and / or a Microsoft Windows Terminal Services client.

[0445] Although the distributed system 1500 in FIG. 15 is shown with four client computing devices, any number of client computing devices can be supported. Figure 15 Other devices (e.g., devices with sensors, etc.) can interact with the server 1512.

[0446] The network(s) 1510 in the distributed system 1500 can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of available protocols, including without limitation TCP / IP (transmission control protocol / Intemet protocol), SNA (systems network architecture), IPX (Internet packet exchange), AppleTalk, and the like. Merely by way of example, the network(s) 1510 can be a LAN, an Ethernet network, a Token-Ring network, and the like; a wide-area network; the Internet; virtual networks, Virtual Private Networks (VPN); intranets; extranets; public switched telephone networks (PSTNs); infrared networks; wireless networks (e.g., networks operating under any Institute of Electrical and Electronic Engineers (IEEE) 802.11 suite of protocols, Bluetooth®, and / or any other wireless protocol); and / or any combination of these and / or other networks.

[0447] The server 1512 can be composed of one or more general purpose computers, specialized server computers (including, by way of example, PC (personal computer) servers, UNIX® servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other arrangement and / or combination. The server 1512 can include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization. One or more flexible pools of logical storage devices can be virtualized to maintain virtual storage devices for the server. Virtual networks can be controlled by the server 1512 with software-defined networking. In various embodiments, the server 1512 can be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the server 1512 can correspond to a server for performing processing as described above in accordance with embodiments of the present disclosure.

[0448] The server 1512 can run an operating system including any of those discussed above, as well as any commercially available server operating systems. The server 1512 can also run any of a variety of additional server applications and / or mid-tier applications, including HTTP (HyperText Transport Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA® servers, database servers, and the like. Exemplary database servers include without limitation those available from Oracle, Microsoft, Sybase, IBM (Digital Equipment Corporation), Informix, and the like. The database servers may

[0449] ​In some implementations, the server 1512 can include one or more applications to analyze and consolidate data feeds and / or event updates from users of the client computing devices 1502, 1504, 1506, and 1508. As an example, data feeds and / or event updates can include, but are not limited to, those received from one or more third party information sources and ongoing information streams, such as feeds, updates, or real-time updates, which can include real-time events related to sensor data applications, financial quotes, network performance measuring tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and so forth. The server 1512 can also include one or more applications to display the data feeds and / or real-time events via one or more display devices of the client computing devices 1502, 1504, 1506, and 1508.

[0450] The distributed system 1500 can also include one or more databases 1514 and 1516. These databases can provide mechanisms for storing information such as user interaction information, usage pattern information, adaptation rule information, and other information used by embodiments of the present disclosure. The databases 1514 and 1516 can reside in a variety of locations. For example, one or more of the databases 1514 and 1516 can reside on a non-transitory storage medium local to (and / or resident in) the server 1512. Alternatively, the databases 1514 and 1516 can be remote from the server 1512 and in communication with the server 1512 via a network-based or dedicated connection. In one set of embodiments, the databases 1514 and 1516 can reside in a storage area network (SAN). Similarly, any necessary files for performing the functions attributed to the server 1512 can be stored locally on the server 1512 and / or remotely as appropriate. In one set of embodiments, the databases 1514 and 1516 can include a relational database adapted to store, update, and retrieve data in response to SQL-formatted commands, such as the Oracle database provided by Oracle.

[0451] In some embodiments, a cloud environment can provide one or more services. Figure 16 is a simplified block diagram of one or more components of a system environment 1600 in which services can be provided as cloud services in accordance with embodiments of the present disclosure. In Figure 16 the embodiment shown in FIG. 16, the system environment 1600 includes one or more client computing devices 1604, 1606, and 1608 that can be used by users to interact with a cloud infrastructure system 1602 that provides cloud services. The cloud infrastructure system 1602 can comprise one or more computers and / or servers that can include those described above in relation to the server 1612.

[0452] It should be appreciated that Figure 16 The cloud infrastructure system 1602 depicted in the figure can have other components than those depicted. Further, Figure 16 The embodiments shown in the figure are merely one example of a cloud infrastructure system that can incorporate embodiments of the present disclosure. In some other embodiments, the cloud infrastructure system 1602 can have more or fewer components than shown in the figure, can combine two or more components, or can have a different configuration or arrangement of components.

[0453] The client computing devices 1604, 1606, and 1608 can be devices similar to those described above. The client computing devices 1604, 1606, and 1608 can be configured to operate a client application such as a web browser, a proprietary client application (e.g., Oracle Forms), or some other application that can be used by a user of the client computing devices to interact with the cloud infrastructure system 1602 to use services provided by the cloud infrastructure system 1602. Although the example system environment 1600 is shown with three client computing devices, any number of client computing devices can be supported. Other devices such as sensors, etc. can interact with the cloud infrastructure system 1602.

[0454] The network(s) 1610 can facilitate communications in the example system environment 1600. Each of the network(s) can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially-available protocols, including those described above for the network(s) 1610.

[0455] In certain embodiments, services provided by the cloud infrastructure system 1602 can include one or more services provided by a cloud application provider. Examples of cloud application providers include Oracle Corporation, Amazon.com, Inc., Google Inc., and Microsoft Corporation. In some embodiments, the services provided by the cloud infrastructure system 1602 can include one or more services provided by a Software as a Service (SaaS) provider. In some embodiments, the services provided by the cloud infrastructure system 1602 can include one or more services provided by a Platform as a Service (PaaS) provider. In some embodiments, the services provided by the cloud infrastructure system 1602 can include one or more services provided by an Infrastructure as a Service (IaaS) provider.

[0456] In certain embodiments, a specific instantiation of a service provided by cloud infrastructure system 1602 can be referred to herein as a "service instance." Generally speaking, any service made available to a user via a communication network (such as the Internet) from a cloud service provider's system is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the user's own local servers and systems. For example, a cloud service provider can host an application, and a user may, via a communication network such as the Internet, subscribe to and use the application via a subscription-based pay structure.

[0457] In some examples, a service in a computer network cloud infrastructure can include protected computer network access to storage, a hosted database, a hosted web server, a software application, or other service provided by the cloud vendor to a user, or as otherwise known in the art. For example, a service can include password-protected access via the Internet to remote storage on the cloud. As another example, a service can include a web-service-based hosted relational database and a scripting language middleware engine for private use by a networked developer. As another example, a service can include access to an email software application hosted on a cloud vendor's web site.

[0458] In certain embodiments, cloud infrastructure system 1602 can include an application suite of products that are delivered via a self-service, subscription-based, elastically scalable, reliable, highly available, and secure model. An example of such a cloud infrastructure system is the Oracle Public Cloud offered by the present assignee.

[0459] Cloud infrastructure system 1602 can also provide computing and analysis services related to "big data." The term "big data" is generally used to describe datasets that can be stored and manipulated by analysts and researchers to visualize large amounts of data, detect trends, and / or otherwise interact with the data in other ways. Such big data and related applications can be hosted and / or manipulated by the infrastructure system at many levels and at different scales. Tens, hundreds, or thousands of processors linked in parallel can act on such data in order to render it or simulate an outside force on the data or what it represents. These datasets can involve structured data, such as data organized in a database or otherwise according to a structured model, and / or unstructured data (e.g., email, images, data blobs (binary large objects), web pages, complex event processing). Cloud infrastructure systems can be better used to perform tasks on big data sets based on demand from enterprises, government agencies, research organizations, private individuals, groups of like-minded individuals or organizations, or other entities by leveraging the ability of embodiments to relatively quickly focus more (or less) computing resources on a target.

[0460] In various embodiments, cloud infrastructure system 1602 can be adapted to automatically provision, manage and track a customer's subscription to services offered by cloud infrastructure system 1602. Cloud infrastructure system 1602 can offer cloud services via different deployment models. For example, services can be offered in a public cloud model where the cloud infrastructure system 1602 is owned by an organization selling cloud services (e.g., owned by Oracle Corporation) and the services are made available to the general public or different industry enterprises. As another example, services can be offered in a private cloud model where the cloud infrastructure system 1602 is operated solely for a single organization and can provide services for this one organization.

[0461] In some embodiments, services provided by cloud infrastructure system 1602 can include one or more services provided under Software as a Service (SaaS) category, Platform as a Service (PaaS) category, Infrastructure as a Service (IaaS) category, or other categories of services including hybrid services. Customers, via subscription orders, can order one or more services provided by cloud infrastructure system 1602. Cloud infrastructure system 1602 then performs processes to provide the services in the customers' subscription orders.

[0462] In some embodiments, services provided by cloud infrastructure system 1602 can include, without limitation, application services, platform services, and infrastructure services. In some examples, application services can be provided by cloud infrastructure system via a SaaS platform. The SaaS platform can be configured to provide cloud services falling under the SaaS category. For example, the SaaS platform can provide the ability to build and deliver on-demand application suites via an integrated development and deployment platform. The SaaS platform can manage and control the underlying software and infrastructure for providing the SaaS services. By utilizing the services provided by the SaaS platform, customers can utilize applications that are executed on the cloud infrastructure system. Customers can acquire the application services without the need for customers to purchase, license, and support the individual components of the applications. Different varieties of SaaS services can be provided. Examples include but are not limited to services for enterprise resource planning, human resource management services, customer relationship management, enterprise integration, and business flexibility management services.

[0463] In some embodiments, platform services can be provided by the cloud infrastructure system 1602 via a PaaS platform. The PaaS platform can be configured to provide cloud services belonging to a PaaS category. Examples of platform services can include without limitation services that enable organizations such as Oracle to deliver an integrated web- scale deployment environment that clients can utilize to host, scale, support, and manage their various on-demand services. The PaaS platform can manage and control the underlying cloud infrastructure, enabling clients to utilize the platform to create completely customized

[0464] By utilizing the services offered by the PaaS platform, customers can employ programming languages and tools supported by the cloud infrastructure system and also control the deployed

[0465] Various different infrastructure services can be provided by an laaS platform in the cloud infrastructure system. The infrastructure services facilitate the management and control of the underlying computing resources, such as storage, networks, and other fundamental computing resources for customers utilizing services offered by the SaaS platform and the PaaS platform.

[0466] In certain embodiments, the cloud infrastructure system 1602 can also include infrastructure resources 1630 for providing the resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 1630 can include pre-integrated and optimized combinations of hardware, such as servers, storage, and networking resources, as well as other resources to execute the services provided by the PaaS platform and the SaaS platform.

[0467] In some embodiments, resources in cloud infrastructure system 1602 can be shared by multiple users and dynamically re-allocated per demand. Additionally, resources can be allocated to users in different time zones. For example, cloud infrastructure system 1602 can enable a first set of users in a first time zone to utilize resources of the cloud infrastructure system for a specified number of hours and then enable the re-allocation of the same resources to another set of users located in a different time zone, thereby maximizing the utilization of resources.

[0468] In certain embodiments, a number of internal shared services 1632 can be provided that are shared by different components or modules of cloud infrastructure system 1602 to enable the provisioning of services by cloud infrastructure system 1602. These internal shared services can include, without limitation, security and identity

[0469] In certain embodiments, cloud infrastructure system 1602 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one embodiment, cloud management functionality can include the ability to provision, manage and track subscriptions of customers that are received by cloud infrastructure system 1602, etc.

[0470] In one embodiment, as depicted in FIG. 16, cloud management functionality can be provided by one or more modules such as order management module 1620, order orchestration module 1622, order provisioning module 1624, order management and monitoring module 1626, and identity management module 1628. These modules can include or can utilize one or more computers and / or servers, which can be general purpose computers, specialized server computers, server farms, server clusters, or any other appropriate arrangement and / or combination. Figure 16

[0471] ​In an exemplary operation, at step 1634, a customer using a client device (such as client computing devices 1604, 1606, or 1608) can interact with the cloud infrastructure system 1602 by requesting one or more services provided by the cloud infrastructure system 1602 and placing subscription orders for one or more services provided by the cloud infrastructure system 1602. In some embodiments, the customer can access cloud user interfaces (UIs) such as cloud UI 1612, cloud UI 1614, and / or cloud UI 1616 and place subscription orders via these UIs. Order information received by the cloud infrastructure system 1602 in response to a customer placing an order may include information identifying the customer and the one or more services provided by the cloud infrastructure system 1602 that the customer intends to subscribe to.

[0472] At step 1636, the order information received from the customer can be stored in the order database 1618. If this is a new order, a new record can be created for that order. In one embodiment, the order database 1618 can be one of several databases operated by the cloud infrastructure system 1618 and operating in conjunction with other system elements.

[0473] At step 1638, the order information can be forwarded to the order management module 1620, which can be configured to perform billing and accounting functions related to the order, such as verifying the order and booking the order upon successful verification.

[0474] At step 1640, information about the order can be transmitted to the order orchestration module 1622, which is configured to orchestrate the supply of services and resources for orders placed by customers. In some cases, the order orchestration module 1622 may use the services of the order supply module 1624 for supply. In some embodiments, the order orchestration module 1622 enables the management of business processes associated with each order and applies business logic to determine whether an order should continue to be supplied.

[0475] like Figure 16 As illustrated in the embodiment, at step 1642, upon receiving a new subscription order, the order orchestration module 1622 sends a request to the order provisioning module 1624 to allocate resources and configure the resources required to fulfill the order. The order provisioning module 1624 enables the allocation of resources for the services ordered by the customer. The order provisioning module 1624 provides an abstraction level between the cloud services provided by the cloud infrastructure system 1600 and the physical implementation layer used to provision the resources for the requested services. This allows the order orchestration module 1622 to be isolated from implementation details, such as whether services and resources are actually provisioned in real time or pre-provisioned and allocated / specified only upon request.

[0476] At step 1644, once the service and resources have been provided, a notification instructing subscribed customers that the requested service is now ready for use can be sent. In some cases, information (e.g., a link) enabling customers to begin using the requested service can be sent to them.

[0477] At step 1646, the order management and monitoring module 1626 can manage and track customer subscription orders. In some cases, the order management and monitoring module 1626 can be configured to collect usage statistics about customers' use of the subscribed services. For example, statistics can be collected on storage usage, data transfer volume, number of users, system startup time, and system downtime.

[0478] In some embodiments, the cloud infrastructure system 1600 may include an identity management module 1628 configured to provide identity services, such as access management and authorization services within the cloud infrastructure system 1600. In some embodiments, the identity management module 1628 may control information about customers who wish to utilize the services provided by the cloud infrastructure system 1602. This information may include information authenticating the identities of these customers and information describing the actions those customers are authorized to perform relative to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 1628 may also include descriptive information about each customer and management of how and by whom the descriptive information is accessed and modified.

[0479] Figure 17 An exemplary computer system 1700 that can be used to implement embodiments of the present disclosure is illustrated. In some embodiments, the computer system 1700 can be used to implement any of the various servers and computer systems described above. Figure 17 As shown, the computer system 1700 includes various subsystems, including a processing unit 1704 that communicates with multiple peripheral subsystems via a bus subsystem 1702. These peripheral subsystems may include a processing acceleration unit 1706, an I / O subsystem 1708, a storage subsystem 1718, and a communication subsystem 1724. The storage subsystem 1718 may include a tangible computer-readable storage medium 1722 and system memory 1710.

[0480] Bus subsystem 1702 provides a mechanism for letting the various components and subsystems of computer system 1700 communicate with each other as desired. Although bus subsystem 1702 is illustrated as a single bus, alternative embodiments of the bus subsystem can utilize multiple buses. Bus subsystem 1702 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and a local bus using any of a variety of bus architectures. By way of example, such architectures can include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, which can be implemented as a Mezzanine bus manufactured to the IEEE P1386.1 standard, etc.

[0481] Processing subsystem 1704 controls the operation of computer system 1700 and can comprise one or more processing units 1732, 1734, etc. Processing units can include one or more processors, including single or multi-core processors, one or more cores of a processor, or combinations thereof. In some embodiments, processing subsystem 1704 can include one or more special-purpose co-processors such as graphics processors, digital signal processors (DSPs), etc. In some embodiments, some or all of the processing units of processing subsystem 1704 can be implemented with customized circuitry such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

[0482] In some embodiments, processing units in processing subsystem 1704 can execute instructions stored in system memory 1710 or on computer-readable storage media 1722. In various embodiments, processing units can execute a variety of programs or code instructions and can maintain multiple concurrently executing programs or processes. Some or all of the program code to be executed can reside on system memory 1710 and / or on computer-readable storage media 1722, potentially including on one or more storage devices, at any given time. Processing subsystem 1704, through

[0483] In certain embodiments, a processing acceleration unit 1706 can be provided for performing customized processing or for offloading some of the processing performed by processing subsystem 1704 so as to accelerate the overall processing performed by computer system 1700.

[0484] I / O subsystem 1708 can include devices and mechanisms for inputting information to computer system 1700 and / or for outputting information from or via computer system 1700. In general, use of the term "input device" is intended to include all possible types of devices and mechanisms for inputting information to computer system 1700. User interface input devices can include, for example, a keyboard, pointing devices such as a mouse or trackball, touchpads, touch screens such as a touch panel or touch display, wheels, a trackwheel, a dial, a button, a switch, a keypad, a thumb-operated keypad, audio input devices such as voice command recognizers, microphones, and other types of input devices. User interface input devices can also include motion sensing and / or gesture recognition devices, such as Microsoft Kinect® motion sensor, Microsoft Xbox® 360 game controller, devices providing an interface for receiving inputs utilizing gestures and spoken commands. User interface input devices can also include eye gesture recognition devices, such as Google Glass® eye gesture recognition device, which detects eye activity (e.g., “blinking” of the user's eyes to enter or exit from a mode in which information can be entered) and

[0485] Other examples of user interface input devices include, without limitation, three dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, and audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode reader 3D scanners, 3D printers, laser rangefinders, and eye gaze tracking devices. Additionally, user interface input devices can include, for example, medical imaging input devices such as computerized tomography scanners, magnetic resonance imaging systems, position emission tomography scanners, and medical ultrasonic imaging devices. User interface input devices can also include, for example, audio input devices such as MIDI keyboards, digital musical instruments and the like.

[0486] ​​​​​User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those utilizing liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is used to encompass all possible types of devices and mechanisms for outputting information from computer system 1700 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0487] Storage subsystem 1718 provides a repository or data storage for storing information used by computer system 1700. Storage subsystem 1718 provides a tangible, non-transient, computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above-described functionality when executed by processing subsystem 1704 can be stored in storage subsystem 1718. The software can be executed by one or more processing units of processing subsystem 1704. Storage subsystem 1718 may also provide a repository for storing data used according to this disclosure.

[0488] The storage subsystem 1718 may include one or more non-transient memory devices, including volatile and non-volatile memory devices. For example... Figure 17 As shown, the storage subsystem 1718 includes system memory 1710 and computer-readable storage medium 1722. System memory 1710 may include multiple memories, including volatile main random access memory (RAM) for storing instructions and data during program execution and non-volatile read-only memory (ROM) or flash memory where fixed instructions are stored. In some implementations, a basic input / output system (BIOS) containing basic routines that help transfer information between elements within computer system 1700, such as during startup, may typically be stored in ROM. RAM typically contains data and / or program modules currently operated and executed by processing subsystem 1704. In some implementations, system memory 1710 may include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0489] As an example rather than a limitation, such as in Figure 17As depicted, system memory 1710 can store application programs 1712, which can include client applications, a web browser, a middle-tier application, a relational database management system (RDBMS), or the like, program data 1714, and an operating system 1716. By Apple and / or a Linux operating system, various commercially available UNIX® or UNIX-like operating systems (including but not limited to the various GNU / Linux operating systems such as Google Chrome OS, Android, iOS, macOS, iOS, OS, 10OS, and Windows® operating systems, among others.

[0490] Computer readable storage media 1722 can store the programming and data structures to provide the functionality of some embodiments. Software (programs, code modules, instructions) that when executed by processing subsystem 1704 cause the processor to provide the functionality described above can be stored in storage subsystem 1718. By way of example, computer readable storage media 1722 can include non-volatile memory, such as a hard disk drive, a magnetic disk drive, an optical disk drive such as a CD ROM, DVD, Blu-ray® or other optical disk storage, or non-volatile solid state memory such as flash memory cards, USB memory sticks, secure digital (SD) cards, DVD disks, digital video tape, and the like. Computer readable storage media 1722 can also include volatile memory such as static or dynamic random access memory (SRAM or DRAM), and the like. Storage subsystem 1718 can be used to store data that is manipulated by processing subsystem 1704 when executing programming and / or software modules such as the operating system, program data 1714, and application programs 1712. Storage subsystem 1718 can also provide a repository for storing data used in accordance with some embodiments.

[0491] In some embodiments, storage subsystem 1700 can also include a computer- readable storage media reader 1720 that can further connect to computer-readable storage media 1722. Optionally, the computer-readable storage media 1722 can be integrated into system memory 1710, and / or combination, with the system memory 1710. The computer-readable storage media 1722 can also include removable storage components, such as flash memory cards (including Secure Digital cards), digital versatile disks (DVDs), memory sticks, and the like.​

[0492] In certain embodiments, computer system 1700 can provide support for executing one or more virtual machines. The computer system 1700 can execute a program, such as an operating system, in order to facilitate the execution of virtual machines. Each virtual machine can be allocated memory, computing (e.g., processor, kernel), I / O, and networking resources. Each virtual machine typically runs its own operating system, which can be the same as or different from operating systems executed by other virtual machines executed by the computer system 1700. Accordingly, multiple operating systems can potentially be concurrently executed by the computer system 1700. Each virtual machine generally operates independently of the other virtual machines.

[0493] Communication subsystem 1724 provides an interface to other computer systems and networks. Communication subsystem 1724 serves as an interface for receiving data from and transmitting data to other systems from the computer system 1700. For example, the communication subsystem 1724 can enable the computer system 1700 to establish a communication channel to one or more client computing devices via the Internet for receiving and sending information to and from the client devices.

[0494] Communication subsystem 1724 can support both wired and / or wireless communication protocols. For example, in certain embodiments, the communication subsystem 1724 can include radio-frequency (RF) transceiver components for accessing wireless voice and / or data networks, Global Positioning System (GPS) receiver components, and / or other components. In some embodiments, communication subsystem 1724 can provide wired communication capability (e.g., using Ethernet) in addition to or instead of a wireless interface.

[0495] Communication subsystem 1724 can receive and transmit data in various forms. For example, in some embodiments, the communication subsystem 1724 can receive input communication in the form of structured and / or unstructured data feeds 1726, event updates 1728, event updates 1730 and the like from one or more users, third-party applications, and / or third-party information sources. For example, the communication subsystem 1724 can be configured to receive data feeds 1726 in real-time from users of social media networks and / or other users of computing devices. feeds, updates, such as web feeds (e.g., Rich Site Summary (RSS) feeds) and / or other communication services, and / or real-time updates from one or more third-party information sources.

[0496] In certain embodiments, communication subsystem 1724 can be configured to receive data that can be continuous in nature or bounded by explicit ends, in the form of continuous data streams, where continuous data streams can comprise event streams 1728 of real-time events and / or event updates 1730. Examples of applications that generate continuous data can include, for example, sensor data applications, financial tickers, network performance measuring tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0497] Communication subsystem 1724 can also be configured to output structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc. to one or more databases that can be in communication with one or more streaming data source computers coupled to computer system 1700.

[0498] Computer system 1700 can be one of various types, including a handheld portable device (e.g., an iPhone® cellular phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head mounted display), a PC, a workstation, a mainframe computer, a kiosk, a server rack, or any other data processing system. Cellular telephone, Computing tablet, PDA), a wearable device (e.g., a Google Head-mounted display), a personal computer, a workstation, a mainframe computer, an information kiosk, or any other data processing system.

[0499] Due to the ever-changing nature of computers and networks, the description of computer system 1700 Figure 17 The description of computer system 1700 described above illustrates the Figure 17 Many other configurations having more or fewer components than the system illustrated in FIG. 1 1 are possible. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and / or methods to implement the various embodiments.

[0500] While specific embodiments of the disclosure have been described above, alternative embodiments can be constructed without departing from the scope of the disclosure. Modifications include any relevant combination of disclosed features. The embodiments of the disclosure are not limited to operation within certain specific data processing environments, but are free to operate within a plurality of data processing environments. Additionally, although the embodiments of the disclosure have been described using a particular series of transactions and steps, it should be clear that the scope of the disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above-described embodiments can be used individually or jointly.

[0501] Furthermore, although embodiments of the present disclosure have been described using a particular combination of hardware and software, it is recognized that other combinations of hardware and software are also within the scope of the present disclosure. Embodiments of the present disclosure can be implemented only in hardware, or only in software, or using combinations thereof. The various processes described herein can be implemented on the same processor or different processors in any combination. Accordingly, where components or modules of the application have been described as being configured to perform certain operations, such configuration can be accomplished, e.g., through design of electronic circuits, through programming of programmable electronic circuits (such as microprocessors) or through any combination of these and / or other techniques. Processes can communicate using a variety of techniques including but not limited to conventional techniques for interprocess communication, and different pairs of processes can use different techniques of communication, or the same pair of processes can use different techniques of communication at different times.

[0502] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. However, it will be apparent to those skilled in the art that modifications, additions and subtractions can be made to the disclosed embodiments without departing from the broader spirit and scope of the disclosure as set forth in the following claims. Thus, while specific embodiments have been described in detail, these particular embodiments are to be regarded as illustrative in nature and not as restrictive. Various modifications and equivalent arrangements are within the scope of the appended claims.

Claims

1. A computer-implemented method comprising: receiving a first stack trace comprising a first sequence of stack frames; generating, based at least in part on the first sequence of stack frames, a first trace signature representing the first sequence of stack frames, wherein the first trace signature corresponds to a first tuple, the first tuple comprises a first node representing the first sequence of stack frames, and wherein the first tuple comprises a binary tree, the binary tree comprises the first node as a root node of the binary tree; receiving one or more subsequent stack traces; and for at least one of the one or more subsequent stack traces, performing operations comprising: determining, from the at least one of the one or more subsequent stack traces, a second stack trace comprising a second sequence of stack frames; generating a second node based at least in part on comparing the first sequence of stack frames to the second sequence of stack frames, wherein generating the second node comprises: determining that the first sequence of stack frames and the second sequence of stack frames share a matching subsequence of stack frames, and a first non-matching subsequence of stack frames included in the first sequence of stack frames is different than a second non-matching subsequence of stack frames included in the second sequence of stack frames; generating a second tuple comprising the second node based at least in part on the first non-matching subsequence of stack frames and the second non-matching subsequence of stack frames; generating a second trace signature representing the second sequence of stack frames, the second trace signature corresponding to the second tuple; and generating, based at least in part on the second trace signature, a subsequent trace signature to represent a subsequent sequence of stack frames.

2. The computer-implemented method of claim 1, further comprising: determining whether the subsequent trace signature has already been generated to represent the subsequent sequence of stack frames included in the at least one of the one or more subsequent stack traces; and wherein the operations are performed on the at least one of the one or more subsequent stack traces based on a determination that the subsequent trace signature has not already been generated.

3. The computer-implemented method of claim 2, wherein, determining whether the subsequent trace signature has already been generated to represent the subsequent sequence of stack frames comprises: determining whether one or more previously generated nodes represent the subsequent sequence of stack frames, the one or more previously generated nodes comprising the first node; and wherein generating the subsequent trace signature to represent the subsequent sequence of stack frames comprises: determining one or more subsequences of stack frames included in the subsequent sequence of stack frames that are not represented by the one or more previously generated nodes; generating one or more additional nodes to represent the one or more subsequences of stack frames included in the subsequent sequence of stack frames; incorporating at least one of the one or more additional nodes into one or more previously generated binary trees of one or more previously generated tuples, the one or more previously generated binary trees comprising the binary tree, and the one or more previously generated tuples comprising the tuple; generating one or more additional binary trees, wherein at least one of the one or more binary trees comprises at least one of the one or more additional nodes; and generating an additional tuple comprising the one or more additional binary trees, wherein the subsequent trace signature corresponds to the additional tuple.

4. The computer-implemented method of claim 3, wherein, each tuple, each node, and each stack frame is uniquely identified by an integer.

5. The computer-implemented method of claim 1, wherein the binary tree is a first binary tree; wherein generating the second node based at least in part on comparing the first sequence of stack frames to the second sequence of stack frames further comprises: generating a second node representing the matching subsequence of stack frames; generating a third node representing the second non-matching subsequence of stack frames included in the second sequence of stack frames; generating a second binary tree including the second node and a third binary tree including the third node; and generating a second tuple including the second binary tree and the third binary tree, wherein the second node and the third node are sibling nodes and the second trace signature corresponds to the second tuple.

6. The computer-implemented method of claim 5, wherein generating the second trace signature further comprises: generating a fourth node representing the matching subsequence of stack frames; generating a fifth node representing the non-matching subsequence of stack frames included in the first sequence of stack frames; and adding the fourth node and the fifth node to the first binary tree as child nodes of the first node.

7. The computer-implemented method of claim 1, further comprising: generating a second trace signature based at least in part on the matching subsequence of stack frames. generating the second trace signature comprises:

8. The computer-implemented method of claim 7, wherein, generating a first segment signature representing the matching subsequence of stack frames; generating a second segment signature representing the second non-matching subsequence of stack frames included in the second sequence of stack frames; and generating the second trace signature, wherein the second trace signature includes the first segment signature and the second segment signature. generating the second trace signature comprises:

9. The computer-implemented method of claim 8, wherein, generating a third segment signature classifying the first non-matching subsequence of stack frames included in the first sequence of stack frames; and modifying the first trace signature to include the first segment signature and the third segment signature.

10. The computer-implemented method of claim 8, the first trace signature is associated with a first counter that is incremented upon receiving any stack trace including the first sequence of stack frames, and the second trace signature is associated with a second counter that is incremented upon receiving any stack trace including the second sequence of stack frames.

11. The computer-implemented method of claim 10, the first segment signature is associated with a third counter that is incremented upon receiving any stack trace including the matching subsequence of stack frames; wherein wherein the second segment signature is associated with a fourth counter that is incremented upon receiving any stack trace including the non-matching subsequence of stack frames included in the second sequence of stack frames; wherein the third counter is copied from the first counter when generating the first segment signature; and wherein the fourth counter is copied from the second counter when generating the second segment signature.

12. A computing system comprising: one or more processors; and a memory accessible by the one or more processors, the memory storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-11. ​ 13. A non-transitory computer-readable medium storing one or more instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1-11.

Images