Performance Event Troubleshooting System
The method and system address the challenge of identifying root causes in time-series data anomalies by using similarity techniques and dynamic updates, ensuring efficient and accurate analysis in real-time to maintain system performance.
Patent Information
- Application Number
- JP2022565810
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-05-05
- Filing Date
- 2021-04-25
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-04-25
AI Technical Summary
Existing systems struggle to efficiently and automatically identify the root cause of anomalies in large volumes of time-series data from various data sources, such as sensor networks and smart grids, which can lead to inefficiencies in digital computer systems.
A method and system for dynamic and automatic root cause analysis using similarity techniques to compare sensor data, identifying potential root causes of anomalies by assigning coefficients to measure comparisons, and dynamically updating the analysis based on real-time data to improve accuracy.
Enables timely and accurate root cause analysis, seamlessly integrating with existing systems to ensure continuous operation of cloud and data center applications without performance loss, and improving accuracy over time as data accumulates.
Smart Images

Figure 0007586613000005 
Figure 0007586613000006 
Figure 0007586613000007
Abstract
Description
[Technical field]
[0001] The present invention relates generally to the field of digital computer systems, and more particularly to performance event troubleshooting. [Background technology]
[0002] Time series data originates from various data sources including sensor networks, smart grids, etc., and petabytes of data are being collected in public and private clouds. The collected time series data may have unexpected changes or patterns that indicate anomalies. By monitoring the data to detect root causes in real time, for example, such anomalies can be prevented from accumulating and affecting the efficiency of computer systems. Summary of the Invention
[0003] Aspects of the invention disclose methods, computer program products and systems for a troubleshooting system. The method includes one or more processors identifying a data source(s), the data source(s) configured to provide sensor data and timestamps of the sensor data as a first set of one or more time series, the sensor data including values of a first group of measurands. The method further includes one or more processors determining that values of one or more second group of measurands of a subset of the sensor data indicate an anomaly. The subset of sensor data is provided by a given data source of the data source(s) and covers a time window, the given data source including a set of components. The method includes one or more processors determining one or more third group of measurands that are candidates for a root cause of the anomaly using a set of one or more similarity techniques for comparing values of the second group of measurands and the third group of measurands over the time window. The third group of measurands are provided by the set of components. For each similarity technique in the set of similarity techniques and for each measurement quantity of the second group, the method includes the one or more processors assigning a set of coefficients to the measurement quantity, each coefficient of the set of coefficients indicative of a result of comparing the respective measurement quantity with the measurement quantities of the third group using a similarity technique. The method further includes the one or more processors using the set of coefficients to determine whether a particular subset of the set of components of the given data source can be identified as a root cause of the anomaly. In response to determining that a particular subset of the set of components of the given data source can be identified as a root cause of the anomaly, the method includes the one or more processors providing the particular subset of the components as a root cause of the anomaly.
[0004] In a further aspect, in response to determining that a particular subset of the set of components of the given data source is not identifiable as a root cause of the anomaly, the method further includes the one or more processors updating a third group of measurements. For each similarity technique in the set of similarity techniques and for each measurement quantity in the second group, the method further includes the one or more processors assigning a set of coefficients to the measurement quantity, each coefficient in the set of coefficients indicating a comparison result of the respective measurement quantity with the updated third group of measurements using a similarity technique. The method further includes the one or more processors using the set of coefficients to determine whether a particular subset of the set of components of the given data source is identifiable as a root cause of the anomaly.
[0005] The present subject matter can enable dynamic and automated root cause analysis methods. The present subject matter can improve root cause analysis on live data. For example, as data accumulates over time, the accuracy of the root cause analysis can improve. Embodiments of the present invention recognize that increased accuracy is advantageous because information that may initially be viewed as an anomaly later turns out to be a deviation that is not an anomaly. Various embodiments of the present invention can perform root cause analysis in real time (e.g., as data sources provide time series data).
[0006] The present subject matter may be seamlessly integrated with existing root cause analysis systems. For example, various embodiments of the present invention may enable an ensemble-based similarity search tool for automated root cause analysis troubleshooting (RAC / TS) in data center storage. Additionally, embodiments of the present invention may provide timely and accurate root cause analysis troubleshooting, thereby ensuring that cloud and data center hosted applications operate without loss of access, data or performance.
[0007] In the following, embodiments of the invention will be described in more detail, by way of example only, with reference to the drawings in which: FIG. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram of a computer system according to an embodiment of the present invention. [Diagram 2] 2 is a flowchart of a method according to an embodiment of the present invention. [Diagram 3] 1 is a diagram illustrating a method according to an embodiment of the present invention. [Figure 4] 1 illustrates a computerized system suitable for implementing one or more method steps according to embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Various embodiments of the present invention are described for purposes of illustration and are not intended to be exhaustive or limited to the disclosed embodiments. Numerous modifications and variations will be apparent to those skilled in the art without departing from the gist and spirit of the described embodiments. The terms used in this specification are selected to best explain the principles of the embodiments, practical applications, or technical improvements beyond those found in the market, or to enable other skilled in the art to understand the embodiments disclosed herein.
[0010] Various embodiments provide a root cause analysis method, a computer system and a computer program product as described by the subject matter of the independent claims. Further advantageous embodiments are described in the dependent claims. The embodiments of the invention may be freely combined with one another if they are not mutually exclusive.
[0011] A time series may be, for example, a sequence of data points measured at successive times, typically uniformly spaced apart in time. A time series may include pairs or tuples (v,t), where "t" is the time at which value "v" is measured. The values v of a time series may be referred to as sensor data. The sensor data of a time series may include values v of a measurand. The measurand may be a physical quantity, quality, condition or characteristic being measured. For example, measurands include, without limitation, temperature, central processing unit (CPU) usage, computational load, Global Mirror secondary write lag (ms / op), etc.
[0012] The measured quantity may be, for example, Global Mirror Secondary Write Lag (ms / op), which is the average number of milliseconds to service a secondary write operation to a Global Mirror. The value may not include the time to service the primary write operation. Embodiments of the invention may monitor the value of Global Mirror Secondary Write Lag to identify delays occurring during the process of writing data to the secondary site (e.g., a detected increase may be an indication of a potential problem).
[0013] One or more time series may have values of the same measurand. For example, two temperature sensors at different locations each transmit a respective time series having a temperature value and an associated timestamp. In another example, two or more time series may be used to determine a value of a single measurand. For example, a measurand that is a ratio of temperature to pressure may be defined using two time series, one of temperature values and the other of pressure values. In another example, each time series of the received time series has a value of a respective distinct measurand. That is, the size of the first set of time series and the first group of measurands are identical, since each measurand of the first group has a respective time series of the first set. The values of the measurand may have a pattern that does not fit a predefined normal behavior of the values of the measurand, which may be referred to as an anomaly or problem.
[0014] The normal behavior of the measurand may be defined by one or more reference values or thresholds. In one example, the reference value may be the maximum possible value of the measurand, beyond which a value of the measurand may indicate an anomaly. In one example, the reference value may include lower and upper limits of an acceptable range of values such that a value of the measurand indicates an anomaly if it is outside the range. In another example, the reference value may be the value of a function or model that models the variation of the value of the measurand over time. In an additional example, an embodiment of the present invention may learn normal behavior from training data by a machine learning model (e.g., the anomaly detection algorithm may be a machine learning model). The trained machine learning model may be an equation, or a set of rules, that predicts anomalies in the input data. The rules may use a comparison to the reference value.
[0015] In various embodiments of the invention, if a component provides a measurand, then a value of the measurand is indicative of the component (e.g., indicative of operational performance of the component). The value of the measurand is received as part of a first set of time series.
[0016] In an example embodiment, the system may be a root cause analysis and performance system. In a further example embodiment, the method may be a root cause analysis and performance method. For example, the anomaly may be a performance problem, a configuration problem, or a software problem (e.g., a bug), or a combination thereof, or the like.
[0017] An embodiment of the invention may determine the third group by selecting measurements from the first group of measurements. In one example, any set of measurements may be selected. In another example, the selection is based on predefined selection criteria. The selection criteria may, for example, require that the selected measurements are correlated with at least a portion of the second group of measurements. The selection criteria may, for example, require that the selected measurements are related to a previously detected problem that is similar or identical to an anomaly. In an additional example, updating the third group may include redetermining the entire third group. The resulting redetermined third group may or may not overlap with the third group. For example, the redetermining may be performed using a different selection criteria.
[0018] Comparing the values of the second group of measurands and the third group of measurands over the time window includes performing a pairwise comparison of the values of the second group of measurands and the third group of measurands over the time window. In an example embodiment, each measurand of the second group may be compared to every measurand of the third group. In one example, the pairwise comparison between the measurands of the second group and the third group may include a comparison of block of ranges of values with block of ranges of values, which may speed up the comparison process.
[0019] In various embodiments, the particular subset of components may include one or more components. The particular subset of more than one components may be components that belong to the same component class (e.g., represent a single problem cause). For example, if the abnormal data source is a storage system, the component class may be both physical and virtual switches and links. Another example of a component class may be physical servers and virtualized servers, etc. An embodiment of the invention may utilize a determination that a particular subset of the set of components of a given data source cannot be identified as a root cause of the anomaly to mean that multiple problem cause components are identified. The multiple problem cause components may not be of the same component class (e.g., do not provide a single problem cause).
[0020] According to one embodiment, updating the third group of measurements includes removing one or more measurements from the third group, or adding one or more measurements to the third group, or both. The updating process may enable automatic and dynamic reactions so that the root cause analysis system can continue. In another example, updating the third group may include prompting a user to provide an update to the third group of measurements. In one example, updating the third group may include determining and taking into account that a selected one of the causal components of the identified problem is (no longer) a root cause.
[0021] According to one embodiment, the measurements to be removed or replaced are provided by selected components of the set of components or causal components of the identified problem. The selected components are components that are not (or will not be) the cause of the anomaly, either because the respective problem has been resolved (after the anomaly has been detected) or because the value of the respective anomalous measurement quantity is caused by another component. The update of the third group of measurements may take into account that the selected component is not the root cause. For example, the user may provide an update of the third group of measurements, taking into account that the selected component is not the root cause.
[0022] According to one embodiment, the method may further include removing the selected component from the set of components for a next process iteration of an embodiment of the present invention (described in more detail with respect to FIGS. 2 and 3). Furthermore, by removing the first most overloaded component, an embodiment of the present invention may test whether the first most overloaded component is a singular bottleneck to the performance of the overall system.
[0023] An embodiment of the present invention recognizes that the fact that a single problem cause / subset of components cannot be identified may be due to the fact that anomalies in a given data source are caused by a combination or cascade of individual problems in different components of a set of components (e.g., multiple problem causes may contribute to the anomaly). In one example, an individual problem may be caused by a single component that affects a measure of other components. For example, the behavior of a measure of multiple components on the same data flow in a storage system may be affected by a problem in one component of the data flow.
[0024] Thus, embodiments of the present invention recognize the advantage of reducing the number of components (set of components) examined by the method by eliminating and / or resolving some problematic components. In an exemplary embodiment, for each component in the set of components, a characteristic that correlates with a majority of anomalies in a given data source is evaluated. For example, in the case of a storage system, the degree of overload may be a characteristic that characterizes a majority of anomalies.
[0025] In another example, the most saturated component of the set of components may be the selected component. A saturated component may be a component that has reached saturation or is overburdened because the respective characteristic values are stationary or the characteristic values are sawtooth. The sawtooth condition does not necessarily indicate that the value has reached a limit, but may indicate that a neighboring component has reached saturation. Various embodiments may determine the neighboring component to be the next component traversed in the data flow.
[0026] Further embodiments of the invention may configure a given data source (e.g., by increasing the respective capacity) to resolve saturation solutions for that selected component. If, after configuring the data source, the selected component is no longer saturated, or if the configuration reveals that saturation is not a real problem, then embodiments of the invention may utilize such information in improving the selection of the third group of measurands for the next iteration. For example, a measurand associated with the selected component may be removed from the third group of measurands.
[0027] According to one embodiment, the method further includes ranking the set of components (or ranking the identified problem causal components) according to a predetermined characteristic of the set of components. In an exemplary embodiment, the selected component is the first ranked component. The first ranked component may be the component that contributes to the detected anomaly. For example, the characteristic is the degree of overload. In this example, the most overloaded component is ranked first. In an embodiment in which the update is based on the selected components, the ranking may be performed after a predetermined number of iterations N, for example after the third group has been updated N times without success (e.g., without a single problem cause being identified).
[0028] According to one embodiment, the subset of components is a single component of the set of components, which may further increase the accuracy of the root cause analysis.
[0029] According to one embodiment, determining whether a particular subset of the set of components can be identified includes combining the respective sets of coefficients for each measurement of the second group and for each measurement of the third group to obtain a combination coefficient and using the combination coefficient for the decision. For example, if the second group includes a single measurement M21, the third group includes two measurements M31 and M32, and the set of similarity techniques includes three techniques ST1, ST2, and ST3, then two sets of coefficients may be obtained from the method, one set of coefficients for the pair (M21, M31) and another set of coefficients for another pair (M21, M32).
[0030] Each of the two coefficient sets has three coefficients, each associated with a respective similarity technique. For example, the set of coefficients for the pair (M21, M31) is C 11 ST1 ,C 11 ST2 ,C 11ST3 ), and the set of coefficients of the pair (M21,M32) may include C 12 ST1 ,C 12 ST2 ,C 12 ST3 ). Each coefficient in the set of coefficients may be, for example, a number. By comparing the values of the set of coefficients, embodiments of the present invention can identify the root cause. For example, if a coefficient or coupling coefficient is much higher than all other coefficients, then the component associated with that coefficient / coupling coefficient is likely to be the root cause. The use of coupling coefficients allows the use of all similarity techniques to determine which is the root cause.
[0031] According to one embodiment, combining the sets of coefficients includes summing the sets of coefficients. The sums of each set of coefficients in the plurality of sets of coefficients may be, for example, weighted summed. Additionally, each similarity technique in the set of similarity techniques is assigned a respective weight. In an example embodiment, the use of weights allows for the use of only a portion of the set of similarity techniques (e.g., by assigning a weight of 0 to non-desired techniques).
[0032] Continuing with the above example, the weighted sum can be defined as follows: The coupling coefficient for the pair (M21,M31) is W ST1 ×C 11 ST1 +W ST2 ×C 11 ST2 +W ST3 ×C 11 ST3 The coupling coefficient of the pair (M21,M32) can be defined as W ST1 ×C 12 ST1 +W ST2 ×C 12 ST2 +W ST3 ×C 12 ST3 The weight W can be defined as ST1, WST2, W ST3 are the weights of the three similarity techniques ST1, ST2, and ST3, respectively.
[0033] In one example, the weights may be user-defined. In another example, the weights may be automatically selected from a predefined weight map that includes a weight associated with each similarity technique. The value of the weight may depend on the type of a given data source, or on the time of execution of the method, or both. For example, a given data source may be a storage area network (SAN) of a data center. An embodiment of the present invention may monitor the SAN of a data center as a dynamic queuing system that operates in distinct regions such as normal and saturated regions. For the saturated region, similarity techniques such as Manhattan distance, Pearson distance, and DTW distance are preferred. In such an example, the weight associated with this technique may be higher than other techniques in the set of techniques.
[0034] According to one embodiment, the method further includes normalizing values of the measurements of the second and third groups of measurements in a time window before performing the comparison. The normalization may be performed to the same range. In one example, an embodiment of the present invention may use min-max normalization to scale all compared measurements to the range [0,1]. For example, the normalization is performed only within the time window. According to one embodiment, the set of similarity techniques includes L1 / Manhattan distance, L2 / Euclidean distance, Dynamic Time Stretching (DTW) distance, Spearman and Pearson metrics.
[0035] According to one embodiment, determining that values of the one or more second group of measurands of the subset of sensor data indicate an anomaly includes receiving an event ticket from the data source, the event ticket indicating the anomaly. For example, the event ticket may indicate a time range (or time window) covering the second set of time series and timestamps of the subset of sensor data. The second set of time series may be time series used to monitor the second group of measurands. The second set of time series may be a subset of the first set of time series. For example, the event ticket may be a log file.
[0036] According to one embodiment, determining that the values of the one or more second group of measures of the subset of sensor data indicate an anomaly is performed in response to receiving an event ticket from the data source, the method further including repeating the method for each event ticket received from or further one of the data sources.
[0037] According to one embodiment, the second group of measurements includes a measurement of the first group of measurements or a combination of measurements of the first group of measurements. According to one embodiment, each time series of the first set of time series includes a value of a respective measurement. In an exemplary embodiment, the number of measurements in the first group is equal to the number of time series in the first set of time series. Various embodiments of the present invention perform the method automatically, which can speed up root cause analysis systems. For example, RCA troubleshooting of complex incidents can often take days or weeks if performed ad-hoc.
[0038] FIG. 1 is a diagram of a computer system 100 according to an exemplary embodiment of the present invention. The computer system 100 may include a set of data sources 101. In an exemplary embodiment, each data source of the set of data sources 101 may be a computer system, and each data source of the set of data sources 101 may be configured to transfer data over a network. For example, the data sources may be a public or private cloud storage system, a storage system addressable over a network by a URL, or any other accessible data source. The data sources may include data regarding one or more sensors. In various embodiments, the sensors may be devices, modules, machines, or subsystems intended to determine and / or monitor values of measurands in a corresponding environment.
[0039] The sensors may collect or obtain measurements at regular or irregular time intervals. The measurements may be provided as a time series. A time series may include, for example, a series of data points (or values) indexed (or listed or graphed) in time order, e.g., a time series includes tuples of a value and an associated timestamp. The timestamp of a value (or data point) indicates the time at which the value is obtained. For example, the values of the time series may be values of a measurand, where the measurand may be a physical quantity, state or characteristic. Thus, each data source of the data sources 101 may provide a time series whose values are values of the measurand, such as temperature, pressure, CPU usage, etc. In one example, the data sources 101 may provide sensor data for a first group of measurands (named "GRP1").
[0040] Computer system 100 includes a monitoring system 103. In various embodiments, monitoring system 103 is configured to detect anomalies in data received from data sources 101. In additional embodiments, monitoring system 103 may be configured to process received time series.
[0041] Typically, hundreds of thousands of monitoring time series and event logs are captured by the monitoring system 103 as multivariate time series at fine granularity (e.g., minutes or seconds). In an example embodiment, the monitoring system 103 can compare the actual behavior of the measurand with the normal behavior of the measurand and generate comparison data. For example, a predefined deviation from the normal behavior may indicate an anomaly. For example, an anomaly may occur due to insufficient memory when insufficient random access memory (RAM) is available to accommodate the data required to perform an operation.
[0042] In one example, the monitoring system 103 may be configured to identify unexpected values of a received time series of measurements. For example, the monitored measurements may include pressure, temperature, and a ratio of temperature to pressure, while the monitored measurements may include at least a portion of measurements of a first group of measurements (GRP1), or a combination of measurements of the first group of measurements (GRP1), or both (e.g., a data source may provide measurements such as temperature, pressure, and CPU usage).
[0043] In one example, the monitoring system 103 may be configured to detect that an anomaly has occurred if the value of the measured quantity of the incoming sample falls outside a normal range. The boundaries of the range may be referred to as thresholds. For example, a score may be calculated using a residual derived from the difference between the received value and a reference value. A score may indicate an anomaly if the score is higher than the first outlier at the highest of the range or lower than the first outlier at the lowest of the range. The use of thresholds allows for the identification of anomalous behavior by the degree of deviation of the data from a normal model.
[0044] In another example, the monitoring system 103 may use analytical methods, such as machine learning models, to detect anomalies. In an example embodiment, the machine learning model may be an auto-encoder. For example, the auto-encoder may be a feed-forward neural network. The auto-encoder may include an input layer having a number of nodes corresponding to each measurand of the first group of measurands GRP1. For example, the number of nodes in the input layer may be the number of measurands of the first group GRP1. The output layer may include the same number of nodes as the input layer and may correspond to the reconstructed values of the first group of measurands GRP1.
[0045] Various embodiments of the present invention may train an autoencoder network on data representing normal behavior of a first group of measurands with the goal of first compressing and then reconstructing input variables. Training may include modifying parameter values to minimize reconstruction error. Training may be performed using a training data set. Embodiments of the present invention may obtain a training data set by collecting data sets of multiple metrics at multiple times. For example, one data set may be obtained from each device, such as a SAN Volume Controller (SVC) device.
[0046] The metric may be a measurement. For example, a training set may be constructed using a number of devices at different times. Each device may provide a multi-dimensional time series. The autoencoder may be trained on multiple multi-dimensional time series (using multiple time windows). For example, only sets with nodes with 4 ports with speeds of 8 Gbps may be filtered. For each entity set, a single host-node-port entity set may be extracted and 35 high priority and aggregate metrics may be filtered in a file that may form the training set. During dimensionality reduction, the network learns the interactions between various variables and reconstructs variables back to the original variables at the output. If the data source is degraded or has problems, embodiments of the invention will see increased errors in the network reconstruction of the input variables. By monitoring the reconstruction errors, embodiments of the invention can detect anomalies.
[0047] Computer system 100 includes a root cause analysis system 105. Root cause analysis system 105 may be configured to generate a set of possible root causes for an anomaly detected by monitoring system 103. The set of possible root causes may include one or more potential root causes of the anomaly.
[0048] The monitoring system 103, the data sources 101, and the root cause analysis system 105 may be interconnected by one or more networks. In one example, the network includes the Internet. In another example, the network includes a wireless link, a telephony connection, a radio connection, or a computer network (e.g., a local area network (LAN) or a wide area network (WAN)). Although shown as a remotely connected system, the monitoring system 103 may be part of the root cause analysis system 105 in another example.
[0049] 2 is a flowchart of a method 200 according to an exemplary embodiment of the present invention. The method 200 may be a root cause analysis and performance method. For illustrative purposes, the method may be implemented in the computer system 100 shown in FIG. 1 (e.g., by the root cause analysis system 105), but is not limited to this implementation.
[0050] In various embodiments, data sources 101 may provide a first set of time series. In an example embodiment, each data source in the data sources may include a set of components. In the case of a storage system, the set of components may include at least one of a server, a virtualization server (e.g., one or more virtual machines (VMs)), a physical network adapter, a virtual network adapter, a SAN fabric, physical switches and links, virtual switches and links, a storage front-end network adapter, a storage front-end virtualization, and a back-end storage.
[0051] The first set of time series may include time series ts1, ts2, ..., tsN. For example, the first set of time series may be streaming data continuously generated by the data sources 101. The root cause analysis system may use the method to incrementally process such data (e.g., without accessing all of the data). Incremental processing allows for stream processing of the received data, thus allowing real-time monitoring and acting on the data in real-time. For example, data generated or acquired in large quantities needs to be analyzed in real-time to facilitate acting on potential load balancing in the network. In another example, the first set of time series may be stored as it is received in order to perform offline analysis of the received sensor data by the root cause analysis system.
[0052] The first set of time series provides values of a first group of measurements. An embodiment of the present invention may monitor the first set of time series (i.e., named "SET1") using monitoring measurements to detect anomalous events. For example, each monitored measurement of the monitored measurements may be a measurement of the first group or a combination of measurements of the first group. In a further example, the monitoring may be performed on a predefined sample of data. The sample of data may be incrementally increased with each received data from the data sources 101. Following the above example, data of time series ts1, ts2, tsN is received and accumulated. The monitoring may be performed periodically on the accumulated data (e.g., the monitoring may be performed every hour for the current hour), or the monitoring may be performed on the data of the current hour and the accumulated data for the hour before the current hour. In another example, the sample of data may be data received at a given time interval and not previously processed (e.g., the monitoring may incrementally process data received for each hour).
[0053] Various embodiments of the present invention may perform monitoring to determine whether values of monitored measurements have normal behavior. For example, each value of the monitored measurements may be compared to respective normal behavior data. In another example, each value of the monitored measurements may be input to a trained machine learning model to predict whether they deviate from normal behavior. The results of the monitoring may be accessible by a root cause analysis system. To that end, the monitoring may be performed by the root cause analysis system or another remote computer system to which the root cause analysis system is connected.
[0054] For example, in step 201 of method 200, the root cause analysis system may determine that values of one or more second group of measurements (e.g., GRP2) of the subset of sensor data indicate an anomaly in a given time range or time window (e.g., morning time). In various embodiments, the determination indicates an anomaly in a subset of time series of the first set of time series SET1 during the time window. For example, the subset of time series may be received from an anomalous data source in the group of data sources 101. In an additional example, the root cause analysis system may receive an event ticket from a remote computer system. The event ticket indicates the second group of measurements, the time window, and the time that the event occurred. The time window may enable focusing the root cause search within the specified time window. In an example embodiment, the time window may be centered around the event time, which may span 4-48 hours before and after the event time.
[0055] In step 203 of method 200, the root cause analysis system may determine one or more third groups (e.g., GRP3) of measurands (i.e., candidates for root causes of the anomaly). In an example embodiment, the third group may be selected from the first group of measurands GRP1 using the second group of measurands GRP2. For example, the root cause analysis system may be configured to search for root causes of the anomaly using an anomaly rule engine database. The anomaly rule engine database includes a set of entries, where each entry in the set of entries describes an anomaly. For example, each entry in the set of entries includes a value for an attribute of the respective anomaly.
[0056] In an example embodiment, attributes of an anomaly may include a number of time series involved in the anomaly, measurements involved in the anomaly, etc. Each entry in the group of entries may be associated with a set of candidate root causes. The root cause analysis system may be configured to identify one or more entries corresponding to the detected anomaly, and each one or more sets of measurements of the candidate root causes may form a third group GRP3. Alternatively, or additionally, embodiments of the invention may prompt the user to provide some or all of the measurements of the third group GRP3. For example, the user may be presented with information indicative of the detected anomaly.
[0057] For example, for a storage system having elevated front-end response time as an anomaly, the third group of metrics may be read / write response time to credit exhaustion, read / write response time to read data rate, read / write response time to back-end read / write response time or latency, and write response time to port to local node response time / latency. For insufficient input anomalies, the metrics may include vdisk response to back-end response, vdisk response to back-end queue, vdisk response to host attributed delay, inter-node (port to local node) vdisk response, and vdisk response to gm secondary write lag.
[0058] Each measure of the third group of measures may be associated with a component of the anomaly data source. For example, a subset of the measures of the third group may be obtained from a network adapter of the anomaly data source. In this manner, the third group of measures may enable multiple potential root cause components to be analyzed. Embodiments of the present invention recognize that a precise selection of the third group may be advantageous and enable the source of a single problem to be discerned.
[0059] In step 205, the method 200 may utilize a set of one or more similarity techniques for pairwise comparison of values of the second group of measurands and the third group of measurands. That is, the method 200 may compare every measurand in the second group GR2 with every measurand in the third group GR3. The number of comparisons performed in step 205 may be N cmp =N st ×N GRP2 ×N GRP3 where N st is the number of similarity techniques, N GRP2 is the number of measurements in the second group GRP2, and N GRP3 is the number of measurements in the third group GRP3. For example, the set of similarity techniques may include L1 / Manhattan, L2 / Euclidean, DTW / Dynamic Time Dilation, Spearman and Pearson metrics. For example, if the second group GRP2 includes two measurements and the third group GRP3 includes three measurements, the number of comparisons to be performed in step 205 may be 5*2*3=30.
[0060] Measure M2 for the second and third groups i and M3 j (i=1,…,N GRP2 , j=1,…,N GRP3 ) can be calculated, for example, using the Euclidean distance method as follows:
number
[0061] Embodiments of the present invention recognize the benefit of normalizing the values of the measurements being compared before performing the comparison, which may allow for an effective similarity comparison. The normalization may be performed to the same range. In one example, min-max normalization may be used, which scales all measurements being compared to the range [0,1], and the normalization is performed only within a time window.
[0062] In step 207, the method 200 performs a step of: for each similarity technique in the set of similarity techniques, i For the measured quantity M2 i To, N st , where each coefficient in the set of coefficients corresponds to each metric M2 i The third group of measurements M3 j For example, the set of coefficients C ij 1,C ij 2,…,C ij Nst For each distinct pair (i,j) of all possible pairs of measurements in the second and third groups, a record may be provided as follows: [Table 1] For example, a set of coefficients C ij 1,C ij 2,…,C ij Nst is the sum of two measures M2 based on the comparison performed by a set of similarity techniques. i ,M3 j The set of coefficients may be a number indicating the level of similarity.
[0063] In step 209, the method 200 calculates the number of pairs (M i , M3 j ) and combine the set of coefficients, resulting in a combined coefficient Coef ij In an exemplary embodiment, the combination is the sum of the coefficients
number
number
[0064] Additionally, in step 211, method 200 may determine whether the coupling coefficient enables identification of a particular subset of the set of components of the anomaly data source as a root cause of the anomaly. For example, the subset of components may be one or more components. Additionally, the subset of components may include a single component and be referred to as a single culprit of the problem. The subset of components may belong to the same component class. The subset of components may enable focused investigation of the anomaly. In an example where the anomaly data source is a storage system, the subset of components may belong to the same component class, where the component class may be, for example, both physical and virtual switches and links. Another example of a component class may be a physical server and a virtualized server, etc.
[0065] For example, outliers of the coupling coefficients may be distinguished from other coupling coefficients. Outliers may include coupling coefficients higher than a predefined threshold. The threshold may be, for example, an average of all coupling coefficients. A further embodiment of the present invention may determine whether the outliers of the coupling coefficients are associated with a third group of measurements coming from a particular subset of components of the set of components. In other words, if the coupling coefficients point to a set of identical problem causes (measurements streaming from components) in two or more similarity techniques of the set of techniques, then the set and the respective source components are most likely to be the root cause of the anomaly. Thus, a deeper investigation into this particular component may be further performed. Thus, in response to determining that a particular subset of the set of components of the anomaly data source may be identified as the root cause of the anomaly (step 211, yes branch), the method 200 provides the particular subset as the root cause (step 213).
[0066] However, determining that the coupling coefficients are not significantly different from one another and that an outlier cannot be identified may indicate that a particular subset of the components cannot be identified. Thus, in response to determining that a particular subset of the set of components of the anomaly data source is not identified as a root cause of the anomaly (step 211, no branch), method 200 may update (at step 215) the third group GRP3 of the measurands, and method 200 may perform steps 205-215 using the updated third group instead of the third group used in the previous iteration.
[0067] The method 200 may perform an update of the third group of measurements by removing or replacing one or more measurements of the third group. In one example, the measurements to be removed or replaced may be determined as follows: A particular component of the anomalous data source may be identified and all measurements provided by the particular component may be removed or replaced in the third group. In this example, the iteration of steps 205-215 may use a set of components excluding the particular component. The particular component may be selected, for example, randomly from the set of components. In another example, the user may be presented with a dialog display to identify the particular component. In another example, the set of components may be ranked using values of a characteristic common to the set of components during a time window. The component ranked first may be the particular component. For example, the characteristic may be a degree of overload.
[0068] 3 is a diagram 300 illustrating a method for root cause analysis according to an exemplary embodiment of the present invention. In various embodiments, the method of diagram 300 may be implemented on, but is not limited to, the computer system 100 previously described in FIG. 1 (e.g., root cause analysis system 105 and optionally one or more computing systems).
[0069] In block 301, the root cause analysis system may receive an event ticket. The event ticket may indicate an abnormal event that has occurred in a given data source. In an example embodiment, the given data source may be a storage system having a set of components, such as a server and a network adapter.
[0070] In block 302, the root cause analysis system can determine an anomaly set of measurements (e.g., key performance indicators (KPIs)) that caused the event. Then, in block 303, the root cause analysis system can determine a candidate set of root causes of the measurements (e.g., select from SET1). In addition, the root cause analysis system can determine a time window that covers the event.
[0071] In block 304, the root cause analysis system can utilize a set of five similarity techniques to perform pairwise measurement comparisons of the anomaly set and the root cause candidate set. The set of similarity techniques includes an ensemble of three distances (i.e., L2 / Euclidean, L1 / Manhattan, DTW / Dynamic Time Dilation) and two correlation measures (i.e., Spearman and Pearson). The ensemble may be reduced or expanded using additional algos including machine learning ML / ANN-based models for pattern recognition herein.
[0072] Further, in block 305, the root cause analysis system determines whether a single common problem cause is identified. If the root cause analysis system, based on the comparison results, points to a set of problems (measurements caused by components) that are the same in two or more ways (e.g., out of five), the root cause analysis system determines that the corresponding set and their respective source components are most likely to be the root cause of the problem (block 305, yes branch). Thus, the root cause analysis system proceeds to block 306 and prompts the user to start a deeper investigation into this particular component.
[0073] If the root cause analysis system points to a different component (no branch out of block 305), the root cause analysis system determines that the cause of the problem is not yet sufficiently defined, and so the root cause analysis system proceeds to block 307 and performs the next troubleshooting step by repeating the comparisons of blocks 303-305 with other pairs of metrics. If the iterations of the comparisons using the different methods point to the same component, the root cause analysis system determines that the corresponding component is likely the cause of the problem.
[0074] If the root cause analysis system has not identified a satisfactory component (i.e., a unique root cause) and all possible pairs of measurements still do not highlight one particular component, then the root cause analysis system may rank the relevant components in order of decreasing degree of overload. In an example embodiment, the root cause analysis system may start with the most overloaded component and then apply a leave-one-out method (block 309). Thus, the root cause analysis system allows for incremental removal of top bottlenecks one at a time, re-performing the comparisons of blocks 303-305 (via process 310) after each removal.
[0075] In a further example, the root cause analysis system may determine a candidate root cause set as described in steps 311-314. In step 311, a data source to run on may be provided, and in step 312, a set of measurements that may be provided by the data source may be determined. In step 313, the root cause analysis system may determine the top k correlated measurements of the set. The root cause analysis system may then use the top k correlated measurements of the set to determine a candidate root cause set (step 314).
[0076] FIG. 4 illustrates a general purpose computing system 400 suitable for implementing at least a portion of the method steps as included in the present disclosure.
[0077] It will be understood that the methods described herein are at least partially non-interactive and automated by a computerized system, such as a server or embedded system. However, in exemplary embodiments, the methods described herein may be implemented in a (partially) interactive system. The methods may further be implemented in software 412, firmware 422, hardware (processor 405), or a combination thereof. In some embodiments, the methods described herein may be implemented in software as an executable program and may be executed by a special purpose or general purpose digital computer, such as a personal computer, a workstation, a minicomputer, or a mainframe computer. In the most general case, system 400 includes a general purpose computer 401.
[0078] In an exemplary embodiment, with respect to the hardware architecture, as shown in FIG. 4, a computer 401 includes a processor 405, a memory (e.g., main memory) 410 coupled to a memory controller 415, and one or more input and / or output (I / O) devices or peripherals 10, 445 communicatively coupled via a local input / output controller 435. The input / output controller 435 may be, but is not limited to, one or more buses or other wired or wireless connections as known in the art. The input / output controller 435 may have additional elements such as controllers, buffers (caches), drivers, repeaters, and receivers to enable communication, which are omitted for simplicity. Additionally, the local interface may include address, control, or data connections, or combinations thereof, to enable appropriate communication between the above-mentioned components. The I / O devices 10, 445 as described herein may generally include any generalized cryptographic or smart card known in the art.
[0079] The processor 405 is specifically a hardware device for executing software stored in the memory 410. The processor 405 may be any custom-made or commercially available processor, a central processing unit (CPU), a coprocessor among several processors associated with the computer 401, a semiconductor-based microprocessor (in the form of a microchip or chipset), a microprocessor, or any device or combination thereof for executing software instructions.
[0080] The memory 410 may include any one or combination of volatile memory elements (e.g., random access memory (RAM) such as DRAM, SRAM, SDRAM, etc.), non-volatile memory elements (e.g., ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM)). It should be noted that the memory 410 may have a distributed architecture, where various components are in separate locations but accessible by the processor 405.
[0081] The software in memory 410 may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions, particularly functions involved in embodiments of the present invention. In the example of Figure 4, the software in memory 410 includes instructions for software 412, e.g., instructions for managing a database, such as a database management system.
[0082] The software in memory 410 also typically includes a suitable operating system (OS) 411. The OS 411 essentially controls the execution of other computer programs, such as software 412 that implements the methods described herein.
[0083] The methods described herein may be in the form of a source program of the software 412, an executable program (object code) of the software 412, a script or other entity that includes a set of instructions to be executed as an aspect of the software 412, or a combination thereof. In the case of a source program, the program must be translated to operate properly in conjunction with the OS 411 via a compiler, assembler, interpreter or the like or a combination thereof that may or may not be included in the memory 410. Additionally, the methods may be written in an object-oriented programming language having classes of data and methods, or a procedural programming language having routines, subroutines or functions or a combination thereof.
[0084] In an exemplary embodiment, a conventional keyboard 450 and mouse 455 may be coupled to the input / output controller 435. Other output devices, such as I / O devices 445, may include input devices, such as, but not limited to, a printer, scanner, microphone, and the like. Finally, the I / O devices 10, 445 may further include devices that communicate both input and output, such as, but not limited to, a network interface card or a modulator / demodulator (for accessing other files, devices, systems, or networks), or a radio frequency or other transceiver, telephone interface, bridge, router, or the like, or a combination thereof. The I / O devices 10, 445 may be any general purpose cryptographic card or smart card known in the art. The system 400 may further include a display controller 425 coupled to the display 430. In an exemplary embodiment, the system 400 may further include a network interface for coupling to a network 465. The network 465 may be an IP-based network for communication between the computer 401 and other external servers, clients, or the like, or a combination thereof, via a broadband connection. The network 465 transmits or receives data between the computer 401 and external systems 30 that may be involved in performing some or all steps of the methods discussed herein. In an exemplary embodiment, the network 465 may be a managed IP network managed by a service provider. The network 465 may be implemented in a wireless manner, for example, using wireless protocols and technologies such as Wi-Fi, WiMAX, etc. The network 465 may also be a packet-switched network, such as a local area network, a wide area network, a metropolitan area network, an Internet network, or other similar types of network environments, or a combination thereof.Network 465 may be a fixed wireless network, a wireless local area network (WLAN), a wireless wide area network (WWAN), a personal area network (PAN), a virtual private network (VPN), an intranet, or other suitable network system and includes devices for receiving and transmitting signals.
[0085] If computer 401 is a PC, workstation, intelligent device, or the like, or a combination thereof, the software in memory 410 may further include a Basic Input Output System (BIOS) 422. The BIOS is a set of essential software routines that initializes and tests the hardware at start-up, starts the OS 411, and supports the transfer of data between hardware devices. The BIOS is stored in ROM so that the BIOS is executed when computer 401 is activated.
[0086] When computer 401 is running, processor 405 is configured to execute software 412 stored in memory 410, to communicate data to and from memory 410, and generally to control the operation of computer 401 in accordance with the software. The methods and OS 411 described herein are loaded by processor 405, possibly buffered within processor 405, and executed therein, in whole or in part, but typically the latter.
[0087] When the systems and methods described herein are implemented in software 412, the methods may be stored on any computer-readable medium, such as storage 420, for use by or in connection with a computer-related system or method, as shown in Figure 4. Storage 420 may include disk storage, such as HDD storage.
[0088] The present invention may be a system, method or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to perform aspects of the present invention.
[0089] A computer readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. Illustrative enumeration of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the above. Computer-readable storage media, as used herein, is not to be construed as a transitory signal per se, such as an electric wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.
[0090] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for storage.
[0091] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object oriented languages such as Smalltalk®, C++ or the like, procedural languages such as the C programming language or similar programming languages. The computer readable program instructions may be executed as a stand-alone software package, entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, the electrical circuitry may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electrical circuitry, which may include, for example, programmable logic circuitry, a field programmable gate array (FPGA), or a programmable logic array (PLA), to perform aspects of the invention.
[0092] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer readable program instructions.
[0093] These computer readable program instructions, when provided to a computer processor or other programmable data processing apparatus, cause the instructions to execute on the computer processor or other programmable data processing apparatus to produce a machine-generated means for implementing the functions / acts identified in the block or blocks of the flowchart diagrams and / or blocks. These computer readable program instructions may also be stored on a computer readable storage medium capable of directing a computer, programmable data processing apparatus or other device or combination thereof to function in a particular manner, such that a computer readable storage medium having instructions stored thereon includes an article of manufacture including instructions that implement aspects of the functions / acts identified in the block or blocks of the flowchart diagrams and / or blocks.
[0094] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to generate a computer implemented process such that the instructions executing on the computer, other programmable data processing apparatus, or other device implement aspects of the functions / actions identified in a block or blocks of the flowchart and / or blocks.
[0095] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order depicted in the drawings. For example, two blocks shown in succession may in fact be completed as a single step simultaneously, substantially simultaneously, or in a partially or fully overlapping manner in time, or the blocks may be executed in the reverse order depending on the functionality involved. It should be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special purpose hardware-based system that performs a particular function or action or that implements a combination of special purpose hardware and computer instructions.
Claims
1. 1. A method comprising: identifying, by one or more processors, a plurality of data sources configured to provide sensor data as a first set of one or more time series, the sensor data including values of a first group of measurands including a plurality of measurands; determining, by one or more processors, that values of a second group of measurands including one or more measurands of the plurality of measurands for a subset of the sensor data are indicative of an anomaly, the subset of the sensor data being provided by a given data source of the plurality of data sources, the given data source including a set of components; determining, by one or more processors, a third group of measurements including one or more of the plurality of measurements as candidates for identifying a root cause of the anomaly; assigning, by one or more processors, a corresponding set of coefficients to the one or more measurements in the second group of measures, each coefficient in the corresponding set of coefficients indicating a comparison of a measurement in the second group of measures with a measurement in the third group of measures using one or more similarity techniques; determining, by one or more processors, using the corresponding set of coefficients, that a particular subset of the set of components of the given data source cannot be identified as a root cause of the anomaly; updating, by one or more processors, the third group of measures; assigning, by one or more processors, an updated set of corresponding coefficients to the one or more measurements included in the second group of measurements based at least in part on the updated third group of measurements; identifying, by one or more processors, a particular subset of the set of components of the given data source as the root cause of the anomaly using the updated corresponding set of coefficients; A method comprising:
2. The plurality of data sources are configured to further provide timestamps of the sensor data as a first set of the one or more time series; the subset of sensor data covers a time window; determining the third group of measures includes using the one or more similarity techniques to compare values of the second group of measures and the third group of measures over the time window, the one or more measures included in the third group of measures being provided by the set of components; assigning a corresponding set of coefficients includes assigning a corresponding set of coefficients for each similarity technique of the one or more similarity techniques and for each measure included in the second group of measures; assigning the updated corresponding sets of coefficients includes assigning an updated corresponding set of coefficients for each similarity technique of the one or more similarity techniques and for each measure included in the second group of measures; The method comprises: providing, by one or more processors, in response to determining that a particular subset of the set of components of the given data source is identifiable as the root cause of the anomaly, the particular subset of components determined to be identifiable as the root cause of the anomaly. The method of claim 1 , comprising:
3. The update of the third group of measurements further comprises: removing, by one or more processors, one or more measurements from the third group of measurements; and adding, by one or more processors, one or more measurements to the third group of measurements. The method of claim 2 , comprising at least said removing step of:
4. The one or more removed measurands are provided by selected components of the set of components, and the method further comprises: removing, by one or more processors, the selected component from the set of components for a next iteration. The method of claim 3 , comprising:
5. ranking, by one or more processors, the set of components in a monotonic sort order according to predefined characteristics of the set of components, the selected component being the first ranked component. The method of claim 4 further comprising:
6. The method of claim 5 , wherein the characteristic is a degree of overload.
7. The method of any one of claims 1 to 6, wherein the particular subset of the set of components is a single component of the set of components.
8. Determining that a particular subset of the set of components of the given data source cannot be identified as a root cause of the anomaly further comprises: for each measure of the second group of measures, combining, by one or more processors, a corresponding set of coefficients to obtain a set of combined coefficients; determining that the particular subset of the set of components of the given data source cannot be identified as a root cause of the anomaly using the set of coupling coefficients; The method according to any one of claims 1 to 7, comprising:
9. The method of claim 8, wherein the step of combining corresponding sets of coefficients comprises the step of summing the corresponding sets of coefficients.
10. The method comprises: normalizing, by one or more processors, the values of the measurements of the second and third groups of measurements prior to comparing the values of the second and third groups of measurements. The method according to any one of claims 1 to 9, comprising:
11. The method of any one of claims 1 to 10, wherein the one or more similarity techniques include at least one of the following: Manhattan distance, Euclidean distance, Dynamic Time Widening (DTW) distance, Spearman metric, and Pearson metric.
12. Determining that values of the second group of measurands of the subset of sensor data are indicative of an anomaly may further comprise: receiving, by one or more processors, an event ticket from said given data source; The method of any one of claims 1 to 11, further comprising:
13. 13. The method of claim 1, wherein determining that values of the second group of measures of the subset of sensor data indicate an anomaly is initiated in response to receiving an event ticket.
14. 14. The method of claim 1, wherein the second group of measurements comprises a measure selected from the group consisting of: (i) a measure of the first group of measurements of the measurements; and (ii) a combination of the measures of the first group of measurements of the measurements.
15. A method according to any preceding claim, wherein each time series of the first set of time series comprises a value of a respective measurand.
16. The computer system one or more processors; one or more computer readable storage media; program instructions stored on the computer-readable storage medium for execution by at least one of the one or more processors; the program instructions comprising: program instructions for identifying a plurality of data sources configured to provide sensor data as a first set of one or more time series, the sensor data including values of a first group of measurands including a plurality of measurands; program instructions for determining that a value of a second group of measurands comprising one or more measurands of the plurality of measurands of a subset of the sensor data indicates an anomaly, the subset of the sensor data being provided by a given data source of the plurality of data sources, the given data source comprising a set of components; determining a third group of measurements including one or more of the plurality of measurements as candidates for identifying a root cause of the anomaly; and program instructions for assigning a corresponding set of coefficients to the one or more measurements in the second group of measures, each coefficient in the corresponding set of coefficients indicating a comparison of a measurement in the second group of measures with a measurement in the third group of measures using one or more similarity techniques; program instructions for determining, using the corresponding set of coefficients, that a particular subset of the set of components of the given data source cannot be identified as a root cause of the anomaly; program instructions for updating the third group of measurands; program instructions for assigning updated corresponding sets of coefficients to the one or more measurements included in the second group of measurements based at least in part on the updated third group of measurements; program instructions for identifying a particular subset of the set of components of the given data source as the root cause of the anomaly using the updated corresponding set of coefficients; 1. A computer system comprising:
17. The method of claim 16, wherein the plurality of data sources are further configured to provide timestamps of the sensor data as a first set of the one or more time series; the subset of sensor data covers a time window; the program instructions for determining the third group of measures include program instructions for using the one or more similarity techniques to compare values of the second group of measures and the third group of measures over the time window, the one or more measures included in the third group of measures being provided by the set of components; the program instructions for assigning a corresponding set of coefficients include program instructions for assigning a corresponding set of coefficients for each similarity technique of the one or more similarity techniques and for each measure included in the second group of measures; the program instructions for assigning the updated corresponding sets of coefficients include program instructions for assigning an updated corresponding set of the coefficients for each similarity technique of the one or more similarity techniques and for each measure included in the second group of measures; The program instructions include: and in response to determining that a particular subset of the set of components of the given data source is identifiable as the root cause of the anomaly, program instructions for providing the determined identifiable particular subset of the components as the root cause of the anomaly.
20. The computer system of claim 16, comprising:
18. 18. The computer system of claim 16 or 17, wherein the particular subset of the set of components is a single component of the set of components.
19. A computer program comprising: identifying a plurality of data sources configured to provide sensor data as a first set of one or more time series, the sensor data including values of a first group of measurands including a plurality of measurands; determining that a value of a second group of measures comprising one or more measures of the plurality of measures of a subset of the sensor data indicates an anomaly, the subset of the sensor data being provided by a given data source of the plurality of data sources, the given data source comprising a set of components; determining a third group of measurements including one or more of the plurality of measurements as candidates for identifying a root cause of the anomaly; and assigning a corresponding set of coefficients to the one or more measurements in the second group of measures, each coefficient in the corresponding set of coefficients indicating a comparison of a measurement in the second group of measures with a measurement in the third group of measures using one or more similarity techniques; determining, using the corresponding set of coefficients, that a particular subset of the set of components of the given data source cannot be identified as a root cause of the anomaly; updating the third group of measures; assigning an updated set of corresponding coefficients to the one or more measurements included in the second group of measurements based at least in part on the updated third group of measurements; and identifying a particular subset of the set of components of the given data source as the root cause of the anomaly using the updated corresponding set of coefficients; A computer program for executing.
20. The method of claim 20, wherein the plurality of data sources are further configured to provide timestamps of the sensor data as a first set of the one or more time series; the subset of sensor data covers a time window; determining the third group of measures includes using the one or more similarity techniques to compare values of the second group of measures and the third group of measures over the time window, the measures included in the third group of measures being provided by the set of components; assigning a corresponding set of coefficients includes assigning a corresponding set of coefficients for each similarity technique of the one or more similarity techniques and for each measure included in the second group of measures; assigning the updated corresponding sets of coefficients includes assigning an updated corresponding set of coefficients for each similarity technique of the one or more similarity techniques and for each measure included in the second group of measures; The computer program causes the computer to: in response to determining that a particular subset of the set of components of the given data source is identifiable as the root cause of the anomaly, providing the determined identifiable particular subset of the components as the root cause of the anomaly.
20. The computer program product of claim 19, further comprising:
Citation Information
Patent Citations
Failure monitoring system, method, and program
JP2007293393A
Monitoring apparatus, monitoring method, and program
JP2015164005A
Operation management system, monitoring server, method and program
JP2019057139A
Fault processing system
US20120324271A1
