Failure prediction in distributed systems

Abstract

Embodiments of the invention relate to systems, methods, and apparatuses for detecting faults in distributed systems. A fault detection platform can identify anomalies in time series data corresponding to historical network messages. The anomalies can be labeled and used to train a first predictive model. At least one other model can be trained using the time series data, anomaly labels, and a supervised machine learning algorithm. A third model can be trained to identify system faults based at least in part on outputs provided by the first and second models. The third model, once trained, can be used to predict future system faults.

Description

Background Technology

[0001] Most modern distributed systems comprise numerous internal components interconnected with multiple external services and / or systems. Fault detection in these systems can be equally complex. Traditionally, fault detection is often reactive, with actions typically taken only after a failure has occurred. Fault prediction systems also heavily rely on the assumption that every component of the system, such as the central processing unit (CPU), memory, and input / output components, is accessible. In distributed systems, an entity typically cannot access every component. Improvements can be made regarding the detection of faults in distributed systems. The embodiments of this disclosure address these and other problems individually and collectively. Summary of the Invention

[0002] One embodiment of the present invention relates to a method. The method may include obtaining time-series data corresponding to historical network messages by a fault detection platform. The method may further include identifying multiple anomalies in the time-series data by the fault detection platform. The method may further include associating one or more of the multiple anomalies with classification labels indicating network faults by the fault detection platform. The method may further include training a first model among multiple models by the fault detection platform using the time-series data. The method may further include training a second model among multiple models by the fault detection platform using the time-series data, one or more anomalies associated with the classification labels, and a supervised machine learning algorithm. The method may further include training a third model by the fault detection platform at least in part based on the outputs of the multiple models and supplementary data. The method may further include predicting future system faults by the fault detection platform using the third model. The method may further include performing one or more corrective actions by the fault detection platform in response to the prediction of future system faults.

[0003] Another embodiment of the present invention relates to a computing device, comprising: one or more processors; and a memory including a computer-readable storage medium storing computer-executable instructions that, when executed by the processor, cause the computing device to perform operations. The operations may include obtaining time-series data corresponding to historical network messages by a fault detection platform. The operations may also include identifying multiple anomalies in the time-series data by the fault detection platform. The operations may further include associating one or more of the multiple anomalies with classification labels indicating network faults by the fault detection platform. The operations may further include training a first model among multiple models by the fault detection platform using the time-series data. The operations may further include training a second model among multiple models by the fault detection platform using the time-series data, one or more anomalies associated with classification labels, and a supervised machine learning algorithm by the fault detection platform. The operations may further include training a third model by the fault detection platform at least in part based on the outputs of the multiple models and supplementary data. The operations may further include predicting future system faults by the fault detection platform using the third model. The operations may further include performing one or more corrective actions by the fault detection platform in response to predicting future system faults.

[0004] More detailed information about embodiments of the present invention can be found in the detailed description and accompanying drawings. Attached Figure Description

[0005] Figure 1 A block diagram of an example distributed system, according to some embodiments, is shown, in which a monitoring computer can be used to predict future system failures.

[0006] Figure 2 The block diagram shown illustrates a method for predicting future system failures according to some embodiments.

[0007] Figure 3 A block diagram of an example monitoring computer according to some embodiments is shown. Detailed Implementation

[0008] Embodiments of the present invention relate to using a secure platform computer to perform joint tasks. In some embodiments, the tasks may be related to generating and utilizing machine learning models. Although the examples herein relate to the generation of machine learning models, it should be understood that a secure platform computer can be used in other situations besides machine learning to provide a secure task execution environment for two or more entities.

[0009] Two or more entities may utilize the secure platform computer disclosed herein to define projects and provide their respective project data. Project data from each entity may be stored by the secure platform computer in a secure memory space (e.g., a designated address space managed by the secure platform computer's chipset). In some embodiments, this secure memory space may be accessible only to the chipset and / or applications and / or modules executing within the secure memory space, and not to other processes and / or systems. By utilizing the secure platform computer, each entity can contribute to the project while ensuring the confidentiality of their data.

[0010] Before discussing specific embodiments of the present invention, some terms may be described in detail.

[0011] The term "computing device" generally refers to a device that performs calculations. A computing device can also provide the ability to communicate remotely with a network. Examples of remote communication capabilities include the use of mobile phone (wireless) networks, wireless data networks (such as 3G, 4G, or similar networks), Wi-Fi, Wi-Max, or any other communication medium that provides access to networks such as the Internet or a private network. Examples of user devices include mobile phones (e.g., cellular phones), PDAs, tablet computers, netbooks, laptops, personal music players, handheld dedicated readers, etc. Other examples of user devices include wearable devices such as smartwatches, fitness trackers, ankle bracelets, rings, earrings, etc., and automobiles with remote communication capabilities. A user device can include any suitable hardware and software for performing such functions, and can also include multiple devices or components (e.g., when two devices are connected to a network—that is, when the other device is used as a modem—and thus able to remotely access the network, the two connected devices can be considered a single user device).

[0012] A "user device" can be a device owned or operated by a user. A user device can be, for example, a smartphone, smartwatch, laptop, desktop computer, or game console. In some cases, a user device can be a web browser, or a combination of hardware, such as a smartphone and a web browser. As another example, a user device can include payment cards, such as credit cards, debit cards, smart cards, etc. A user device can be configured with payment credentials, which can be sent during a transaction.

[0013] "Time series data" refers to a sequence of data points indexed by time. Each data point may include any appropriate number of data attributes. Each data point may be associated with a timestamp or other suitable time indicator (or the order of other data points in the sequence).

[0014] Machine learning algorithms can be used to build mathematical models based on sample data called "training data" in order to make predictions or decisions without being explicitly programmed to perform the task. Some machine learning algorithms include supervised learning algorithms (e.g., classification algorithms, regression algorithms, decision trees, random forests, etc. that utilize labeled training data), semi-supervised learning algorithms (e.g., algorithms that utilize training data, some of which are labeled while some are unlabeled), unsupervised learning algorithms (e.g., cluster analysis algorithms, k-nearest neighbors, Apriori, etc.), and reinforcement learning algorithms (e.g., Markov decision processes, etc.).

[0015] A "machine learning model" can be a mathematical representation of a real-world process. In some embodiments, a machine learning model is a mathematical model generated (e.g., trained) using training data and machine learning algorithms. Some example models include artificial neural networks, recurrent neural networks, decision trees, Bayesian networks, etc.

[0016] "Anomaly" refers to a situation or event that deviates from the standard, normal, or expected. For example, a response to a message (e.g., an authorization request message) is typically received within a certain time period after the message is sent (e.g., an authorization response message). If no response is received within some threshold of that elapsed time period, such an event can be considered anomaly. As another non-limiting example, some messages have expected data field values ​​(e.g., values ​​received in historical messages and / or conforming to the message's specifications). When a message is received with data field values ​​that differ from these expected data field values, the event can be considered anomaly. "Multiple anomalies" can refer to multiple anomalies that can be identified within a collection of time-series data.

[0017] A “classification label” can refer to any suitable identifier that categorizes examples of time series data. For example, a classification label can be used to indicate that examples of time series data include one or more anomalies (e.g., unexpected data field values, unexpected message reception, unexpected delays in message reception, etc.).

[0018] The "Autoregressive Integrated Moving Average (ARIMA) model" is a generalization of the Autoregressive Moving Average (ARMA) model. The category of models including ARIMA and ASMA can be called "autoregressive algorithms." These models are fitted to time series data to better understand the data or predict future points in the sequence (called "forecasts"). The AR part of ARIMA indicates that the evolutionary variable of interest regresses to its own lagged (e.g., previous) values. The MA part indicates that the regression error is actually a linear combination of error terms that occurred simultaneously and at different times in the past. The I part indicates that the data value has been replaced (e.g., transformed) to the difference between its value and its previous value. The purpose of each of these features is to make the model fit the data as well as possible.

[0019] A "mixture model" refers to a statistical model that utilizes multiple mixture components. A typical finite-dimensional mixture model is a hierarchical model consisting of the following components: 1) N observed random variables, each according to a mixture distribution of K components, where the components belong to the same family of parameter distributions (e.g., all normal, all Zipfian, etc.) but have different parameters; 2) N random latent variables specifying the identity of the mixture components for each observation, each according to a K-dimensional categorical distribution; 3) a set of K mixture weights, which are probabilities that sum to 1; and 4) a set of K parameters, each specifying the parameters of the corresponding mixture component. In many cases, each "parameter" is actually a set of parameters. For example, if the mixture components are Gaussian distributed, each component will have a mean and variance. If the mixture components are categorically distributed (e.g., when each observation is from tokens of a finite alphabet of size V), there will be a vector of V probabilities that sum to 1. In a Bayesian setting, the mixture weights and parameters themselves will be random variables, and a prior distribution will be placed on the variables. In such cases, the weights can typically be viewed as K-dimensional random vectors plotted according to a Dirichlet distribution (the conjugate prior of a categorical distribution), and the parameters will be distributed according to their respective conjugate priors. Mixture models can use any suitable number of arbitrary distributions. Some example distributions are binomial, multinomial, negative binomial, Poisson, exponential, log-normal, multivariate normal, multivariate student's-t, a vector of Bernoulli distribution values, and so on.

[0020] A "stochastic process model" is a mathematical model typically defined as a family of random variables indexed by a mathematical set such that each random variable is uniquely associated with an element in that set. Random variables can be associated with a set of numbers or indexed by a set of numbers. These random variables can be viewed at different points in time because they can change over time. Some example stochastic process models may include random walks, martingales, Markov processes, Lévy processes, Gaussian processes, random fields, update processes, and branching processes.

[0021] A recurrent neural network (RNN) is a machine learning algorithm that is loosely modeled after the human brain and designed to recognize patterns. Neural networks can classify or cluster new data. They help group unlabeled data based on similarity between example inputs, and / or they can classify data when labeled datasets have already been used to train the model. An RNN is a type of neural network where connections between nodes form a directional graph along a time series. This allows RNNs to exhibit temporal dynamics. When necessary, RNNs can use internal memory to process the sequence of inputs.

[0022] An Application Programming Interface (API) can be an interface or communication protocol between a client and a server. In some embodiments, an API can define the format of a specific request and its corresponding response. APIs can take many forms, but typically include specifications for routines, data structures, object classes, variables, or remote calls. APIs can be used in network-based systems, operating systems, database systems, computer hardware or software libraries, and so on.

[0023] A "payment credential" may include any suitable information associated with an account (e.g., a payment account and / or a payment device associated with the account). This information may be directly related to the account or derived from account-related information. Instances of account information may include a PAN (primary account number or "account number"), username, expiry date, CVV (card verification value), dCVV (dynamic card verification value), CVV2 (card verification value 2), CVC3 card verification value, and so on. CVV2 is generally understood to be a static verification value associated with a payment device. CVV2 values ​​are typically visible to the user (e.g., a consumer), while CVV and dCVV values ​​are typically embedded in memory or authorization request messages and are not easily known to the user (although they are known to the issuer and payment processor). A payment credential may be any information identifying a payment account or associated with a payment account. A payment credential can be provided to make payments from a payment account. A payment credential may also include a username, expiry date, gift card number or code, and any other suitable information.

[0024] A "resource provider" can be an entity that provides resources such as goods, services, information, and / or access. Examples of resource providers include merchants, access devices, secure data access points, etc. A "merchant" can typically be an entity that participates in transactions and can sell goods or services or provide access to goods or services. A resource provider may operate a computer to perform operations, and the computer may also be referred to as a "resource provider computer."

[0025] An "acquiring party" can typically be a business entity that has a business relationship with a particular merchant or other entity (e.g., a commercial bank). Some entities can perform the functions of both an issuer and an acquirer. Some embodiments may cover such a single entity as an issuer-acquiring party. The acquirer can operate an acquiring party computer, which may also generally be referred to as a "transfer computer".

[0026] An "authorizing entity" can be the entity that requests authorization. Examples of authorizing entities include issuers, government agencies, document repositories, access administrators, etc. An "issuer" can typically refer to a commercial entity that maintains user accounts (e.g., a bank). Issuers may also issue payment credentials to consumers stored on user devices such as cellular phones, smart cards, tablets, or laptops. An authorizing entity may operate a computer to perform operations; this computer may also be referred to as an "authorizing entity computer."

[0027] An "access device" can be any suitable device that provides access to a remote system. An access device can also be used to communicate with a merchant's computer, transaction processing computer, authentication computer, or any other suitable system. Access devices can typically be located anywhere suitable, such as at the merchant's location. Access devices can take any suitable form. Some examples of access devices include POS or point-of-sale devices (e.g., POS terminals), cellular phones, PDAs, personal computers (PCs), tablet PCs, handheld dedicated readers, set-top boxes, electronic cash registers (ECRs), automated teller machines (ATMs), virtual cash registers (VCRs), kiosks, security systems, access systems, etc. Access devices can use any suitable contact or contactless operating mode to send or receive data to or from a user's mobile device, or associate with a user's mobile device. In some embodiments where the access device may include a POS terminal, any suitable POS terminal can be used, and it may include a reader, a processor, and a computer-readable medium. The reader may include any suitable contact or contactless operating mode. For example, an exemplary card reader may include a radio frequency (RF) antenna, an optical scanner, a barcode reader, or a magnetic stripe reader to interact with payment devices and / or mobile devices. In some embodiments, a cellular phone, tablet computer, or other dedicated wireless device used as a POS terminal may be referred to as a mobile point of sale or “mPOS” terminal.

[0028] An "authorization request message" can be an electronic message requesting authorization for a transaction. In some embodiments, the authorization request message is sent to the transaction processing computer and / or the issuer of the payment card to request transaction authorization. Authorization request messages according to some embodiments may conform to ISO 8583, a standard for systems exchanging information about electronic transactions associated with payments made by a user using a payment device or payment account. The authorization request message may include an issuer account identifier that can be associated with the payment device or payment account. The authorization request message may also include additional data elements corresponding to the "transaction data," including, for example only, a service code, CVV (card verification value), dCVV (dynamic card verification value), PAN (primary account number or "account number"), payment token, username, expiration date, etc. The authorization request message may include additional "transaction data," such as any information associated with the current transaction, such as the transaction amount, merchant identifier, merchant location, acquiring bank identification number (BIN), card acceptor ID, information identifying the purchased item, and any other information that can be used to determine whether the transaction is identified and / or authorized.

[0029] An "authorization response message" can be a message responding to an authorization request. In some cases, an authorization response message can be an electronic message response to an authorization request message generated by the issuing financial institution or a transaction processing computer. For example only, an authorization response message may include one or more of the following status indicators: approved – the transaction is approved; rejected – the transaction is not approved; or call center – further information is pending, and the merchant must call the toll-free authorization number. An authorization response message may also include an authorization code, which can be a code returned by the credit card issuing bank to the merchant's access device (e.g., a POS device) in response to the authorization request message in an electronic message (directly or via the transaction processing computer), indicating that the transaction has been approved. This code can serve as evidence of authorization. As described above, in some embodiments, the transaction processing computer may generate or forward authorization response messages to the merchant.

[0030] "Memory" can be any suitable one or more devices capable of storing electronic data. Suitable memory can include a non-transient computer-readable medium storing instructions that can be executed by a processor to implement a desired method. Examples of memory can include one or more memory chips, disk drives, etc. Such memory can be operated using any suitable electrical, optical, and / or magnetic modes of operation.

[0031] "Processor" can refer to any suitable one or more data computing devices. A processor can include one or more microprocessors that work together to perform the desired function. A processor can include a CPU, which includes at least one high-speed data processor sufficient to execute program components for performing user and / or system-generated requests. The CPU can be a microprocessor such as AMD's Athlon, Duron, and / or Opteron; IBM and / or Motorola's PowerPC; IBM and Sony's Cell processor; Intel's Celeron, Itanium, Pentium, Xeon, and / or XScale; and / or similar processors.

[0032] A "server computer" can include a powerful computer or cluster of computers. For example, a server computer can be a mainframe, a small cluster of computers, or a group of servers operating like cells. In one example, a server computer can be a database server coupled to a web server. A server computer can be coupled to a database and can include any hardware, software, other logic, or combination of the foregoing for serving requests from one or more client computers. A server computer can include one or more computing devices and can use any of a variety of computing architectures, arrangements, and compilations to serve requests from one or more client computers.

[0033] Figure 1 A block diagram of an example distributed system 100, in which a monitoring computer 102 (e.g., a computing device for a fault detection system) can be used to predict future system failures according to some embodiments, is shown. Figure 1 The transaction processing system is shown, but any suitable system with any suitable number of components may be used. System 100 may include a monitoring computer 102, a user device 106, an access device 108, a resource provider computer 110, a transmission computer 112, a processing network computer 114, and an authorization entity computer 116.

[0034] Figure 1 A user 104 is shown operating a user device 106 (e.g., a payment card). User 104 can use the user device 106 to pay for goods or services at, for example, a merchant's resource provider. The merchant can operate a resource provider computer 110 and / or access device 108. The resource provider computer 110 can communicate with an authorizing entity computer 116 operated by the issuer via a transmission computer 112 operated by the acquirer and a processing network computer 114 operating as part of a payment processing network.

[0035] A payment processing network may include data processing subsystems, networks, and operations used to support and deliver authorization services, exception handling services, and clearing and settlement services. An exemplary payment processing network may include VisaNet™. For example, a payment processing network like VisaNet™ is capable of processing credit card transactions, debit card transactions, and other types of commercial transactions. VisaNet™ specifically includes a VIP system (Visa Integrated Payment System) that processes authorization requests, and a Base II system that performs clearing and settlement services. The payment processing network can use any suitable wired or wireless network, including the Internet.

[0036] A typical payment transaction process may involve the use of a user device 106 at access device 108, as described below. User device 106 may be a payment card that can be swiped or tapped at access device 108 to initiate a transaction. User device 106 and access device 108 may exchange card data (e.g., provided by user device 106) and terminal data (e.g., provided by access device 108). In response to this exchange, access device 108 may generate a transaction message to proceed with the payment. In some embodiments, the transaction message generated by access device 108 may contain any suitable data, such as any suitable transaction data. For example, the transaction message may include data from track 2 (card data) provided by user device 106, including payment credentials, merchant data provided by access device 108, etc. The transaction message may further indicate the amount associated with the transaction (e.g., the payment price).

[0037] Resource provider computer 110 can receive this information from access device 108 via an external communication interface (e.g., via an application programming interface). Resource provider computer 110 can then generate an authorization request message and electronically send it to transmission computer 112, the authorization request message including at least a portion of the information received from access device 108. Transmission computer 112 can then receive, process, and forward the authorization request message to processing network computer 114 for authorization.

[0038] Generally, prior to a credit card or debit card transaction, the processing network computer 114 has established an agreement with each issuer regarding how to authorize transactions from that issuer. In some cases, such as when the transaction amount is below a threshold, the processing network computer 114 can be configured to authorize the transaction based on its information about the user account without generating an authorization request message and sending it to the authorization entity computer 116. In other cases, such as when the transaction amount is above a threshold, the processing network computer 114 can receive the authorization request message, identify the issuer associated with the user device 106, and forward the authorization request message for the transaction to the authorization entity computer 116 for verification and authorization. Once the transaction is authorized, the authorization entity computer 116 can generate an authorization response message (which may include an authorization code indicating whether the transaction is approved or rejected) and send this electronic message to the processing network computer 114 via its external communication interface. Then, the processing network computer 114 can forward the authorization response message to the transmission computer 112, which can then send an electronic message including an authorization instruction to the resource provider computer 110, which can then forward the message to the access device 108.

[0039] At the end of the day or at some other suitable time interval, the clearing and settlement process between the transaction execution resource provider computer 110, the transmission computer 112, the processing network computer 114 and / or the authorized entity computer 116 can be carried out.

[0040] In some embodiments, monitoring computer 102 (also referred to as a “fault detection platform”) may communicatively connect to any suitable combination of resource provider computer 110, transmission computer 112, processing network computer 114, and authorization entity computer 116 (collectively, “system components”). Each system component may be configured to send authorization request messages not only to the next component in the sequence as described above, but also to monitoring computer 102. Similarly, each component may send a corresponding authorization response message to the next component in the sequence and to monitoring computer 102. Thus, in some embodiments, monitoring computer 102 may collect network messages (e.g., authorization request messages, authorization response messages, etc.) from any suitable combination of system components. Each network message may be timestamped (e.g., by the sending computer or by the receiving computer) and may be considered an example of time-series data. Although network messages are utilized in the examples provided herein, it should be understood that similar techniques can be used to utilize any suitable data to detect future faults, rather than necessarily utilizing network messages specifically.

[0041] In some embodiments, the monitoring computer 102 may be configured to train and / or maintain multiple models, including two or more machine learning models (e.g., a first model, a second model, etc.) configured to predict future system failures based at least in part on network messages provided as input.

[0042] In some embodiments, monitoring computer 102 can detect anomalies in historical network messages by feeding them to a machine learning model that has been previously trained (e.g., by monitoring computer 102) to identify anomalies from network messages. In some embodiments, the machine learning model may have been previously trained (e.g., by monitoring computer 102) using supervised learning techniques and a training dataset comprising multiple network messages labeled as anomalous or non-anomalous. Subsequent network messages may be fed to the trained model to identify anomalies in those messages. Once identified, the specific network message considered anomalous can be labeled (e.g., using classification labels indicating anomalies). This labeled data may be used by some of the machine learning algorithms (e.g., supervised machine learning algorithms) to train one or more additional models (e.g., a second model, any suitable model trained using supervised machine learning algorithms, etc.).

[0043] Each of the multiple machine learning models trained by the monitoring computer 102 can utilize the same or different machine learning algorithms (e.g., supervised machine learning, semi-supervised machine learning, unsupervised machine learning, etc.). For example, a first model among the multiple models can be trained to identify a first set of anomalies using time series data and a machine learning algorithm (e.g., Autoregressive Integrated Moving Average (ARIMA)). A second model among the multiple models can be trained to identify a second set of anomalies using time series data, one or more anomalies associated with classification labels (e.g., time series data labeled with classification labels indicating anomalies), and supervised machine learning algorithms (e.g., multivariate regression, random forest, decision tree, etc.). In some cases, the monitoring computer 102 can utilize historical network messages to detect one or more anomalies (e.g., anomalies, multiple anomalies), and then can label said one or more anomalies (e.g., using classification labels indicating the occurrence of anomalies).

[0044] The monitoring computer 102 can train and maintain another machine learning model (referred to as a "combined model" or "third model"), which is trained to predict future system failures based at least in part on the outputs provided by other machine learning models (e.g., at least in part on the outputs provided by the first and second models discussed above, or using the outputs provided by any suitable number of models). In some embodiments, the third model can be trained using the outputs of other machine learning models and supplementary data (e.g., call center data indicating the reason for a customer's call, or any suitable data not provided by the model).

[0045] When the combined model predicts a system failure, the monitoring computer 102 can be configured to perform one or more corrective actions. For example, in response to determining that a system failure will occur within a specific component, the monitoring computer 102 can be configured to perform corrective actions, such as notifying the component (e.g., sending a notification such as an email or text message, push notification, etc.) to indicate the action to be taken. For example, the notification may instruct the component to switch to a backup server, restart the server, switch to a backup connection, etc. In some embodiments, the specific corrective action suggested may depend on the specific type of the predicted failure.

[0046] Figure 2 The block diagram shown illustrates a method 200 for predicting future system failures according to some embodiments. Method 200 can be... Figure 1 The monitoring computer 102 performs the operation.

[0047] Method 200 can begin with step 1, and can start from components of the distributed system (e.g., Figure 1 The distributed system 100 (any suitable component) collects time-series data. "Time-series data" can refer to a sequence of time-indexed data points (e.g., messages). Each data point (message) may include any suitable number of data attributes (e.g., data fields). Each data point may be associated with a timestamp or other suitable time indicator (or the order relative to other data points in the sequence). In some embodiments, time-series data may be collected once and / or periodically, at regular intervals and / or within a certain time period. As a non-limiting example, each data point may be a network packet (e.g., a message, such as an authorization request message and / or a corresponding authorization response message), an error detected by a system component, a delay between related messages, time since a previous action, or as combined above. Figure 1 Any suitable data relating to the transaction as described.

[0048] In step 2, the monitoring computer 102 can store the received time-series data in a data storage area for later use. In some embodiments, the monitoring computer 102 can store related data points in a common storage container. For example, messages sent and / or received by a specific system component can be stored in one storage container, while messages sent and / or received by another system component can be stored in different storage containers. In some embodiments, the monitoring computer 102 can receive multiple types of data from system components and can combine multiple types of data in storage containers associated with system components.

[0049] In step 3, the monitoring computer 102 may execute one or more processes to enrich the time-series collected data. For example, the monitoring computer 102 may calculate the elapsed time between related messages (e.g., the elapsed time between the sending of an authorization request message and the receiving of a corresponding authorization response message).

[0050] In step 4, the monitoring computer 102 may perform one or more processes for transforming the initially received time-series data into one or more different formats. In some embodiments, one or more data transformations may be applied to the time-series data. Data transformations may be used to remove noise and / or improve the signal provided in the time-series data. In some embodiments, the monitoring computer 102 may utilize a predefined list of transformations corresponding to reformatting and / or transforming the time-series data to suit multiple predetermined machine learning algorithms.

[0051] As an example, time series data with a quadratic growth trend can be made linear by taking the square root of the data points (e.g., applying a square root transformation). In some embodiments, an exponential time series distribution can be made linear by taking the logarithm of the data point values ​​(e.g., applying a logarithmic transformation). In some embodiments, monitoring computer 102 can transform time series data using box-cox transformations (e.g., configurable data transformation methods that support square root, logarithmic, and other related transformations). Power transformations can be used by monitoring computer 102 to remove shifts from the data distribution to make the distribution more normal (Gaussian). For time series datasets, this can have the effect of keeping changes over time invariant. Another example transformation can include different transformations that can be used to remove systemic structure from time series data. For example, a trend can be removed by subtracting the previous value from each value in the time series. Monitoring computer 102 can normalize the data by rescaling the time series data from its original range to a new range (e.g., from a range of 1-100 to a range of 0-1). Normalization transformations can be used to transform time series data with a Gaussian distribution. In some embodiments, this transformation can have the effect of transforming time series data to a mean of zero or a centered standard deviation of 1. The transformations discussed herein are merely illustrative examples, and any suitable transformation can be used by the monitoring computer 102 according to any suitable predefined rules to transform time series data into a suitable / optimal format for a particular set of models trained and maintained by the monitoring computer 102.

[0052] In step 5, monitoring computer 102 can execute any suitable number of processes for performing time series data aggregation. "Time series data aggregation" can refer to the aggregation of time series data points within a specified time period (referred to as "granularity") (e.g., for a system component). As a non-limiting example, the granularity of time series data aggregation could be 30 minutes. Therefore, in this example, monitoring computer 102 can perform the operation of aggregating all data points (e.g., from a specific system component, from all system components, etc.) within each 30-minute time period of the time series dataset. Monitoring computer 102 can aggregate the entire time series into any suitable number of corresponding 30-minute time periods. In some embodiments, monitoring computer 102 can be configured to combine the values ​​of time series data within a given time period. For example, the average of the time series data points can be calculated and correlated with a specific time period. By aggregating time series data, monitoring computer 102 can significantly reduce the computational requirements needed for later model training.

[0053] In step 6, any resulting data, such as additional data fields, transformed datasets, and / or aggregated data, can be stored in the data storage area. In some embodiments, this data can be stored together with or associated with the received original time-series data. For example, time-series data, enriched time-series data, transformed time-series data, and aggregated time-series data can be stored together as tuples or in any suitable data container.

[0054] In step 7, the monitoring computer 102 may be configured to perform any suitable process for detecting anomalies in the stored data. An "anomaly" refers to an outlier data point that is at least partially an outlier based on some standard or common signal in the dataset. For example, an unexpected spike or drop in a message can indicate an anomaly.

[0055] In some embodiments, one or more anomaly detection algorithms can be used to detect anomalies in time series data. For example, supervised machine learning algorithms can be used to detect anomalies in a dataset. Supervised machine learning algorithms can utilize a training dataset that includes normal samples and samples showing anomalies. Each sample can be labeled as either anomaly or non-anomaly. Some example supervised algorithms may include supervised neural networks, parameterization of the trained model, support vector machine learning, Bayesian networks, decision trees, k-nearest neighbor algorithms, etc. As a non-limiting example, the k-nearest neighbor algorithm can be used to calculate approximate distances between different points (e.g., different data points, input vectors, etc.) and then assign unlabeled points to the category of their k-nearest neighbors. If a data point's neighbors are anomalies, that data point can also be labeled / classified as anomaly. Otherwise, the data point can be labeled as non-anomaly.

[0056] In some embodiments, unsupervised machine learning algorithms can be used for anomaly detection. Unsupervised machine learning algorithms do not require training data. They assume two things: 1) only a small subset of the data is anomalous, and 2) any anomaly is statistically distinct from normal samples. Based on these assumptions, similar and frequently occurring data samples can be grouped (e.g., clusters) and assumed to be normal, while other data samples that occur occasionally (e.g., data points not near clusters) can be considered anomalous. Some example unsupervised machine learning algorithms include k-means, c-means, self-organizing maps, single-class support vector machines, expectation-maximization meta-algorithms, and adaptive resonance theory.

[0057] In some embodiments, a seasonal trend decomposition procedure can be used to segment the time series data signal into three parts: seasonality, trend, and residue. If a deviation in the residue is detected and a threshold is applied, any data point that violates that threshold can be considered an anomaly. Each data sample from previously stored training data can be labeled as an anomaly and / or non-anomaly. In some embodiments, these labeled data samples can also be stored.

[0058] In step 8, multiple additional machine learning models and / or statistical models (collectively, “Models”) can be trained and / or fitted using the time series data and / or the time series data enriched, transformed, and / or aggregated in steps 3-5, respectively. Any suitable number of models can be trained / fitted. However, in some embodiments, the models may include ARIMA models, mixture models, stochastic process models, recurrent neural networks, or any suitable combination of two or more models listed above. Each of these models may be an instance of a “first model” and / or a “second model”.

[0059] In some embodiments, (enriched, transformed, and / or aggregated) time series data can be used in conjunction with one or more autoregressive algorithms (e.g., ARIMA, ARMA, etc.) to fit one or more models to the time series data. One or more models can be fitted to the time series data to better understand the data and / or predict future points in the sequence (referred to as "forecasts"). The model fitted to the time series data can be referred to as a "statistical model." Any suitable number and type of statistical models can be fitted to the time series data to forecast (predict) subsequent data points (e.g., anomalies indicating system failures).

[0060] In some embodiments, past points in time series data can influence present and future points in time. ARIMA is one such model. ARIMA can use multiple lagged observations of a time series to forecast (predict) observations. Weights can be applied to each of the past observations, and these weights can vary based on their relevance. This is known as autoregression, which is the process of regressing a variable back to its past values. In ARIMA models, integral properties can be used to reduce the seasonality of time series data. For example, ARIMA models can utilize the degree of differencing that removes seasonality. ARIMA models can also use moving averages to remove nondeterministic or random shifts from the time series. The coefficients of an ARIMA model can be calculated recursively, and the model can be selected based on estimates calculated from the model that are closest to the actual observations.

[0061] In some embodiments, one or more mixture models can be fitted to (e.g., received, enriched, transformed, and / or aggregated) time series data. Statistical models utilizing multiple mixture components are employed. A typical finite-dimensional mixture model is a hierarchical model consisting of: 1) N observed random variables, each according to a mixture distribution of K components, wherein the components belong to the same family of parameter distributions (e.g., all normal, all Zipfian, etc.) but have different parameters; 2) N random latent variables specifying the identity of the mixture components at each observation, each according to a K-dimensional categorical distribution; 3) a set of K mixture weights, which are probabilities that sum to 1; and 4) a set of K parameters, each specifying the parameters of a corresponding mixture component. In some embodiments, the mixture model can utilize any suitable number of arbitrary distributions. Some example distributions are vectors of binomial, multinomial, negative binomial, Poisson, exponential, log-normal, multivariate normal, multivariate student's-t, and Bernoulli distribution values, etc.

[0062] As a non-limiting example, a Gaussian mixture model (GMM) (e.g., a probabilistic model) can be fitted to (received, enriched, transformed, or aggregated) time-series data. A GMM may assume that the observed data consists of a mixture of several Gaussian distributions, which may also be given different mixing weights. The final distribution of the time-series data can be obtained by multiplying each mixture component by its associated mixing weight and then summing them together. A GMM can be in the form of unsupervised learning. However, supervised learning GMMs can be used alternatively or in combination. Data can be modeled using any suitable number of mixture models, each utilizing any suitable number of distributions.

[0063] In some embodiments, a stochastic process model can be fitted to time series data. A stochastic process model can be a mathematical model typically limited to a family of random variables indexed by a mathematical set such that each random variable is uniquely associated with an element in that set. Random variables can be associated with a set of numbers or indexed by a set of numbers. These random variables can be viewed at different points in time because they can change over time. Some example stochastic process models may include random walks, martingales, Markov processes, Lévy processes, Gaussian processes, random fields, update processes, and branching processes.

[0064] For example, Markov processes can be fitted to (e.g., received, enriched, transformed, or aggregated) time-series data. For instance, a Markov chain is a mathematical system constituting a set of random variables that transition from one state to another according to certain probability rules. These transitions can satisfy Markov properties, which state that the probability of transitioning to any particular state depends only on the current state and the elapsed time, and not on the previous sequence of states. Markov chains are memoryless. A transition matrix quantifying the probability of transitioning from one state to another can be generated for a Markov chain. This transition matrix can be generated, at least in part, based on the time-series data and labels. One or more states of a Markov chain can represent fault states (e.g., anomalies). By utilizing the transition matrix, the probability of a transition to a fault state (anomaly) occurring can be calculated. If this probability exceeds a threshold, it can be said that the Markov chain predicts a fault. Markov chains can be utilized in a manner similar to supervised or unsupervised learning.

[0065] In some embodiments, recurrent neural networks (RNNs) can be generated to model time-series data (e.g., received, enriched, transformed, or aggregated). An RNN is a machine learning algorithm that can classify or cluster new data. RNNs can group unlabeled data (e.g., new time-series data) based on similarity between example inputs, and / or they can classify data when labeled datasets have been used to train the model. In some embodiments, an RNN can be trained using labeled time-series data (e.g., instances labeled as “abnormal” or “non-abnormal”) to identify whether anomalies are likely to occur in the future based on the time-series data.

[0066] In some embodiments, each generated model (e.g., an ARIMA model, a mixture model, a stochastic process model, an RNN, or any suitable combination of two or more of the above) can be used to provide a corresponding output indicating the probability of an anomaly occurring within a specific future time period, such as the next 5 minutes. A combined model can be trained using a training dataset that includes the outputs of each model. A “combined model” refers to a machine learning model that takes as input at least two outputs provided by any suitable combination of the corresponding machine learning model and / or statistical model. In some embodiments, the model outputs and labeled time-series data can be combined to form a training dataset for the combined model. The combined model can be trained using any suitable supervised machine learning algorithm to classify whether future data indicates the occurrence and / or probability of a fault. In some embodiments, the output of the combined model can be a value indicating the probability of a fault. If the output violates a predefined threshold, the input data can be identified as indicating a system fault. In some embodiments, the training dataset can include supplementary data, such as multiple calls from a call center (e.g., an IT help call center), which can be used as another signal to determine whether an error has occurred. In some embodiments, the combined model can be trained to classify and / or cluster instances of the training dataset. Therefore, the output of the combined model can indicate not only the probability of a failure occurring, but also the type of failure predicted.

[0067] In step 9, once the various models have been trained, new time series data can be collected (e.g., from...). Figure 1 (Components). In some embodiments, similar operations can be performed to enrich, transform, and / or aggregate new time series data, as described in steps 3-5 above. The new time series data (received, enriched, transformed, and aggregated) can be provided as input to models (excluding combined models) to generate outputs from each of these models.

[0068] In step 10, the model's output can be combined with the new time series data.

[0069] In step 11, in some embodiments, supplementary data (e.g., multiple IT support calls) may be received. New time-series data (including outputs from other models), and in some cases, supplementary data, may be provided as input to the combined model. If the output indicates (e.g., within a specific time period, such as the next ten minutes, for a specific component) the probability of a failure (of a specific type of failure) occurring (exceeding a predefined threshold), a notification may be sent to the entity associated with the component corresponding to the possible failure. In some embodiments, a predefined mapping may be used to look up specific actions to be taken. In some embodiments, the mapping may identify specific actions to be taken (e.g., restarting a specific server, switching to a backup server, switching to a backup connection, etc.) based at least in part on the specific type of failure predicted. The notification may be in any suitable format (e.g., email message, text message, push notification, etc.).

[0070] In step 12, new time-series data can be labeled as “abnormal” or “non-abnormal” at least in part based on the combined output (e.g., “abnormal” if a threshold is violated; “non-abnormal” if the threshold is not violated). Examples of labeled time-series data can be added to the training dataset, and the model can be incrementally updated, retrained, and / or refitted later using these labeled examples.

[0071] Figure 3 A block diagram of an example monitoring computer 102 according to some embodiments is shown. The monitoring computer 102 may include a processor 304. The processor 304 may be coupled to system memory 306 and external communication interface 308.

[0072] The computer-readable medium 310 may also be operatively coupled to the processor 304. The computer-readable medium 310 may include software executable by the processor 304. For example, the computer-readable medium 310 may include multiple software modules executable by the processor 304, such as a data processing module 312, an anomaly detection manager 314, a model manager 316, a combined model manager 318, a fault monitoring engine 320, and a notification engine 322. In some embodiments, Figure 3 Each or some subsets of the modules can be communicatively connected to the data storage area 324.

[0073] Data storage area 324 can be implemented using various data structures, such as arrays, hash graphs, (linked) lists, constructed text files (e.g., XML), tables, etc. Such data structures can be stored in memory and / or in structured files. Data storage area 324 can be configured to store any suitable data associated with providing the functionality of monitoring computer 102 as discussed above. For example, data storage area 324 can store any suitable number of machine learning models, training datasets, one or more combined models, historical system data (e.g., historical network messages), etc.

[0074] Data processing module 312 can be configured to cause processor 304 to receive any suitable data from any suitable source. For example, data processing module 312 can be configured to receive any suitable number of training datasets. In some embodiments, the training dataset may include any suitable number of historical network messages (e.g., authorization request messages, authorization response messages, etc.) between components of a distributed system. In some embodiments, the training dataset may include time-series data. These historical network messages may be any suitable messages exchanged between such components. In some embodiments, historical network messages may be timestamped or otherwise indexed / sorted by time. In some instances, each network message may be associated with a response time, which indicates the amount of time elapsed before a response to the message is received. In some embodiments, data processing module 312 may combine multiple related messages and / or supplement messages with supplementary data. For example, data processing module 312 may receive an authorization request message and subsequently receive a corresponding authorization response message. In some embodiments, data processing module 312 may store the data from the authorization request message together with the data received in the authorization response message. If the time interval between the sending of the authorization request message (by a system component) and the receiving of the authorization response message (by a system component) is unknown, the data processing module 312 can calculate this time interval and associate it with the authorization request message and / or the authorization response message. In some embodiments, the data processing module 312 may be configured to cause the processor 304 to perform the above-mentioned combination. Figure 2 Any suitable operation discussed in steps 1-5. In some embodiments, data processing module 312 may store historical network messages (e.g., also referred to as time-series data) in a suitable data storage area, such as data storage area 324.

[0075] In some embodiments, the anomaly detection manager 314 may be configured to cause the processor 304 to detect anomalies in stored data (e.g., time-series data retrieved from data storage area 324). An "anomaly" refers to an outlier data point that is at least partially an outlier based on some standard or common signals of the dataset. The anomaly detection manager 314 may be configured to cause the processor 304 to execute one or more anomaly detection algorithms to detect anomalies in the time-series data. For example, the anomaly detection manager 314 may acquire, train, and / or maintain a machine learning model that can be used to detect anomalies in new time-series data inputs. The machine learning model may be trained using training data stored in data storage area 324. In some embodiments, instances in the training data may be pre-labeled as indicating anomalies or not, and the model may be trained using supervised machine learning algorithms. Some example supervised algorithms may include supervised neural networks, parameterization of the training model, support vector machine learning, Bayesian networks, decision trees, k-nearest neighbor algorithms, etc. In some embodiments, the k-nearest neighbor algorithm may be used to compute approximate distances between different points (e.g., different data points, input vectors, etc.), and then unlabeled instances may be assigned to the category of their k-nearest neighbors. If an instance's neighboring items are anomalous, the instance can also be marked / classified as anomalous. Otherwise, the data point can be marked as non-anomaly. In some embodiments, the anomaly detection manager 314 can use unsupervised machine learning algorithms such as k-means, c-means, self-organizing maps, single-class support vector machines, expectation-maximization meta-algorithms, and adaptive resonance theory to classify / cluster the data in order to identify and mark the instance as indicating anomaly or not indicating anomaly. In either case, the anomaly detection manager 314 can store the currently marked data in data storage area 324.

[0076] In some embodiments, the model manager 316 may be configured to cause the processor 304 to train the above-mentioned combination. Figure 2 Any suitable machine learning model discussed. Alternatively, model manager 316 can be configured to cause processor 304 to fit a statistical model to (receive, enrich, transform, and aggregate) labeled time series data. For example, model manager 316 can generate ARIMA models, mixture models, stochastic process models, and combinations thereof as described above. Figure 2The recurrent neural network discussed herein. Each model can model labeled time-series data, and each model can provide an output indicating the probability of a future failure occurring within a certain time period, such as the next five minutes, as indicated by the time-series data instance. Model manager 316 can be configured to feed each labeled example of time-series data to various models to generate a corresponding model output. Model manager 316 can store these model outputs as part of the labeled time-series data in data storage area 324. Each of the models can be stored in data storage area 324 for later use.

[0077] In some embodiments, the combined model manager 318 may be configured to cause the processor 304 to generate, train, and maintain a combined model. The combined model can be trained using any suitable supervised machine learning algorithm and time-series data labeled and supplemented with various model outputs (referred to as the “derived dataset”). In some embodiments, the combined model manager 318 may enrich the derived dataset with any suitable supplementary data, such as multiple calls, emails, texts, complaints, etc., received within a certain time period (regarding timestamps associated with a given time-series data instance). The combined model manager 318 may use the derived dataset (and in some cases, supplementary data) as training data to train the combined model (using a supervised machine learning algorithm) to predict future system failures. The combined model may be stored in data storage 324 for later use.

[0078] In some embodiments, the fault detection engine 320 may be configured to cause the processor 304 to receive new input data (e.g., new time-series data from the data processing module 312). The new input data has been enriched, transformed, and / or aggregated by the data processing module 312. The fault detection engine 320 may feed the new input data to each model trained by the model manager 316 to produce multiple outputs. The fault detection engine 320 may combine the outputs with the new input data and provide the combination to the combined model trained by the combined model manager 318. In some embodiments, the fault detection engine 320 may additionally retrieve values ​​from any suitable source corresponding to multiple emails, texts, calls, etc. received within a certain time period (e.g., the last 30 minutes). If the value is retrieved, it may also be provided as input to the combined model. The output of the combined model may be provided to the notification engine 322. In some embodiments, the fault detection engine 320 may mark the new input data as anomalous or non-anomalous, and may store the marked input data as new examples of training data in the data storage area 324. Therefore, the training dataset in data storage area 324 can grow, and each of the models and / or combined models can be updated and / or retrained over time to produce more accurate outputs.

[0079] In some embodiments, the notification engine 322 may be configured to cause the processor 304 to provide one or more notifications to one or more system components in response to receiving an indication (e.g., from the fault monitoring engine 320) that a fault has been predicted and / or detected (e.g., the combined model has output a value indicating anomalous behavior). The notification may include any suitable electronic communication. For example, the notification may be provided via email, text message, push notification, pop-up window, or any suitable interface hosted by the notification engine 322. In some embodiments, the notification may include corrective actions. The notification engine 322 may access a predefined list of corrective actions stored in the data storage area 324. Each corrective action may be associated with a fault type and / or system component. The notification engine 322 may use the predicted / detected fault type and the specific system component associated with the fault to determine the associated corrective action. The notification engine 322 may provide an indication that a corrective action should be performed to the affected system component (or an agent of the system component). In some embodiments, corrective actions may include, but are not limited to, restarting a server, switching to a backup server, switching to a backup communication channel, performing a virus scan, calling a technician, or powering off a computing device.

[0080] Technological advantages

[0081] The embodiments of the present invention offer several advantages. For example, using the techniques discussed above, faults can be detected from any suitable component of a distributed system by utilizing the response time between network messages as an indicator of faults. Furthermore, by employing a combination of various statistical and machine learning models, the prediction of future faults can be made more accurate, as different predictive techniques can be combined. Each of these models is updated and / or retrained over time, making the combined model discussed herein more accurate over time. By using these various models as input to the combined model, the system is more likely to detect future faults compared to a conventional system that only utilizes statistical or machine learning models.

[0082] Any computing device described herein may be an example of a computer system that can be used to implement any of the entities or components described above. Subsystems of such a computer system may be interconnected via a system bus. Additional subsystems include printers, keyboards, storage devices, and monitors coupled to a display adapter. Peripheral devices and input / output (I / O) devices are coupled to an I / O controller and may be connected to the computer system via any of the many means known in the art, such as a serial port. For example, I / O ports or external interfaces may be used to connect the computer device to a wide area network (e.g., the Internet), a mouse input device, or a scanner. Interconnection via a system bus allows the central processing unit to communicate with each subsystem and control the execution of instructions from system memory or storage devices, as well as the exchange of information between subsystems. System memory and / or storage devices may embody computer-readable media.

[0083] As described, the services of the present invention may involve implementing one or more functions, processes, operations, or method steps. In some embodiments, the functions, processes, operations, or method steps may be implemented as the result of executing an instruction set or software code by a suitably programmed computing device, microprocessor, data processor, etc. The instruction set or software code may be stored in memory or other forms of data storage elements accessed by the computing device, microprocessor, etc. In other embodiments, the functions, processes, operations, or method steps may be implemented by firmware or a dedicated processor, integrated circuit, etc.

[0084] Any software component or function described in this application may be implemented as software code executed by a processor using any suitable computer language, such as Java, C++, or Perl, using conventional or object-oriented technologies. The software code may be stored as a series of instructions or commands on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), magnetic media such as a hard disk drive or floppy disk, or optical media such as a CD-ROM. Any such computer-readable medium may reside on or within a single computing device, and may exist on different computing devices or within a system or network.

[0085] Although certain exemplary embodiments have been described in detail and shown in the accompanying drawings, it should be understood that such embodiments are merely illustrative of the invention and not limiting, and that the invention is not limited to the specific arrangements and constructions shown and described, as various other modifications will be apparent to those skilled in the art.

[0086] As used herein, unless explicitly indicated otherwise, the use of “a / an” or “the” is intended to mean “at least one”.

Claims

1. A computer-implemented method, comprising: The fault detection platform obtains time-series data corresponding to historical network messages. The time-series data includes related time-series data instances, which include a first instance of time-series data and a second instance of time-series data. The related time-series data instances are associated at least in part with related network messages including the historical network messages. The fault detection platform identifies multiple anomalies in the time series data based at least in part on the time periods that have elapsed between the relevant time series data instances. The training dataset is generated by the fault detection platform by associating the relevant time series data instances corresponding to one or more of the plurality of anomalies with corresponding classification labels indicating network faults. The fault detection platform uses the relevant time series data instances to train the first machine learning model among multiple machine learning models; The fault detection platform uses the training dataset and supervised machine learning algorithm to train the second machine learning model among the multiple machine learning models; The fault detection platform trains a third machine learning model based at least in part on the outputs of the plurality of machine learning models; The fault detection platform uses the third machine learning model to predict future network faults; as well as The fault detection platform performs one or more corrective actions in response to the prediction of the future network fault.

2. The computer-implemented method according to claim 1, wherein the historical network messages of the time series data correspond to a distributed system comprising multiple computing devices.

3. The computer-implemented method according to claim 1, wherein the time-series data includes response times corresponding to the historical network messages.

4. The computer-implemented method according to claim 1, wherein the historical network message of the time series data includes an authorization request message and a corresponding authorization response message.

5. The computer-implemented method according to claim 1, wherein identifying an exception among the plurality of exceptions includes determining a time period between authorization request messages during which no corresponding authorization response message was received.

6. The computer-implemented method of claim 1 further includes transforming the time series data before training the first machine learning model.

7. The computer-implemented method according to claim 1, wherein at least one of the first machine learning model or the second machine learning model is a recurrent neural network.

8. The computer-implemented method of claim 1, wherein the first machine learning model is an autoregressive integral moving average (ARIMA) model fitted to the time series data.

9. The computer-implemented method of claim 1, wherein the first machine learning model is trained using a stochastic process with Markov properties.

10. The computer-implemented method of claim 1, wherein identifying the plurality of anomalies utilizes an unsupervised learning technique using the time series data.

11. A computing device for a fault detection platform, the computing device comprising: processor; as well as A memory, including a computer-readable storage medium storing computer-executable instructions that, when executed by the processor, cause the computing device to perform operations, including: Obtain time-series data corresponding to historical network messages in a distributed network. The time-series data includes related time-series data instances, which include a first instance of time-series data and a second instance of time-series data. The related time-series data instances are associated at least in part with related network messages including the historical network messages. Multiple anomalies in the time series data are identified, at least in part, based on the time intervals that have elapsed between the relevant time series data instances. A training dataset is generated by associating the relevant time-series data instances corresponding to one or more of the plurality of anomalies with corresponding classification labels indicating network failures. The first machine learning model among multiple machine learning models is trained using the relevant time series data instances. The second machine learning model among the plurality of machine learning models is trained using the training dataset and the supervised machine learning algorithm; The third machine learning model is trained at least in part based on the outputs of the plurality of machine learning models; Predicting future system failures using the third machine learning model; and One or more corrective actions are performed in response to the prediction of a future system failure.

12. The computing apparatus of claim 11, wherein the plurality of machine learning models comprises a first set of machine learning models trained using at least one time series forecasting method, a second set of machine learning models trained using at least one random distribution of the time series data, a third set of machine learning models trained using at least one Markov process, and a fourth set of machine learning models comprising one or more recurrent neural networks.

13. The computing device of claim 11, wherein executing the computer-executable instructions further causes the computing device to perform additional operations, the additional operations including transforming the time series data into a fixed dataset, wherein the time series data is transformed using a difference method.

14. The computing device of claim 11, wherein the historical network messages of the time series data include a plurality of authorization request messages.

15. The computing device of claim 11, wherein the time-series data includes a first value and a second value, the first value indicating a first time when an authorization request message is sent from a first computing component to a second computing component, and the second value indicating a second time when an authorization response message corresponding to the authorization request message is received, or indicating that the authorization response message corresponding to the authorization request message has not been received.

16. The computing apparatus of claim 11, wherein at least one output of the plurality of machine learning models includes a probability value indicating that input data provided to the third machine learning model predicts a failure of at least one computing component of the distributed network.

17. The computing apparatus of claim 11, wherein at least one output of the plurality of machine learning models includes additional classification labels.

18. The computing device of claim 11, wherein the one or more corrective actions include sending a notification to at least one computing component of the distributed network, the notification including information indicating a fault in the at least one computing component.

19. The computing device of claim 11, wherein the one or more corrective actions include requesting at least one computing component of the distributed network to perform a system reset.

20. The computing device of claim 11, wherein executing the computer-executable instructions further causes the computing device to perform additional operations, the additional operations including aggregating and transforming the time series data using a window function.

Images