A cloud-edge system fault prediction method based on neural Hawkes process

Abstract

The present invention discloses a cloud-edge system fault prediction method based on a neural Hawkes process. The fault time and fault type are represented by vectors respectively, and then the initial representation of the fault is obtained by vector summation. The initial representation of the fault is processed based on a graph neural network, and the mutual influence relationship between the fault types is attached to the fault vector to obtain the final fault vector representation. A fault prediction model is established based on the Hawkes process, and the model parameters are optimized through a data-driven method. Finally, the fault vector representation output by the graph neural network is predicted based on the Hawkes fault prediction model. The present invention adopts the above method to accurately model the mutual influence relationship between fault types, making the fault representation richer and more accurate, and making fault prediction in future cloud-edge systems more accurate.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a cloud-edge system fault prediction method based on a neural Hawkes process. Background Art

[0002] The cloud-edge-end system is a new type of cloud computing system consisting of three components: the "cloud," the "edge," and the "end." The "cloud" is the central node in cloud computing, the "edge" is the edge of cloud computing, and the "end" is a terminal device such as a mobile phone or camera. In a cloud-edge-end system, numerous devices and complex network topologies can lead to various types of failures during system operation. These failures are often interconnected; one type of failure occurring at one moment can trigger another type of failure at a later moment. For example, an access error to a device's memory can cause a process to terminate. To ensure the normal operation of the system, it is necessary to predict and correct potential failures based on historical failures.

[0003] The research on fault detection in systems is quite extensive, and the common method is to use the Hawkes process to solve it, such as the literature - Hawkes process research for Granger causality discovery between faults. The Hawkes process is used to establish the incentive function of historical events to future events, the likelihood function is used to model the relationship between faults, and the optimization methods such as EM or hill climbing are used to optimize the model parameters. This method has the following two problems: (1) It ignores the characteristics between faults, which makes it impossible to correctly model the relationship between faults; (2) It is based on the assumption that faults depend on a specific distribution. However, in actual cloud-edge systems, faults often do not obey a specific distribution, which makes the existing methods often have deviations in fault prediction. Summary of the Invention

[0004] To address the above problems, the present invention aims to propose a data-driven neural Hawkes process for fault detection. This method is based on the characteristics of the faults and learns the associations between faults based on a large amount of historical data, thereby providing more accurate and efficient fault prediction.

[0005] A cloud-edge system fault prediction method based on the neural Hawkes process represents the fault time and fault type as vectors, and then obtains the initial representation of the fault by vector summation; the initial representation of the fault is processed based on the graph neural network, and the mutual influence relationship between the fault types is attached to the fault vector to obtain the final fault vector representation; a fault prediction model is established based on the Hawkes process, and the model parameters are optimized through a data-driven method; the fault vector representation output by the graph neural network is predicted based on the fault prediction model.

[0006] Furthermore, time coding technology is used to represent the fault time.i ,t i ), whose time encoding is a d-dimensional vector, where the value of the j-th dimension is

[0007]

[0008] Where a i Indicates the fault type, t i Indicates the failure time; ω j It is a parameter used to convert the timestamp into phase.

[0009] Furthermore, the graph neural network is obtained by learning historical fault sequences, which are a set of initial representations of faults;

[0010] H={h1,h2,…,h i ,…,h n}

[0011] Where H represents the historical fault sequence; h i represents the initial representation of the i-th fault;

[0012] With faults as nodes and relationships between faults as edges, a fully connected graph is established as the input of the graph neural network. l is used to represent the current layer of the graph neural network, and the node i in the l+1th layer network is represented by Through the neighboring nodes of the l-th layer network node i Indicates calculated;

[0013]

[0014]

[0015]

[0016] Where N i represents the neighbor set of node i; σ represents the sigmoid activation function; || represents the splicing operation; a and W are the parameters of the graph neural network; represents the representation of the network node i at layer l;

[0017] In order to enable the attention mechanism to obtain stable node representations, a multi-head attention mechanism is used to output representations of multiple nodes in parallel. The final output of each layer is the concatenation of the outputs of each attention head.

[0018]

[0019] Where Y represents the number of heads in the multi-head attention; W y Indicates the parameters corresponding to the y-th multi-head attention; splicing result That is, the fault vector representation of the final output of the graph neural network.

[0020] Furthermore, a fault prediction model is established based on the Hawkes process, specifically: at the future time t, the conditional intensity λ of a certain type of fault k (t) is defined as:

[0021]

[0022] Where μ represents a hyperparameter, which is related to the overall safety level of the system; S represents the fault sequence of historical fault events, and each fault representation is obtained by the graph neural network; a j Indicates the fault type, t j represents the failure time; α k,j Indicates historical fault events h j Related to the possible failure events The distance between them; k represents the kernel function of the time decay effect;

[0023] The conditional intensity functions of all K types of faults are summed to obtain the conditional intensity functions of all types of faults.

[0024]

[0025] Convert the conditional intensity function of the failure at the future time t into a probability;

[0026]

[0027] Where, Indicates the time t from the last fault occurrence n The probability that no failure occurs at future time t;

[0028] For the future failure time t ′ n+1 , calculated by the following formula.

[0029]

[0030] The type of fault predicted to occur is the fault type with the greatest condition intensity among all fault types;

[0031]

[0032] Finally, the possible failure at the future moment is obtained as (t ′ n+1 ,k ′ n+1 ).

[0033] Furthermore, the model parameters are optimized through a data-driven approach, specifically using the ADAM stochastic gradient descent optimization method.

[0034] The present invention adopts the above-mentioned cloud-edge system fault prediction method, which has the following advantages: First, time coding is used to represent timestamps, and the representation of fault types is integrated to make the fault representation richer and more accurate; second, graph neural networks are used to model the connections between faults, which can accurately model the mutual influence relationship between complex and changeable fault types in cloud-edge systems; third, the use of neural Hawkes process for prediction can quickly and accurately determine the parameters of the model through data-driven methods, which facilitates the optimization of model parameters and makes the fault prediction results in future cloud-edge systems more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a structural diagram of the cloud-edge system fault prediction model of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0037] This technical solution consists of four main parts: the first is to initialize the fault representation; the second is to learn the relationship between faults through graph neural networks to obtain a vector representation of the fault; the third is to use the Hawkes process to predict the type and time of fault occurrence; and the fourth is to optimize the model parameters.

[0038] 1. Initialize the fault indication.

[0039] The fault sequence S = {(a1, t1), (a2, t2), …, (a i ,t i ),…,(a n ,t n )},a i represents the i-th fault type, t i Indicates the timestamp of the occurrence of the i-th fault. All historical fault types constitute a set L = {label1, label2,…, label K}, K represents the total number of faults.

[0040] Each type of fault has an initial feature vector e i Collect the log data of the system equipment when this type of fault occurs, including the device number associated with the fault, the throughput of the network equipment when the fault occurs, the CPU utilization of the associated equipment, the memory utilization, etc., and compress it into a vector of fixed length p. Assume that the feature is a p-dimensional vector, that is, e i =[x1,x2,…xp ], where each dimension is the detection value of the corresponding indicator (floating point number). Use a linear transformation to compress it into a p-dimensional vector (d <p)

[0041] e′i=Wiei+bi

[0042] Where, is the parameter matrix, is the parameter vector.

[0043] After setting the initial feature vector representing the fault type, since time information is crucial for fault prediction, this solution uses time coding technology to represent time. i ,t i ), whose time encoding is a d-dimensional vector, where the value of the j-th dimension is

[0044]

[0045] In the formula, ω j 、w j is the conversion parameter used to convert the timestamp into phase;

[0046] Finally, the failure event (a i ,t i ) can be expressed as the sum of fault code and time code.

[0047]

[0048] Where h i Indicates the failure time*a i ,t i ), where the sum of the two vectors is the sum in each dimension, i.e. h i With e i ′ 、 are both d-dimensional vectors.

[0049] 2. Learn the connections between faults through graph neural networks.

[0050] After obtaining the initial vector representation of the fault sequence H={h1,h2,…h n After that, graph neural networks are used to learn and describe the connections between faults. The connections between fault events are represented using a graph G = (V, E). V is a set of nodes, each representing a fault. E is a set of edges, representing the connections between faults. Assuming that each fault has a causal relationship with all other faults, graph G is a fully connected graph.

[0051] Use l to represent the current layer of the graph neural network, and the representation of node i in the l+1 layer network The representation of node i in the lth layer network and its neighboring nodes j∈N i The expression is calculated

[0052]

[0053]

[0054]

[0055] Where N i represents the neighbor set of node i, σ is the sigmoid activation function; || represents the splicing operation; a and W are the parameters of the graph neural network.

[0056] In order to enable the attention mechanism to obtain a stable representation of the nodes, the multi-head attention mechanism is used to output the representation of multiple nodes in parallel. Finally, the output of each layer is the result of splicing the outputs of each attention head.

[0057]

[0058] Where Y is the number of heads in the multi-head attention, W y is the parameter corresponding to the y-th multi-head attention.

[0059] The graph G represents the association between faults, and the nodes output by the graph neural network are represented as the representation of each historical fault, that is, the final The fault representations obtained through the graph attention network can show the relationship between different faults. For example, a memory leak on a cloud node at time t1 has a high similarity with a subsequent task failure on that cloud node at time t2, but has a low similarity with a task failure on another cloud node at time t3.

[0060] 3. Use Hawkes process to predict the type and time of fault occurrence.

[0061] After obtaining the representation of each fault, for each type of fault that may occur in the cloud-edge system at a certain time t in the future, the time code at time t and the fault representation of each fault type are added to obtain the representation of each possible fault at the future time t. Where K is the number of fault types. k The intensity of the condition λ k (t) is defined as:

[0062]

[0063] Where μ represents a hyperparameter, which is set to 0 here because the overall safety level of the system is unknown; S represents the failure sequence of historical failure events; α k,j Indicates historical fault events h j Related to the possible failure events The distance between them; κ represents the kernel function of the time decay effect.

[0064] Among them, α k,j It is calculated by cosine similarity and κ is calculated by exponential function.

[0065]

[0066]

[0067] Where β is a hyperparameter indicating the decay speed, which is set to 0.1 here.

[0068] The conditional intensity functions for all types of faults are summed up to obtain the conditional intensity functions for all types of faults.

[0069]

[0070] Next, the conditional intensity function of a failure occurring at future time t is converted into a probability.

[0071]

[0072] Where, Indicates the time t from the last fault occurrence n The probability that no failure occurs at future time t.

[0073] For the future failure time t ′ n+1 , calculated by the following formula.

[0074]

[0075] The type of fault predicted to occur is the fault type with the greatest condition intensity among all fault types.

[0076]

[0077] 4. Optimization of model parameters.

[0078] The model is trained by maximum likelihood estimation. L ], given the conditional intensity function λ(t), the logarithmic likelihood is

[0079]

[0080] Where n is the number of all faults in the sequence.

[0081] The sequence S is obtained by resampling all the data. Assuming that M sequences are sampled, the total likelihood is the sum of the likelihoods calculated on all sequences. The objective function of training can be obtained by maximizing the total likelihood.

[0082]

[0083] According to the above objective function, appropriate optimization methods can be used to optimize the parameters. Here, the optimization method uses the ADAM stochastic gradient descent optimization method, so the data does not need to follow the pre-assumed distribution. After training and making the model converge, the model can be used to monitor and predict system faults. For example, the system predicts the future time t p When a task on cloud node A fails, the system can move the task to other cloud nodes or edge nodes in advance to reduce the risk of task failure in the system.

[0084] The above are specific embodiments of the present invention, but the scope of protection of the present invention should not be limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed by the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection defined in the claims.

Claims

1. A cloud-edge system fault prediction method based on neural Hawkes process, characterized by: include: The fault time and fault type are represented by vectors, and then the initial representation of the fault is obtained by vector summation; The initial representation of the fault is processed based on a graph neural network, and the mutual influence relationship between fault types is added to the fault vector to obtain the final fault vector representation; Establish a fault prediction model based on the Hawkes process and optimize the model parameters through data-driven methods; Predict the fault vector representation output by the graph neural network based on the fault prediction model; The graph neural network is obtained by learning historical fault sequences, which are the set of initial representations of faults. ； Where, Represents a historical fault sequence; Indicates the Initial indication of a fault; With faults as nodes and relationships between faults as edges, a fully connected graph is established as the input of the graph neural network; Represents the current layer of the graph neural network, Nodes in a layer network Representation Through the Layer network nodes Neighboring nodes Indicates calculated; ； ； ； Where, Representation node The set of neighbors of Represents the sigmoid activation function; Represents a splicing operation; are the parameters of the graph neural network; Indicates the Layer network nodes the expression; In order to enable the attention mechanism to obtain stable node representations, a multi-head attention mechanism is used to output representations of multiple nodes in parallel. The final output of each layer is the concatenation of the outputs of each attention head. ； Where, Indicates the number of heads in multi-head attention; Indicates the Parameters corresponding to multi-head attention; splicing results That is, the fault vector representation of the final output of the graph neural network; Optimize model parameters through data-driven methods, specifically using the ADAM stochastic gradient descent optimization method; The model is trained by maximum likelihood estimation; for Fault sequence on , given the conditional intensity function , the log-likelihood is ； Where n is the number of all faults in the sequence; sequence It is obtained by resampling all the data. Assuming that M sequences are sampled, the total likelihood is the sum of the likelihoods calculated on all sequences. Maximizing the total likelihood can obtain the objective function of training; ； According to the above objective function, appropriate optimization methods can be used to optimize the parameters. Here, the optimization method adopts the ADAM stochastic gradient descent optimization method, so the data does not need to obey the pre-assumed distribution. After training, the model converges and can be used to monitor and predict system faults.

2. The cloud-edge system fault prediction method according to claim 1 is characterized in that: Use time coding technology to represent the fault time. , whose time encoding is a d-dimensional vector, where The value of the dimension is ； Where, Representation node The type of fault, Representation node Failure time; It is the conversion parameter used to convert the timestamp into phase.

3. The cloud-edge system fault prediction method according to claim 2, characterized in that: A fault prediction model is established based on the Hawkes process, specifically: The intensity of the conditions for a certain type of failure to occur at future time t Defined as: ； Where, represents a hyperparameter that is related to the overall security of the system; Fault sequences representing historical fault events, where each fault representation is obtained using a graph neural network; Indicates the fault type. Indicates the fault time; Indicates historical fault events Related to the possible failure events the distance between them; Kernel function representing the time decay effect; The conditional intensity functions of all K types of faults are summed to obtain the conditional intensity functions of all types of faults. ； Convert the conditional intensity function of the failure at the future time t into a probability; ； Where, Indicates the time from the last fault occurrence To the future There is no probability of failure; Time to future failures , calculated by the following formula; ； The type of fault predicted to occur is the fault type with the greatest condition intensity among all fault types; ； Finally, the possible failures in the future are obtained as .

Images