A network node equipment operation failure early warning system based on big data

Abstract

The present invention discloses a network node equipment operation fault early warning system based on big data, which relates to the field of network information technology. The system comprises: a node log acquisition module, a key information extraction module, a fault prediction module and an early warning module. The present invention mines characteristic information associated with faults by performing big data analysis on the operation logs of each node in the network, and analyzes the characteristic information in real time, predicts potential faults according to their trends, effectively discovers potential risks in advance, and safeguards the stability of network operation.

Description

Technical Field

[0001] The present invention relates to the field of network information technology, and in particular to a network node equipment operation failure early warning system based on big data. Background Art

[0002] The stable operation of network node equipment is crucial to ensuring the quality of network services. With the development of Internet technology, the complexity of business continues to increase, the network topology is becoming more and more complex, and the information exchange between nodes is becoming more frequent. In the network, if any node has an abnormal indicator, it is possible that the abnormality will continue to accumulate and expand during the link propagation process, causing the entire network to paralyze.

[0003] However, traditional network monitoring systems mainly monitor key indicators within nodes and can only respond after a failure occurs, lacking an effective prediction mechanism to detect potential risks in advance. Therefore, there is an urgent need for an early warning system that can analyze a large amount of network data in real time and accurately predict possible failures of network node devices. Summary of the invention

[0004] In response to the above-mentioned deficiencies in the prior art, the present invention provides a network node equipment operation fault warning system based on big data, which solves the problem that the prior art can usually only judge and respond after a node in the network fails, and lacks an effective prediction mechanism to detect potential risks in advance.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A network node equipment operation failure early warning system based on big data, comprising:

[0007] The node log acquisition module is used to read the operation logs of each node in the network and form the original big data log set;

[0008] The key information extraction module is used to obtain the characteristic information associated with the fault in the operation log of each node from the original big data log set to form a characteristic information set;

[0009] The fault prediction module is used to predict whether each node has a potential fault based on the feature information set;

[0010] The early warning module is used to issue early warnings for nodes with potential failures.

[0011] The beneficial effects of the present invention are as follows: the present invention conducts big data analysis on the operation logs of each node in the network, mines characteristic information associated with faults, and analyzes the characteristic information in real time, predicts potential faults according to their trends, effectively discovers potential risks in advance, and safeguards the stability of network operation.

[0012] Furthermore, the operation log read by the node log acquisition module includes at least the following information items:

[0013] Communication response time of nodes;

[0014] CPU usage of the node;

[0015] The CPU temperature of the node;

[0016] Memory usage of the node;

[0017] Real-time power consumption of the node.

[0018] The beneficial effect of the above further scheme is that the above information items are key information items for measuring the operating status of network node devices. If a node fails or is about to fail, the above information items of the node device usually change in advance.

[0019] Furthermore, the key information extraction module stores a plurality of fault-related feature items, so as to extract feature information associated with the fault from each node operation log of the original big data log set according to each fault-related feature item.

[0020] Furthermore, during network operation, the number of faulty nodes is periodically counted, and when each N faulty nodes are recorded, the faulty node count is cleared, and each fault-related feature item stored in the key information extraction module is updated through machine learning; N is a positive integer.

[0021] Furthermore, the method for the key information extraction module to update each stored fault-related feature item through machine learning includes the following steps:

[0022] A1. Obtain the operation log of each node in the network including N faulty nodes in the current counting cycle of faulty nodes;

[0023] A2. Calculate the correlation between each information item in the operation log and the fault event based on the data obtained in A1;

[0024] A3. Each information item with a correlation greater than the first threshold is taken as a new fault correlation feature item and stored in the key information extraction module.

[0025] The beneficial effect of the above further scheme is that different networks usually have different characteristics when implementing different functions, and the sensitive information associated with the fault is usually different. The present invention can periodically and adaptively track the fault-related feature items of the operation log, and extract the feature information associated with the fault from each node operation log of the original big data log set, which not only improves the accuracy, but also avoids redundant information from participating in the judgment of fault prediction, thereby avoiding interference and glitches.

[0026] Furthermore, A2 calculates the correlation between each information item of the operation log and the fault event by the following formula:

[0027]

[0028] Among them, i is the serial number of the information item, j is the serial number of the node, and r i is the correlation between the i-th information item and the fault event, x i,j is the value of the i-th information item in the running log of the j-th node, is the average value of the i-th information item in the running log of each node, y j is the fault value of the jth node. If the jth node fails, then y j is 1, if the jth node is not faulty, then y j is 0, is the average value of the fault value of each node, and M is the total number of nodes to which the data obtained by A1 belongs.

[0029] The beneficial effect of the above further solution is: through the above formula, based on the big data sample, the correlation between each information item of the operation log and the fault event is quantified. The design of this formula also completes the normalization of the correlation by itself, and its value range is in the closed interval [0,1]. The larger the value, the higher the correlation.

[0030] Furthermore, the fault prediction module uses a trend prediction neural network whose number is equal to the number of nodes in the network to predict whether each node has a potential fault based on the feature information of each node in the feature information set;

[0031] Each of the trend prediction neural networks comprises:

[0032] The multi-scale recursive layer is used to perform multi-scale time recursive operations on the feature information of each node in the feature information set to obtain the time domain trend information vector of each node;

[0033] The fully connected layer is used to calculate the time domain trend feature value of each node based on the time domain trend information vector of each node;

[0034] The logic judgment layer is used to perform logic judgment on the time domain trend characteristic value of each node and predict whether each node has potential faults.

[0035] Furthermore, the multi-scale recursive layer includes K recursive operation units, where K is the number of fault-related feature items stored in the key information extraction module; the expression of the nth recursive operation unit is:

[0036]

[0037] Among them, c n(t) is the nth element value of the time domain trend information vector at time t, σ n is the nth scale update coefficient, α n is the harmonic coefficient of the nth scale, c n (t-1) is the nth element value of the time domain trend information vector at time t-1, s n (t) is the value of the nth information item in the feature information of the multi-scale recursive layer at time t, s n (t-1) is the value of the nth information item in the feature information input to the multi-scale recursive layer at time t-1.

[0038] Furthermore, the expression of the fully connected layer is:

[0039]

[0040] Among them, f(t) is the output data of the fully connected layer at time t, w(n) is the nth weight coefficient, and b is the bias coefficient;

[0041] The logic judgment layer outputs 1 when f(t) is greater than the second threshold, indicating that there is a potential fault; when f(t) is less than or equal to the second threshold, the logic judgment layer outputs 0, indicating that there is no potential fault.

[0042] The beneficial effects of the above further scheme are as follows: different update coefficients and harmonization coefficients are set for each associated feature item in the feature information, and the time domain change law of the feature information is identified and analyzed at multiple scales. After weighted processing by the fully connected layer, the time domain trend feature value of the node is formed, and the potential fault risk is predicted through logical judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A structural diagram of a network node device operation failure early warning system based on big data provided by an embodiment of the present invention;

[0044] Figure 2 A structural diagram of a trend prediction neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0046] like Figure 1 As shown, in one embodiment of the present invention, a network node device operation failure early warning system based on big data includes:

[0047] The node log acquisition module is used to read the operation logs of each node in the network and form the original big data log set;

[0048] The key information extraction module is used to obtain the characteristic information associated with the fault in the operation log of each node from the original big data log set to form a characteristic information set;

[0049] The fault prediction module is used to predict whether each node has a potential fault based on the feature information set;

[0050] The early warning module is used to issue early warnings for nodes with potential failures.

[0051] In this embodiment, the operation log read by the node log acquisition module includes at least the following information items:

[0052] Communication response time of nodes;

[0053] CPU usage of the node;

[0054] The CPU temperature of the node;

[0055] Memory usage of the node;

[0056] Real-time power consumption of the node.

[0057] The above-mentioned information items are key information items for measuring the operating status of network node devices. If a node fails or is about to fail, the above-mentioned information items of the node device usually change in advance.

[0058] The key information extraction module stores multiple fault-related feature items, so as to extract feature information associated with the fault from each node operation log of the original big data log set according to each fault-related feature item.

[0059] During network operation, this embodiment periodically counts the number of faulty nodes, clears the faulty node count every time N faulty nodes are recorded, and updates each fault-related feature item stored in the key information extraction module through machine learning; N is a positive integer.

[0060] The method for updating each stored fault-related feature item by the key information extraction module through machine learning includes the following steps:

[0061] A1. Obtain the operation log of each node in the network including N faulty nodes in the current counting cycle of faulty nodes;

[0062] A2. Calculate the correlation between each information item in the operation log and the fault event based on the data obtained in A1;

[0063] A3. Each information item with a correlation greater than the first threshold is taken as a new fault correlation feature item and stored in the key information extraction module.

[0064] Different networks usually have different characteristics when implementing different functions, and the sensitive information associated with faults is usually different. The present invention can periodically and adaptively track the fault-related feature items of the operation log, and extract the feature information associated with the fault from each node operation log of the original big data log set, which not only improves the accuracy, but also avoids redundant information from participating in the judgment of fault prediction, thereby avoiding interference and glitches.

[0065] Step A2 calculates the correlation between each information item in the operation log and the fault event by the following formula:

[0066]

[0067] Among them, i is the serial number of the information item, j is the serial number of the node, and r i is the correlation between the i-th information item and the fault event, x i,j is the value of the i-th information item in the running log of the j-th node, is the average value of the i-th information item in the running log of each node, y j is the fault value of the jth node. If the jth node fails, then y j is 1, if the jth node is not faulty, then y j is 0, is the average value of the fault value of each node, and M is the total number of nodes to which the data obtained by A1 belongs.

[0068] Through the above formula, based on the big data sample, the correlation between each information item in the operation log and the fault event is quantified. The design of this formula also completes the normalization of the correlation. Its value range is in the closed interval [0,1]. The larger the value, the higher the correlation.

[0069] The fault prediction module uses a trend prediction neural network whose number is equal to the number of nodes in the network, and predicts whether each node has a potential fault based on the feature information of each node in the feature information set;

[0070] like Figure 2 As shown, each trend prediction neural network includes:

[0071] The multi-scale recursive layer is used to perform multi-scale time recursive operations on the feature information of each node in the feature information set to obtain the time domain trend information vector of each node;

[0072] The fully connected layer is used to calculate the time domain trend feature value of each node based on the time domain trend information vector of each node;

[0073] The logic judgment layer is used to perform logic judgment on the time domain trend characteristic value of each node and predict whether each node has potential faults.

[0074] The multi-scale recursive layer includes K recursive operation units, where K is the number of fault-related feature items stored in the key information extraction module; the expression of the nth recursive operation unit is:

[0075]

[0076] Among them, c n (t) is the nth element value of the time domain trend information vector at time t, σ n is the nth scale update coefficient, α n is the harmonic coefficient of the nth scale, c n (t-1) is the nth element value of the time domain trend information vector at time t-1, s n (t) is the value of the nth information item in the feature information of the multi-scale recursive layer at time t, s n (t-1) is the value of the nth information item in the feature information input to the multi-scale recursive layer at time t-1.

[0077] The expression of the fully connected layer is:

[0078]

[0079] Among them, f(t) is the output data of the fully connected layer at time t, w(n) is the nth weighting coefficient, and b is the bias coefficient.

[0080] The logic judgment layer outputs 1 when f(t) is greater than the second threshold, indicating that there is a potential fault; when f(t) is less than or equal to the second threshold, it outputs 0, indicating that there is no potential fault.

[0081] The trend prediction neural network of this embodiment sets different update coefficients and harmonic coefficients for each associated feature item in the feature information, performs multi-scale recognition and analysis on the time domain variation law of the feature information, forms the time domain trend feature value of the node after weighted processing by the fully connected layer, and predicts potential fault risks through logical judgment.

[0082] In summary, the present invention performs big data analysis on the operation logs of each node in the network, mines feature information associated with faults, and analyzes the feature information in real time, predicts potential faults according to their trends, effectively discovers potential risks in advance, and safeguards the stability of network operation.

[0083] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0084] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A network node equipment operation failure early warning system based on big data, characterized in that: include: The node log acquisition module is used to read the operation logs of each node in the network and form the original big data log set; The key information extraction module is used to obtain the characteristic information associated with the fault in the operation log of each node from the original big data log set. Composition feature information set ； A fault prediction module is used to use a trend prediction neural network whose number is equal to the number of nodes in the network to predict whether each node has a potential fault according to the feature information of each node in the feature information set; The early warning module is used to issue early warnings for nodes with potential failures; Wherein, each of the trend prediction neural networks comprises: The multi-scale recursive layer is used to perform multi-scale time recursive operations on the feature information of each node in the feature information set to obtain the time domain trend information vector of each node; The fully connected layer is used to calculate the time domain trend feature value of each node based on the time domain trend information vector of each node; The logic judgment layer is used to perform logic judgment on the time domain trend characteristic value of each node to predict whether there is a potential fault in each node; The multi-scale recursive layer includes K recursive operation units, where K is the number of fault-related feature items stored in the key information extraction module; the expression of the nth recursive operation unit is: c n (t)=σ n ·[a n ·s n (t)+(1-a n )·s n (t-1)|+(1-σ n )·c n (t-1) Among them, c n (t) is the nth element value of the time domain trend information vector at time t, σ n is the nth scale update coefficient, α n is the harmonic coefficient of the nth scale, c n (t-1) is the nth element value of the time domain trend information vector at time t-1, s n (t) is the value of the nth information item in the feature information of the multi-scale recursive layer at time t, s n (t-1) is the value of the nth information item in the feature information input to the multi-scale recursive layer at time t-1.

2. According to the big data-based network node equipment operation failure early warning system of claim 1, it is characterized in that: The operation log read by the node log acquisition module includes at least the following information items: Communication response time of nodes; CPU usage of the node; The CPU temperature of the node; Memory usage of the node; Real-time power consumption of the node.

3. According to the big data-based network node equipment operation failure early warning system of claim 1, it is characterized in that: The key information extraction module stores a plurality of fault-related feature items, so as to extract feature information associated with the fault from each node operation log of the original big data log set according to each fault-related feature item.

4. The network node equipment operation failure early warning system based on big data according to claim 3 is characterized in that: During network operation, the number of faulty nodes is periodically counted. Every time N faulty nodes are recorded, the faulty node count is reset, and each fault-related feature item stored in the key information extraction module is updated through machine learning; N is a positive integer.

5. The network node equipment operation failure early warning system based on big data according to claim 4 is characterized in that: The method in which the key information extraction module updates each stored fault-related feature item through machine learning comprises the following steps: A1. Obtain the operation log of each node in the network including N faulty nodes in the current counting cycle of faulty nodes; A2. Calculate the correlation between each information item in the operation log and the fault event based on the data obtained in A1; A3. Each information item with a correlation greater than the first threshold is taken as a new fault correlation feature item and stored in the key information extraction module.

6. The network node equipment operation failure early warning system based on big data according to claim 5 is characterized in that: A2 calculates the correlation between each information item of the operation log and the fault event by the following formula: Among them, i is the serial number of the information item, j is the serial number of the node, and r i is the correlation between the i-th information item and the fault event, x i,j is the value of the i-th information item in the running log of the j-th node, is the average value of the i-th information item in the running log of each node, y j is the fault value of the jth node. If the jth node fails, then y j is 1, if the jth node is not faulty, then y j is 0, is the average value of the fault value of each node, and M is the total number of nodes to which the data obtained by A1 belongs.

7. The network node equipment operation failure early warning system based on big data according to claim 1 is characterized in that: The expression of the fully connected layer is: Among them, f(t) is the output data of the fully connected layer at time t, w(n) is the nth weight coefficient, and b is the bias coefficient; The logic judgment layer outputs 1 when f(t) is greater than the second threshold, indicating that there is a potential fault; when f(t) is less than or equal to the second threshold, the logic judgment layer outputs 0, indicating that there is no potential fault.

Images