Network fault monitoring method and system based on data monitoring

Abstract

The invention discloses a network fault monitoring method and system based on data monitoring, and relates to the field of data processing, the system comprises a monitoring center, and the monitoring center is in communication connection with a data acquisition module, a data preprocessing module, an anomaly detection module and a feedback module; network data are collected through the data collection module, the collected network data are preprocessed through the data preprocessing module, standard network data are obtained, anomaly detection is conducted on the obtained standard network data through the anomaly detection module, and whether abnormal standard network data sample points exist or not is judged. And the feedback module performs feedback according to a judgment result, if an abnormal standard network data sample point exists, the network data is abnormal and the network has a fault, otherwise, the network data is normal and the network does not have a fault, and whether the network has a fault is accurately judged, so that the real-time performance and the accuracy of network fault monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a network fault monitoring method and system based on data monitoring. Background Art

[0002] With the rapid development of the Internet, the scale and complexity of networks are increasing. Enterprises, factories and individuals all rely on the network for daily business and communication. Therefore, ensuring the reliability and performance of the network has become crucial. During the operation of the network, monitoring data is the basis for ensuring the stability and efficient operation of the network. By real-time monitoring and analyzing network data, network faults can be discovered and handled in a timely manner to ensure the continuity of network services and user experience. Traditional network fault monitoring methods rely on manual experience and are difficult to cover all possible fault conditions, affecting the accuracy of fault detection. How to detect anomalies in standard network data, determine whether there are abnormal standard network data sample points, and then determine whether there is a fault in the network to ensure the accuracy of fault detection is a problem we need to solve. To this end, a network fault monitoring method and system based on data monitoring are provided. Summary of the invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a network fault monitoring method and system based on data monitoring. The object of the present invention can be achieved by the following technical solutions: A network fault monitoring system based on data monitoring includes a monitoring center, wherein the monitoring center is communicatively connected with a data acquisition module, a data preprocessing module, an abnormality detection module and a feedback module; The data acquisition module is used to collect network data; The data preprocessing module is used to preprocess the collected network data to obtain standard network data; The anomaly detection module is used to perform anomaly detection on the obtained standard network data to determine whether there are abnormal standard network data sample points; The feedback module is used to provide feedback according to the judgment result.

[0004] Furthermore, the process of the data acquisition module collecting network data includes: The data acquisition module is composed of several data acquisition terminals. The data acquisition terminals are configured to generate corresponding configuration channels. The corresponding servers are linked through the generated configuration channels. The data acquisition terminals obtain network data in the server through the configuration channels and send the obtained network data to the data preprocessing module.

[0005] Furthermore, the process of preprocessing the collected network data by the data preprocessing module includes: The data preprocessing module is provided with a data scanning unit and a data preprocessing unit. The collected network data is scanned by the data scanning unit to obtain the status of the network data. The status of the network data includes a repeated status, a missing status and a normal status. If the status of the network data is a repeated status, the network data is sent to the data preprocessing unit, and the data preprocessing unit deletes the repeated network data. If the status of the network data is a missing status, the network data is sent to the data preprocessing unit, and the data preprocessing unit deletes the missing network data. If the status of the network data is a normal status, the network data is not processed. The preprocessing is completed, and the network data after the preprocessing is recorded as standard network data.

[0006] Furthermore, the process of the anomaly detection module performing anomaly detection on the obtained standard network data includes: Construct a two-dimensional coordinate system, quantify the obtained standard network data, and map the quantified results in the coordinate system to obtain corresponding standard network data sample points; Randomly select k standard network data sample points, k∈(a, b), k, a and b are all positive integers, and record the selected k standard network data sample points as the center points of the standard network data population. One center point of the standard network data population represents one population; Obtain the straight-line distance and three-line distance between each standard network data sample point and each standard network data population center point; According to the straight-line distance and three-line distance between each standard network data sample point and each standard network data population center point, the distribution distance between each standard network data sample point and each standard network data population center point is obtained; Compare the distribution distances between each standard network data sample point and each standard network data population center point, and distribute each standard network data sample point to the population where the standard network data population center point with the smallest distribution distance is located; Obtain the average values ​​of the horizontal coordinates and vertical coordinates of the standard network data sample points and the center point of the standard network data population in each population, and use the coordinates formed by the average values ​​of the horizontal coordinates and vertical coordinates of the standard network data sample points and the center point of the standard network data population as the new center point of the standard network data population; Obtain the distribution distance between the standard network data sample point and the new standard network data population center point, compare the distribution distance between each standard network data sample point and each new standard network data population center point, distribute each standard network data sample point to the population where the new standard network data population center point with the smallest distribution distance is located, obtain the average value of the horizontal coordinate and the vertical coordinate of the standard network data sample point and the new standard network data population center point in each population, and use the coordinates composed of the average value of the horizontal coordinate and the vertical coordinate of the standard network data sample point and the new standard network data population center point as the latest standard network data population center point, and so on, until the straight-line distance between the latest standard network data population center point and the previous standard network data population center point is less than or equal to R, where R is a positive number; The distribution distance between each standard network data sample point and other standard network data sample points in its population is obtained, and the distribution distance threshold is set according to the actual situation. If there is a standard network data sample point and the distribution distance between other standard network data sample points in its population is greater than the distribution distance threshold, then this standard network data sample point is recorded as an abnormal standard network data sample point. If there is no standard network data sample point and the distribution distance between other standard network data sample points in its population is greater than the distribution distance threshold, then there is no abnormal standard network data sample point.

[0007] Furthermore, the process of obtaining the straight-line distance between each standard network data sample point and each standard network data population center point includes: Randomly select a standard network data sample point and a standard network data population center point, record the selected standard network data sample point and the standard network data population center point as the first standard network data sample point and the first standard network data population center point, respectively, and obtain the coordinates of the first standard network data sample point and the first standard network data population center point; Obtaining a horizontal distance and a vertical distance between the first standard network data sample point and the first standard network data population center point according to the obtained coordinates of the first standard network data sample point and the first standard network data population center point; Obtaining a straight-line distance between the first standard network data sample point and the first standard network data population center point according to the obtained horizontal distance and vertical distance between the first standard network data sample point and the first standard network data population center point; In this way, the straight-line distances between the remaining standard network data sample points and the remaining standard network data population center points are obtained.

[0008] Furthermore, the process of obtaining the three-line distance between each standard network data sample point and each standard network data population center point includes: Obtaining the middle distance between the first standard network data sample point and the first standard network data population center point according to the longitudinal distance and the straight-line distance between the first standard network data sample point and the first standard network data population center point; According to the obtained intermediate distance, horizontal distance and vertical distance between the first standard network data sample point and the center point of the first standard network data population, the three-line distance between the first standard network data sample point and the center point of the first standard network data population is obtained.

[0009] Furthermore, the process of the feedback module providing feedback according to the judgment result includes: If there are abnormal standard network data sample points, the network data is abnormal, the network fails, and a red alarm is issued to remind relevant personnel to handle it; If there are no abnormal standard network data sample points, the network data is normal and the network is not faulty.

[0010] The present invention also discloses a monitoring method of a network fault monitoring system based on data monitoring, comprising the following steps: Step 1: Acquire network data, and pre-process the acquired network data to obtain standard network data; Step 2: Perform anomaly detection on the obtained standard network data to determine whether there are abnormal standard network data sample points; Step 3: Provide feedback based on the judgment results.

[0011] Compared with the prior art, the beneficial effects of the present invention are: collecting network data, preprocessing the collected network data, obtaining standard network data, performing anomaly detection on the obtained standard network data, judging whether there are abnormal standard network data sample points, and providing feedback based on the judgment results. If there are abnormal standard network data sample points, the network data is abnormal and the network fails. Otherwise, the network data is normal and the network does not fail. Whether the network fails is accurately judged, thereby improving the real-time and accuracy of network fault monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0013] like Figure 1 As shown, a network fault monitoring system based on data monitoring includes a monitoring center, and the monitoring center is communicatively connected with a data acquisition module, a data preprocessing module, an anomaly detection module and a feedback module; The data acquisition module is used to collect network data; It should be further explained that, in the specific implementation process, the process of the data acquisition module collecting network data includes: The data acquisition module is composed of several data acquisition terminals. The data acquisition terminals are configured to generate corresponding configuration channels. The corresponding servers are linked through the generated configuration channels. The data acquisition terminals obtain network data in the server through the configuration channels and send the obtained network data to the data preprocessing module.

[0014] The data preprocessing module is used to preprocess the collected network data to obtain standard network data; It should be further explained that, in a specific implementation, the process of the data preprocessing module preprocessing the collected network data includes: The data preprocessing module is provided with a data scanning unit and a data preprocessing unit. The collected network data is scanned by the data scanning unit to obtain the status of the network data. The status of the network data includes a repeated status, a missing status and a normal status. If the status of the network data is a repeated status, the network data is sent to the data preprocessing unit, and the data preprocessing unit deletes the repeated network data. If the status of the network data is a missing status, the network data is sent to the data preprocessing unit, and the data preprocessing unit deletes the missing network data. If the status of the network data is a normal status, the network data is not processed. The preprocessing is completed, and the network data after the preprocessing is recorded as standard network data.

[0015] The anomaly detection module is used to perform anomaly detection on the obtained standard network data to determine whether there are abnormal standard network data sample points; It should be further explained that, in a specific implementation process, the process of the anomaly detection module performing anomaly detection on the obtained standard network data includes: Construct a two-dimensional coordinate system, quantify the obtained standard network data, and map the quantified results in the coordinate system to obtain corresponding standard network data sample points; Randomly select k standard network data sample points, k∈(a, b), k, a and b are all positive integers, and record the selected k standard network data sample points as the center points of the standard network data population. One center point of the standard network data population represents one population; Obtain the straight-line distance between each standard network data sample point and each standard network data population center point; It should be further explained that, in the specific implementation process, the process of obtaining the straight-line distance between each standard network data sample point and each standard network data population center point includes: Randomly select a standard network data sample point and a standard network data population center point, record the selected standard network data sample point and the standard network data population center point as the first standard network data sample point and the first standard network data population center point, obtain the coordinates of the first standard network data sample point and the first standard network data population center point, and record the coordinates of the obtained first standard network data sample point and the first standard network data population center point as (x 1 ,y 1 ) and (x 2 ,y 2 ); Obtaining the horizontal distance between the first standard network data sample point and the center point of the first standard network data population, subtracting the horizontal coordinate of the first standard network data sample point from the horizontal coordinate of the center point of the first standard network data population, taking the absolute value of the subtraction result, obtaining the horizontal distance between the first standard network data sample point and the center point of the first standard network data population, and recording the obtained horizontal distance between the first standard network data sample point and the center point of the first standard network data population as H; in, ; Obtaining the longitudinal distance between the first standard network data sample point and the center point of the first standard network data population, subtracting the longitudinal coordinate of the first standard network data sample point from the longitudinal coordinate of the center point of the first standard network data population, taking the absolute value of the subtraction result, obtaining the longitudinal distance between the first standard network data sample point and the center point of the first standard network data population, and recording the obtained longitudinal distance between the first standard network data sample point and the center point of the first standard network data population as Z; in, ; According to the obtained horizontal distance and vertical distance between the first standard network data sample point and the center point of the first standard network data population, the straight-line distance between the first standard network data sample point and the center point of the first standard network data population is obtained, and the straight-line distance between the obtained first standard network data sample point and the center point of the first standard network data population is recorded as J; in, ; By analogy, the straight-line distances between the remaining standard network data sample points and the remaining standard network data population center points are obtained; Obtain the three-line distance between each standard network data sample point and each standard network data population center point; It should be further explained that, in the specific implementation process, the process of obtaining the three-line distance between each standard network data sample point and each standard network data population center point includes: Obtaining the middle distance between the first standard network data sample point and the center point of the first standard network data population, obtaining the middle distance between the first standard network data sample point and the center point of the first standard network data population according to the longitudinal distance and the straight-line distance between the first standard network data sample point and the center point of the first standard network data population, and recording the obtained middle distance between the first standard network data sample point and the center point of the first standard network data population as N; in, ; According to the obtained intermediate distance, horizontal distance and vertical distance between the first standard network data sample point and the center point of the first standard network data population, the three-line distance between the first standard network data sample point and the center point of the first standard network data population is obtained, and the three-line distance between the obtained first standard network data sample point and the center point of the first standard network data population is recorded as SX; in, ; By analogy, the three-line distances between the remaining standard network data sample points and the remaining standard network data population center points are obtained; Obtain the distribution distance between each standard network data sample point and each standard network data population center point; It should be further explained that, in the specific implementation process, the process of obtaining the distribution distance between each standard network data sample point and each standard network data population center point includes: According to the obtained three-line distance and straight-line distance between the first standard network data sample point and the center point of the first standard network data population, the distribution distance between the first standard network data sample point and the center point of the first standard network data population is obtained, and the distribution distance between the obtained first standard network data sample point and the center point of the first standard network data population is recorded as FP; in, ; By analogy, the distribution distances between the remaining standard network data sample points and the remaining standard network data population center points are obtained; Compare the distribution distances between each standard network data sample point and each standard network data population center point, and distribute each standard network data sample point to the population where the standard network data population center point with the smallest distribution distance is located; Calculate the average values ​​of the horizontal coordinates and vertical coordinates of the standard network data sample points and the center point of the standard network data population in each population, and use the coordinates formed by the average values ​​of the horizontal coordinates and vertical coordinates of the standard network data sample points and the center point of the standard network data population as the new center point of the standard network data population; Calculate the distribution distance between the standard network data sample point and the new standard network data population center point, compare the distribution distance between each standard network data sample point and each new standard network data population center point, distribute each standard network data sample point to the population where the new standard network data population center point with the smallest distribution distance is located, calculate the average value of the horizontal coordinate and the vertical coordinate of the standard network data sample point and the new standard network data population center point in each population, and use the coordinate composed of the average value of the horizontal coordinate and the vertical coordinate of the standard network data sample point and the new standard network data population center point as the latest standard network data population center point, and so on, and perform several rounds of repeated calculations until the straight-line distance between the latest standard network data population center point and the previous standard network data population center point is less than or equal to R, where R is a positive number; Calculate the distribution distance between each standard network data sample point and other standard network data sample points in its population, set the distribution distance threshold according to the actual situation, if there is a standard network data sample point and other standard network data sample points in its population whose distribution distance is greater than the distribution distance threshold, then this standard network data sample point is recorded as an abnormal standard network data sample point, if there is no standard network data sample point and other standard network data sample points in its population whose distribution distance is greater than the distribution distance threshold, then there is no abnormal standard network data sample point.

[0016] The feedback module is used to provide feedback according to the judgment result; It should be further explained that, in the specific implementation process, the process of the feedback module providing feedback according to the judgment result includes: If there are abnormal standard network data sample points, the network data is abnormal, the network fails, and a red alarm is issued to remind relevant personnel to handle it; If there are no abnormal standard network data sample points, the network data is normal and the network is not faulty.

[0017] The present invention also discloses a monitoring method of a network fault monitoring system based on data monitoring, comprising the following steps: Step 1: Acquire network data, and pre-process the acquired network data to obtain standard network data; Step 2: Perform anomaly detection on the obtained standard network data to determine whether there are abnormal standard network data sample points; Step 3: Provide feedback based on the judgment results.

[0018] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A network fault monitoring system based on data monitoring, including a monitoring center, characterized in that: The monitoring center is communicatively connected to a data acquisition module, a data preprocessing module, an anomaly detection module and a feedback module; The data acquisition module is used to collect network data; The data preprocessing module is used to preprocess the collected network data to obtain standard network data; The anomaly detection module is used to perform anomaly detection on the obtained standard network data to determine whether there are abnormal standard network data sample points; The feedback module is used to provide feedback according to the judgment result.

2. The network fault monitoring system based on data monitoring according to claim 1 is characterized in that: The process of collecting network data by the data collection module includes: The data acquisition module is composed of several data acquisition terminals. The data acquisition terminals are configured to generate corresponding configuration channels. The corresponding servers are linked through the generated configuration channels. The data acquisition terminals obtain network data in the server through the configuration channels and send the obtained network data to the data preprocessing module.

3. The network fault monitoring system based on data monitoring according to claim 2 is characterized in that: The process of preprocessing the collected network data by the data preprocessing module includes: The data preprocessing module is provided with a data scanning unit and a data preprocessing unit. The collected network data is scanned by the data scanning unit to obtain the status of the network data. The status of the network data includes a repeated status, a missing status and a normal status. If the status of the network data is a repeated status, the network data is sent to the data preprocessing unit, and the data preprocessing unit deletes the repeated network data. If the status of the network data is a missing status, the network data is sent to the data preprocessing unit, and the data preprocessing unit deletes the missing network data. If the status of the network data is a normal status, the network data is not processed. The preprocessing is completed, and the network data after the preprocessing is recorded as standard network data.

4. The network fault monitoring system based on data monitoring according to claim 3 is characterized in that: The process of the anomaly detection module performing anomaly detection on the obtained standard network data includes: Construct a two-dimensional coordinate system, quantify the obtained standard network data, and map the quantified results in the coordinate system to obtain corresponding standard network data sample points; Randomly select k standard network data sample points, k∈(a, b), k, a and b are all positive integers, and record the selected k standard network data sample points as the center points of the standard network data population. One center point of the standard network data population represents one population; Obtain the straight-line distance and three-line distance between each standard network data sample point and each standard network data population center point; According to the straight-line distance and three-line distance between each standard network data sample point and each standard network data population center point, the distribution distance between each standard network data sample point and each standard network data population center point is obtained; Compare the distribution distances between each standard network data sample point and each standard network data population center point, and distribute each standard network data sample point to the population where the standard network data population center point with the smallest distribution distance is located; Obtain the average values ​​of the horizontal coordinates and vertical coordinates of the standard network data sample points and the center point of the standard network data population in each population, and use the coordinates formed by the average values ​​of the horizontal coordinates and vertical coordinates of the standard network data sample points and the center point of the standard network data population as the new center point of the standard network data population; Obtain the distribution distance between the standard network data sample point and the new standard network data population center point, compare the distribution distance between each standard network data sample point and each new standard network data population center point, distribute each standard network data sample point to the population where the new standard network data population center point with the smallest distribution distance is located, obtain the average value of the horizontal coordinate and the vertical coordinate of the standard network data sample point and the new standard network data population center point in each population, and use the coordinates composed of the average value of the horizontal coordinate and the vertical coordinate of the standard network data sample point and the new standard network data population center point as the latest standard network data population center point, and so on, until the straight-line distance between the latest standard network data population center point and the previous standard network data population center point is less than or equal to R, where R is a positive number; The distribution distance between each standard network data sample point and other standard network data sample points in its population is obtained, and the distribution distance threshold is set according to the actual situation. If there is a standard network data sample point and the distribution distance between other standard network data sample points in its population is greater than the distribution distance threshold, then this standard network data sample point is recorded as an abnormal standard network data sample point. If there is no standard network data sample point and the distribution distance between other standard network data sample points in its population is greater than the distribution distance threshold, then there is no abnormal standard network data sample point.

5. The network fault monitoring system based on data monitoring according to claim 4 is characterized in that: The process of obtaining the straight-line distance between each standard network data sample point and each standard network data population center point includes: Randomly select a standard network data sample point and a standard network data population center point, record the selected standard network data sample point and the standard network data population center point as the first standard network data sample point and the first standard network data population center point, respectively, and obtain the coordinates of the first standard network data sample point and the first standard network data population center point; Obtaining a horizontal distance and a vertical distance between the first standard network data sample point and the first standard network data population center point according to the obtained coordinates of the first standard network data sample point and the first standard network data population center point; Obtaining a straight-line distance between the first standard network data sample point and the first standard network data population center point according to the obtained horizontal distance and vertical distance between the first standard network data sample point and the first standard network data population center point; In this way, the straight-line distances between the remaining standard network data sample points and the remaining standard network data population center points are obtained.

6. The network fault monitoring system based on data monitoring according to claim 5 is characterized in that: The process of obtaining the three-line distance between each standard network data sample point and each standard network data population center point includes: Obtaining the middle distance between the first standard network data sample point and the first standard network data population center point according to the longitudinal distance and the straight-line distance between the first standard network data sample point and the first standard network data population center point; According to the obtained intermediate distance, horizontal distance and vertical distance between the first standard network data sample point and the center point of the first standard network data population, the three-line distance between the first standard network data sample point and the center point of the first standard network data population is obtained.

7. The network fault monitoring system based on data monitoring according to claim 6 is characterized in that: The process of the feedback module providing feedback according to the judgment result includes: If there are abnormal standard network data sample points, the network data is abnormal, the network fails, and a red alarm is issued to remind relevant personnel to handle it; If there are no abnormal standard network data sample points, the network data is normal and the network is not faulty.

8. A network fault monitoring method based on data monitoring, specifically applied to a network fault monitoring system based on data monitoring as claimed in any one of claims 1 to 7, characterized in that: The following steps are involved: Step 1: Acquire network data, and pre-process the acquired network data to obtain standard network data; Step 2: Perform anomaly detection on the obtained standard network data to determine whether there are abnormal standard network data sample points; Step 3: Provide feedback based on the judgment results.

Images