Network environment detection method based on unsupervised learning

Abstract

The invention discloses a network environment detection method based on unsupervised learning, and relates to the field of unsupervised learning, and the method comprises the steps: obtaining network flow data in a server; copying the acquired network flow data, and performing primary network environment detection on the acquired network flow data to judge whether the coordinates of the network flow data sample points are abnormal or not; performing secondary network environment detection on the copied network flow data, and judging whether the network environment is abnormal or not according to the results of the primary network environment detection and the secondary network environment detection; if the results of the two times of network environment detection are normal, the network environment is normal, and the network environment in the server is detected twice by adopting different methods, so that the correctness of the detection results is ensured, and the security of the network environment is ensured.

Description

Technical Field

[0001] This invention relates to the field of unsupervised learning, specifically a network environment detection method based on unsupervised learning. Background Technology

[0002] With the development of the digital age, more and more information is stored on servers. Many organizations and enterprises rely on servers for business operations. If the network environment on the server is abnormal but not detected, it may lead to problems such as information leakage, property loss and corporate crisis. Therefore, accurate detection of the network environment to ensure whether the network environment is abnormal has important theoretical and practical significance. How to perform multiple tests on the network environment to ensure the accuracy of the test results and guarantee the security of the network environment is a problem we need to solve. To this end, we present a network environment detection method based on unsupervised learning. Summary of the Invention

[0003] To address the aforementioned technical problems, the present invention aims to provide a network environment detection method based on unsupervised learning.

[0004] The objective of this invention can be achieved through the following technical solution: a network environment detection method based on unsupervised learning, comprising the following steps: Step S1: Obtain network traffic data within the server; Step S2: Copy the acquired network traffic data, perform a network environment check on the acquired network traffic data, and determine whether the coordinates of the network traffic data sample points are abnormal. Step S3: Perform a second network environment check on the copied network traffic data, and determine whether the network environment is abnormal based on the results of the first and second network environment checks.

[0005] Furthermore, the process of obtaining network traffic data within the server includes: Set up several network traffic acquisition terminals, configure each terminal, generate a corresponding configuration channel, and connect to the corresponding server. The network traffic acquisition terminal obtains network traffic data from the corresponding server through the configuration channel.

[0006] Furthermore, the process of performing a network environment check on the acquired network traffic data includes: A two-dimensional coordinate system is constructed, the acquired network traffic data is quantified, and the quantization result is mapped onto the coordinate system to obtain the corresponding network traffic data sample points. Set up network traffic data sample point clusters, and denote the number of network traffic data sample point clusters as k. Divide the network traffic data sample points into k clusters, and randomly select one network traffic data sample point from each network traffic data sample point cluster as the initial cluster center network traffic data sample point. The sum of the absolute values ​​of the differences between the x-coordinate and y-coordinate of the network traffic data sample point and the x-coordinate and y-coordinate of the initial cluster center network traffic data sample point is denoted as the broken-line distance between the network traffic data sample point and the initial cluster center network traffic data sample point. The distance between the network traffic data sample points and the initial cluster center network traffic data sample points is denoted as the straight-line distance; The average of the broken line distance and the straight line distance between the network traffic data sample point and the initial cluster center network traffic data sample point is recorded as the cluster distance between the network traffic data sample point and the initial cluster center network traffic data sample point; Calculate the broken line distance and straight line distance between each network traffic data sample point and each initial cluster center network traffic data sample point, and obtain the clustering distance between the network traffic data sample point and the initial cluster center network traffic data sample point through the calculated broken line distance and straight line distance; Compare the clustering distance between each network traffic data sample point and each initial cluster center network traffic data sample point, and assign each network traffic data sample point to the cluster of the initial cluster center network traffic data sample point with the smallest clustering distance; Calculate the average of the x-coordinate and y-coordinate of the network traffic data sample points within each cluster and the initial cluster center network traffic data sample points, and use the coordinates formed by the calculated averages as the new cluster center network traffic data sample points. Calculate the clustering distance between network traffic data sample points and new cluster center network traffic data sample points. Assign each network traffic data sample point to the cluster containing the new cluster center network traffic data sample point with the smallest clustering distance. Calculate the average of the x-coordinate and y-coordinate of the network traffic data sample points and the new cluster center network traffic data sample point within each cluster. Use the calculated average as the latest cluster center network traffic data sample point. Repeat this process for several rounds. The coordinates of each network traffic data sample point are compared with those of other network traffic data samples in its group. Based on the comparison results, it is determined whether there are any coordinate anomalies in the network traffic data sample points. If there are coordinate anomalies in the network traffic data sample points, the network environment detection is considered abnormal. If there are no coordinate anomalies in the network traffic data sample points, the network environment detection is considered normal.

[0007] Furthermore, the process of calculating the piecewise linear distance between each network traffic data sample point and each initial cluster center network traffic data sample point includes: Obtain the coordinates of the network traffic data sample points and the coordinates of the initial cluster center network traffic data sample points. Obtain the absolute value of the difference between the x-coordinate and y-coordinate of the network traffic data sample points and the x-coordinate and y-coordinate of the initial cluster center network traffic data sample points. Add the absolute values ​​of the differences between the obtained x-coordinate and y-coordinate of the network traffic data sample points and the x-coordinate and y-coordinate of the initial cluster center network traffic data sample points to obtain the broken line distance between the network traffic data sample points and the initial cluster center network traffic data sample points.

[0008] Furthermore, the process of calculating the straight-line distance between each network traffic data sample point and each initial cluster center network traffic data sample point includes: The absolute values ​​of the differences between the x-coordinates and y-coordinates of the obtained network traffic data sample points and the x-coordinates and y-coordinates of the initial cluster center network traffic data sample points are squared respectively. The results are added together, and the square root of the sum is taken. The result of the square root operation is the straight-line distance between the network traffic data sample points and the initial cluster center network traffic data sample points.

[0009] Furthermore, the process of performing secondary network environment detection on the copied network traffic data includes: A two-dimensional coordinate system is constructed, the copied network traffic data is quantized, and the quantization result is mapped into the coordinate system to obtain the corresponding secondary network traffic data sample points. The maximum value between the absolute values ​​of the difference between the x-coordinates and the absolute values ​​of the difference between the y-coordinates of two secondary network traffic data sample points is recorded as the quadratic broken-line distance between the two secondary network traffic data sample points. The average of the quadratic broken line distance and the straight line distance between two secondary network traffic data sample points is recorded as the reachable distance between the two secondary network traffic data sample points; Calculate the quadratic polyline distance and straight-line distance between each secondary network traffic data sample point and other secondary network traffic data sample points, and obtain the reachable distance between each secondary network traffic data sample point and other secondary network traffic data sample points through the calculated quadratic polyline distance and straight-line distance; Set a reachability threshold between secondary network traffic data sample points and other secondary network traffic data sample points, count the number of other secondary network traffic data sample points within the reachability threshold, set a threshold for the number of other secondary network traffic data sample points within the reachability threshold, and record secondary network traffic data sample points whose number of other secondary network traffic data sample points within the reachability threshold exceeds the threshold as core network traffic data sample points. If the reachability distance between two core network traffic data sample points is less than the reachability distance threshold, then the two core network traffic data sample points will be assigned to the same group. If any secondary network traffic data sample points and core network traffic data sample points that have not been assigned to any group are found to be abnormal in the secondary network environment detection; If there are no remaining secondary network traffic data sample points and core network traffic data sample points that have not been assigned to any group, then the secondary network environment detection is normal.

[0010] Furthermore, the process of calculating the quadratic piecewise linear distance between each secondary network traffic data sample point and other secondary network traffic data sample points includes: Obtain the coordinates of each secondary network traffic data sample point and the coordinates of other secondary network traffic data sample points. Obtain the absolute value of the difference between the x-coordinate and y-coordinate of each secondary network traffic data sample point and the x-coordinate and y-coordinate of other secondary network traffic data sample points. Compare the absolute values ​​of the differences between the x-coordinate and y-coordinate. If the absolute value of the difference between the x-coordinate and y-coordinate is greater than the absolute value of the difference between the y-coordinate and y-coordinate, then the absolute value of the difference between the x-coordinate and y-coordinate is the calculated quadratic broken-line distance between each secondary network traffic data sample point and other secondary network traffic data sample points. If the absolute value of the difference between the x-coordinate and y-coordinate is less than the absolute value of the difference between the y-coordinate and y-coordinate, then the absolute value of the difference between the y-coordinate and y-coordinate is the calculated quadratic broken-line distance between each secondary network traffic data sample point and other secondary network traffic data sample points.

[0011] Furthermore, the process of determining whether the network environment is abnormal based on the results of the first and second network environment checks includes: The network environment is normal if both the first and second network environment tests are normal. The network environment is abnormal when the first network environment test is normal and the second network environment test is abnormal, the first network environment test is abnormal and the second network environment test is normal, or both the first and second network environment tests are abnormal.

[0012] Compared with existing technologies, the beneficial effects of this invention are as follows: Several network traffic acquisition terminals are set up to acquire network traffic data within the server. The acquired network traffic data is copied, and a first network environment check is performed on the acquired network traffic data to determine if the coordinates of the network traffic data sample points are abnormal. A second network environment check is performed on the copied network traffic data, and the network environment is determined to be abnormal based on the results of the first and second network environment checks. If both network environment checks are normal, the network environment is considered normal. By employing different methods to perform two checks on the network environment within the server, the accuracy of the check results is ensured, and the security of the network environment is guaranteed. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0014] like Figure 1 As shown, a network environment detection method based on unsupervised learning includes the following steps: Step S1: Obtain network traffic data within the server; Step S2: Copy the acquired network traffic data, perform a network environment check on the acquired network traffic data, and determine whether the coordinates of the network traffic data sample points are abnormal. Step S3: Perform a second network environment check on the copied network traffic data, and determine whether the network environment is abnormal based on the results of the first and second network environment checks.

[0015] It should be further explained that, in the specific implementation process, the process of obtaining network traffic data within the server includes: Set up several network traffic acquisition terminals, configure each terminal, generate a corresponding configuration channel, and connect to the corresponding server. The network traffic acquisition terminal obtains network traffic data from the corresponding server through the configuration channel.

[0016] It should be further explained that, in the specific implementation process, the process of performing a network environment check on the acquired network traffic data includes: A two-dimensional coordinate system is constructed, the acquired network traffic data is quantified, and the quantization result is mapped onto the coordinate system to obtain the corresponding network traffic data sample points. Set up a network traffic data sample point cluster, and record the number of network traffic data sample point clusters as k. Divide the network traffic data sample points into k clusters, and randomly select a network traffic data sample point from each network traffic data sample point cluster as the initial cluster center network traffic data sample point. The initial cluster center network traffic data sample point is the representative network traffic data sample point of the network traffic data sample point cluster. The sum of the absolute values ​​of the differences between the x-coordinate and y-coordinate of the network traffic data sample point and the x-coordinate and y-coordinate of the initial cluster center network traffic data sample point is denoted as the broken-line distance between the network traffic data sample point and the initial cluster center network traffic data sample point. The distance between the network traffic data sample points and the initial cluster center network traffic data sample points is denoted as the straight-line distance; The average of the broken line distance and the straight line distance between the network traffic data sample point and the initial cluster center network traffic data sample point is recorded as the cluster distance between the network traffic data sample point and the initial cluster center network traffic data sample point; Calculate the clustering distance between each network traffic data sample point and each initial cluster center network traffic data sample point; It should be further explained that, in the specific implementation process, the calculation process of the clustering distance between the network traffic data sample points and the initial cluster center network traffic data sample points includes: Obtain the coordinates of the network traffic data sample point, denoted as (x1, y1), and obtain the coordinates of the initial cluster center network traffic data sample point, denoted as (x2, y2). Using the coordinates of the network traffic data sample point and the initial cluster center network traffic data sample point, obtain the absolute value of the difference between the x-coordinate of the network traffic data sample point and the x-coordinate of the initial cluster center network traffic data sample point, and the absolute value of the difference between the y-coordinate of the network traffic data sample point and the y-coordinate of the initial cluster center network traffic data sample point, denoted as h and z respectively. in, , ; The broken-line distance between the network traffic data sample points and the initial cluster center network traffic data sample points is obtained and denoted as hz; in, ; The squares of the differences between the x-coordinate of the network traffic data sample point and the x-coordinate of the initial cluster center network traffic data sample point are obtained by using the coordinates of the network traffic data sample point and the coordinates of the initial cluster center network traffic data sample point. These squares are denoted as hc and zc, respectively. in, , ; The square of the straight-line distance between the network traffic data sample point and the initial cluster center network traffic data sample point is obtained and denoted as j. 2 ; in, ; Then j is the straight-line distance between the network traffic data sample point and the initial cluster center network traffic data sample point; in, , ; The clustering distance between the network traffic data sample points and the initial cluster center network traffic data sample points is obtained by the broken line distance and the straight line distance between the network traffic data sample points and the initial cluster center network traffic data sample points, and is denoted as ju; in, ; Compare the clustering distance between each network traffic data sample point and each initial cluster center network traffic data sample point, and assign each network traffic data sample point to the cluster of the initial cluster center network traffic data sample point with the smallest clustering distance; Calculate the average of the x-coordinate and y-coordinate of the network traffic data sample points within each cluster and the initial cluster center network traffic data sample points. Use the coordinates formed by the average of the x-coordinate and y-coordinate of the network traffic data sample points and the initial cluster center network traffic data sample points as the new cluster center network traffic data sample points. Calculate the clustering distance between network traffic data sample points and new cluster center network traffic data sample points. Compare the clustering distance between each network traffic data sample point and each new cluster center network traffic data sample point. Assign each network traffic data sample point to the cluster containing the new cluster center network traffic data sample point with the smallest clustering distance. Calculate the average of the x-coordinate and y-coordinate of the network traffic data sample points and the new cluster center network traffic data sample points within each cluster. Use the average of the x-coordinate and y-coordinate of the network traffic data sample points and the new cluster center network traffic data sample points as the latest cluster center network traffic data sample point. Repeat this process for several rounds. The coordinates of each network traffic data sample point are compared with those of other network traffic data samples in its group. When the coordinates of a network traffic data sample are significantly different from those of other network traffic data samples in its group, the network traffic data sample is determined to be an abnormal network traffic data sample, and an abnormal network environment detection is detected. A network environment detection is considered normal when the coordinates of a network traffic data sample that does not exist differ significantly from the coordinates of other network traffic data samples within its group.

[0017] It should be further explained that, in the specific implementation process, the process of performing a secondary network environment check on the copied network traffic data includes: A two-dimensional coordinate system is constructed, the copied network traffic data is quantized, and the quantization result is mapped into the coordinate system to obtain the corresponding secondary network traffic data sample points. The maximum value between the absolute values ​​of the difference between the x-coordinates and the absolute values ​​of the difference between the y-coordinates of two secondary network traffic data sample points is recorded as the quadratic broken-line distance between the two secondary network traffic data sample points. The average of the quadratic broken-line distance and the straight-line distance between two quadratic network traffic data sample points is denoted as the reachable distance between the two quadratic network traffic data sample points. Calculate the reachability distance between each secondary network traffic data sample point and other secondary network traffic data sample points; For example: The coordinates of secondary network traffic data sample point A are (x3, y3), and the coordinates of secondary network traffic data sample point B are (x4, y4). Calculate the straight-line distance between secondary network traffic data sample point A and secondary network traffic data sample point B, denoted as zx; in, ; The absolute values ​​of the differences between the x-coordinates of secondary network traffic data sample point A and the x-coordinates of secondary network traffic data sample point B, and the absolute values ​​of the differences between the y-coordinates of secondary network traffic data sample point A and the y-coordinates of secondary network traffic data sample point B are obtained from the coordinates of secondary network traffic data sample point A and the coordinates of secondary network traffic data sample point B, and are denoted as eh and ez respectively. in, , ; Obtain the quadratic polyline distance between secondary network traffic data sample point A and secondary network traffic data sample point B, denoted as ehz; in, ; The reachable distance between secondary network traffic data sample point A and secondary network traffic data sample point B is obtained by the straight-line distance and the secondary broken-line distance between them, and is denoted as kd. in, ; Set a reachability threshold between secondary network traffic data sample points and other secondary network traffic data sample points, denoted as n. Count the number of other secondary network traffic data sample points within the reachability threshold. Set a threshold for the number of other secondary network traffic data sample points within the reachability threshold, denoted as m. Secondary network traffic data sample points with more than m other secondary network traffic data sample points within the reachability threshold are recorded as core network traffic data sample points. If the reachability distance between two core network traffic data sample points is less than n, then the two core network traffic data sample points will be assigned to the same group. If any secondary network traffic data sample points and core network traffic data sample points that have not been assigned to any group are found to be abnormal in the secondary network environment detection; If there are no remaining secondary network traffic data sample points and core network traffic data sample points that have not been assigned to any group, then the secondary network environment detection is normal.

[0018] It should be further explained that, in the specific implementation process, the process of determining whether the network environment is abnormal based on the results of the first and second network environment checks includes: The network environment is normal if both the first and second network environment tests are normal. The network environment is abnormal when the first network environment test is normal and the second network environment test is abnormal, the first network environment test is abnormal and the second network environment test is normal, or both the first and second network environment tests are abnormal.

[0019] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A network environment detection method based on unsupervised learning, characterized in that, Includes the following steps: Step S1: Obtain network traffic data within the server; Step S2: Copy the acquired network traffic data, perform a network environment check on the acquired network traffic data, and determine whether the coordinates of the network traffic data sample points are abnormal. Step S3: Perform a second network environment check on the copied network traffic data, and determine whether the network environment is abnormal based on the results of the first and second network environment checks; The process of obtaining network traffic data from the server includes: Set up several network traffic acquisition terminals, configure each network traffic acquisition terminal, generate a corresponding configuration channel, and connect to the corresponding server. The network traffic acquisition terminal obtains network traffic data in the corresponding server through the configuration channel. The process of performing a network environment test on the acquired network traffic data includes: A two-dimensional coordinate system is constructed, the acquired network traffic data is quantified, and the quantization result is mapped into the coordinate system to obtain the corresponding network traffic data sample points. Set up network traffic data sample point clusters, and denote the number of network traffic data sample point clusters as k. Divide the network traffic data sample points into k clusters, and randomly select one network traffic data sample point from each network traffic data sample point cluster as the initial cluster center network traffic data sample point. Calculate the broken line distance and straight line distance between each network traffic data sample point and each initial cluster center network traffic data sample point, and obtain the clustering distance between the network traffic data sample point and the initial cluster center network traffic data sample point through the calculated broken line distance and straight line distance; Compare the clustering distance between each network traffic data sample point and each initial cluster center network traffic data sample point, and assign each network traffic data sample point to the cluster of the initial cluster center network traffic data sample point with the smallest clustering distance; Calculate the average of the x-coordinate and y-coordinate of the network traffic data sample points within each cluster and the initial cluster center network traffic data sample points. Use the coordinates formed by the calculated average values ​​as the new cluster center network traffic data sample points. Repeat this process for several rounds. The coordinates of each network traffic data sample point are compared with those of other network traffic data samples in its group. Based on the comparison results, it is determined whether there are any coordinate anomalies in the network traffic data sample points. If there are coordinate anomalies in the network traffic data sample points, the network environment detection is considered abnormal. If there are no coordinate anomalies in the network traffic data sample points, the network environment detection is considered normal. The process of performing secondary network environment checks on copied network traffic data includes: A two-dimensional coordinate system is constructed, the copied network traffic data is quantized, and the quantization result is mapped into the coordinate system to obtain the corresponding secondary network traffic data sample points. The maximum value between the absolute values ​​of the difference between the x-coordinates and the absolute values ​​of the difference between the y-coordinates of two secondary network traffic data sample points is recorded as the quadratic broken-line distance between the two secondary network traffic data sample points. The average of the quadratic broken line distance and the straight line distance between two secondary network traffic data sample points is recorded as the reachable distance between the two secondary network traffic data sample points; Calculate the quadratic polyline distance and straight-line distance between each secondary network traffic data sample point and other secondary network traffic data sample points, and obtain the reachable distance between each secondary network traffic data sample point and other secondary network traffic data sample points through the calculated quadratic polyline distance and straight-line distance; Set a reachability threshold between secondary network traffic data sample points and other secondary network traffic data sample points, count the number of other secondary network traffic data sample points within the reachability threshold, set a threshold for the number of other secondary network traffic data sample points within the reachability threshold, and record secondary network traffic data sample points whose number of other secondary network traffic data sample points within the reachability threshold exceeds the threshold as core network traffic data sample points. If the reachability distance between two core network traffic data sample points is less than the reachability distance threshold, then the two core network traffic data sample points will be assigned to the same group. If any secondary network traffic data sample points and core network traffic data sample points that have not been assigned to any group are found to be abnormal in the secondary network environment detection; If there are no remaining secondary network traffic data sample points and core network traffic data sample points that have not been assigned to any group, then the secondary network environment detection is normal. The process of determining whether the network environment is abnormal based on the results of a primary network environment test and a secondary network environment test includes: The network environment is normal if both the first and second network environment tests are normal. The network environment is abnormal when the first network environment test is normal and the second network environment test is abnormal, the first network environment test is abnormal and the second network environment test is normal, or both the first and second network environment tests are abnormal. The process of calculating the piecewise linear distance between each network traffic data sample point and each initial cluster center network traffic data sample point includes: Obtain the coordinates of the network traffic data sample points and the coordinates of the initial cluster center network traffic data sample points. Obtain the absolute value of the difference between the x-coordinate and y-coordinate of the network traffic data sample points and the x-coordinate and y-coordinate of the initial cluster center network traffic data sample points. Add the absolute values ​​of the differences between the obtained x-coordinate and y-coordinate of the network traffic data sample points and the x-coordinate and y-coordinate of the initial cluster center network traffic data sample points to obtain the broken line distance between the network traffic data sample points and the initial cluster center network traffic data sample points. The process of calculating the quadratic piecewise linear distance between each secondary network traffic data sample point and other secondary network traffic data sample points includes: Obtain the coordinates of each secondary network traffic data sample point and the coordinates of other secondary network traffic data sample points. Obtain the absolute value of the difference between the x-coordinate and y-coordinate of each secondary network traffic data sample point and the x-coordinate and y-coordinate of other secondary network traffic data sample points. Compare the absolute values ​​of the differences between the x-coordinate and y-coordinate. If the absolute value of the difference between the x-coordinate and y-coordinate is greater than the absolute value of the difference between the y-coordinate and y-coordinate, then the absolute value of the difference between the x-coordinate and y-coordinate is the calculated quadratic broken-line distance between each secondary network traffic data sample point and other secondary network traffic data sample points. If the absolute value of the difference between the x-coordinate and y-coordinate is less than the absolute value of the difference between the y-coordinate and y-coordinate, then the absolute value of the difference between the y-coordinate and y-coordinate is the calculated quadratic broken-line distance between each secondary network traffic data sample point and other secondary network traffic data sample points.

2. The network environment detection method based on unsupervised learning according to claim 1, characterized in that, The process of calculating the straight-line distance between each network traffic data sample point and each initial cluster center network traffic data sample point includes: The absolute values ​​of the differences between the x-coordinates and y-coordinates of the obtained network traffic data sample points and the x-coordinates and y-coordinates of the initial cluster center network traffic data sample points are squared respectively. The results are added together, and the square root of the sum is taken. The result of the square root operation is the straight-line distance between the network traffic data sample points and the initial cluster center network traffic data sample points.

Images