Network environment detection method based on unsupervised learning

Abstract

The invention discloses a network environment detection method based on unsupervised learning, and relates to the field of unsupervised learning, and the method comprises the steps: obtaining network flow data in a server; copying the acquired network flow data, and performing primary network environment detection on the acquired network flow data to judge whether the coordinates of the network flow data sample points are abnormal or not; performing secondary network environment detection on the copied network flow data, and judging whether the network environment is abnormal or not according to the results of the primary network environment detection and the secondary network environment detection; if the results of the two times of network environment detection are normal, the network environment is normal, and the network environment in the server is detected twice by adopting different methods, so that the correctness of the detection results is ensured, and the security of the network environment is ensured.

Description

Technical Field

[0001] The invention relates to the field of unsupervised learning, and in particular to a network environment detection method based on unsupervised learning. Background Art

[0002] With the development of the digital age, more and more information is stored in servers. Many organizations and enterprises rely on servers for business operations. If the network environment in the server is abnormal but not detected, it may lead to information leakage, property loss, and corporate crisis. Therefore, accurate detection of the network environment to ensure whether the network environment is abnormal has important theoretical and practical significance.

[0003] How to perform multiple tests on the network environment to ensure the accuracy of the test results and ensure the security of the network environment is a problem we need to solve. To this end, a network environment detection method based on unsupervised learning is now provided. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a network environment detection method based on unsupervised learning.

[0005] The purpose of the present invention can be achieved by the following technical solution: A network environment detection method based on unsupervised learning, comprising the following steps: Step S1: Obtain network traffic data in the server; Step S2: copying the acquired network traffic data, and performing a network environment detection on the acquired network traffic data to determine whether the coordinates of the network traffic data sample points are abnormal; Step S3: Perform a secondary network environment detection on the copied network traffic data, and determine whether the network environment is abnormal based on the results of the primary network environment detection and the secondary network environment detection.

[0006] Furthermore, the process of obtaining network traffic data in the server includes: Set up several network flow collection terminals, configure each network flow collection terminal, generate a corresponding configuration channel, link the corresponding server, and the network flow collection terminal obtains the network flow data in the corresponding server through the configuration channel.

[0007] Furthermore, the process of performing a network environment detection on the acquired network traffic data includes: Construct a two-dimensional coordinate system, quantify the acquired network traffic data, and map the quantified results in the coordinate system to obtain the corresponding network traffic data sample points; Set a network flow data sample point cluster, and record the number of network flow data sample point clusters as k, divide the network flow data sample points into k clusters, and randomly select a network flow data sample point from each network flow data sample point cluster and record it as the initial cluster center network flow data sample point; The sum of the absolute values ​​of the differences between the horizontal coordinate and the vertical coordinate of the network flow data sample point and the horizontal coordinate and the vertical coordinate of the network flow data sample point of the initial cluster center is recorded as the polyline distance between the network flow data sample point and the network flow data sample point of the initial cluster center; The distance between the network flow data sample point and the initial cluster center network flow data sample point is recorded as the straight-line distance; The average of the polyline distance and the straight-line distance between the network flow data sample point and the initial cluster center network flow data sample point is recorded as the cluster distance between the network flow data sample point and the initial cluster center network flow data sample point; Calculate the polyline distance and straight-line distance between each network flow data sample point and each initial cluster center network flow data sample point, and obtain the cluster distance between the network flow data sample point and the initial cluster center network flow data sample point through the calculated polyline distance and straight-line distance; Compare the clustering distance between each network flow data sample point and each initial cluster center network flow data sample point, and assign each network flow data sample point to the group where the initial cluster center network flow data sample point with the smallest clustering distance is located; Calculate the average values ​​of the horizontal coordinates and vertical coordinates of the network flow data sample points in each group and the network flow data sample points of the initial cluster center, and use the coordinates composed of the average values ​​obtained as the new network flow data sample points of the cluster center; Calculate the cluster distance between the network flow data sample point and the new cluster center network flow data sample point, assign each network flow data sample point to the group where the new cluster center network flow data sample point with the smallest cluster distance is located, calculate the average value of the horizontal coordinate and the vertical coordinate of the network flow data sample point in each group and the new cluster center network flow data sample point, and use the calculated average value as the latest cluster center network flow data sample point, and so on, and repeat the calculation for several rounds; The coordinates of each network traffic data sample point are compared with those of other network traffic data samples in its group, and whether there is an abnormality in the coordinates of the network traffic data sample point is determined based on the comparison result. If there is an abnormality in the coordinates of the network traffic data sample point, then the network environment detection is abnormal; if there is no abnormality in the coordinates of the network traffic data sample point, then the network environment detection is normal.

[0008] Furthermore, the process of calculating the polyline distance between each network flow data sample point and each initial cluster center network flow data sample point includes: The coordinates of the network flow data sample point and the coordinates of the network flow data sample point of the initial cluster center are obtained, the absolute value of the difference between the horizontal coordinate and the vertical coordinate of the network flow data sample point and the horizontal coordinate and the vertical coordinate of the network flow data sample point of the initial cluster center is obtained, and the absolute value of the difference between the horizontal coordinate and the vertical coordinate of the obtained network flow data sample point and the horizontal coordinate and the vertical coordinate of the network flow data sample point of the initial cluster center is added to obtain the broken line distance between the network flow data sample point and the network flow data sample point of the initial cluster center.

[0009] Furthermore, the process of calculating the straight-line distance between each network flow data sample point and each initial cluster center network flow data sample point includes: The absolute values ​​of the differences between the horizontal and vertical coordinates of the obtained network traffic data sample points and the horizontal and vertical coordinates of the network traffic data sample points of the initial cluster center are squared respectively, the results of the operations are added, and the result of the addition is squared, and the result of the square root operation is the straight-line distance between the network traffic data sample point and the network traffic data sample point of the initial cluster center.

[0010] Furthermore, the process of performing a secondary network environment detection on the copied network traffic data includes: Construct a two-dimensional coordinate system, quantify the copied network traffic data, and map the quantified results in the coordinate system to obtain the corresponding secondary network traffic data sample points; The maximum value of the absolute value of the difference between the horizontal coordinate and the vertical coordinate of two secondary network flow data sample points is recorded as the secondary polyline distance between the two secondary network flow data sample points; The average of the secondary polyline distance and the straight-line distance between two secondary network flow data sample points is recorded as the reachable distance between the two secondary network flow data sample points; Calculate the secondary polyline distance and straight-line distance between each secondary network flow data sample point and other secondary network flow data sample points, and obtain the reachable distance between each secondary network flow data sample point and other secondary network flow data sample points through the calculated secondary polyline distance and straight-line distance; Set a reachable distance threshold between the secondary network flow data sample point and other secondary network flow data sample points, count the number of other secondary network flow data sample points within the reachable distance threshold, set a threshold for the number of other secondary network flow data sample points within the reachable distance threshold, and record the secondary network flow data sample points whose number of other secondary network flow data sample points within the reachable distance threshold is greater than the threshold as core network flow data sample points; If the reachable distance between two core network traffic data sample points is less than the reachable distance threshold, the two core network traffic data sample points are assigned to the same group; If there are remaining secondary network traffic data sample points and core network traffic data sample points that are not assigned to any group, the secondary network environment detection is abnormal; If there are no remaining secondary network traffic data sample points and core network traffic data sample points that are not assigned to any group, the secondary network environment detection is normal.

[0011] Furthermore, the process of calculating the secondary polyline distance between each secondary network flow data sample point and other secondary network flow data sample points includes: Obtain the coordinates of each secondary network flow data sample point and the coordinates of other secondary network flow data sample points, obtain the absolute value of the difference between the horizontal coordinate and the vertical coordinate of each secondary network flow data sample point and the horizontal coordinate and the vertical coordinate of other secondary network flow data sample points, compare the absolute value of the difference between the horizontal coordinate and the vertical coordinate, if the absolute value of the difference between the horizontal coordinate is greater than the absolute value of the difference between the vertical coordinate, then the absolute value of the difference between the horizontal coordinate is the calculated secondary polyline distance between each secondary network flow data sample point and the other secondary network flow data sample points, if the absolute value of the difference between the horizontal coordinate is less than the absolute value of the difference between the vertical coordinate, then the absolute value of the difference between the vertical coordinate is the calculated secondary polyline distance between each secondary network flow data sample point and the other secondary network flow data sample points.

[0012] Further, the process of determining whether the network environment is abnormal according to the results of the first network environment detection and the second network environment detection includes: When both the first network environment test and the second network environment test are normal, the network environment is normal; When the primary network environment detection is normal and the secondary network environment detection is abnormal, the primary network environment detection is abnormal and the secondary network environment detection is normal, and both the primary network environment detection and the secondary network environment detection are abnormal, the network environment is abnormal.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: a number of network traffic collection terminals are set up, network traffic data in the server is obtained through the set network traffic collection terminals, the obtained network traffic data is copied, a network environment detection is performed on the obtained network traffic data to determine whether the coordinates of the network traffic data sample points are abnormal, a secondary network environment detection is performed on the copied network traffic data, and whether the network environment is abnormal is determined according to the results of the primary network environment detection and the secondary network environment detection; if the results of the two network environment detections are normal, the network environment is normal, and different methods are used to perform two detections on the network environment in the server, thereby ensuring the correctness of the detection results and the security of the network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0015] like Figure 1 As shown, a network environment detection method based on unsupervised learning includes the following steps: Step S1: Obtain network traffic data in the server; Step S2: copying the acquired network traffic data, and performing a network environment detection on the acquired network traffic data to determine whether the coordinates of the network traffic data sample points are abnormal; Step S3: Perform a secondary network environment detection on the copied network traffic data, and determine whether the network environment is abnormal based on the results of the primary network environment detection and the secondary network environment detection.

[0016] It should be further explained that, in the specific implementation process, the process of obtaining the network traffic data in the server includes: Set up several network flow collection terminals, configure each network flow collection terminal, generate a corresponding configuration channel, link the corresponding server, and the network flow collection terminal obtains the network flow data in the corresponding server through the configuration channel.

[0017] It should be further explained that, in the specific implementation process, the process of performing a network environment detection on the acquired network traffic data includes: Construct a two-dimensional coordinate system, quantify the acquired network traffic data, and map the quantified results in the coordinate system to obtain the corresponding network traffic data sample points; Set a network flow data sample point cluster, and record the number of network flow data sample point clusters as k, divide the network flow data sample points into k clusters, randomly select a network flow data sample point from each network flow data sample point cluster and record it as an initial cluster center network flow data sample point, and the initial cluster center network flow data sample point is a representative network flow data sample point of the network flow data sample point cluster; The sum of the absolute values ​​of the differences between the horizontal coordinate and the vertical coordinate of the network flow data sample point and the horizontal coordinate and the vertical coordinate of the network flow data sample point of the initial cluster center is recorded as the polyline distance between the network flow data sample point and the network flow data sample point of the initial cluster center; The distance between the network flow data sample point and the initial cluster center network flow data sample point is recorded as the straight-line distance; The average of the polyline distance and the straight-line distance between the network flow data sample point and the initial cluster center network flow data sample point is recorded as the cluster distance between the network flow data sample point and the initial cluster center network flow data sample point; Calculate the clustering distance between each network flow data sample point and each initial cluster center network flow data sample point; It should be further explained that, in the specific implementation process, the calculation process of the clustering distance between the network flow data sample point and the initial cluster center network flow data sample point includes: Obtain the coordinates of the network flow data sample point, recorded as (x1, y1), obtain the coordinates of the network flow data sample point of the initial cluster center, recorded as (x2, y2), obtain the absolute value of the difference between the horizontal coordinate of the network flow data sample point and the horizontal coordinate of the network flow data sample point of the initial cluster center, and the absolute value of the difference between the vertical coordinate of the network flow data sample point and the vertical coordinate of the network flow data sample point of the initial cluster center through the coordinates of the network flow data sample point and the coordinates of the network flow data sample point of the initial cluster center, recorded as h and z respectively; in, , ; Get the polyline distance between the network flow data sample point and the network flow data sample point of the initial cluster center, recorded as hz; in, ; Obtain the square of the difference between the horizontal coordinate of the network flow data sample point and the horizontal coordinate of the network flow data sample point at the initial cluster center, and the square of the difference between the vertical coordinate of the network flow data sample point and the vertical coordinate of the network flow data sample point at the initial cluster center through the coordinates of the network flow data sample point and the coordinates of the network flow data sample point at the initial cluster center, and record them as hc and zc respectively; in, , ; Get the square of the straight-line distance between the network traffic data sample point and the initial cluster center network traffic data sample point, denoted as j 2 ; in, ; Then j is the straight-line distance between the network flow data sample point and the network flow data sample point of the initial cluster center; in, , ; The cluster distance between the network flow data sample point and the initial cluster center network flow data sample point is obtained by the broken line distance and straight line distance between the network flow data sample point and the initial cluster center network flow data sample point, which is recorded as ju; in, ; Compare the clustering distance between each network flow data sample point and each initial cluster center network flow data sample point, and assign each network flow data sample point to the group where the initial cluster center network flow data sample point with the smallest clustering distance is located; Calculate the average value of the horizontal coordinate and the vertical coordinate of the network flow data sample point and the initial cluster center network flow data sample point in each group, and use the coordinate formed by the average value of the horizontal coordinate and the vertical coordinate of the network flow data sample point and the initial cluster center network flow data sample point as the new cluster center network flow data sample point; Calculate the cluster distance between the network flow data sample point and the new cluster center network flow data sample point, compare the cluster distance between each network flow data sample point and each new cluster center network flow data sample point, assign each network flow data sample point to the group where the new cluster center network flow data sample point with the smallest cluster distance is located, calculate the average value of the horizontal coordinate and the vertical coordinate of the network flow data sample point and the new cluster center network flow data sample point in each group, and use the coordinate composed of the average value of the horizontal coordinate and the vertical coordinate of the network flow data sample point and the new cluster center network flow data sample point as the latest cluster center network flow data sample point, and so on, and perform several rounds of repeated calculations; Compare the coordinates of each network traffic data sample point with the coordinates of other network traffic data samples in its group. When the coordinates of a network traffic data sample are significantly different from those of other network traffic data samples in its group, the network traffic data sample is determined to be an abnormal network traffic data sample, and a network environment detection abnormality is detected. When there is no significant difference between the coordinates of a network traffic data sample and the coordinates of other network traffic data samples in its group, a network environment detection is normal.

[0018] It should be further explained that, in the specific implementation process, the process of performing secondary network environment detection on the copied network traffic data includes: Construct a two-dimensional coordinate system, quantify the copied network traffic data, and map the quantified results in the coordinate system to obtain the corresponding secondary network traffic data sample points; The maximum value of the absolute value of the difference between the horizontal coordinate and the vertical coordinate of two secondary network flow data sample points is recorded as the secondary polyline distance between the two secondary network flow data sample points; The average of the secondary polyline distance and the straight line distance between two secondary network flow data sample points is recorded as the reachable distance between the two secondary network flow data sample points. Calculate the reachable distance between each secondary network flow data sample point and other secondary network flow data sample points; For example: The coordinates of the secondary network flow data sample point A are (x3, y3), and the coordinates of the secondary network flow data sample point B are (x4, y4); Calculate the straight-line distance between the secondary network flow data sample point A and the secondary network flow data sample point B, denoted as zx; in, ; Obtain the absolute value of the difference between the horizontal coordinate of the secondary network flow data sample point A and the horizontal coordinate of the secondary network flow data sample point B and the absolute value of the difference between the vertical coordinate of the secondary network flow data sample point A and the vertical coordinate of the secondary network flow data sample point B through the coordinates of the secondary network flow data sample point A and the coordinates of the secondary network flow data sample point B, and record them as eh and ez respectively; in, , ; Get the secondary polyline distance between the secondary network flow data sample point A and the secondary network flow data sample point B, denoted as ehz; in, ; The reachable distance between the secondary network flow data sample point A and the secondary network flow data sample point B is obtained by the straight-line distance and the secondary polyline distance between the secondary network flow data sample point A and the secondary network flow data sample point B, which is recorded as kd; in, ; Set the reachable distance threshold between the secondary network flow data sample point and other secondary network flow data sample points, denoted as n, count the number of other secondary network flow data sample points within the reachable distance threshold, set the threshold of the number of other secondary network flow data sample points within the reachable distance threshold, denoted as m, and record the secondary network flow data sample points whose number of other secondary network flow data sample points within the reachable distance threshold is greater than m as core network flow data sample points; If the reachable distance between two core network traffic data sample points is less than n, the two core network traffic data sample points are assigned to the same group; If there are remaining secondary network traffic data sample points and core network traffic data sample points that are not assigned to any group, the secondary network environment detection is abnormal; If there are no remaining secondary network traffic data sample points and core network traffic data sample points that are not assigned to any group, the secondary network environment detection is normal.

[0019] It should be further explained that, in the specific implementation process, the process of judging whether the network environment is abnormal according to the results of the first network environment detection and the second network environment detection includes: When both the first network environment test and the second network environment test are normal, the network environment is normal; When the primary network environment detection is normal and the secondary network environment detection is abnormal, the primary network environment detection is abnormal and the secondary network environment detection is normal, and both the primary network environment detection and the secondary network environment detection are abnormal, the network environment is abnormal.

[0020] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A network environment detection method based on unsupervised learning, characterized in that: The following steps are involved: Step S1: Obtain network traffic data in the server; Step S2: copy the acquired network traffic data, and perform a network environment detection on the acquired network traffic data to determine whether the coordinates of the network traffic data sample points are abnormal; Step S3: Perform a secondary network environment detection on the copied network traffic data, and determine whether the network environment is abnormal based on the results of the primary network environment detection and the secondary network environment detection.

2. A network environment detection method based on unsupervised learning according to claim 1, characterized in that: The process of obtaining network traffic data within the server includes: Set up several network flow collection terminals, configure each network flow collection terminal, generate a corresponding configuration channel, link the corresponding server, and the network flow collection terminal obtains the network flow data in the corresponding server through the configuration channel.

3. A network environment detection method based on unsupervised learning according to claim 2, characterized in that: The process of performing a network environment detection on the acquired network traffic data includes: Construct a two-dimensional coordinate system, quantify the acquired network traffic data, and map the quantified results in the coordinate system to obtain the corresponding network traffic data sample points; Set a network flow data sample point cluster, and record the number of network flow data sample point clusters as k, divide the network flow data sample points into k clusters, and randomly select a network flow data sample point from each network flow data sample point cluster and record it as the initial cluster center network flow data sample point; Calculate the polyline distance and straight-line distance between each network flow data sample point and each initial cluster center network flow data sample point, and obtain the cluster distance between the network flow data sample point and the initial cluster center network flow data sample point through the calculated polyline distance and straight-line distance; Compare the clustering distance between each network flow data sample point and each initial cluster center network flow data sample point, and assign each network flow data sample point to the group where the initial cluster center network flow data sample point with the smallest clustering distance is located; Calculate the average values ​​of the horizontal and vertical coordinates of the network flow data sample points in each group and the network flow data sample points of the initial cluster center, and use the coordinates composed of the average values ​​as the new network flow data sample points of the cluster center, and repeat the calculation for several rounds in this way; The coordinates of each network traffic data sample point are compared with those of other network traffic data samples in its group, and whether there is an abnormality in the coordinates of the network traffic data sample point is determined based on the comparison result. If there is an abnormality in the coordinates of the network traffic data sample point, then the network environment detection is abnormal; if there is no abnormality in the coordinates of the network traffic data sample point, then the network environment detection is normal.

4. A network environment detection method based on unsupervised learning according to claim 3, characterized in that: The process of calculating the polyline distance between each network flow data sample point and each initial cluster center network flow data sample point includes: The coordinates of the network flow data sample point and the coordinates of the network flow data sample point of the initial cluster center are obtained, the absolute value of the difference between the horizontal coordinate and the vertical coordinate of the network flow data sample point and the horizontal coordinate and the vertical coordinate of the network flow data sample point of the initial cluster center is obtained, and the absolute value of the difference between the horizontal coordinate and the vertical coordinate of the obtained network flow data sample point and the horizontal coordinate and the vertical coordinate of the network flow data sample point of the initial cluster center is added to obtain the broken line distance between the network flow data sample point and the network flow data sample point of the initial cluster center.

5. A network environment detection method based on unsupervised learning according to claim 4, characterized in that: The process of calculating the straight-line distance between each network flow data sample point and each initial cluster center network flow data sample point includes: The absolute values ​​of the differences between the horizontal and vertical coordinates of the obtained network traffic data sample points and the horizontal and vertical coordinates of the network traffic data sample points of the initial cluster center are squared respectively, the results of the operations are added, and the result of the addition is squared, and the result of the square root operation is the straight-line distance between the network traffic data sample point and the network traffic data sample point of the initial cluster center.

6. The network environment detection method based on unsupervised learning according to claim 3 is characterized in that: The process of performing secondary network environment detection on the copied network traffic data includes: Construct a two-dimensional coordinate system, quantify the copied network traffic data, and map the quantified results in the coordinate system to obtain the corresponding secondary network traffic data sample points; The maximum value of the absolute value of the difference between the horizontal coordinate and the vertical coordinate of two secondary network flow data sample points is recorded as the secondary polyline distance between the two secondary network flow data sample points; The average of the secondary polyline distance and the straight-line distance between two secondary network flow data sample points is recorded as the reachable distance between the two secondary network flow data sample points; Calculate the secondary polyline distance and straight-line distance between each secondary network flow data sample point and other secondary network flow data sample points, and obtain the reachable distance between each secondary network flow data sample point and other secondary network flow data sample points through the calculated secondary polyline distance and straight-line distance; Set a reachable distance threshold between the secondary network flow data sample point and other secondary network flow data sample points, count the number of other secondary network flow data sample points within the reachable distance threshold, set a threshold for the number of other secondary network flow data sample points within the reachable distance threshold, and record the secondary network flow data sample points whose number of other secondary network flow data sample points within the reachable distance threshold is greater than the threshold as core network flow data sample points; If the reachable distance between two core network traffic data sample points is less than the reachable distance threshold, the two core network traffic data sample points are assigned to the same group; If there are remaining secondary network traffic data sample points and core network traffic data sample points that are not assigned to any group, the secondary network environment detection is abnormal; If there are no remaining secondary network traffic data sample points and core network traffic data sample points that are not assigned to any group, the secondary network environment detection is normal.

7. A network environment detection method based on unsupervised learning according to claim 6, characterized in that: The process of calculating the secondary polyline distance between each secondary network flow data sample point and other secondary network flow data sample points includes: Obtain the coordinates of each secondary network flow data sample point and the coordinates of other secondary network flow data sample points, obtain the absolute value of the difference between the horizontal coordinate and the vertical coordinate of each secondary network flow data sample point and the horizontal coordinate and the vertical coordinate of other secondary network flow data sample points, compare the absolute value of the difference between the horizontal coordinate and the vertical coordinate, if the absolute value of the difference between the horizontal coordinate is greater than the absolute value of the difference between the vertical coordinate, then the absolute value of the difference between the horizontal coordinate is the calculated secondary polyline distance between each secondary network flow data sample point and the other secondary network flow data sample points, if the absolute value of the difference between the horizontal coordinate is less than the absolute value of the difference between the vertical coordinate, then the absolute value of the difference between the vertical coordinate is the calculated secondary polyline distance between each secondary network flow data sample point and the other secondary network flow data sample points.

8. The network environment detection method based on unsupervised learning according to claim 6, characterized in that: The process of judging whether the network environment is abnormal based on the results of the first network environment detection and the second network environment detection includes: When both the first network environment test and the second network environment test are normal, the network environment is normal; When the primary network environment detection is normal and the secondary network environment detection is abnormal, the primary network environment detection is abnormal and the secondary network environment detection is normal, and both the primary network environment detection and the secondary network environment detection are abnormal, the network environment is abnormal.

Images