Encrypted traffic anomaly detection method of power Internet of Things and related device
Through information entropy screening and Partial Seeded K-Means algorithm dynamically characterize the security baseline of encrypted traffic. Combined with the comparison of closest neighbor similarity in the cluster, the problems of difficult and high delay in encryption traffic detection in the existing technology are solved, and efficient anomalies are achieved. Traffic identification and dynamic adaptation.
Patent Information
- Application Number
- CN202410975774.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-26
AI Technical Summary
Existing encrypted traffic detection technology is difficult to capture malicious traffic in an encrypted environment, and it has high analysis delay and computing overhead, which makes application difficult.
The feature filtering method based on information entropy and the Partial Seeded K-Means algorithm are used to dynamically characterize the security baseline of normal service encrypted traffic, and abnormal detection is performed through the in-cluster closest similarity comparison method.
It realizes efficient identification of abnormal traffic in encrypted traffic, has good scenario adaptability and scalability, has low requirements for training data, can dynamically adjust the security baseline, and reduces calculation overhead and false alarm rate.
Smart Images

Figure CN119030742A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to a flow detection method, and specifically to a method and related device for detecting anomaly in encrypted flow of electric power Internet of Things. Background Art
[0002] Network traffic detection technology is of great significance to network management, service quality assurance and network security. With the continuous development of encryption technology, the amount and proportion of encrypted traffic in Internet traffic are increasing. Since the characteristics of encrypted traffic will change, traditional traffic detection methods are usually difficult to reproduce in an encrypted environment. Therefore, how to perform effective traffic detection on encrypted traffic and detect malicious and abnormal traffic has become an important challenge in the field of network security.
[0003] At present, encrypted traffic detection technology mainly includes three technical routes: encryption algorithm differentiation of encrypted traffic, malicious feature detection of encrypted traffic based on machine learning, and malicious ciphertext detection of encrypted traffic based on cryptography. However, these existing technical routes are still difficult to capture malicious traffic, and there are high analysis delays and computing overheads, which makes some methods still difficult to apply. Summary of the invention
[0004] In response to the technical problems that existing encrypted traffic detection technologies have difficulty in capturing malicious traffic and are difficult to apply, this application provides a method and related devices for detecting anomalies in encrypted traffic in the power Internet of Things.
[0005] In order to achieve the above objectives, this application adopts the following technical solutions:
[0006] In the first aspect, the present application proposes a method for detecting anomalies in encrypted traffic of power Internet of Things, comprising:
[0007] Obtain the network traffic data to be detected;
[0008] According to the effective cluster center, determine the feature vector v of the network traffic data to be detected new Candidate cluster labels of
[0009] The cluster whose label is the candidate cluster label in the encrypted traffic session vector set is taken as the candidate cluster, and the nearest neighbor sample in the cluster is determined from the candidate cluster;
[0010] According to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Is it abnormal?
[0011] The method for obtaining the effective cluster center and the encrypted traffic session vector set includes:
[0012] Take the encrypted traffic data without abnormalities as samples, divide the samples into sessions, obtain the sample set S, and filter out the effective feature attribute set based on information entropy;
[0013] The sample set S is converted into an encrypted traffic session vector set V according to the valid feature attribute set;
[0014] Use the Partial Seeded K-Means algorithm to divide the encrypted traffic session vector set V into multiple clusters and compare them with the feature vector v new The cluster center with the closest distance is taken as the effective cluster center.
[0015] Furthermore, the encrypted traffic data without abnormalities is used as a sample, and a valid feature attribute set is screened out based on information entropy, including:
[0016] Perform multiple random sampling on the sample set S to obtain multiple sampling samples;
[0017] For each sample, calculate the information entropy of the feature attribute:
[0018]
[0019] Among them, H(c i ) is the characteristic attribute c i The information entropy in a sample is is the characteristic attribute c i The probability of a value appearing in the total sample, j = 1, 2, ..., k, where k is the number of different values;
[0020] Arrange the characteristic entropy of the characteristic attributes in ascending order to obtain the ordered sequence C of the characteristic attributes sampled for the tth time t ; Where w is the total number of feature attributes, and the maximum value of t is the total number of random sampling times P;
[0021] According to the ordered sequence C of the characteristic attributes of each sampling t , get the ordered sequence set W of feature attributes corresponding to P random samplings;
[0022] The effective feature attribute set C is obtained by the following formula valid :
[0023]
[0024] Among them, p(c j ) is the characteristic attribute c j The ordered sequence C of characteristic attributes of each sample in the set W t The probability of appearing in the first T feature attributes with the smallest information entropy is , where T is the threshold of the number of valid feature attributes.
[0025] Furthermore, the value range of the effective feature attribute number threshold T is [0.5w, 0.9w].
[0026] Furthermore, the Partial Seeded K-Means algorithm is used to divide the encrypted traffic session vector set V into multiple clusters, including:
[0027] (1-1) Split the encrypted traffic session vector set V into a sample subset D' divided by known label clusters and a sample subset D-D' divided by unknown clusters;
[0028] (1-2) Calculate each known label cluster C i The cluster center μ of the samples in (1≤i≤l) i ; Where l is the number of clusters with known labels;
[0029] (1-3) Calculate each sample x in the sample subset D-D' j With cluster center μ i The distance between the cluster center μ i The sample with the largest value in the minimum distance is used as the new cluster center μ l+1 , and μ l+1 as the new cluster center;
[0030] (1-4) Repeat steps (1-3) until kl samples are selected to form a new cluster center set {μ l+1 ,μ l+2 ,...,μ k};
[0031] (1-5) Cluster center μ i and the cluster center set {μ l+1 ,μ l+2 ,...,μ k}Merge as the initial cluster center set;
[0032] (1-6) Calculate each sample x in the sample subset D-D' j and the cluster centers μ in the initial cluster center set i The distance d ji ;
[0033] (1-7) Determine the sample x based on the closest cluster center j The cluster label λ j , and the sample x j into the corresponding cluster C λj ;
[0034] (1-8) Calculate the new cluster center μ i ', if μ i '≠μ i , μ iUpdate to μ i 'value;
[0035] (1-9) Repeat steps (1-6) to (1-8) until the values in the k clusters are no longer updated, and the cluster division C = {C 1 ,C 2 ,...,C k}.
[0036] Furthermore, according to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Whether it is abnormal, including:
[0037] (2-1) Determine the feature vector v based on the closest cluster center new The candidate cluster labels λ new :
[0038] λ new =argmin i∈{1,2,...,k} d(v new ,μ i )
[0039] (2-2) From the candidate cluster Filter out the distance feature vector v new The nearest δ vectors form the nearest sample set V in the cluster close ;
[0040] (2-3) The feature vector v new Merge candidate cluster Repeat step (2-2) to obtain the nearest sample set V in the cluster close Each sample v i The nearest sample set of , and calculate the feature vector v new and sample v i The intra-cluster compactness of Then execute steps (2-4), otherwise, λ new =None, execute steps (2-5); where Compactness(v new ) is the feature vector v new After clustering, the feature vector v new The intra-cluster compactness of is the maximum value of the intra-cluster density of the samples in the set of the nearest neighboring samples in the cluster;
[0041] (2-4) Calculate the eigenvector v new and the nearest sample set V in the cluster close Medium sample v i The separation between the centers of the other k-1 clusters; if Then execute step (2-5), otherwise, λ new =None, execute step (2-5); where Seperation(v new ) is the feature vector v new The separation from other cluster centers, It is the minimum value of the separation between the sample in the nearest sample set within the cluster and the center of other clusters;
[0042] (2-5) If λ new =None, then the eigenvector v new is the abnormal encrypted traffic, otherwise, the feature vector v new Encrypt traffic for non-anomalous traffic.
[0043] Further,
[0044]
[0045] Where: d(v i ,v j ) is v i and v j The Euclidean distance of .
[0046] In the second aspect, the present application proposes a power Internet of Things encrypted traffic anomaly detection system, comprising:
[0047] An acquisition module, used to acquire network traffic data to be detected;
[0048] Candidate module, used to determine the feature vector v of the network traffic data to be detected based on the effective cluster center new Candidate cluster labels of
[0049] A neighbor module, used for taking a cluster whose label in the encrypted traffic session vector set is a candidate cluster label as a candidate cluster, and determining the nearest neighbor sample in the cluster from the candidate cluster;
[0050] The judgment module is used to determine the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Is it abnormal?
[0051] The method for obtaining the effective cluster center and the encrypted traffic session vector set includes:
[0052] Take the encrypted traffic data without abnormalities as samples, divide the samples into sessions, obtain the sample set S, and filter out the effective feature attribute set based on information entropy;
[0053] The sample set S is converted into an encrypted traffic session vector set V according to the valid feature attribute set;
[0054] Use the Partial Seeded K-Means algorithm to divide the encrypted traffic session vector set V into multiple clusters and compare them with the feature vector v new The cluster center with the closest distance is taken as the effective cluster center.
[0055] Furthermore, according to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Whether it is abnormal, including:
[0056] (2-1) Determine the feature vector v based on the closest cluster center new The candidate cluster labels λ new :
[0057] λ new =argmin i∈{1,2,...,k} d(v new ,μ i )
[0058] (2-2) From the candidate cluster C λnew Filter out the distance feature vector v new The nearest δ vectors form the nearest sample set V in the cluster close ;
[0059] (2-3) The feature vector v new Merge into candidate cluster C λnew Repeat step (2-2) to obtain the nearest sample set V in the cluster close Each sample v i The nearest sample set of , and calculate the feature vector v new and sample v i The compactness within the cluster; if Compactness(v new )≤argmax vi∈Vclose Compactness(v i ), then execute steps (2-4), otherwise, λ new =None, execute steps (2-5); where Compactness(v new ) is the feature vector v new After clustering, the feature vector v new The intra-cluster compactness, argmax vi∈Vclose Compactness(v i ) is the maximum value of the intra-cluster compactness of the samples in the set of the nearest neighboring samples in the cluster;
[0060] (2-4) Calculate the eigenvector v new and the nearest sample set V in the cluster closeMedium sample v i The separation between the centers of the other k-1 clusters; if Seperation(v new )≥argmin vi∈Vclose Seperation(v i ), then execute step (2-5), otherwise, λ new =None, execute step (2-5); where Seperation(v new ) is the feature vector v new Separation from other cluster centers, argmin vi∈Vclose Seperation(v i ) is the minimum value of the separation between the sample in the nearest sample set within the cluster and the center of other clusters;
[0061] (2-5) If λ new =None, then the eigenvector v new is the abnormal encrypted traffic, otherwise, the feature vector v new Encrypt traffic for non-anomalous traffic.
[0062] In a third aspect, the present application provides an electronic device, comprising:
[0063] Memory for storing computer programs;
[0064] A processor is used to implement the steps of the above-mentioned method for detecting anomalies in encrypted traffic of the power Internet of Things when executing the computer program.
[0065] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for detecting anomalies in encrypted traffic in the power Internet of Things are implemented.
[0066] Compared with the prior art, this application has the following beneficial effects:
[0067] This application proposes a method for detecting anomalies in encrypted traffic in the power Internet of Things. According to the distribution law of the encrypted traffic data of normal business itself, a method based on information entropy is used to screen the effective feature set of encrypted traffic sessions for machine learning. A Partial Seeded K-Means unsupervised clustering learning method is also designed to dynamically characterize the security baseline of normal business encrypted traffic. Finally, a method based on the nearest neighbor similarity comparison within the cluster is designed to perform baseline detection on abnormal encrypted traffic. This application has the advantages of good scene adaptability and scalability, as well as low requirements for training data. In practical applications, when the business scene changes, the security baseline can be dynamically adjusted by relearning, and the effectiveness of using clustering to characterize the security baseline of normal business encrypted traffic can be improved by utilizing the prior knowledge of the business and application labels corresponding to some sessions in the normal encrypted traffic.
[0068] This application also proposes a power Internet of Things encrypted traffic anomaly detection system, electronic device and computer storage medium, which have all the advantages of the above-mentioned power Internet of Things encrypted traffic anomaly detection method. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0070] Figure 1 This is a first flow chart of the method for detecting anomaly in encrypted traffic of the power Internet of Things in this application;
[0071] Figure 2 This is a second flow chart of the method for detecting anomaly in encrypted traffic of the power Internet of Things in this application;
[0072] Figure 3 This is a connection diagram of the power Internet of Things encrypted traffic anomaly detection system for this application. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.
[0074] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0075] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0076] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
[0077] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.
[0078] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0079] IoT access gateways can be divided into high-end and low-end types. Among them, the high-end type uses SSL, national secret SSL and other technologies to realize the identity authentication, network access control and transmission channel encryption of IoT terminals or edge IoT agents, and is a protective device to ensure the security of power IoT terminal access. The low-end type is an isolation gateway device that integrates two security protection measures, SSL, national secret SSL security access and network gate isolation based on dual-machine circuit switching. It is mainly deployed inside the substation and is used for local wireless terminal access to the substation management information regional network. Compared with the high-end IoT access gateway, the single-machine procurement cost is lower.
[0080] At present, encrypted traffic detection technology is mainly divided into three technical routes, namely, encryption algorithm differentiation of encrypted traffic, malicious feature detection of encrypted traffic based on machine learning, and malicious ciphertext detection of encrypted traffic based on cryptography:
[0081] (1) Differentiate the encryption algorithms of encrypted traffic. There are many different encryption algorithms used in current encrypted traffic. Different encryption methods will produce different characteristics and require different identification methods. Currently, there is no universal method that can be applied to all types of encrypted traffic. Therefore, it is necessary to first distinguish the encrypted traffic according to different levels of precision, for example, distinguishing between AES, DES, 3DES, Grain and other cryptographic systems, to provide support for subsequent abnormal traffic and malicious traffic detection.
[0082] (2) Detection of malicious features of encrypted traffic based on machine learning. Different types of traffic have different network behavior patterns, and this information is intuitively reflected in their data packets. For example, malicious encrypted traffic and benign encrypted traffic will generate different data packet header information when performing handshake protocols, and different types of malicious encrypted traffic also have differences in their average packet length, inter-packet interval, etc. These differences in behavior patterns are an important basis for detecting malicious traffic in encrypted traffic. Therefore, the behavior patterns of malicious traffic can be extracted from the collected encrypted traffic, further summarized as the characteristics of malicious traffic, and identified using machine learning models.
[0083] Malicious feature detection of encrypted traffic based on machine learning is to extract malicious features from encrypted traffic, thereby constructing a malicious feature set, and inputting it into the training model as a training set, and obtaining the expected detection accuracy through methods such as model design and parameter tuning. This type of method has the advantages of high efficiency and wide applicability. Therefore, malicious feature detection of encrypted traffic based on machine learning has become a current research hotspot. First, it is necessary to collect the required data set. There are usually two methods: public traffic data set and private traffic data set. The public traffic data set and private traffic data set include both malicious or abnormal encrypted traffic and benign or normal encrypted traffic; then, the collected traffic data is preprocessed, including traffic cleaning, traffic segmentation, feature set construction and traffic conversion; finally, the data set is used as input, and the machine model is used to learn the malicious features of malicious traffic, and malicious traffic or abnormal traffic is detected through iterative training.
[0084] However, the core of malicious feature detection of encrypted traffic based on machine learning lies in high-quality datasets, just as the open source datasets ImageNet and COCO provide important support for the rapid development of computer vision. However, it is difficult to capture malicious traffic, and the technical difficulty and engineering workload of manual reproduction and correct labeling are huge. Therefore, encrypted traffic detection lacks some high-quality open source datasets that contain correct labels and have detailed classification on malicious traffic. In addition, the emergence of new protocols such as TLS1.3 has further reduced the proportion of plaintext in the header of encrypted traffic data packets, which has brought further challenges to malicious feature detection of encrypted traffic based on machine learning. Furthermore, the rapid progress of adversarial sample technology in the field of machine learning has further reduced the difficulty of traffic obfuscation and disguise. Attackers can disguise attack traffic as benign or normal traffic by learning benign or normal traffic data and adding appropriate noise, and even mislead machine learning detection models to identify normal traffic as malicious traffic; the ability of malicious feature detection models for encrypted traffic based on machine learning will gradually decline over time. At the same time, the distribution of normal / abnormal, benign / malicious traffic in the training / test set must conform to the distribution in the real environment, otherwise the actual application effect of the machine learning detection model will be greatly reduced.
[0085] (3) Cryptography-based detection of malicious ciphertext in encrypted traffic. This detection method can divide the searchable encryption system into public key searchable encryption system and symmetric searchable encryption system. The difficulties of searchable encryption technology include ciphertext retrieval and ciphertext calculation. Among them, ciphertext retrieval can be divided into single keyword retrieval, multi-keyword retrieval, fuzzy keyword retrieval and interval retrieval. Ciphertext calculation can be divided into homomorphic encryption and function (attribute) encryption. Usually, encrypted traffic detection works in the application layer or TCP / IP layer, and requires deep integration in user data protection, encryption rule protection, key management, and retrieval algorithm efficiency. First of all, although searchable encryption technology can prevent user data leakage, it cannot guarantee that users can evade detection through obfuscation after mastering the detection rules. When applying searchable encryption technology to traffic detection, it is also necessary to protect the encryption rules from leakage. In addition, the problem of detection efficiency also restricts the application of searchable encryption in malicious traffic detection. Secondly, searchability requires keys, and rule protection also requires keys. The generation and management of these two keys is not a simple combination of protocols. It requires strict cryptographic security protocols to ensure it. It also needs to be deeply integrated with the network traffic deep packet inspection (DPI) mechanism on the network middlebox (such as firewalls and intrusion detection devices). Finally, the detection of malicious traffic also requires efficient algorithms to ensure the practicality of the detection solution.
[0086] Based on the above situation, the present application proposes a method and related device for detecting anomalies in encrypted traffic of the power Internet of Things. The present application is described in detail below in conjunction with the embodiments and drawings.
[0087] like Figure 1 As shown, it is a first flow chart of a method for detecting anomaly of encrypted traffic in the power Internet of Things of this application, which may include:
[0088] S101, obtaining network traffic data to be detected.
[0089] S102, determining the feature vector v of the network traffic data to be detected according to the effective cluster center new The candidate cluster labels.
[0090] S103: taking a cluster in the encrypted traffic session vector set whose label is the candidate cluster label as a candidate cluster, and determining the nearest neighbor sample in the cluster from the candidate cluster.
[0091] S104, according to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Is it abnormal?
[0092] It should be noted that intra-cluster compactness is usually used to measure the compactness of sample points in the same cluster, that is, similarity. In clustering algorithms such as K-Means, intra-cluster compactness can be measured by calculating the average distance from sample points in the cluster to the cluster center. Inter-cluster separation is used to measure the degree of difference between different clusters. In cluster analysis, inter-cluster separation is usually measured by calculating the average distance between sample points in different clusters or the average distance between cluster centers. For example, the average inter-cluster distance can be used as an indicator of inter-cluster separation, which reflects the degree of separation between different clusters. The larger the average inter-cluster distance, the greater the difference between different clusters, that is, the higher the inter-cluster separation. Intra-cluster compactness reflects the similarity between samples in the same cluster, while inter-cluster separation reflects the degree of difference between different clusters. By comprehensively considering these two indicators, the quality of clustering effect can be comprehensively evaluated, so as to select the best clustering model and parameter settings.
[0093] The method for obtaining the effective cluster center and the encrypted traffic session vector set may include:
[0094] (1) Take the non-abnormal encrypted traffic data as the sample, divide the sample into sessions, obtain the sample set S, and filter out the effective feature attribute set based on information entropy;
[0095] It should be noted that in network traffic analysis, a session is usually defined as a set of data packets with the same five-tuple (source IP address, source port, destination IP address, destination port, transport layer protocol, such as TCP or UDP), which represent a complete communication process.
[0096] (2) Convert the sample set S into the encrypted traffic session vector set V according to the valid feature attribute set;
[0097] (3) Using the Partial Seeded K-Means algorithm, the encrypted traffic session vector set V is divided into multiple clusters, and the feature vector v new The cluster center with the closest distance is taken as the effective cluster center.
[0098] It should be noted that this application is equivalent to using the Partial Seeded K-Means algorithm to learn the baseline of normal business encrypted traffic. The baseline refers to the normal behavior benchmark of industrial control system users, equipment, applications, and traffic. Baseline-based encrypted traffic detection is to establish a baseline for the encrypted network communication traffic of normal business and compare it with the encrypted network communication traffic generated in real time. When the network communication line to be tested deviates from the baseline and exceeds the preset threshold, an alarm message is sent to the user system, and the user evaluates and decides on the effectiveness of the alarm. The baseline-based encrypted traffic detection method is different from the detection method based on malicious / abnormal encrypted traffic feature matching or supervised machine learning. It only needs to collect and learn the encrypted traffic of normal business to build a security baseline, and the security baseline can be dynamically constructed by relearning according to business changes. In addition, the baseline-based encrypted traffic detection method has the ability to detect business encrypted traffic anomalies caused by abnormal terminal equipment or manual misoperation, as well as unknown attacks on internal application systems by attackers attempting to penetrate the IoT access gateway based on business encrypted traffic.
[0099] This application uses an unsupervised learning method to characterize the baseline of normal business encrypted traffic, and uses the baseline to detect abnormal encrypted traffic. In response to the problem of feature selection for encrypted traffic baseline learning, a feature screening method based on information entropy is proposed to select features with stronger characterization capabilities for the normal business encrypted traffic baseline from normal business encrypted traffic. In response to the problem of normal business encrypted traffic baseline learning algorithm, a Partial Seeded K-Means algorithm is proposed to make full use of the prior knowledge of business and application labels corresponding to some sessions in normal encrypted traffic, and improve the effectiveness of using clustering to characterize the baseline of normal business encrypted traffic. In response to the problem of how to effectively use the baseline to detect abnormal encrypted traffic, a detection method based on nearest neighbor similarity comparison within the cluster is proposed to avoid the high computational overhead caused by re-clustering when distinguishing each encrypted traffic session to be detected.
[0100] like Figure 2 As shown, it is a second flow chart of a method for detecting anomaly of encrypted traffic in the power Internet of Things of this application, which may include:
[0101] S201, effective feature set screening based on information entropy
[0102] The present invention adopts an unsupervised learning method based on clustering, and the quality of the baseline of clustering characterization directly affects the final detection effect. There are two factors that affect the quality of clustering: one is that the data dimension is too high. When there are too many characteristic attributes of the data samples involved in the training, the operation time of the clustering algorithm will increase exponentially, and there are also higher requirements for the hardware configuration. In particular, when some redundant characteristic attributes are involved in the training, it will seriously affect the clustering effect, and then affect the accuracy of the later detection. Second, the attributes of each sample involved in the clustering also have different abilities to classify data. Feature sets with stronger data classification capabilities should be screened and retained for training and detection.
[0103] The feature extraction method based on information entropy is:
[0104] Information entropy refers to the probability of occurrence in discrete random events, and its purpose is to deal with the problem of measuring the amount of information. The lower the information entropy, the more orderly the event is. On the contrary, if the information entropy is higher, it means that the event is more chaotic. The formula for calculating information entropy is as follows:
[0105]
[0106] Where H(X) is the sample X = {x i , (i=1,2,...,n)}, p(x i ) is x i The probability of appearing in the total sample. When the probability of occurrence is 1, the information entropy is equal to 0, that is, the sample is more orderly; when the probability of occurrence is close to 0 (but not 0), H(X) is equal to the total number of samples, the information entropy is the largest, that is, the sample is more chaotic.
[0107] Normal sensor terminals that communicate with the power IoT access gateway usually only carry relatively simple and fixed application functions and have relatively monotonous and fixed behavior patterns. Therefore, it can be considered that the characteristic attribute values of normal business encrypted traffic are relatively fixed and ordered, that is, the information entropy of the characteristic attributes is low. When an abnormality occurs or a network attack occurs, the characteristic attribute values of the encrypted traffic generated by the terminal will deviate from the fixed or ordered values of normal business, and the values will be more random and disordered. Taking the simplest flood attack as an example, the number of uplink data packets and total data packets between units will change dramatically. It should be noted that the sensor terminal is a device deployed inside or near the main body of the monitored object to collect and measure the status, electrical and environmental quantities of the measuring equipment. It has simple data processing, control and communication functions as needed, and is generally not configured with edge computing functions.
[0108] The feature attribute screening method adopted in this application is to screen the n attributes with the lowest information entropy of the candidate feature set that can most effectively characterize the normal data baseline through statistical methods based on the normal business encrypted traffic data used for training.
[0109] Assume C = {c i , (i=1,2,...,w)} is the complete set of candidate characteristic attributes of encrypted traffic, c i is the i-th attribute, and w is the total number of characteristic attributes. The encrypted traffic data of normal services is divided into samples by session, and the sample set S = {s i ,(i=1,2,...,m)},s i is the ith session sample, and m is the total number of samples. S is randomly sampled in units of 1000. In order to ensure the sampling effect, the sampling times P should preferably satisfy that the sampled samples cover at least half of the total samples:
[0110]
[0111] Calculate a certain attribute c based on a sample set sampled once i Information entropy in one sampling:
[0112]
[0113] Feature attribute c i The set of values in a sampling is k is the number of different values, It is the probability of a certain feature attribute value appearing in the total sample (that is, the ratio of the number of times a certain feature attribute value appears in a sampling sample to the total number of samples).
[0114] For each sample sampling, the calculation formula of the information entropy of a sampling is used to calculate the information entropy of w characteristic attributes. According to the ascending order of information entropy from small to large, the ordered sequence C of the characteristic attributes of the tth sampling can be obtained. t = <c’ i , (i=1,2,...,w)>, then P samplings can obtain a set of ordered sequences of P feature attributes W={C t ,(t=1,2,...,P)}.
[0115] Set a threshold T for the number of valid feature attributes, and use more than half and no more than 90% of the attributes as valid feature attributes for baseline learning:
[0116]
[0117] Among them, C valid represents the effective feature attribute set obtained through screening, p(c j ) represents the characteristic attribute c j In the set W = {C t ,(t=1,2,...,P)}, the ordered sequence C of characteristic attributes of each sampling tThe probability of appearing in the first T feature attributes with the smallest information entropy. valid It contains the T characteristic attributes with the highest statistical probability of occurrence and the smallest information entropy in P samplings. When the probability of occurrence of two characteristic attributes is the same, C valid Prioritize feature attributes with lower information entropy.
[0118] The definition of the candidate feature set is shown in Table 1:
[0119] Table 1 Candidate set of encrypted traffic session feature attributes
[0120]
[0121]
[0122]
[0123] The candidate set of encrypted traffic session feature attributes used in this application is shown in Table 1. Before using the clustering algorithm to characterize the baseline of normal business encrypted traffic, the effective feature set screening method is first used to obtain an effective feature attribute set C containing T effective features. valid The value range of T is generally [0.5w, 0.9w], w is the total number of feature attributes, and in practical applications, the value of T can be adjusted according to actual business needs, the clustering effect of training data, and the computational time and space load overhead.
[0124] S202, Baseline Learning Method Based on Partial Seeded K-Means Algorithm
[0125] K-Means algorithm is one of the most classic unsupervised learning algorithms and is a clustering method based on partitioning. According to the similarity between specific standard objects, the specified objects are organized into two or more clusters. The basic process of K-Means algorithm is: a set of sample objects is divided into k clusters according to attributes. The algorithm initially determines the value k of the target number of clusters and k initial cluster centers, and classifies the sample objects with the closest distance into one cluster; through an iterative method, the cluster centers are updated one by one until the cluster center no longer changes. In practical applications, the clustering effect of K-Means algorithm is largely affected by the initial cluster center selection method.
[0126] Considering that sample data with known labels can be used to assist in the selection of the initial cluster center, clustering performance can be improved. For example, some scholars proposed to constrain the sample entry into the cluster by constructing two constraint sets, Must-list and cannot-link, but did not constrain the selection of the initial center point. Others proposed the Seeded / Constrained K-Means algorithm, which constrains the selection of the initial center point through the seed, and can also choose to constrain the sample when entering the cluster, but it requires that each cluster needs to have a seed in advance for use.
[0127] In practical applications, only part of the sample data has known labels. Therefore, it is impossible to ensure that each cluster has sample data with known labels as seeds. To solve this problem, this application proposes a Partial Seeded K-Means algorithm. The specific steps of the algorithm are as follows:
[0128] Input: Given a sample set D = {x 1 ,x 2 ,...,x m}, the number of clusters k, the number of clusters with known labels l, l ≤ k, the sample subset D'={x 1 ,x 2 ,...,x m}, the sample subset D-D' divided by the unknown cluster.
[0129] (1-1) Calculate each known label cluster C i The cluster center of the samples in (1≤i≤l)
[0130] (1-2) Calculate each sample x in D-D' j (1≤j≤mn) and the known label cluster center μ i (1≤i≤l), select the sample with the largest value among the minimum distances to the known label cluster center as the new initial cluster center μ l+1 , and μ l+1 as the center of the new known cluster.
[0131] (1-3) Repeat steps (1-2) until kl samples are selected to form a new initial cluster center set {μ l+1 ,μ l+2 ,...,μ k}; Set μ i (1≤i≤l) and {μ l+1 ,μ l+2 ,...,μ k}Merge, a total of k cluster centers are used as the initial cluster center set.
[0132] (1-4) Calculate each sample x in D-D' j (1≤j≤mn) and each cluster center μ i The distance d of (1≤i≤k) ji =||x j -μ i || 2 .
[0133] (1-5) Determine the sample x based on the closest cluster center j The cluster label λ of (1≤j≤mn) j =argmin i∈{1,2,...,k-l} d ji , and 1≤j≤mn, and the sample x j into the corresponding clusters,
[0134] (1-6) Calculate the new cluster center If μ i '≠μ i , μ i Update to μ i ' value.
[0135] (1-7) Repeat steps (1-4) to (1-6) until the values in the k clusters are no longer updated.
[0136] Output: Cluster partition C as clustering result = {C 1 ,C 2 ,...,C k}.
[0137] The Partial Seeded K-Means algorithm uses sample clusters with some known labels to divide samples into several subsets, that is, using Partial Seed to determine the initial cluster center. Considering the limitations of prior labels, the encrypted traffic of a normal business may contain multiple unknown applications, and the encrypted traffic session of the business may have multiple modes. Therefore, the seed is no longer used for constraints when the samples are subsequently clustered. In other words, samples with known labels may be classified into the cluster where the original label is located during the clustering process, or they may be classified into a newly generated cluster.
[0138] When using the Partial Seeded K-Means algorithm to learn the baseline of normal business encrypted traffic, first use the effective feature set screening method to obtain a set C containing T effective feature attributes. valid Then, the session set S = {s i ,(i=1,2,...,m)}, according to the effective feature attribute set C valid Transformed into an encrypted traffic session vector set V containing T attribute values = {vi ,(i=1,2,...,m)}, where s i is the i-th sample, m is the total number of samples; finally, V={v i ,(i=1,2,...,m)} as input, and select the target cluster number k and the number of clusters with known labels l, and run the algorithm. In actual applications, l can be determined based on the number of known normal business and application labels, and the session vectors corresponding to these businesses and applications can be marked with seeds. You can also adjust the k value for multiple trainings and observe the clustering results to obtain the k clusters with the best clustering effect, and use them as the baseline for subsequent anomaly detection.
[0139] It should be noted that the vector distance calculation method used in this application is the Euclidean distance, and the specific calculation method is:
[0140]
[0141] Among them, v i and v j Represents two vectors containing valid feature attributes of encrypted traffic sessions, v i (c t ) represents the vector v i In c t The value of a feature attribute.
[0142] S203, anomaly detection based on nearest neighbor similarity comparison within a cluster
[0143] The Partial Seeded K-Means algorithm is used to divide the valid feature vector set V of normal business encrypted traffic training data used for baseline learning into C = {C 1 ,C 2 ,...,C k}, and the centers of the k clusters are {μ 1 ,μ 2 ,...,μ k}. new is the effective feature vector of the new unknown encrypted traffic to be detected. new The following steps can be used to detect whether the encrypted traffic represented by the proxy is abnormal.
[0144] Input: C = {C 1 ,C 2 ,...,C k} and the centers of the k clusters {μ 1 ,μ 2 ,...,μ k}, unknown encrypted traffic vector v to be detected new .
[0145] (2-1) Determine v based on the closest cluster center new The candidate cluster labels λ new =argmin i∈{1,2,...,k} d(v new ,μ i ).
[0146] (2-2) According to the calculation formula of Euclidean distance, from Filter out the distance v new The set of the nearest δ vectors V close = {v i ,(i=1,2,...,δ)}, where, V close Called v new The set of nearest neighbor samples in the cluster, Represents the training sample vector set (i.e., the valid feature vector set V in this application) with label λ new The preset value range of δ is generally 5≤δ≤10, which can be adjusted according to the number of actual training samples in practical applications; Capacity When it is less than the preset value of δ,
[0147] (2-3) v new Incorporation Repeat step 2 to get V close = {v i ,(i=1,2,...,δ)}, each sample v i The nearest sample set of , and calculate v new and v i The intra-cluster compactness of :
[0148]
[0149] if (ie v new After entering the cluster new The intra-cluster density of is less than the maximum intra-cluster density of the samples in the set of the nearest neighboring samples in the cluster), then continue with step (2-4); otherwise, new =None, jump to output.
[0150] (2-4) Calculate v new and V close = {v i ,(i=1,2,...,δ)} i The separation between the other k-1 cluster centers is:
[0151]
[0152] if (ie v new If the separation degree between the sample and other cluster centers is greater than the minimum separation degree between the sample and other cluster centers in the nearest sample set within the cluster), then jump to the output; otherwise, new =None.
[0153] Output: According to v new The candidate cluster labels λ new Judge, if λ new =None then v new The encrypted traffic session represented is abnormal encrypted traffic and triggers an alarm, otherwise it is normal encrypted traffic.
[0154] For unknown encrypted traffic sessions to be detected, this application does not use v new Instead of adding the whole sample set and re-calculating the cluster to detect anomalies, we select δ nearest neighbor samples from the candidate cluster and judge the similarity between the intra-cluster compactness and inter-cluster separation of the nearest neighbor samples to determine v new Is it an exception? Therefore, the computational complexity is O(N 2 ) is reduced to O(N).
[0155] In practical applications, network traffic data packets can be obtained, including encrypted traffic data packets and non-encrypted traffic data packets, normal encrypted traffic data packets and abnormal encrypted traffic data packets, training data packets and data packets to be detected. The data packets captured by the data packet acquisition module are processed, and the data packet flows and sessions are classified according to the five-tuple <source IP, source port, destination IP, destination port, transport layer protocol> information in the data packet. Then the sessions classified by the data packet processing module are filtered, and for complete sessions that comply with the encrypted communication protocol agreed upon by the power IoT access gateway and the sensor terminal and have no data packet loss, the values corresponding to the characteristic attributes are extracted and recorded according to the candidate set of encrypted traffic session characteristic attributes described in Table 1. The normal business encrypted traffic security baseline learning process, by using the method of the present application, uses training data to screen and retain a characteristic attribute set with stronger data classification capabilities, and converts the sessions extracted and recorded by the encrypted traffic session feature extraction module into a one-dimensional vector form containing only valid characteristic attribute values. Using the one-dimensional vector of the valid characteristic attribute values of all training data sessions, k cluster divisions of the clustered training data are obtained, and used as a security baseline. To determine whether there is an anomaly in the encrypted traffic to be detected, first use the effective feature set obtained by the effective feature set screening module to convert the session of the encrypted traffic to be detected into a one-dimensional vector form that only contains valid feature attribute values. Then use the security baseline obtained by the baseline learning module to determine whether there is an anomaly in the encrypted traffic to be detected based on the security baseline.
[0156] In summary, compared with the method for detecting malicious encrypted traffic features using supervised learning, this application has the following advantages: (1) Better scenario adaptability: A large number of new power system business applications use encrypted communication technology, generating a large amount of encrypted traffic. This application is aimed at learning the normal encrypted traffic of these businesses, and characterizes the security baseline through clustering methods to achieve the identification of anomalies; while the supervised learning method focuses on learning various types of malicious encrypted traffic and then implements a blacklist-like mechanism. In actual applications, it may cause a large number of false alarms due to insufficient learning of the normal encrypted traffic of the target business environment. (2) Better scalability: This application uses an online learning method. In actual applications, users can specify a security baseline learning period. After the learning period is completed, it enters a baseline-based anomaly detection period. When the actual application business scenario changes, it can be adapted by re-entering the learning period to construct a new security baseline; the malicious encrypted traffic feature detection method based on supervised learning usually uses an offline learning method. When it is necessary to learn the newly recognized malicious encrypted traffic, the classifier needs to be retrained and updated offline. In particular, some detection methods using deep learning have higher difficulty and cost in incremental learning. (3) Lower requirements for training data: The malicious encrypted traffic that the malicious encrypted traffic feature detection method based on supervised learning relies on is difficult to capture, and the technical difficulty and engineering workload of manual reproduction and correct labeling are huge. In addition, new malicious encrypted traffic emerges in an endless stream; however, the training data of this application only relies on the encrypted traffic of normal business that is easier to obtain in large quantities, and based on the prior knowledge of business and application labels, some encrypted traffic is labeled with seed labels. The requirements for training data and the computational difficulty of training pre-data processing are lower.
[0157] In addition, the present application abandons the blacklist-like mechanism for learning various types of malicious encrypted traffic, and instead learns the encrypted traffic of normal business. The clustering method is used to characterize the security baseline to achieve the recognition of anomalies. The data used for machine learning training only relies on the encrypted traffic of normal business that is easier to obtain in large quantities, avoiding the technical difficulty and huge engineering workload of malicious traffic being difficult to capture, artificially reproduce and mark, and can greatly reduce the difficulty and cost of the detection model training stage. In addition, some data can be labeled according to the user's prior knowledge of business and application labels to guide the initial cluster center screening of clustering algorithm training, effectively improving the performance of the clustering algorithm. When using the security baseline for anomaly detection, there is no need to re-calculate the clustering calculation, but a method based on the nearest similarity comparison within the cluster is used, which greatly reduces the computational time and space overhead and the equipment hardware performance requirements, thereby saving the equipment implementation cost, thereby generating direct economic benefits. The present application can effectively reduce the threat risk brought by encrypted traffic to the power grid side and thus reduce the overall power grid security management and operation costs, and therefore can also generate indirect economic benefits.
[0158] Furthermore, the construction of new power systems is accelerating, and a large number of new businesses are facing the need for safe grid connection. It is necessary to achieve secure access and confidential communication of various new IoT terminals, and to ensure that data encryption does not bring new blind spots to security monitoring. This application effectively makes up for the shortcomings of traditional means of monitoring plaintext traffic, and proposes a method, device and equipment for bypassing mirrored encrypted traffic for power IoT access gateways, using unsupervised learning methods to characterize the security baseline of normal business encrypted traffic, and using the security baseline to detect abnormal encrypted traffic, ensuring the access security and behavioral reliability of various types of IoT terminal devices, which is of great significance for ensuring the safe and stable operation of the power grid and providing reliable power supply for economic and social development. In addition, the application of this application in the power system can effectively prevent covert attacks implemented through encrypted traffic, support the observable, measurable, adjustable and controllable needs for encrypted traffic in the power Internet of Things, and serve as a model and demonstration for the network security protection work of the energy industry and other large enterprises that are also facing new network security protection challenges.
[0159] Based on the above-mentioned power Internet of Things encrypted traffic anomaly detection method, Figure 3 As shown, the present application also proposes a power Internet of Things encrypted traffic anomaly detection system, which may include:
[0160] An acquisition module, used to acquire network traffic data to be detected;
[0161] Candidate module, used to determine the feature vector v of the network traffic data to be detected based on the effective cluster center new Candidate cluster labels of
[0162] A neighbor module, used for taking a cluster whose label in the encrypted traffic session vector set is a candidate cluster label as a candidate cluster, and determining the nearest neighbor sample in the cluster from the candidate cluster;
[0163] The judgment module is used to determine the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Is it abnormal?
[0164] The method for obtaining the effective cluster center and the encrypted traffic session vector set includes:
[0165] Take the encrypted traffic data without abnormalities as samples, divide the samples into sessions, obtain the sample set S, and filter out the effective feature attribute set based on information entropy;
[0166] The sample set S is converted into an encrypted traffic session vector set V according to the valid feature attribute set;
[0167] Use the Partial Seeded K-Means algorithm to divide the encrypted traffic session vector set V into multiple clusters and compare them with the feature vector v new The cluster center with the closest distance is taken as the effective cluster center.
[0168] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0169] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.
[0170] An electronic device provided in an embodiment of the present application includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of the method for detecting anomalies in encrypted traffic of the power Internet of Things described in any of the above embodiments are implemented.
[0171] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection (including wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s).
[0172] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the method for detecting anomalies in encrypted traffic in the power Internet of Things as described in any of the above embodiments are implemented.
[0173] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.
[0174] For the description of the relevant parts of the power Internet of Things encrypted flow anomaly detection system, electronic device, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the power Internet of Things encrypted flow anomaly detection method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.
[0175] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for detecting abnormal encrypted traffic in power Internet of Things, characterized in that: include: Obtain the network traffic data to be detected; According to the effective cluster center, determine the feature vector v of the network traffic data to be detected new Candidate cluster labels of The cluster whose label is the candidate cluster label in the encrypted traffic session vector set is taken as the candidate cluster, and the nearest neighbor sample in the cluster is determined from the candidate cluster; According to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Is it abnormal? The method for obtaining the effective cluster center and the encrypted traffic session vector set includes: Take the encrypted traffic data without abnormalities as samples, divide the samples into sessions, obtain the sample set S, and filter out the effective feature attribute set based on information entropy; The sample set S is converted into an encrypted traffic session vector set V according to the valid feature attribute set; Use the Partial Seeded K-Means algorithm to divide the encrypted traffic session vector set V into multiple clusters and compare them with the feature vector v new The cluster center with the closest distance is taken as the effective cluster center.
2. According to claim 1, the method for detecting abnormal encrypted traffic in the power Internet of Things is characterized in that: The method uses the normal encrypted traffic data as a sample and selects a valid feature attribute set based on information entropy, including: Perform multiple random sampling on the sample set S to obtain multiple sampling samples; For each sample, calculate the information entropy of the feature attribute: Among them, H(c i ) is the characteristic attribute c i The information entropy in a sample is is the characteristic attribute c i The probability of a value appearing in the total sample, j = 1, 2, ..., k, where k is the number of different values; Arrange the characteristic entropy of the characteristic attributes in ascending order to obtain the ordered sequence C of the characteristic attributes sampled for the tth time t ; Where w is the total number of feature attributes, and the maximum value of t is the total number of random sampling times P; According to the ordered sequence C of the characteristic attributes of each sampling t , get the ordered sequence set W of feature attributes corresponding to P random samplings; The effective feature attribute set C is obtained by the following formula valid : Among them, p(c j ) is the characteristic attribute c j The ordered sequence C of characteristic attributes of each sample in the set W t The probability of appearing in the first T feature attributes with the smallest information entropy, where T is the threshold of the number of valid feature attributes.
3. According to claim 2, the method for detecting abnormal encrypted traffic in the power Internet of Things is characterized in that: The value range of the effective feature attribute number threshold T is [0.5w, 0.9w].
4. According to claim 3, the method for detecting abnormal encrypted traffic in the power Internet of Things is characterized in that: The PartialSeeded K-Means algorithm is used to divide the encrypted traffic session vector set V into multiple clusters, including: (1-1) Split the encrypted traffic session vector set V into a sample subset D' divided by known label clusters and a sample subset D-D' divided by unknown clusters; (1-2) Calculate each known label cluster C i The cluster center μ of the samples in (1≤i≤l) i ; Where l is the number of clusters with known labels; (1-3) Calculate each sample x in the sample subset D-D' j With cluster center μ i The distance between the cluster center μ i The sample with the largest value in the minimum distance is used as the new cluster center μ l+1 , and μ l+1 as the new cluster center; (1-4) Repeat steps (1-3) until kl samples are selected to form a new cluster center set {μ l+1 ,μ l+2 ,...,μ k }; (1-5) Cluster center μ i and the cluster center set {μ l+1 ,μ l+2 ,...,μ k }Merge as the initial cluster center set; (1-6) Calculate each sample x in the sample subset D-D' j and the cluster centers μ in the initial cluster center set i The distance d ji ; (1-7) Determine the sample x based on the closest cluster center j The cluster label λ j , and the sample x j Assign to the corresponding cluster (1-8) Calculate the new cluster center μ i ', if μ i '≠μ i , μ i Update to μ i 'value; (1-9) Repeat steps (1-6) to (1-8) until the values in the k clusters are no longer updated, and the cluster partition C = {C1, C2, ..., C k }.
5. According to claim 4, the method for detecting abnormal encrypted traffic in the power Internet of Things is characterized in that: According to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Whether it is abnormal, including: (2-1) Determine the feature vector v based on the closest cluster center new The candidate cluster labels λ new : l new =argmin i∈{1,2,...,k} d(v new ,m i ) (2-2) From the candidate cluster Filter out the distance feature vector v new The nearest δ vectors form the nearest sample set V in the cluster close ; (2-3) The feature vector v new Merge candidate cluster Repeat step (2-2) to obtain the nearest sample set V in the cluster close Each sample v i The nearest sample set of , and calculate the feature vector v new and sample v i The intra-cluster compactness of Then execute steps (2-4), otherwise, λ new =None, execute steps (2-5); where Compactness(v new ) is the feature vector v new After clustering, the feature vector v new The intra-cluster compactness of is the maximum value of the intra-cluster density of the samples in the set of the nearest neighboring samples in the cluster; (2-4) Calculate the eigenvector v new and the nearest sample set V in the cluster close Medium sample v i The separation between the centers of the other k-1 clusters; if Then execute step (2-5), otherwise, λ new =None, execute step (2-5); where Seperation(v new ) is the feature vector v new The separation from other cluster centers, It is the minimum value of the separation between the sample in the nearest sample set within the cluster and the center of other clusters; (2-5) If λ new =None, then the eigenvector v new is the abnormal encrypted traffic, otherwise, the feature vector v new Encrypt traffic for non-anomalous traffic.
6. According to claim 5, the method for detecting abnormal encrypted traffic in the power Internet of Things is characterized in that: Where: d(v i ,v j ) is v i and v j The Euclidean distance of .
7. A power Internet of Things encrypted traffic anomaly detection system, characterized in that: include: An acquisition module, used to acquire network traffic data to be detected; Candidate module, used to determine the feature vector v of the network traffic data to be detected based on the effective cluster center new Candidate cluster labels of A neighbor module, used for taking a cluster whose label in the encrypted traffic session vector set is a candidate cluster label as a candidate cluster, and determining the nearest neighbor sample in the cluster from the candidate cluster; The judgment module is used to determine the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Is it abnormal? The method for obtaining the effective cluster center and the encrypted traffic session vector set includes: Take the encrypted traffic data without abnormalities as samples, divide the samples into sessions, obtain the sample set S, and filter out the effective feature attribute set based on information entropy; The sample set S is converted into an encrypted traffic session vector set V according to the valid feature attribute set; Use the Partial Seeded K-Means algorithm to divide the encrypted traffic session vector set V into multiple clusters and compare them with the feature vector v new The cluster center with the closest distance is taken as the effective cluster center.
8. According to claim 7, the power Internet of Things encrypted flow anomaly detection system is characterized in that: According to the feature vector v new The similarity between the closeness within the cluster and the separation between clusters with the nearest neighboring samples in the cluster is used to determine the feature vector v new Whether it is abnormal, including: (2-1) Determine the feature vector v based on the closest cluster center new The candidate cluster labels λ new : l new =argmin i∈{1,2,...,k} d(v new ,m i ) (2-2) From the candidate cluster Filter out the distance feature vector v new The nearest δ vectors form the nearest sample set V in the cluster close ; (2-3) The feature vector v new Merge candidate cluster Repeat step (2-2) to obtain the nearest sample set V in the cluster close Each sample v i The nearest sample set of , and calculate the feature vector v new and sample v i The intra-cluster compactness of Then execute steps (2-4), otherwise, λ new =None, execute steps (2-5); where Compactness(v new ) is the feature vector v new After clustering, the feature vector v new The intra-cluster compactness, argmax vi∈Vclose Compactness(v i ) is the maximum value of the intra-cluster compactness of the samples in the set of the nearest neighboring samples in the cluster; (2-4) Calculate the eigenvector v new and the nearest sample set V in the cluster close Medium sample v i The separation between the centers of the other k-1 clusters; if Then execute step (2-5), otherwise, λ new =None, execute step (2-5); where Seperation(v new ) is the feature vector v new The separation from other cluster centers, It is the minimum value of the separation between the sample in the nearest sample set within the cluster and the center of other clusters; (2-5) If λ new =None, then the eigenvector v new is the abnormal encrypted traffic, otherwise, the feature vector v new Encrypt traffic for non-anomalous traffic.
9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the method for detecting anomaly of encrypted traffic in the power Internet of Things as described in any one of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for detecting anomalies in encrypted traffic of the power Internet of Things as described in any one of claims 1 to 6 are implemented.
