Method for identifying similar network behavior users based on unsupervised variational autoencoder

Abstract

The application discloses a similar network behavior user identification method based on an unsupervised variational autoencoder, and relates to the technical field of network security. The method is based on super-large network communication metadata, uses an unsupervised variational autoencoder model, learns deep law features of user network behaviors from massive user network flow metadata, abstracts network behaviors of users into high-dimensional vector representations of essential behavior modes, and finally identifies users by using the similarity between vectors. Specifically, the method comprises network flow metadata, network behavior statistical feature extraction, behavior feature image processing, deep learning model training, deep feature extraction and similar behavior user identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a similar network behavior user identification method based on an unsupervised variational autoencoder algorithm. BACKGROUND

[0002] In recent years, with the rapid development of computer communication technology and the continuous popularization of the Internet, Internet users are growing at an alarming rate. According to the 53rd Statistical Report on Internet Development in China released by the China Internet Network Information Center (CNNIC), as of December 2023, the number of Internet users in China has reached 1.092 billion. The rapid development of the Internet has greatly facilitated people's social life, but at the same time, various malicious behaviors such as software ransom, illegal speech, data theft, and cyber attacks are emerging in an endless stream. Malicious users hide among the massive user compliant behaviors, posing a huge challenge to network space security and governance. Network user identification is one of the key issues in network space security governance. Previous work on network user identification can be broadly divided into two categories: one is to directly calculate the similarity between user portrait feature vectors, and the other is to train a model through machine learning methods to identify users. Similarity comparison requires a lot of human involvement, and the extracted features often cannot represent the essence of user behavior. Secondly, regardless of which machine learning or deep learning algorithm is used, it relies on labeled training datasets, which cannot meet the needs of a periodically updated and sufficiently rich and balanced training dataset. Moreover, the accuracy of the model trained on a small sample set in an experimental environment cannot be guaranteed in a real network environment. Finally, there is still a lack of a more universal identification method for different types of network users.

[0003] With the continuous popularization of the Internet, the number of network users has increased dramatically, resulting in an explosive growth in network traffic. For example, in a medium-sized city, network traffic has reached tens of Tbps. User behavior is mixed in massive network traffic, making it difficult to distinguish. In existing network user identification techniques, one approach is to directly calculate the similarity between user portrait feature vectors, and the other is to train a model through machine learning methods to identify users. The above technical solutions often have many limitations. Machine learning-based methods often work well on laboratory data, but their effectiveness in real network environments is unknown. The vector similarity method based on user portraits requires a lot of human involvement and relies on expert experience for feature extraction. The extracted feature vectors often cannot represent the essence of user behavior. SUMMARY

[0004] The technical problem solved by the present application is to provide a similar network behavior user identification method based on an unsupervised variational autoencoder algorithm to solve the problems of the background art.

[0005] The present application adopts the following technical solutions to solve the above technical problems:

[0006] A similar network behavior user identification method based on an unsupervised variational autoencoder, based on super large-scale network communication metadata, uses an unsupervised variational autoencoder model to learn deep regularity features of user network behavior from massive user network traffic metadata, abstracts the user's network behavior into high-dimensional vector representation of essential behavior patterns, and finally identifies users using the similarity between vectors; specifically including network traffic metadata, network behavior statistical feature extraction, behavior feature image processing, deep learning model training, deep feature extraction, and similar behavior user identification.

[0007] As a further preferred scheme of the similar network behavior user identification method based on the unsupervised variational autoencoder of the present application, the network traffic metadata is processed as follows:

[0008] The collected message data is merged into communication session metadata, and each session communication flow is represented as follows:

[0009] Flow i (sip i ,dip i ,spt i ,dpt i ,transprtl i ,appprtl i ,stime i ,etime i ,upkts i ,ubytes i ,dpkts i ,dbytes i )

[0010] Among them, the i-th session communication is defined as Flow i , and is abbreviated as f i , which is respectively one-to-one corresponding to (source IP, destination IP, source port, destination port, transport layer protocol, application layer protocol, start time, end time, uplink packet number, uplink byte number, downlink packet number, downlink byte number) of the i-th session flow.

[0011] As a further preferred embodiment of the similar network behavior user identification method based on unsupervised variational autoencoder of the present invention, the network behavior statistical feature extraction involves the following specific steps:

[0012] Based on the communication metadata, a sliding window with the same size and step size is used to perform time-series processing on the communication metadata. Let the sliding window be w.

[0013] To address the complex and redundant network behaviors of network users, a sliding window is used as the basic descriptive unit. Statistical features within this basic unit are calculated as a primary expression of user network behavior. For user j in the k-th time window (u j ,w k The behavioral characteristics of ) are represented as:

[0014]

[0015] Where c1, c2, c3, ..., c n The statistical features are extracted based on communication metadata; to comprehensively describe user network behavior as much as possible, 42-dimensional features are designed from aspects such as size, quantity, duration, and entropy; let user u j In w k Intra-connection session set is The feature calculation formula is expressed as follows:

[0016] ① Frequency of calls:

[0017] ② Communication time:

[0018] ③ Average communication time:

[0019] ④~⑥ Number of communications lasting less than 60 seconds, more than 300 seconds, and between 60 and 300 seconds:

[0020]

[0021] ⑦ Standard deviation of communication time:

[0022] ⑧ Number of bags carried upwards:

[0023] ⑨ Number of downlink packets:

[0024] ⑩ Number of bytes in the uplink:

[0025] Downlink bytes:

[0026] Average number of bytes of uplink packets:

[0027] Average number of bytes of downlink packets:

[0028] Maximum number of bytes of uplink: c 14 (u j ,w k ) = max(ubytes i )

[0029] Maximum number of bytes of downlink: c 15 (u j ,w k ) = max(dbytes i )

[0030] Standard deviation of bytes of uplink:

[0031] Standard deviation of bytes of downlink:

[0032] Number of connections with 0 bytes of uplink, number of connections with 0-1M bytes of uplink, number of connections with 1-10M bytes of uplink, number of connections with more than 10M bytes of uplink:

[0033]

[0034] Number of connections with 0 bytes of downlink, number of connections with 0-1M bytes of downlink, number of connections with 1-10M bytes of downlink, number of connections with more than 10M bytes of downlink:

[0035]

[0036] Number of connections per unit time:

[0037] Number of source IPs: c 27 (u j ,w k ) = |set(sip i )|

[0038] Number of destination IPs: c 28 (u j ,wk ) = | set(dip i )|

[0039] Source port is the regular port of the number of Tonglian frequency:

[0040] Destination port is the regular port of the number of Tonglian frequency:

[0041] The destination IP belongs to the number of Tonglian frequency:

[0042] The destination IP belongs to the Tonglian time:

[0043] The destination IP belongs to the number of uplink bytes:

[0044] The destination IP belongs to the number of downlink bytes:

[0045] Application layer is the number of Tonglian frequency of ssh protocol:

[0046] Application layer is the Tonglian time of ssh protocol:

[0047] The maximum byte of uplink corresponds to the Tonglian time: 37 (u j ,w k ) = (etime i -stime i ) × A(max(ubytes i ))

[0048] The maximum byte of downlink corresponds to the Tonglian time: 38 (u j ,w k ) = (etime i -stime i ) × A(max(dbytes i ))

[0049] Source IP entropy value: 39 (u j ,w k ) = -∑P(sipi ) x log(2, P(dip i ))

[0050] Destination IP entropy value: c 40 (u j ,w k ) = -∑P(dip i ) x log(2, P(dip i ))

[0051] Destination port entropy value: c 41 (u j ,w k ) = -∑P(dpt i ) x log(2, P(dpt i ))

[0052] Application layer protocol entropy value: c 42 (u j ,w k ) = -∑P(appprtl i ) x log(2, P(appprtl i ))

[0053] In the above formula, A(x) is an indicator function, which is 1 when the corresponding condition is met, otherwise 0; P(sip t ), P(dip t ), P(dpt t ), P(appprtl t ) respectively represent the probability of any sip i , dip i , dpt i , appprtl i appearing in the source IP set, destination IP set, destination port set, application layer protocol set of the specified user u j within the time window w k , and the ordered characteristics of the user's online behavior are represented by the entropy value.

[0054] As a further preferred scheme of the application, the behavior feature is image processed, and the specific operation steps are as follows:

[0055] Let the sliding window sequence be {w1, w2, w3, …, w k , …, w n-1 , w n}, for any user u j , if w kThe internal is network active, then calculate its c1~c 42 The eigenvector, on the contrary, is complemented with a zero vector; The eigenvectors in each window are spliced in the vertical direction to form a unified n x 42-dimensional matrix primary representation, so as to be used for training and application of the deep learning model; By referring to the idea of image recognition, the user primary representation matrix is converted into a value in the range of 0~255 in a normalized manner, and a corresponding gray image is generated according to the brightness relationship, and the normalization formula is represented as:

[0056]

[0057] Wherein, max(c i ) and min(c i ) represent the maximum and minimum values of the i-th feature of the sample u j , represents the value of the i-th feature in the window k, represents the new value after conversion.

[0058] As a further preferred scheme of the present application, the deep learning model training is as follows:

[0059] The variational autoencoder model based on the convolutional neural network (CNN) assumes that the real observation data follows a certain implicit distribution, and generates samples similar to the input data by probabilistic modeling of the implicit representation of the observation data; Considering the difficulty of obtaining labeled samples, from the perspective of unsupervised learning, the variational autoencoder model is constructed by using the convolutional network on the global massive network user {u1, u2, u3, …, u m} sample set, and the data is learned by taking the model reconstruction loss and the KL divergence regular term as the minimum, so as to capture the essential features of the data;

[0060] The adopted variational autoencoder is composed of an encoder and a decoder; the encoder is composed of 3 convolutional layers and 2 fully connected layers, each convolutional layer adopts 128, 64 and 32 filters respectively, the convolution kernel size is [3, 3], and the moving step is 1; each convolutional layer is followed by a max pooling layer with a size of [2, 2] to reduce the spatial dimension of the feature map; each convolutional layer adopts ReLU as the activation function; after the operation of all convolutional layers, the output feature map is compressed and flattened into a one-dimensional shape by using the Flatten operation, then two fully connected layers are used to estimate the mean and variance of the latent variable Z respectively, and then the encoding end output of 1024-dimensional vector is generated according to the distribution of Z; the decoder is reciprocal to the encoder, composed of 3 reverse convolutional layers and 2 fully connected layers, taking the output of the encoder as the input of the decoder, converting the input into the output size of the third convolutional layer of the encoder through a fully connected layer and a Reshape operation, then connecting 3 reverse convolutional layers, each reverse convolutional layer adopts 32, 64 and 128 filters respectively, the convolution kernel size is [3, 3], and the moving step is 1; each reverse convolutional layer is followed by an up operation with a size of [2, 2] to improve the spatial dimension of the feature map; after the reverse convolutional operation, a fully connected layer and a Reshape operation are used to output a grayscale image with the same size as the input sample of the encoder.

[0061] As a further preferred scheme of the similar network behavior user identification method based on the unsupervised variational autoencoder of the present application, deep feature extraction, the specific operation steps are as follows:

[0062] The encoder part of the pre-trained variational autoencoder model is used to encode the user n x 42-dimensional primary representation as input, abstract the deep features of the user network behavior, and simplify the representation into a 1024-dimensional vector.

[0063] As a further preferred scheme of the similar network behavior user identification method based on the unsupervised variational autoencoder of the present application, similar behavior user identification, the specific operation steps are as follows: for the global network user {u1, u2, u3, …, u m On the basis of 1024-dimensional vector representation, the cosine similarity method is used to identify the user with similar network behavior. i The formula of the user with similar network behavior is:

[0064]

[0065] S(u i ,u j ) value is between -1 and 1, the larger the value is, the more similar the network behavior of the user is. i j ​The more similar the network behaviors are, the greater the difference is; when the identification threshold is 0.9, when the cosine similarity is >= 0.85, the behavior similar users are identified and recommended.

[0066] Compared with the prior art, the above technical scheme has the following technical effects:

[0067] 1. The application discloses a similar network behavior user identification method based on an unsupervised variational autoencoder algorithm.

[0068] 2. The application does not need to perform deep analysis on the traffic message, and performs relevant traffic behavior statistical feature extraction based on the interconnection metadata to form the primary representation of the network behavior of the user, thereby accelerating the efficiency and having certain real-time performance.

[0069] 3. The application adopts the unsupervised variational autoencoder model, and trains on the global network user sample, thereby overcoming the pain point of difficult acquisition of the supervised learning sample, the extracted features are closer to the nature of the network behavior of the user, and the different types of network users can be uniformly represented and converted, and the application has better universality.

[0070] 4. The application identifies the network user from the perspective of the similar behavior mode based on the analysis of the interconnection metadata, and avoids the influence of the unidentifiability of the encrypted traffic. DETAILED DESCRIPTION

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0072] Figure 1 The application is a similar network behavior user identification method based on an unsupervised variational autoencoder. DETAILED DESCRIPTION

[0073] The technical solutions of the present application will be further described in detail below with reference to the drawings:

[0074] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort are within the protection scope of the present application. The purpose and effects of the present application will become more apparent from the following detailed description of the preferred embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0075] In the context of strengthening network space security governance, aiming at the problem of network user identification, based on super large-scale network communication metadata, using an unsupervised variational autoencoder model, learning deep law features of user network behavior from massive user network traffic metadata, then abstracting the user's apparent network behavior into high-dimensional vector representation of essential behavior patterns, finally using the similarity between vectors to identify users, achieving the purpose of the patent; the method described in the patent contains 6 steps of network traffic metadata, network behavior statistical feature extraction, behavior feature image processing, deep learning model training, deep feature extraction, and similar behavior user identification. The specific operation steps are as follows:

[0076] 1. Network traffic metadata:

[0077] Collect packet data to merge into communication session metadata, and each session communication flow is represented as follows:

[0078] Flow i (sip i ,dip i ,spt i ,dpt i ,transprtl i ,appprtl i ,stime i ,etime i ,upkts i ,ubytes i ,dpkts i ,dbytes i )

[0079] For the i-th session communication, we define it as Flow i , which is abbreviated as f i , which is respectively one-to-one corresponding to (source IP, destination IP, source port, destination port, transport layer protocol, application layer protocol, start time, end time, uplink packet number, uplink byte number, downlink packet number, downlink byte number).

[0080] 2. Network behavior statistical feature extraction: On the basis of the communication metadata, the sliding window with the same size and step size is used to process the time series of the communication metadata, and the sliding window is denoted as w. For the complex and redundant network behavior of the network user, the statistical features in the basic unit are calculated as the primary expression of the user network behavior, taking the sliding window as the basic description unit. Denote the behavior feature of the user j in the kth time window (u j ,w k ) as:

[0081]

[0082] wherein c1, c2, c3, …, c n are the statistical features extracted based on the communication metadata. In this patent, in order to describe the network behavior of the user as comprehensively as possible, 42-dimensional features are designed from the aspects of size, quantity, duration, and entropy value. Let the user u j in the w k communication session set be The feature calculation formula is:

[0083] ① Communication frequency:

[0084] ② Communication time:

[0085] ③ Average communication time:

[0086] ④-⑥ Communication frequency when the time is less than 60 seconds, greater than 300 seconds, and between 60-300 seconds:

[0087]

[0088] ⑦ Standard deviation of communication time:

[0089] ⑧ Uplink packet quantity:

[0090] ⑨ Downlink packet quantity:

[0091] ⑩ Uplink byte quantity:

[0092] Downlink byte quantity:

[0093] Uplink packet average byte quantity:

[0094] Downlink packet average byte quantity:​

[0095] Max uplink bytes: c 14 (u j ,w k ) = max(ubytes i )

[0096] Max downlink bytes: c 15 (u j ,w k ) = max(dbytes i )

[0097] Uplink byte standard deviation:

[0098] Downlink byte standard deviation:

[0099] Uplink byte frequency of 0, uplink byte frequency of 0-1M, uplink byte frequency of 1-10M, uplink byte frequency of more than 10M:

[0100]

[0101] Downlink byte frequency of 0, downlink byte frequency of 0-1M, downlink byte frequency of 1-10M, downlink byte frequency of more than 10M:

[0102]

[0103] Uplink frequency per unit time:

[0104] Source IP number: c 27 (u j ,w k ) = |set(sip i )|

[0105] Destination IP number: c 28 (u j ,w k ) = |set(dip i )|

[0106] Uplink frequency of regular source port:

[0107] The number of connection frequencies of the destination port is:

[0108] The number of connection frequencies of the destination IP belonging to overseas is:

[0109] The connection time of the destination IP belonging to overseas is:

[0110] The number of uplink bytes of the destination IP belonging to overseas is:

[0111] The number of downlink bytes of the destination IP belonging to overseas is:

[0112] The number of connection frequencies of the application layer being ssh protocol is:

[0113] The connection time of the application layer being ssh protocol is: The connection time corresponding to the maximum uplink bytes is: 37 (u j ,w k )=(etime i -stime i )×A(max(ubytes i ))

[0114] The connection time corresponding to the maximum downlink bytes is: 38 (u j ,w k )=(etime i -stime i )×A(max(dbytes i ))

[0115] The source IP entropy value is: 39 (u j ,w k )=-∑P(sip i )×log(2,P(sip i ))

[0116] The destination IP entropy value is: 40 (u j ,wk ) = -∑P(dip i ) × log(2, P(dip i ))

[0117] Destination port entropy value: c 41 (u j ,w k ) = -∑P(dpt i ) × log(2, P(dpt i ))

[0118] Application layer protocol entropy value: c 42 (u j ,w k ) = -∑P(appprtl i ) × log(2, P(appprtl i )).

[0119] In the above formula, A(x) is an indicator function, which is 1 when the corresponding condition is met, otherwise 0; P(sip t ), P(dip t ), P(dpt t ), P(appprtl i ) respectively represent the probability of any sip i , dip i , dpt i , appprtl i appearing in the source IP set, destination IP set, destination port set, application layer protocol set of the specified user u j within the time window w k , which shows the ordered characteristics of user online behavior through entropy value.

[0120] 3. Behavior feature image processing: record the sliding window sequence as {w1, w2, w3, …, w k , …, w n-1 , w n}, for any user u j , if there is network activity within w k , calculate its c1~c 42 dimensional feature vector, otherwise, complete it with zero vector. The feature vectors in each window are spliced in the vertical direction to form a unified n × 42-dimensional matrix primary representation, which can be used for training and application of deep learning model. Drawing on the idea of image recognition, the user primary representation matrix is converted to a value in the range of 0~255 in a normalized manner, and the corresponding gray image is generated according to the brightness relationship, and the normalization formula is represented as:

[0121]

[0122] where max(c i ) and min(c i ) represent the maximum and minimum values of the i-th feature of the sample u j , respectively, represents the value of the i-th feature within the window k, represents the new value after conversion.

[0123] 4. Deep learning model training: a variational autoencoder model based on convolutional neural network (CNN, Convolutional Neural Network), the core of which is to assume that real observation data follows a certain implicit distribution, and to generate samples similar to input data by probabilistic modeling of the implicit representation of observation data. Considering the difficulty of obtaining labeled samples, from the perspective of unsupervised learning, on the global massive network user {u1, u2, u3, …, u m} sample set, a variational autoencoder model is constructed using a convolutional network, with the goal of minimizing the model reconstruction loss and the KL divergence regular term, to learn the data and capture the essence of the data. The variational autoencoder used in this patent consists of an encoder and a decoder. The encoder consists of 3 convolutional layers and 2 fully connected layers, each convolutional layer uses 128, 64 and 32 filters respectively, and the convolution kernel size is [3, 3] and the moving step is 1. Each convolutional layer is followed by a max-pooling layer with a size of [2, 2] to reduce the spatial dimension of the feature map. Each convolutional layer uses ReLU as the activation function. After all convolutional layer operations are completed, the output feature map is compressed and flattened into a one-dimensional shape using the Flatten operation, and then two fully connected layers are used to estimate the mean and variance of the latent variable Z, and then the encoder output of 1024-dimensional vector is generated according to the distribution of Z. The decoder is reciprocal to the encoder, consisting of 3 reverse convolutional layers and 2 fully connected layers, taking the output of the encoder as input, and converting the input to the output size of the third convolutional layer of the encoder through a fully connected layer and a Reshape operation, followed by 3 reverse convolutional layers, each reverse convolutional layer uses 32, 64, 128 filters respectively, and the convolution kernel size is [3, 3] and the moving step is 1. Each reverse convolutional layer is followed by an up-sampling operation with a size of [2, 2] to increase the spatial dimension of the feature map. After the reverse convolution operation is completed, a fully connected layer and a Reshape operation are used to output a grayscale image with the same size as the input sample of the encoder.

[0124] 5. Deep feature extraction: Use the encoder part of the pre-trained variational autoencoder model to encode the user's n x 42-dimensional primary representation, abstract the deep features of the user's network behavior, and simplify the representation to a 1024-dimensional vector.

[0125] 6. Similar behavior user identification: Based on the 1024-dimensional vector representation, use the cosine similarity method to identify users with similar network behavior. m} and 1024-dimensional vector representation, use the cosine similarity method to identify users with similar network behavior. i The formula is:

[0126]

[0127] S(u i ,u j ) value between -1 to 1, the greater the value, the more similar the network behavior of (u i ,u j ), on the contrary, the greater the difference. With 0.9 as the recognition threshold, when the cosine similarity >= 0.85, it is identified as a similar behavior user for recommendation.

[0128] The specific implementation of the patent will be further described in detail below, with practical operation examples to explain the patent, so as to better understand and reproduce the patent method. Figure 1 The overall framework of the similar network behavior user identification method based on unsupervised variational autoencoder is as follows: First, the global traffic data is aggregated and merged into a common connection metadata containing five tuples and related attributes; Based on the metadata, use sliding window to time sequence the traffic of each user, and in each window, use statistical analysis method to extract the features of complex and redundant traffic to form a primary unified representation; Map the behavior features to the interval of 0 to 255 and convert them into grayscale images with 1 channel; Construct a variational autoencoder model based on convolutional neural network as the basic unit and use grayscale image data for unsupervised training to learn the deep regularity features of network behavior; Use the Encoder capability of the variational autoencoder model to abstract deeper vector representation based on the primary representation of the user, and then use the cosine similarity to measure the similarity between any two users to achieve the purpose of similar network behavior user identification. Based on the seven-day common connection metadata of global users, the effect is verified on the five network behavior user datasets of scanning, DDoS, botnet, etc. The accuracy of similar behavior user identification is about 93%, and the average recall rate is about 60%.

[0129] Those skilled in the art can understand that the above description is only preferred examples of the present application and is not used to limit the present application, although the present application is described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions recorded in the foregoing examples or make equivalent replacements for part of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. All technical features in the embodiments can be freely combined according to actual needs.

[0130] Finally, it should be noted that the above description is only the preferred embodiment of the present application and is not used to limit the present application, although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments or make equivalent replacements for part of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying similar network behavior users based on an unsupervised variational autoencoder, characterized in that: Based on the super large network communication metadata, using unsupervised variational autoencoder model, learning the deep law characteristics of user network behavior from massive user network traffic metadata, then abstracting the user's network behavior to high-dimensional vector representation of the essential behavior mode, finally using the similarity between vectors to identify users; Specifically contains network traffic metadata, network behavior statistical feature extraction, behavior feature image processing, deep learning model training, deep feature extraction, similar behavior user identification; The network behavior statistical feature extraction specifically includes the following operation steps: On the basis of the communication metadata, a sliding window with the same size and step length is used to process the time series of the communication metadata, and the sliding window is denoted as w; For the complex and redundant network behavior of network users, the sliding window is taken as the basic description unit, and the statistical features in the basic unit are calculated as the primary expression of user network behavior. The behavior feature of user j in the kth time window (u j ,w k ) is represented as: wherein c1, c2, c3, …, c n Based on the extracted statistical features of the communication metadata; in order to describe the user network behavior as comprehensively as possible, 42-dimensional features are designed from size, quantity, duration, and entropy value; let user u j In the w k The set of internal communication sessions is The feature calculation formula is represented as: ①Interconnect frequency: ②Communication time: ③ Average connection time: (4) The frequency of communication less than 60 seconds, the frequency of communication greater than 300 seconds, and the frequency of communication between 60 and 300 seconds: ⑦Standard deviation of the time of communication:

8. Number of uplink packets: (9) Number of Downlink Packets:

10. Number of uplink bytes: Downlink bytes: Average number of bytes of uplink packets: Downlink packet average byte number: Max uplink bytes: c 14 (u j ,w k ) = max(ubytes i ) Downlink max bytes: c 15 (u j ,w k ) = max(dbytes i ) Upstream byte standard deviation: Downlink byte standard deviation: The number of uplink bytes is equal to 0, the number of uplink bytes is between 0 and 1M, the number of uplink bytes is between 1 and 10M, the number of uplink bytes is greater than 10M: The number of downlink bytes is equal to 0, the number of downlink bytes is between 0 and 1M, the number of downlink bytes is between 1 and 10M, the number of downlink bytes is greater than 10M: Frequency of communication per unit time: Source IP count: c 27 (u j ,w k ) = | set (sip i )| Number of destination IPs: c 28 (u j ,w k ) = | set(dip i )| Source port is the regular port of the Tonglian frequency: Destination port is the frequency of the regular port: Purpose IP belongs to the overseas communication frequency: Purpose IP belongs to the overseas time of communication: Purpose IP belongs to the number of uplink bytes of overseas: Purpose IP belongs to the number of bytes of downlink in the overseas: The application layer is ssh protocol is tonglian frequency: The application layer is ssh protocol is the time of communication: Uplink maximum bytes corresponding to the communication time: c 37 (u j ,w k ) = (etime i -stime i ) x A(max(ubytes i )) Downlink maximum bytes corresponding to the time of communication: c 38 (u j ,w k ) = (etime i -stime i ) × A(max(dbytes i )) Source IP entropy value: c 39 (u j ,w k ) = -∑P(sip i ) x log(2, P(sip i )) Destination IP entropy value: c 40 (u j ,w k ) = -∑P(dip i ) x log(2, P(dip i )) Destination port entropy value: c 41 (u j ,w k ) = -∑P(dpt i ) x log(2, P(dpt i ) Application layer protocol entropy value: c 42 (u j ,w k ) = -∑P(appprtl i ) x log(2, P(appprtl i ) In the above formula, A(x) is an indicator function, which is 1 when the corresponding condition is met, otherwise 0; P(sip t ), P(dip i ), P(dpt t ), P(appprtl t ) respectively represent the probability of any sip k , dip j , dpt i , appprtl i appearing in the source IP set, the destination IP set, the destination port set, and the application layer protocol set of the specified user u i within the time window w i , which reflects the ordered characteristics of the user's online behavior in the form of entropy value. 2.The method of claim 1, wherein: Network traffic metadata, the specific operation steps are as follows: The message data is merged into communication session metadata, and each communication session flow is represented as follows: Flow i (sip i ,dip i ,spt i ,dpt i ,transprtl i ,appprtl i ,stime i ,etime i ,upkts i ,ubytes i ,dpkts i ,dbytes i ) wherein, for the i-th session flow, is defined as Flow i , and is abbreviated as f i , respectively corresponds to the source IP, destination IP, source port, destination port, transport layer protocol, application layer protocol, start time, end time, number of uplink packets, number of uplink bytes, number of downlink packets, and number of downlink bytes of the i-th session flow, respectively. 3.The method of claim 1, wherein: Behavior feature image processing, the specific operation steps are as follows: The sliding window sequence is denoted as {w1, w2, w3, …, w k ,…,w n-1 ,w n} for any user u j , if there is network activity in w k , the c1-c 42 dimensional feature vector is calculated, otherwise, it is completed with a zero vector; the feature vectors in each window are spliced in the vertical direction to form a unified n x 42-dimensional matrix primary representation, which is used as a deep learning model for training and application; by referring to the idea of image recognition, the user primary representation matrix is converted into a value in the range of 0-255 in a normalized manner, and a corresponding gray image is generated according to the brightness relationship, and the normalization formula is represented as: where max(c i ) and min(c i ) represent the maximum and minimum values of the i-th feature of the sample u j represents the value of the i-th feature within the window k, represents the new value after conversion.​ 4. The method of claim 1, wherein the method is based on an unsupervised variational autoencoder. Deep learning model training, the specific operation steps are as follows: The core of the convolutional neural network (CNN) based variational autoencoder model is to assume that the real observation data follows a certain implicit distribution, to model the implicit representation of the observation data by probability, and to generate samples similar to the input data. Considering the difficulty of obtaining labeled samples, from the perspective of unsupervised learning, a variational autoencoder model is constructed by using a convolutional network on a global massive network user {u1, u2, u3, …, u m} sample set, the data is learned by taking the model reconstruction loss and the KL divergence regularization term as the minimum, and the purpose of capturing the essential features of the data is achieved. The variational autoencoder used is composed of encoder and decoder; The encoder is composed of 3 convolutional layers and 2 fully connected layers, each convolutional layer uses 128, 64 and 32 filters respectively, the convolution kernel size is [3, 3], and the moving step is 1; Each convolutional layer is followed by a max pooling layer with a size of [2, 2] to reduce the spatial dimension of the feature map; Each convolutional layer uses ReLU as the activation function; After all convolutional layer operations are completed, the output feature map is compressed and flattened to one-dimensional shape using the Flatten operation, then two fully connected layers are used to estimate the mean and variance of the latent variable Z, and then the encoder output of 1024-dimensional vector is generated according to the distribution of Z; The decoder and the encoder are reciprocal, composed of 3 reverse convolutional layers and 2 fully connected layers, taking the output of the encoder as the input of the decoder, passing through a fully connected layer and a reshape operation, converting the input to the output size of the third convolutional layer of the encoder, then connecting 3 reverse convolutional layers, each reverse convolutional layer uses 32, 64 and 128 filters respectively, the convolution kernel size is [3, 3], and the moving step is 1; Each reverse convolutional layer is followed by an up operation with a size of [2, 2] to improve the spatial dimension of the feature map; After the reverse convolutional operation is completed, a fully connected layer and a reshape operation are used to output a grayscale image with the same size as the input sample of the encoder.

5. The method of claim 1, wherein the method is based on an unsupervised variational autoencoder. Deep feature extraction, the specific operation steps are as follows: Using the encoder part of the pre-trained variational autoencoder model, the user's n x 42-dimensional primary representation is input for encoding processing to abstract the deep features of user network behavior, which is simplified to a 1024-dimensional vector.

6. The method of claim 1, wherein the method is based on an unsupervised variational autoencoder. The specific operation steps of the similar behavior user identification are as follows: for the global network user {u1, u2, u3,..., uN}, on the basis of 1024-dimensional vector representation, the cosine similarity method is used to identify the user u m with similar network behavior, and the formula is as follows: i with similar network behavior, and the formula is as follows: S(u i ,u j ) value between -1 to 1, the greater the value, the more similar (u i ,u j ) network behavior, on the contrary, the greater the difference; with 0.9 as the identification threshold, when the cosine similarity >= 0.85, as the behavior similar user identification recommendation.

Images