Method, device and equipment for detecting anonymous traffic
By extracting flow context relevance and destination IP address distribution features in anonymous traffic detection, and combining kernel density estimation algorithm and classification model, risk windows are screened and classified, solving the problems of high computational overhead and low efficiency caused by unclear anonymous traffic features and large data volume, thus achieving efficient anonymous traffic detection.
Patent Information
- Application Number
- CN202411639502.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2044-11-15
AI Technical Summary
In existing technologies, anonymous traffic features are not obvious and the amount of network traffic data is large, resulting in high computational overhead and low efficiency in traffic data screening.
By acquiring traffic data within a preset time window, extracting flow context relevance features and destination IP address distribution features, using a kernel density estimation algorithm to determine risk values, and then using a trained classification model to classify the traffic data within the risk window.
It effectively reduces computational overhead, improves the accuracy and speed of anonymous traffic detection, simplifies the method, and has good versatility.
Smart Images

Figure CN119628876B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer data processing, and particularly relates to a detection method, device and equipment of anonymous traffic. BACKGROUND
[0002] In the current technical environment, methods for identifying anonymous network traffic mainly include feature-based methods and statistical-based methods.
[0003] Among them, the feature-based method relies on extracting traffic features from the unencrypted protocol header, and on this basis, the type of traffic is identified through pattern recognition or keyword matching technology. The statistical-based method focuses on analyzing the statistical features such as message length and time interval in the interactive process of encrypted traffic. These features can be trained and identified through machine learning technology to help detect traffic of various encrypted protocols.
[0004] ShadowSocks, as a typical anonymous proxy system, is usually deployed on virtual private servers in cloud data centers around the world. The traffic data of anonymous proxy SS is mainly composed of encrypted data based on the TCP (Transmission Control Protocol) protocol, and does not involve the negotiation and exchange of keys in the communication process between the client and the remote server.
[0005] Because the protocol header of SS traffic lacks obvious distinguishing features and does not have differential length direction features like the methods mentioned above, traditional traffic identification methods face challenges in detecting SS anonymous traffic. Moreover, due to the huge amount of traffic data on the network backbone route, the computational cost of checking all traffic one by one is very high, and the number of known illegal addresses is limited, which cannot be effectively investigated.
[0006] Therefore, in the related art, the anonymous traffic features are not obvious, and the amount of network traffic data is large, resulting in high computational cost and low traffic data investigation efficiency. SUMMARY
[0007] The purpose of the present application is to provide a detection method, device and equipment of anonymous traffic, to solve the problem of large computational cost and low traffic data investigation efficiency caused by the fact that the anonymous traffic features are not obvious and the amount of network traffic data is large in the related art.
[0008] In a first aspect, the present application provides a detection method of anonymous traffic, comprising:
[0009] obtaining each piece of traffic data in a preset time window;
[0010] respectively, to obtain flow context correlation features and destination IP address distribution features of each of the traffic data;
[0011] Based on the obtained flow context correlation features and destination IP address distribution features, a kernel density estimation algorithm is used to determine the risk value of each of the traffic data;
[0012] If the risk value of any of the traffic data is greater than a preset risk threshold, the preset time window is determined as a risk window;
[0013] The traffic data in the risk window are classified by using the trained classification model to obtain a traffic classification result.
[0014] In a possible implementation, the feature extraction is performed on each of the traffic data to obtain flow context correlation features of each of the traffic data, including:
[0015] For any traffic data in the preset time window, the following steps are performed:
[0016] For any one of a plurality of preset context relationship types, if the relationship between the current traffic data and the remaining traffic data other than the current traffic data meets the any one of the relationship types, the feature parameter of the current traffic data is updated, wherein the feature parameter corresponds to the any one of the relationship types;
[0017] The flow context correlation features of the current traffic data are determined according to the plurality of feature parameters of the current traffic data.
[0018] In a possible implementation, the feature extraction is performed on each of the traffic data to obtain destination IP address distribution features of each of the traffic data, including:
[0019] The probability distribution of each destination IP address in the preset time window is determined according to the destination IP address of each of the traffic data in the preset time window;
[0020] Based on the probability distribution, an entropy value of the preset time window is obtained, and the entropy value is taken as the destination IP address distribution feature of any of the traffic data in the preset time window.
[0021] In a possible implementation, the kernel density estimation algorithm is used to determine the risk value of each of the traffic data based on the obtained flow context correlation features and destination IP address distribution features, including:
[0022] For any traffic data in the preset time window, the following steps are performed:
[0023] Based on the flow context relevance features and destination IP address distribution features of the current traffic data, the feature matrix vector of the current traffic data is determined;
[0024] The difference between the feature matrix vector of the current traffic data and the feature matrix vector of each traffic data in the preset time window is used as the distance between the current traffic data and each traffic data in the preset time window.
[0025] The distance is standardized using a preset bandwidth to obtain the standardized distance between the current traffic data and each traffic data in the preset time window.
[0026] Based on the obtained standardized distances and Gaussian kernel functions, kernel density estimation is performed on each traffic data to obtain the risk value of the current traffic data.
[0027] In one possible implementation, the step of performing kernel density estimation on each traffic data based on the obtained standardized distances and Gaussian kernel functions to obtain the risk value of the current traffic data includes:
[0028] The obtained standardized distances are input into a Gaussian kernel function to obtain the influence of each traffic data point on the current traffic data within the preset time window;
[0029] The influence of each traffic data point in the preset time window on the current traffic data is summed to obtain the total influence of the preset time window.
[0030] The sum of influence is normalized using the preset bandwidth and the number of all traffic data in the preset time window to obtain the kernel function value of the current traffic data;
[0031] The kernel function value of the current traffic data is used as the risk value of the current traffic data.
[0032] In one possible implementation, the classification model is trained using the following steps:
[0033] Obtain training samples, which include a set of sample traffic data and a set of labeled tags for the sample traffic data;
[0034] The sample traffic data set is input into a classification model for classification prediction processing to obtain a set of predicted labels.
[0035] The obtained predicted label set and labeled label set are compared to determine the loss function value, and the network parameters of the classification model are updated based on the loss function value.
[0036] In one possible implementation, the method further includes:
[0037] When training the classification model, traffic data whose predicted labels meet the high confidence condition are added to the training samples, and the classification model is iteratively trained using the updated training samples until there is no traffic data that meets the high confidence condition or the preset number of iterations is reached; wherein, the high confidence condition is that the absolute value of the predicted label is greater than a preset confidence threshold.
[0038] In one possible implementation, classifying the traffic data in the risk window using a trained classification model to obtain a traffic classification result includes:
[0039] For any traffic data point in the risk window, perform the following steps:
[0040] The current traffic data is classified and predicted using the decision function of the trained classification model to obtain the predicted label.
[0041] If the predicted label is greater than or equal to 0, then the current traffic data is determined to be risky traffic;
[0042] If the predicted label is less than 0, then the current traffic data is determined to be normal traffic.
[0043] Secondly, this application provides an anonymous traffic detection device, the device comprising:
[0044] The traffic data acquisition module is configured to acquire traffic data for each data point within a preset time window;
[0045] The feature determination module is configured to extract features from each of the traffic data to obtain the flow context relevance features and destination IP address distribution features of each of the traffic data.
[0046] The risk value determination module is configured to determine the risk value of each traffic data based on the obtained context relevance features of each flow and the distribution features of the destination IP address, using a kernel density estimation algorithm.
[0047] The risk window determination module is configured to determine the preset time window as a risk window if the risk value of any of the traffic data is greater than a preset risk threshold.
[0048] The traffic data classification module is configured to classify the traffic data in the risk window using a trained classification model to obtain the traffic classification result.
[0049] In one possible implementation, feature extraction is performed on each of the traffic data to obtain the flow context relevance features of each traffic data, and the feature determination module is configured to:
[0050] For any traffic data point within the preset time window, perform the following steps:
[0051] For any one of the multiple preset context relationship types, if the relationship between the current traffic data and the remaining traffic data (excluding the current traffic data) satisfies any one of the relationship types, then the feature parameters of the current traffic data are updated; wherein, the feature parameters correspond to any one of the relationship types.
[0052] Based on multiple feature parameters of the current traffic data, the flow context relevance features of the current traffic data are determined.
[0053] In one possible implementation, feature extraction is performed on each of the traffic data to obtain the destination IP address distribution characteristics of each traffic data, and the feature determination module is configured as follows:
[0054] Based on the destination IP address of each traffic data item within the preset time window, determine the probability distribution of each destination IP address within the preset time window;
[0055] Based on the probability distribution, the entropy value of the preset time window is obtained, and the entropy value is used as the destination IP address distribution feature of any traffic data in the preset time window.
[0056] In one possible implementation, the risk value of each traffic data item is determined using a kernel density estimation algorithm based on the obtained context relevance features and destination IP address distribution features. The risk value determination module is configured to:
[0057] For any data point within the preset time window, perform the following steps:
[0058] Based on the flow context relevance features and destination IP address distribution features of the current traffic data, the feature matrix vector of the current traffic data is determined;
[0059] The difference between the feature matrix vector of the current traffic data and the feature matrix vector of each traffic data in the preset time window is used as the distance between the current traffic data and each traffic data in the preset time window.
[0060] The distance is standardized using a preset bandwidth to obtain the standardized distance between the current traffic data and each traffic data in the preset time window.
[0061] Based on the obtained standardized distances and Gaussian kernel functions, kernel density estimation is performed on each traffic data to obtain the risk value of the current traffic data.
[0062] In one possible implementation, the step of performing kernel density estimation on each traffic data based on the obtained standardized distances and Gaussian kernel functions to obtain the risk value of the current traffic data, wherein the risk value determination module is configured to:
[0063] The obtained standardized distances are input into a Gaussian kernel function to obtain the influence of each traffic data point on the current traffic data within the preset time window;
[0064] The influence of each traffic data point in the preset time window on the current traffic data is summed to obtain the total influence of the preset time window.
[0065] The sum of influence is normalized using the preset bandwidth and the number of all traffic data in the preset time window to obtain the kernel function value of the current traffic data;
[0066] The kernel function value of the current traffic data is used as the risk value of the current traffic data.
[0067] In one possible implementation, the traffic data classification module is configured to train a classification model using the following steps:
[0068] Obtain training samples, which include a set of sample traffic data and a set of labeled tags for the sample traffic data;
[0069] The sample traffic data set is input into a classification model for classification prediction processing to obtain a set of predicted labels.
[0070] The obtained predicted label set and labeled label set are compared to determine the loss function value, and the network parameters of the classification model are updated based on the loss function value.
[0071] In one possible implementation, the traffic data classification module is further configured to:
[0072] When training the classification model, traffic data whose predicted labels meet the high confidence condition are added to the training samples, and the classification model is iteratively trained using the updated training samples until there is no traffic data that meets the high confidence condition or the preset number of iterations is reached; wherein, the high confidence condition is that the absolute value of the predicted label is greater than a preset confidence threshold.
[0073] In one possible implementation, the traffic data classification module is configured to classify the traffic data in the risk window using a trained classification model to obtain a traffic classification result.
[0074] For any traffic data point in the risk window, perform the following steps:
[0075] The current traffic data is classified and predicted using the decision function of the trained classification model to obtain the predicted label.
[0076] If the predicted label is greater than or equal to 0, then the current traffic data is determined to be risky traffic;
[0077] If the predicted label is less than 0, then the current traffic data is determined to be normal traffic.
[0078] Thirdly, this application provides an electronic device, comprising:
[0079] Processor and memory;
[0080] The memory is used to store the processor-executable instructions;
[0081] The processor is configured to execute the instructions to implement the method for detecting anonymous traffic as provided in any of the first aspects of this application.
[0082] Fourthly, this application provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform an anonymous traffic detection method as described in any of the first aspects of this application.
[0083] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method for detecting anonymous traffic as described in any of the claims provided in the first aspect of this application.
[0084] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:
[0085] The anonymous traffic detection method provided in this application can filter out risky data in a preset time window based on effective features and kernel density estimation algorithm, and divide the risk window accordingly. The traffic data in the risk window is classified and identified by a classification model to obtain the classification result. It is not necessary to classify and identify the traffic data in all time windows, which can effectively reduce the computational overhead, speed up the identification speed, and improve the accuracy of anonymous traffic detection. Moreover, the technical solution provided in this application is simple and has good versatility.
[0086] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. Based on common knowledge in the art, the above preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this application. Attached Figure Description
[0087] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0088] Figure 1 A schematic diagram illustrating an application scenario of an anonymous traffic detection method provided in this application embodiment;
[0089] Figure 2 A schematic diagram illustrating the overall process of an anonymous traffic detection method provided in this application embodiment;
[0090] Figure 3 A schematic diagram of a sliding window provided in an embodiment of this application;
[0091] Figure 4 A flowchart illustrating the process of determining flow context relevance features provided in an embodiment of this application;
[0092] Figure 5 A flowchart illustrating the process of determining the distribution characteristics of destination IP addresses provided in an embodiment of this application;
[0093] Figure 6 A flowchart illustrating the process of determining the risk value of each data point using a kernel density estimation algorithm, as provided in this embodiment of the application.
[0094] Figure 7 A flowchart illustrating the process of determining the risk value of current traffic data provided in this application embodiment;
[0095] Figure 8 A schematic diagram illustrating the process of training a classification model provided in an embodiment of this application;
[0096] Figure 9 A schematic diagram of the structure of an anonymous traffic detection device provided in an embodiment of this application;
[0097] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0098] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0099] Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0100] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.
[0101] The following explains the technical terms and technologies involved in this application:
[0102] Anonymous network traffic technology: Anonymization technology aims to hide a user's identity and location, preventing the tracking of user behavior through network traffic analysis. It is typically used to protect user privacy and prevent network attackers or monitoring agencies from discovering their true identity and location.
[0103] Shadowsocks (SS) is an open-source proxy tool designed to help users bypass internet censorship, especially in countries and regions where certain websites are blocked. It achieves access to restricted content by encrypting traffic and concealing the user's access behavior. Shadowsocks uses the SOCKS5 protocol, allowing users to access network resources through a remote proxy server. It bypasses firewalls or network restrictions by encrypting user traffic and forwarding it through a proxy server. When a user uses a Shadowsocks client locally, the client encrypts the traffic and sends it to the configured Shadowsocks server. The server decrypts the traffic, forwards the request to the target website, and then encrypts the returned data again before sending it back to the user.
[0104] Virtual Private Server (VPS): This is a virtualization technology that divides a physical server into multiple independent virtual servers, each with its own operating system and resources (such as CPU, memory, storage, etc.). Although multiple VPSs share the hardware of the same physical server, they behave like independent servers in use.
[0105] Kernel density estimation is a nonparametric statistical method used to estimate the probability density function of a random variable. It works by "expanding" each data point into a smooth distribution and then superimposing these distributions to obtain the overall probability density distribution of the data. Kernel density estimation provides insight into the overall distribution of data and can be used to detect anomalous patterns or irregular traffic in network traffic.
[0106] Semi-supervised learning: Semi-supervised learning is a machine learning method that falls between supervised and unsupervised learning. It uses a small amount of labeled data and a large amount of unlabeled data for model training, which can reduce labeling costs and improve model performance.
[0107] Self-trained learning is a commonly used semi-supervised learning method that aims to improve model performance by utilizing a large amount of unlabeled data and a small amount of labeled data. Self-trained learning iteratively uses the model's predictions on unlabeled data as "pseudo-labels" and adds these pseudo-labeled data to the training set, gradually enhancing the model's learning ability.
[0108] Encryption and anonymization technologies, such as SSH, TOR, and VPN, provide an effective way to encrypt user communications and conceal their true identities. These technologies play a crucial role in protecting the privacy of legitimate users, enabling them to securely bypass geographical restrictions and access restricted content. However, due to their high anonymity and encryption, they are also widely exploited by malicious cyber actors, becoming important tools for various illegal activities. These illegal activities include data theft, the dissemination of illegal pornography, cyberattacks, and transactions on the dark web. Therefore, the identification and monitoring of this encrypted traffic is particularly important, as it is of paramount importance for maintaining network security and combating cybercrime.
[0109] In the current technological environment, methods for identifying anonymous network traffic mainly include feature-based methods and statistical methods.
[0110] Feature-based methods rely on extracting traffic features from unencrypted protocol headers, and then using pattern recognition or keyword matching techniques to identify the type of traffic. Statistical methods, on the other hand, focus on analyzing statistical features such as message length and time intervals during the interaction of encrypted traffic. These features can be trained and identified using machine learning techniques to help detect traffic from various encrypted protocols.
[0111] ShadowSocks, a typical anonymous proxy system, is usually deployed on virtual private servers in cloud data centers around the world. The traffic data of the anonymous proxy SS mainly consists of encrypted data based on the TCP (Transmission Control Protocol) protocol; key negotiation and exchange are not involved in the communication between the client and the remote server.
[0112] Because the protocol header of SS traffic lacks obvious distinguishing features and does not have differentiated length and direction features like the methods mentioned earlier, traditional traffic identification methods face challenges in detecting anonymous SS traffic. Moreover, due to the extremely large amount of traffic data on network backbone routes, checking all traffic individually is computationally very costly, and the known illegal addresses are limited, making effective screening impossible.
[0113] Therefore, the related technologies suffer from problems such as unclear anonymous traffic characteristics and a large amount of network traffic data, resulting in high computational overhead and low efficiency in traffic data screening.
[0114] In view of this, this application provides a method, apparatus and device for detecting anonymous traffic, in order to solve the problems in related technologies where the characteristics of anonymous traffic are not obvious and the amount of network traffic data is huge, resulting in high computational overhead and low efficiency of traffic data screening.
[0115] The inventive concept of this application can be summarized as follows: First, feature extraction is performed on each traffic data in a preset time window to obtain the flow context relevance features and destination IP address distribution features of each traffic data; then, based on the flow context relevance features and destination IP address distribution features, a kernel density estimation algorithm is used to determine the risk value of each traffic data; by comparing the risk value of the traffic data with a preset risk threshold, a risk window is selected; finally, a trained classification model is used to classify the traffic data in the risk window to obtain the traffic classification result.
[0116] In summary, the anonymous traffic detection method provided in this application can filter out risky data within a preset time window based on effective features and kernel density estimation algorithm, and divide the risk window accordingly. The traffic data in the risk window is then classified and identified using a classification model to obtain the classification result. It is not necessary to classify and identify the traffic data in all time windows, which can effectively reduce computational overhead, speed up the identification process, and improve the accuracy of anonymous traffic detection. Furthermore, the technical solution provided in this application is simple and has good versatility.
[0117] After introducing the main inventive concepts of the embodiments of this application, the following is a brief description of the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.
[0118] For ease of understanding, the following detailed description, in conjunction with the accompanying drawings, illustrates an anonymous traffic detection method provided in this application:
[0119] like Figure 1 The diagram illustrates an application scenario of an anonymous traffic detection method provided in this embodiment of the application. The diagram includes: a network 10, a server 20, and a storage device 30. The server 20 obtains traffic data within a preset time window via the network. Using the method provided in this embodiment, the traffic data within the preset time window can be processed for detection, and the accuracy of determining whether the traffic is risky can be determined, thus improving the efficiency and accuracy of traffic detection.
[0120] The description in this application focuses on a single server only. However, those skilled in the art should understand that the illustrated network 10, server 20, and memory 30 are intended to illustrate the operation of the electronic devices, servers, and memory involved in the technical solutions of this application. The detailed description of a single server and memory is at least for ease of explanation and does not imply any limitation on the number, type, or location of servers. It should be noted that adding additional modules to or removing individual modules from the illustrated environment will not change the underlying concept of the exemplary embodiments of this application. Furthermore, although detailed descriptions are provided for ease of explanation... Figure 1 The diagram shows a bidirectional arrow from memory 30 to server 20, but those skilled in the art will understand that the sending and receiving of the aforementioned data also needs to be achieved through network 10.
[0121] It should be noted that the memory in the embodiments of this application can be, for example, a cache system, hard disk storage, memory storage, etc. Furthermore, the method for processing optically dispersed data proposed in this application is not only applicable to… Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of this application do not impose any limitations.
[0122] Based on the above description, the overall process of the anonymous traffic detection method provided in this application embodiment is as follows: Figure 2 As shown, it includes the following:
[0123] In step 201, each flow data point within a preset time window is acquired.
[0124] In one possible implementation, embodiments of this application use a sliding window to collect and acquire traffic data, such as... Figure 3 As shown, the sliding window can be set to a window size w. flow =10s, sliding step size b=w flow =10s, each sliding window is used as a preset time window, and the traffic data within the window is then used for subsequent calculations.
[0125] It should be added that this application also requires preprocessing of the traffic data within the preset time window; the specific preprocessing steps will not be elaborated here.
[0126] In step 202, feature extraction is performed on each traffic data to obtain the flow context relevance features and destination IP address distribution features of each traffic data.
[0127] In one possible implementation, feature extraction is performed on each traffic data segment to obtain the flow context relevance features for each traffic data segment. The process is as follows: Figure 4 As shown, it can be implemented as follows:
[0128] For any traffic data point within a preset time window, perform the following steps:
[0129] In step 401, for any one of the multiple preset context relationship types, if the relationship between the current traffic data and the remaining traffic data other than the current traffic data satisfies any one of the relationship types, then the feature parameters of the current traffic data are updated; wherein, the feature parameters correspond to any one of the relationship types.
[0130] In step 402, the flow context relevance features of the current flow data are determined based on multiple feature parameters of the current flow data.
[0131] In this application's embodiments, there are multiple preset context relationship types. Let f and g represent any two traffic flows, then the five preset context relationship types are as follows:
[0132] R S_DP f.sip=g.sip, f.dip=g.dip, f.dport=g.dport indicates that the same source host requests the same target service;
[0133] R S_D f.sip = g.sip, f.dip = g.dip, indicating that the same source host requests one or more services from the destination IP;
[0134] R DP f.dip = g.dip, d.fport = g.dport indicates that one or more source hosts are requesting the same target service;
[0135] R D f.dip = g.dip, indicating that one or more source hosts are requesting the same destination IP;
[0136] R S f.sip = g.sip, receives the same destination IP on one or more destination IPs or target services.
[0137] When a user accesses a target service using the anonymous proxy system SS, all connections initiated by the host application to the target service are proxied, causing the context of the stream to change.
[0138] For example, when a user accesses a webpage containing images, videos, and advertisements, the service needs to be provided by a web server, image server, video server, and advertisement server. Without using the anonymous proxy system Shadowsocks (SS), the user's host needs to establish connections with n target hosts simultaneously. If the anonymous proxy system SS is used, the host will establish multiple connections with the SS server, which will then send requests to the n target hosts. Therefore, satisfying R... S_D and R S_DP The amount of traffic data will be significantly higher than that in non-SS processes.
[0139] When multiple users on a network use the same Shadowsocks (SS) service simultaneously, traffic from different hosts to multiple different target services is handled by the anonymous proxy system SS. However, when SS is not used, traffic between different hosts and different target servers is unrelated. Therefore, when using SS, the following condition is met: D and R DP The amount of traffic data will be significantly higher than that in non-SS processes.
[0140] Based on the changes in flow context characteristics caused by using the anonymous proxy system SS and not using the anonymous proxy system SS as described above, embodiments of this application determine whether the flow involves the anonymous proxy system SS by obtaining the flow context relevance characteristics of each flow data.
[0141] Within the preset time window w flowWithin the window, use the following formula to calculate the correlation between any currently processed flow f and the preceding and following flows G = g1, g2, ...;
[0142]
[0143] For example, each traffic data f will obtain a feature vector [S_DP, S_D, DP, D, S]. f and each other traffic data in the window are calculated using the above formula. If the relationship in the above formula is satisfied, then +1 is added. For example, if traffic data f and 10 traffic data in the window satisfy the relationship S_DP and the relationship S_D, 20 data satisfy the relationship DP and the relationship D, and 0 data satisfy the relationship S, then the feature of traffic data f will be [10, 10, 20, 20, 0]. This feature is the flow context relevance feature.
[0144] In one possible implementation, feature extraction is performed on each traffic data segment to obtain the destination IP address distribution characteristics of each traffic data segment. The process is as follows: Figure 5 As shown, it can be implemented as follows:
[0145] In step 501, the probability distribution of each destination IP address in the preset time window is determined based on the destination IP address of each traffic data in the preset time window.
[0146] In step 502, based on the probability distribution, the entropy value of the preset time window is obtained, and the entropy value is used as the destination IP address distribution feature of any traffic data in the preset time window.
[0147] When a host uses the anonymous proxy system SS, the network connection that the client should establish with the target server changes to a connection between the client and the SS server. This causes more traffic data to be concentrated on a single host. Therefore, the distribution of destination IP addresses for traffic data generated by a host using the anonymous proxy system SS exhibits a certain degree of centralization, unlike the dispersed distribution of destination IP addresses for traffic data generated by a host not using the anonymous proxy system SS. This application's embodiments measure the degree of dispersion of destination IP addresses by calculating the entropy of the set of destination IP addresses accessed by the host within a given time period.
[0148] For example, let dipSet = {ip1, ip2, ..., ip...} n} represents the destination IP address of the host connection within the preset time window. i Let P = {p1, p2, ..., p...} be the i-th destination IP address. i ,...,p n} is the probability distribution of dipSet, p i It is an IP address i The probability, entropy is The higher the entropy value, the more dispersed the IP addresses of the destination hosts accessed by the host, and the lower the probability that the host uses the anonymous proxy system SS; the lower the entropy value, the more concentrated the IP addresses of the destination hosts accessed by the host, and the higher the probability that the host uses the anonymous proxy system SS.
[0149] Based on user habits, this application proposes effective features, including flow context relevance features and destination IP address distribution features. Based on these effective features, risky traffic and normal traffic can be distinguished.
[0150] In step 203, based on the obtained contextual relevance features of each flow and the distribution features of the destination IP address, a kernel density estimation algorithm is used to determine the risk value of each flow data.
[0151] In one possible implementation, based on the obtained contextual relevance features of each flow and the distribution features of destination IP addresses, a kernel density estimation algorithm is used to determine the risk value of each flow data. The process is as follows: Figure 6 As shown, it can be implemented as follows:
[0152] For any data point within a preset time window, perform the following steps:
[0153] In step 601, the feature matrix vector of the current traffic data is determined based on the flow context relevance features and destination IP address distribution features of the current traffic data.
[0154] For example, if the current traffic data has the flow context relevance feature [10,10,20,20,0] and the destination IP address distribution feature, i.e., the entropy, is [5], then the feature matrix vector of the current traffic data is [10,10,20,20,0,5].
[0155] In step 602, the difference between the feature matrix vector of the current traffic data and the feature matrix vector of each traffic data in the preset time window is used as the distance between the current traffic data and each traffic data in the preset time window.
[0156] In step 603, the distance is standardized using a preset bandwidth to obtain the standardized distance between the current traffic data and each traffic data in the preset time window.
[0157] In step 604, based on the obtained standardized distances and Gaussian kernel functions, kernel density estimation is performed on each flow data to obtain the risk value of the current flow data.
[0158] In one possible implementation, kernel density estimation is performed on each traffic data point based on the obtained standardized distances and Gaussian kernel functions to obtain the risk value of the current traffic data. The process is as follows: Figure 7As shown, it can be implemented as follows:
[0159] In step 701, the obtained standardized distances are input into a Gaussian kernel function to obtain the influence of each flow data point on the current flow data within a preset time window.
[0160] In step 702, the influence of each traffic data point in the preset time window on the current traffic data is summed to obtain the total influence of the preset time window.
[0161] In step 703, the sum of influence is normalized using the preset bandwidth and the number of all traffic data in the preset time window to obtain the kernel function value of the current traffic data.
[0162] In step 704, the kernel function value of the current traffic data is used as the risk value of the current traffic data.
[0163] In one possible implementation, embodiments of this application use the following kernel density estimation algorithm formula to determine the risk value of the current traffic data:
[0164]
[0165] Where x represents the feature matrix vector of the current traffic data, x i This represents the feature matrix vector of the i-th traffic data point within a preset time window. Represents the Gaussian kernel function. Indicates the preset bandwidth. This represents the standard deviation of all traffic data within a preset time window. This represents the mean of all traffic data within a preset time window, where n or N represents the number of traffic data points within that preset time window. This indicates the risk value of the current traffic data.
[0166] In step 204, if the risk value of any traffic data exceeds the preset risk threshold, then the preset time window is determined as the risk window.
[0167] For example, setting risk thresholds When the value of one or more traffic data within a preset time window exceeds the risk threshold, the preset time window is designated as a risk window, and all data within the risk window is designated as suspected risk data and used for the next step of classification model processing.
[0168] In step 205, the traffic data in the risk window is classified using the trained classification model to obtain the traffic classification result.
[0169] In one possible implementation, the embodiments of this application employ the following steps to train a classification model, the process of which is as follows: Figure 8As shown, it can be implemented as follows:
[0170] In step 801, training samples are obtained, which include a set of sample traffic data and a set of labeled samples of traffic data.
[0171] In step 802, the sample traffic data set is input into the classification model for classification prediction processing to obtain the predicted label set.
[0172] In step 803, the obtained predicted label set and labeled label set are compared to determine the loss function value, and the network parameters of the classification model are updated based on the loss function value.
[0173] For example, the sample traffic data of the known label is X. known Its label is Y known Traffic data for unknown tags is X. unknown Its predicted label is Y unknown All data X T =[X known ,X unknown ], Y T =[Y known ,Y unknown [X], where +1 represents a positive class, indicating risky traffic, and -1 represents a negative class, indicating normal traffic. This application's embodiments use X. known and the corresponding label Y known To train the initial support vector machine (SVM) model f(x) = ω T x+b, where ω is the slope and b is the bias, input a sample traffic data and its corresponding label, compare the obtained predicted label set and the labeled label set to determine the loss function value, update the network parameters of the classification model based on the loss function value, and finally determine the optimal ω and b through training to obtain the optimal support vector machine (SVM) model.
[0174] In one possible implementation, when training the classification model, this embodiment of the application also adds traffic data whose predicted labels meet the high confidence condition to the training samples, and iteratively trains the classification model using the updated training samples until there is no traffic data that meets the high confidence condition or the preset number of iterations is reached; wherein, the high confidence condition is that the absolute value of the predicted label is greater than a preset confidence threshold.
[0175] For example, traffic data with an unknown label is X. unknown In this process, traffic data with high confidence levels are selected as samples. Confidence is measured by the absolute value of the output of the decision function; a larger value indicates greater distance from the hyperplane and higher confidence. This embodiment utilizes the formula |ω T x j+b|>0.9, select the traffic data with high confidence, and add the high-confidence traffic to the sample traffic dataset with known labels and the labeled label set X. known Y known Each new sample traffic data point is considered a new sample traffic data point. Training continues until no more high-confidence traffic data can be added to the sample traffic dataset, or until a preset number of iterations (e.g., 100,000) is reached, at which point training is complete.
[0176] The above-mentioned method for training the classification model is a semi-supervised learning algorithm. It can effectively classify and determine unknown traffic data when only a portion of labeled data is available, thus solving the industry's pain point of being unable to effectively determine actual data due to a lack of data.
[0177] In one possible implementation, the traffic data within the risk window is classified using a trained classification model to obtain the traffic classification result. This can be implemented as follows:
[0178] For any traffic data point in the risk window, perform the following steps:
[0179] The current traffic data is classified and predicted using the decision function of the trained classification model to obtain the predicted label.
[0180] If the predicted label is greater than or equal to 0, the current traffic data is determined to be risky traffic;
[0181] If the predicted label is less than 0, then the current traffic data is determined to be normal traffic.
[0182] For example, using a trained classification model to analyze traffic data X with unknown labels. unknown Perform an initial prediction to obtain the predicted label Y. unknown For any traffic data x with an unknown label j ∈X unknown Through the decision function of the classification model Make a prediction if the predicted label If it is positive, add 1; otherwise, subtract 1 from the negative category. That is, risk traffic + 1 and normal traffic - 1.
[0183] In one possible implementation, when a traffic data point is marked as +1 (i.e., risky traffic) after being processed by the classification model, the system can perform a series of processes such as tracking and alerting as needed.
[0184] In summary, the anonymous traffic detection method provided in this application can filter out risky data within a preset time window based on effective features and a kernel density estimation algorithm, and divide the risk window accordingly. The traffic data within the risk window is then classified and identified using a classification model to obtain the classification result. This eliminates the need to classify and identify traffic data across all time windows, effectively reducing computational overhead, accelerating identification speed, and improving the accuracy of anonymous traffic detection. Furthermore, the technical solution provided in this application is simple and has good versatility.
[0185] Based on the same inventive concept, embodiments of this application also provide an anonymous traffic detection device, such as... Figure 9 As shown, the device 900 includes:
[0186] The traffic data acquisition module 901 is configured to acquire each piece of traffic data within a preset time window;
[0187] The feature determination module 902 is configured to extract features from each of the traffic data to obtain the flow context relevance features and destination IP address distribution features of each of the traffic data.
[0188] The risk value determination module 903 is configured to determine the risk value of each traffic data based on the obtained context relevance features of each flow and the distribution features of the destination IP address, using a kernel density estimation algorithm.
[0189] The risk window determination module 904 is configured to determine the preset time window as a risk window if the risk value of any of the traffic data is greater than a preset risk threshold.
[0190] The traffic data classification module 905 is configured to classify the traffic data in the risk window using a trained classification model to obtain the traffic classification result.
[0191] In one possible implementation, feature extraction is performed on each of the traffic data to obtain the flow context relevance features of each traffic data, and the feature determination module is configured to:
[0192] For any traffic data point within the preset time window, perform the following steps:
[0193] For any one of the multiple preset context relationship types, if the relationship between the current traffic data and the remaining traffic data (excluding the current traffic data) satisfies any one of the relationship types, then the feature parameters of the current traffic data are updated; wherein, the feature parameters correspond to any one of the relationship types.
[0194] Based on multiple feature parameters of the current traffic data, the flow context relevance features of the current traffic data are determined.
[0195] In one possible implementation, feature extraction is performed on each of the traffic data to obtain the destination IP address distribution characteristics of each traffic data, and the feature determination module is configured as follows:
[0196] Based on the destination IP address of each traffic data item within the preset time window, determine the probability distribution of each destination IP address within the preset time window;
[0197] Based on the probability distribution, the entropy value of the preset time window is obtained, and the entropy value is used as the destination IP address distribution feature of any traffic data in the preset time window.
[0198] In one possible implementation, the risk value of each traffic data item is determined using a kernel density estimation algorithm based on the obtained context relevance features and destination IP address distribution features. The risk value determination module is configured to:
[0199] For any traffic data point within the preset time window, perform the following steps:
[0200] Based on the flow context relevance features and destination IP address distribution features of the current traffic data, the feature matrix vector of the current traffic data is determined;
[0201] The difference between the feature matrix vector of the current traffic data and the feature matrix vector of each traffic data in the preset time window is used as the distance between the current traffic data and each traffic data in the preset time window.
[0202] The distance is standardized using a preset bandwidth to obtain the standardized distance between the current traffic data and each traffic data in the preset time window.
[0203] Based on the obtained standardized distances and Gaussian kernel functions, kernel density estimation is performed on each traffic data to obtain the risk value of the current traffic data.
[0204] In one possible implementation, the step of performing kernel density estimation on each traffic data based on the obtained standardized distances and Gaussian kernel functions to obtain the risk value of the current traffic data, wherein the risk value determination module is configured to:
[0205] The obtained standardized distances are input into a Gaussian kernel function to obtain the influence of each traffic data point on the current traffic data within the preset time window;
[0206] The influence of each traffic data point in the preset time window on the current traffic data is summed to obtain the total influence of the preset time window.
[0207] The sum of influence is normalized using the preset bandwidth and the number of all traffic data in the preset time window to obtain the kernel function value of the current traffic data;
[0208] The kernel function value of the current traffic data is used as the risk value of the current traffic data.
[0209] In one possible implementation, the traffic data classification module is configured to train a classification model using the following steps:
[0210] Obtain training samples, which include a set of sample traffic data and a set of labeled tags for the sample traffic data;
[0211] The sample traffic data set is input into a classification model for classification prediction processing to obtain a set of predicted labels.
[0212] The obtained predicted label set and labeled label set are compared to determine the loss function value, and the network parameters of the classification model are updated based on the loss function value.
[0213] In one possible implementation, the traffic data classification module is further configured to:
[0214] When training the classification model, traffic data whose predicted labels meet the high confidence condition are added to the training samples, and the classification model is iteratively trained using the updated training samples until there is no traffic data that meets the high confidence condition or the preset number of iterations is reached; wherein, the high confidence condition is that the absolute value of the predicted label is greater than a preset confidence threshold.
[0215] In one possible implementation, the traffic data classification module is configured to classify the traffic data in the risk window using a trained classification model to obtain a traffic classification result.
[0216] For any traffic data point in the risk window, perform the following steps:
[0217] The current traffic data is classified and predicted using the decision function of the trained classification model to obtain the predicted label.
[0218] If the predicted label is greater than or equal to 0, then the current traffic data is determined to be risky traffic;
[0219] If the predicted label is less than 0, then the current traffic data is determined to be normal traffic.
[0220] The following reference Figure 10 To describe an electronic device 130 according to this embodiment of the present application. Figure 10 The electronic device 130 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0221] like Figure 10 As shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).
[0222] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.
[0223] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.
[0224] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0225] Electronic device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 130, and / or with any device that enables electronic device 130 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, electronic device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in electronic device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0226] In an exemplary embodiment, this application also provides a computer-readable storage medium including instructions, such as a memory 132 including instructions, which can be executed by a processor 131 of an electronic device 130 to complete the aforementioned method for detecting anonymous traffic. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.
[0227] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor 131, implements the anonymous traffic detection method provided in this application.
[0228] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0229] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0230] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0231] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0232] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for detecting anonymous traffic, characterized in that, The method includes: Retrieve traffic data for each segment within a preset time window; For any traffic data within the preset time window, the following steps are performed: For any one of the multiple preset context relationship types, if the relationship between the current traffic data and the remaining traffic data (excluding the current traffic data) satisfies any one of the relationship types, then the feature parameters of the current traffic data are updated; wherein, the feature parameters correspond to any one of the relationship types; based on the multiple feature parameters of the current traffic data, the flow context relevance features of the current traffic data are determined. The various preset context relationship types include: This indicates that the same source host is requesting the same target service. This indicates that the same source host requests one or more services from the destination IP. This indicates that one or more source hosts are requesting the same target service. This indicates that one or more source hosts are requesting the same destination IP. , receiving the same destination IP on one or more destination IPs or target services; in, These are any two data points. For traffic data The source host IP, For traffic data The source host IP, For traffic data The destination IP address of the request For traffic data The destination IP address of the request For traffic data The target service, For traffic data The target service; Based on the destination IP address of each traffic data in the preset time window, determine the probability distribution of each destination IP address in the preset time window; based on the probability distribution, obtain the entropy value of the preset time window, and use the entropy value as the distribution feature of the destination IP address of any traffic data in the preset time window; Obtain the vector corresponding to the flow context relevance feature of each traffic data and the vector corresponding to the IP address distribution feature within a preset time window. Concatenate the vector corresponding to the flow context relevance feature of the current traffic data and the vector corresponding to the IP address distribution feature within the preset time window of the current traffic data to obtain the feature matrix vector of the current traffic data. Based on the feature matrix vector, use a kernel density estimation algorithm to determine the risk value of each traffic data. If the risk value of any of the traffic data is greater than the preset risk threshold, then the preset time window is determined as a risk window; The traffic data in the risk window is classified using a trained classification model to obtain the traffic classification result.
2. The method according to claim 1, characterized in that, The step of determining the risk value of each traffic data point based on the feature matrix vector using a kernel density estimation algorithm includes: For any traffic data point within the preset time window, perform the following steps: The difference between the feature matrix vector of the current traffic data and the feature matrix vector of each traffic data in the preset time window is used as the distance between the current traffic data and each traffic data in the preset time window. The distance is standardized using a preset bandwidth to obtain the standardized distance between the current traffic data and each traffic data in the preset time window. Based on the obtained standardized distances and Gaussian kernel functions, kernel density estimation is performed on each traffic data to obtain the risk value of the current traffic data.
3. The method according to claim 2, characterized in that, The step of performing kernel density estimation on each traffic data based on the obtained standardized distances and Gaussian kernel functions to obtain the risk value of the current traffic data includes: The obtained standardized distances are input into a Gaussian kernel function to obtain the influence of each traffic data point on the current traffic data within the preset time window; The influence of each traffic data point in the preset time window on the current traffic data is summed to obtain the total influence of the preset time window. The sum of influence is normalized using the preset bandwidth and the number of all traffic data in the preset time window to obtain the kernel function value of the current traffic data; The kernel function value of the current traffic data is used as the risk value of the current traffic data.
4. The method according to claim 1, characterized in that, The classification model is trained using the following steps: Obtain training samples, which include a set of sample traffic data and a set of labeled tags for the sample traffic data; The sample traffic data set is input into a classification model for classification prediction processing to obtain a set of predicted labels. The obtained predicted label set and labeled label set are compared to determine the loss function value, and the network parameters of the classification model are updated based on the loss function value.
5. The method according to claim 4, characterized in that, The method further includes: When training the classification model, traffic data whose predicted labels meet the high confidence condition are added to the training samples, and the classification model is iteratively trained using the updated training samples until there is no traffic data that meets the high confidence condition or the preset number of iterations is reached; wherein, the high confidence condition is that the absolute value of the predicted label is greater than a preset confidence threshold.
6. The method according to any one of claims 1 to 5, characterized in that, The step of classifying the traffic data in the risk window using a trained classification model to obtain traffic classification results includes: For any traffic data point in the risk window, perform the following steps: The current traffic data is classified and predicted using the decision function of the trained classification model to obtain the predicted label. If the predicted label is greater than or equal to 0, then the current traffic data is determined to be risky traffic; If the predicted label is less than 0, then the current traffic data is determined to be normal traffic.
7. A device for detecting anonymous traffic, characterized in that, The device includes: The traffic data acquisition module is configured to acquire traffic data for each data point within a preset time window; The feature determination module is configured to perform the following steps for any traffic data within the preset time window: for any one of multiple preset context relationship types, if the relationship between the current traffic data and the remaining traffic data (excluding the current traffic data) satisfies any one of the relationship types, then update the feature parameters of the current traffic data; wherein the feature parameters correspond to the any one of the relationship types; determine the flow context relevance features of the current traffic data based on the multiple feature parameters of the current traffic data; The various preset context relationship types include: This indicates that the same source host is requesting the same target service. This indicates that the same source host requests one or more services from the destination IP. This indicates that one or more source hosts are requesting the same target service. This indicates that one or more source hosts are requesting the same destination IP. , receiving the same destination IP on one or more destination IPs or target services; in, These are any two data points. For traffic data The source host IP, For traffic data The source host IP, For traffic data The destination IP address of the request For traffic data The destination IP address of the request For traffic data The target service, For traffic data The target service; The feature determination module is further configured to determine the probability distribution of each destination IP address in the preset time window based on the destination IP address of each traffic data in the preset time window; obtain the entropy value of the preset time window based on the probability distribution, and use the entropy value as the destination IP address distribution feature of any traffic data in the preset time window; The risk value determination module is configured to obtain the vector corresponding to the flow context relevance feature of each traffic data and the vector corresponding to the IP address distribution feature within a preset time window; concatenate the vector corresponding to the flow context relevance feature of the current traffic data and the vector corresponding to the IP address distribution feature within the preset time window of the current traffic data to obtain the feature matrix vector of the current traffic data; and determine the risk value of each traffic data based on the feature matrix vector using a kernel density estimation algorithm. The risk window determination module is configured to determine the preset time window as a risk window if the risk value of any of the traffic data is greater than a preset risk threshold. The traffic data classification module is configured to classify the traffic data in the risk window using a trained classification model to obtain the traffic classification result.
8. The apparatus according to claim 7, characterized in that, Based on the feature matrix vector, a kernel density estimation algorithm is used to determine the risk value of each traffic data point. The risk value determination module is configured as follows: For any data point within the preset time window, perform the following steps: The difference between the feature matrix vector of the current traffic data and the feature matrix vector of each traffic data in the preset time window is used as the distance between the current traffic data and each traffic data in the preset time window. The distance is standardized using a preset bandwidth to obtain the standardized distance between the current traffic data and each traffic data in the preset time window. Based on the obtained standardized distances and Gaussian kernel functions, kernel density estimation is performed on each traffic data to obtain the risk value of the current traffic data.
9. The apparatus according to claim 8, characterized in that, The risk value determination module is configured to perform kernel density estimation on each traffic data based on the obtained standardized distances and Gaussian kernel functions to obtain the risk value of the current traffic data. The obtained standardized distances are input into a Gaussian kernel function to obtain the influence of each traffic data point on the current traffic data within the preset time window; The influence of each traffic data point in the preset time window on the current traffic data is summed to obtain the total influence of the preset time window. The sum of influence is normalized using the preset bandwidth and the number of all traffic data in the preset time window to obtain the kernel function value of the current traffic data; The kernel function value of the current traffic data is used as the risk value of the current traffic data.
10. The apparatus according to claim 7, characterized in that, The traffic data classification module is configured to train a classification model using the following steps: Obtain training samples, which include a set of sample traffic data and a set of labeled tags for the sample traffic data; The sample traffic data set is input into a classification model for classification prediction processing to obtain a set of predicted labels. The obtained predicted label set and labeled label set are compared to determine the loss function value, and the network parameters of the classification model are updated based on the loss function value.
11. The apparatus according to claim 10, characterized in that, The traffic data classification module is also configured to: When training the classification model, traffic data whose predicted labels meet the high confidence condition are added to the training samples, and the classification model is iteratively trained using the updated training samples until there is no traffic data that meets the high confidence condition or the preset number of iterations is reached; wherein, the high confidence condition is that the absolute value of the predicted label is greater than a preset confidence threshold.
12. The apparatus according to any one of claims 7 to 11, characterized in that, The traffic data in the risk window is classified using a trained classification model to obtain a traffic classification result. The traffic data classification module is configured as follows: For any traffic data point in the risk window, perform the following steps: The current traffic data is classified and predicted using the decision function of the trained classification model to obtain the predicted label. If the predicted label is greater than or equal to 0, then the current traffic data is determined to be risky traffic; If the predicted label is less than 0, then the current traffic data is determined to be normal traffic.
13. An electronic device, characterized in that, include: Processor and memory; The memory is used to store the processor-executable instructions; The processor is configured to execute the instructions to implement the method for detecting anonymous traffic as described in any one of claims 1-6.
14. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method for detecting anonymous traffic as described in any one of claims 1-6.
Citation Information
Patent Citations
Method and system of malicious http detection
CN107483488A
Information security risk prediction method and device, computing equipment and storage medium
CN111181757A