A method and related device for identifying malicious traffic

Abstract

The embodiment of the present application provides a malicious traffic identification method and related devices, wherein a malicious traffic identification method may include: determining the reception time of the first alarm traffic; obtaining multiple second alarm traffic corresponding to the first alarm traffic within a target time period based on a preset strategy; the target time period is a time period determined based on the reception time; the similarity between each second alarm traffic in the multiple second alarm traffic and the first alarm traffic is greater than a preset threshold; extracting features from the multiple second alarm traffic to obtain first feature information; based on the first feature information, determining whether the first alarm traffic is malicious traffic. By implementing the embodiment of the present application, the accuracy of malicious traffic identification in the existing network can be improved by using a multi-stream backtracing method.

Description

[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on December 31, 2020, with application number 202011639885.1 and application name “A method for identifying malicious traffic and related devices”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a malicious traffic identification method and related devices. Background Art

[0003] As the most important protocol at present, the HyperText Transfer Protocol (HTTP) is widely used on the Internet. In order to facilitate communication and conceal malicious behavior, various malware, such as Trojans, often use HTTP communication methods, which mainly refers to the communication between the controlled node and the Trojan's command and control server (CC / C2). Since Trojans are updated and iterated very quickly, the updated Trojans will have obvious differences from the previous communication traffic. There are currently two ways to detect malicious traffic using HTTP: 1) Detection from the traffic layer, that is, detection by extracting features from the traffic; 2) Detection from the host behavior, that is, detection by extracting features from the infected host behavior. Among them, there are two main methods for processing the extracted features: 1) Unsupervised clustering detection method; 2) Supervised model-based detection method.

[0004] However, whether detecting from the features corresponding to the traffic layer or from the features corresponding to the host behavior, the unsupervised clustering detection method and the supervised model detection method only consider the features of a single stream, that is, the features of an HTTP stream, and do not consider the multi-stream network behavior features of malicious CC communications, that is, the features of multiple HTTP streams. The current detection method does not have enough basic information richness and cannot effectively and accurately identify whether the traffic is malicious traffic. Moreover, the behavior of many rogue software is similar to the characteristic behavior of CC traffic at the single-stream level. Using only the single-stream feature analysis method cannot effectively distinguish between rogue software and malware.

[0005] Therefore, how to more accurately detect malicious traffic in the network is an urgent problem to be solved. Summary of the invention

[0006] The embodiments of the present application provide a malicious traffic identification method and related devices to improve the accuracy of malicious traffic identification.

[0007] In a first aspect, an embodiment of the present application provides a method for identifying malicious traffic, which may include:

[0008] Determine the receiving time of the first alarm flow; obtain multiple second alarm flows corresponding to the first alarm flow within a target time period based on a preset strategy; the target time period is a time period determined based on the receiving time; the similarity between each of the multiple second alarm flows and the first alarm flow is greater than a preset threshold; perform feature extraction on the multiple second alarm flows to obtain first feature information; based on the first feature information, determine whether the first alarm flow is malicious traffic.

[0009] In the implementation of the first aspect, the malicious traffic identification device can trace back from the time of receiving a single traffic (i.e., the first alarm traffic) according to a preset strategy to obtain multiple traffics (i.e., multiple second alarm traffics) that match the single traffic. Then, feature extraction is performed on the multiple traffics traced back to obtain feature information, so that the malicious traffic identification device can classify the above-mentioned single traffic according to the feature information, thereby determining whether the single traffic is malicious traffic. Among them, the similarity between the multiple second alarm traffics and the first alarm traffic is greater than a preset threshold. This method of classifying a single traffic according to the feature information of multiple traffics that are similar to a single traffic enables the malicious traffic identification device to fully consider the characteristics of the multi-flow network behavior of the malicious CC communication traffic when identifying the traffic, thereby more accurately detecting and distinguishing the malicious traffic in the existing network. This avoids the randomness of the detection of a single HTTP stream in the existing technology during the detection process due to the relatively complex traffic conditions in the existing network. In addition, the embodiment of the present application observes the communication behavior of the traffic from a multi-stream perspective, traces back multiple alarm flows to different clusters based on one or more preset strategies, uses the statistical feature information of different clusters to which each alarm flow belongs, and judges the positive and negative (i.e., whether the alarm flow is malicious flow) based on the feature information, thereby eliminating accidental errors. This observation of the overall communication behavior of malicious traffic within a certain period of time can judge malicious samples from a behavioral perspective, making the final multi-stream judgment result more robust and also behaviorally interpretable. Moreover, the embodiment of the present application detects multi-stream traffic from the characteristics corresponding to the traffic layer or from the characteristics corresponding to the host behavior. The richness of basic information is sufficient for the malicious traffic identification device to effectively and accurately identify whether the traffic is malicious traffic. Therefore, the communication traffic of rogue software and the communication traffic of malicious software can be distinguished from the characteristics of multiple streams, thereby improving the accuracy of malicious traffic identification.

[0010] In a possible implementation, the target time period is a time period of a preset length backward from the receiving time as the starting point, or a time period of a preset length forward from the receiving time as the end point. In an embodiment of the present application, the receiving time of the first alarm flow can be used as the endpoint, and a time period of a preset length forward or backward can be taken to ensure that as many second alarm flows similar to the first alarm flow as possible are obtained.

[0011] In a possible implementation, the preset strategy includes: one or more of a first strategy, a second strategy, and a third strategy, wherein the first strategy is a strategy for obtaining the multiple second alarm flows based on the Internet Protocol IP address and the User Agent UA information of the first alarm flow; the second strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and a preset generalization rule; the third strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and the Hypertext Transfer Protocol HTTP Header information of the first alarm flow. By implementing the embodiments of the present application, a variety of traffic backtracking methods can accurately trace back to multiple flows of the same source as the first alarm flow, so that it can be identified whether the first alarm flow is malicious flow based on the behavioral characteristics of the multiple flows, thereby improving the accuracy of identifying malicious flow.

[0012] In one possible implementation, the preset strategy includes the first strategy; the collecting of multiple second alarm flows corresponding to the first alarm flow within the target time period according to the preset strategy includes: obtaining the IP address and UA information of the first alarm flow; collecting the HTTP flow with the same UA information as the first alarm flow among the multiple HTTP flows sent by the IP address within the target time period as the second alarm flow. By implementing the embodiment of the present application, by tracing back the flows with the same source IP address and UA information as the first alarm information, it is possible to trace back to multiple flows sent by the same software, the same service device, or the same application, thereby determining whether the first alarm flow is malicious flow based on the behavioral characteristics of the multiple flows traced back, thereby improving the accuracy of identifying malicious flow.

[0013] In a possible implementation, the preset strategy includes the second strategy, and the collecting of multiple second alarm flows corresponding to the first alarm flow in the target time period according to the preset strategy includes: obtaining the IP address of the first alarm flow; collecting multiple first HTTP flows sent by the IP address in the target time period; generalizing the multiple first HTTP flows according to the preset generalization rule to obtain multiple second HTTP flows, wherein the preset generalization rule is to uniformly replace the target string corresponding to each of the multiple first HTTP flows with a preset standard; and filtering out the target second HTTP flows whose similarity with the first alarm flow is greater than a preset threshold from the multiple second HTTP flows as the second alarm flow. In the implementation of the embodiment of the present application, after generalization, the method of calculating the similarity between flows is used to further determine multiple flows (sent by the same software, different applications) in the same cluster as the first alarm flow (the similarity exceeds the preset threshold), and then determining whether the first alarm flow is malicious flow according to the behavioral characteristics of the multiple flows, thereby improving the accuracy of identifying malicious flow.

[0014] In a possible implementation, the preset strategy includes the third strategy, and the collecting of multiple second alarm flows corresponding to the first alarm flow in the target time period according to the preset strategy includes: obtaining the IP address and the HTTP Header information of the first alarm flow; collecting multiple third HTTP flows sent by the IP address in the target time period; performing N-gram processing on the HTTP Header corresponding to each of the multiple third HTTP flows to obtain a first matrix, the first matrix including the HTTP Header sequence information corresponding to each third HTTP flow; performing dimensionality reduction processing on the first matrix, extracting the target HTTP Header sequence information matching the HTTP Header information of the first alarm flow in the first matrix after dimensionality reduction processing; based on the target HTTP Header sequence information, obtaining the third HTTP flow corresponding to the target HTTP Header sequence information as the second alarm flow. In the implementation of the embodiment of the present application, by extracting the HTTP Header sequence information in the flow for backtracking, it is possible to backtrack to multiple flows sent by different applications in the same software, and then determine whether the first alarm flow is malicious flow according to the behavioral characteristics of the multiple flows, thereby improving the accuracy of identifying malicious flow.

[0015] In one possible implementation, the first feature information is a feature representation vector; the feature extraction of the multiple second alarm flows to obtain the first feature information includes: feature extraction of the multiple second alarm flows to obtain behavior feature information corresponding to the multiple second alarm flows, the behavior feature information including: one or more of connection behavior features, request difference features, and request response features; according to the behavior feature information, the feature representation vector is obtained. By implementing the embodiment of the present application and extracting behavior features of multi-flow flows, it is possible to distinguish the flows corresponding to rogue software and the flows corresponding to malware, thereby improving the accuracy of malicious flow identification.

[0016] In one possible implementation, judging whether the first alarm traffic is malicious traffic based on the first characteristic information includes: performing detection through a backtracking model based on the first characteristic information to obtain a first detection result; performing detection through a baseline model based on the multiple second alarm traffic to obtain a second detection result, wherein the baseline model is a detection model pre-trained based on historical traffic; judging whether the first alarm traffic is malicious traffic based on the first detection result and the second detection result. By implementing the embodiment of the present application, by comprehensively considering the first detection result detected through the backtracking model and the second detection result detected through the baseline model, it is finally determined whether the first alarm traffic is malicious traffic, which greatly improves the accuracy of malicious traffic identification.

[0017] In one possible implementation, the method further includes: if the first alarm traffic is malicious traffic, performing a preset generalization process on the first alarm traffic to obtain a generalized first alarm traffic; and classifying the generalized first alarm traffic to determine the type of malicious traffic that matches the first alarm traffic. By implementing the embodiment of the present application, by classifying the first alarm traffic after generalization, the type of malicious traffic that matches the first alarm traffic can be determined, so as to better maintain network security.

[0018] In one possible implementation, before determining the reception time of the first alarm traffic, it also includes: receiving multiple fourth HTTP streams; performing feature extraction on each of the multiple fourth HTTP streams according to a preset feature extraction rule to obtain a second feature set, wherein the second feature set includes: second feature information corresponding to the multiple fourth HTTP streams respectively; based on the second feature set, the first alarm traffic is screened out from the multiple fourth HTTP streams through a first classification model. By implementing the embodiment of the present application, the first alarm traffic suspected of malicious traffic is screened out from multiple fourth HTTP streams (i.e., single-stream filtering) through a first classification model according to single-stream traffic features (such as manual features and / or representation learning features), which can effectively reduce the storage and detection of a large number of irrelevant data streams during the detection process and improve the analysis efficiency of malicious traffic.

[0019] In one possible implementation, the second feature information includes manual feature information and / or representation learning feature information; wherein the manual feature information includes: one or more of the domain name readability feature, uniform resource locator URL structure feature, behavior indication feature, and HTTP Header feature corresponding to the fourth HTTP stream; the representation learning feature information includes the high-dimensional feature corresponding to the fourth HTTP stream. When implementing the embodiment of the present application, when implementing single-stream filtering to extract the first alarm flow of suspected malicious traffic in the existing network traffic, it can be achieved by identifying the manual features and / or representation learning features corresponding to the traffic, for example: extracting one or more of the domain name readability feature, uniform resource locator URL structure feature, behavior indication feature, and HTTP Header feature corresponding to the multiple fourth HTTP streams; for another example: extracting the high-dimensional features corresponding to the multiple fourth HTTP streams based on the representation learning model. The accuracy of single-stream filtering in identifying the first alarm flow of suspected malicious traffic is improved, and the analysis efficiency of malicious traffic is improved.

[0020] In a second aspect, an embodiment of the present application provides a malicious traffic identification device, including:

[0021] A determination unit, used to determine a reception time of the first alarm flow;

[0022] A tracing unit, configured to obtain a plurality of second alarm flows corresponding to the first alarm flow within a target time period according to a preset strategy; the target time period is a time period determined based on the receiving time; and the similarity between each of the plurality of second alarm flows and the first alarm flow is greater than a preset threshold;

[0023] An extraction unit, configured to extract features from the plurality of second alarm flows to obtain first feature information;

[0024] A judging unit is used to judge whether the first alarm traffic is malicious traffic based on the first characteristic information.

[0025] In one possible implementation, the preset strategy includes: one or more of a first strategy, a second strategy, and a third strategy, wherein the first strategy is a strategy for obtaining the multiple second alarm flows based on the Internet Protocol IP address and user agent UA information of the first alarm flow; the second strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and a preset generalization rule; the third strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and the Hypertext Transfer Protocol HTTP Header information of the first alarm flow.

[0026] In one possible implementation, the preset strategy includes the first strategy; the backtracking unit is specifically used to: obtain the IP address and UA information of the first alarm traffic; collect multiple HTTP streams sent by the IP address within the target time period, and the HTTP stream with the same UA information as the first alarm traffic is the second alarm traffic.

[0027] In one possible implementation, the preset strategy includes the second strategy, and the backtracking unit is specifically used to: obtain the IP address of the first alarm traffic; collect multiple first HTTP streams sent by the IP address within the target time period; generalize the multiple first HTTP streams according to the preset generalization rule to obtain multiple second HTTP streams, and the preset generalization rule is to uniformly replace the target string corresponding to each first HTTP stream in the multiple first HTTP streams using a preset standard; from the multiple second HTTP streams, filter out the target second HTTP stream whose similarity with the first alarm traffic is greater than a preset threshold as the second alarm traffic.

[0028] In one possible implementation, the preset strategy includes the third strategy, and the backtracking unit is specifically used to: obtain the IP address and the HTTP Header information of the first alarm traffic; collect multiple third HTTP streams sent by the IP address within the target time period; perform N-gram processing on the HTTP Header corresponding to each of the multiple third HTTP streams to obtain a first matrix, wherein the first matrix includes HTTP Header sequence information corresponding to each third HTTP stream; perform dimensionality reduction processing on the first matrix to extract the target HTTP Header sequence information that matches the HTTP Header information of the first alarm traffic in the first matrix after dimensionality reduction processing; based on the target HTTP Header sequence information, obtain the third HTTP stream corresponding to the target HTTP Header sequence information as the second alarm traffic.

[0029] In one possible implementation, the first feature information is a feature representation vector; the extraction unit is specifically used to: perform feature extraction on the multiple second alarm flows to obtain behavior feature information corresponding to the multiple second alarm flows, the behavior feature information including: one or more of connection behavior features, request difference features, and request response features; and obtain the feature representation vector based on the behavior feature information.

[0030] In one possible implementation, the judgment unit is specifically used to: perform detection through a backtracking model based on the first feature information to obtain a first detection result; perform detection through a baseline model based on the multiple second alarm flows to obtain a second detection result, wherein the baseline model is a detection model pre-trained based on historical flows; and determine whether the first alarm flow is malicious flow based on the first detection result and the second detection result.

[0031] In one possible implementation, the device also includes: a generalization unit, which is used to perform preset generalization processing on the first alarm traffic to obtain a generalized first alarm traffic if the first alarm traffic is malicious traffic; and a classification unit, which is used to classify the generalized first alarm traffic to determine the malicious traffic type that matches the first alarm traffic.

[0032] In one possible implementation, the device also includes an alarm traffic unit, which is used to: receive multiple fourth HTTP streams before determining the receiving time of the first alarm traffic; perform feature extraction on each of the multiple fourth HTTP streams according to a preset feature extraction rule to obtain a second feature set, wherein the second feature set includes: second feature information corresponding to the multiple fourth HTTP streams respectively; based on the second feature set, filter out the first alarm traffic from the multiple fourth HTTP streams through a first classification model.

[0033] In one possible implementation, the second feature information includes manual feature information and / or representation learning feature information; wherein the manual feature information includes: one or more of domain name readability features, uniform resource locator URL structure features, behavior indication features, and HTTP Header features corresponding to the fourth HTTP stream; and the representation learning feature information includes high-dimensional features corresponding to the fourth HTTP stream.

[0034] In a third aspect, an embodiment of the present application provides a service device, the service device includes a processor, and the processor is configured to support the service device to implement the corresponding functions in the malicious traffic identification method provided in the first aspect. The service device may also include a memory, the memory is used to couple with the processor, and the memory stores the necessary program instructions and data of the service device. The service device may also include a communication interface for the service device to communicate with other devices or communication networks.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing computer software instructions used for a malicious traffic identification device provided in the second aspect above, which includes a program designed for executing the above aspect.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed by a computer, the computer can execute the process performed by the malicious traffic identification device in the second aspect above.

[0037] In a sixth aspect, the present application provides a chip system, which includes a processor for supporting a terminal device to implement the functions involved in the first aspect, for example, generating or processing the information involved in the malicious traffic identification method. In a possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for a data sending device. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0039] Figure 1 It is a schematic diagram of the architecture of a malicious traffic identification system provided in an embodiment of the present application.

[0040] Figure 2 It is a flow chart of a malicious traffic identification method provided in an embodiment of the present application.

[0041] Figure 3 It is a schematic diagram of a framework for malicious traffic identification provided in an embodiment of the present application.

[0042] Figure 4 It is a schematic diagram of feature extraction provided in an embodiment of the present application.

[0043] Figure 5 It is a flow chart of tracing back traffic according to a first strategy provided in an embodiment of the present application.

[0044] Figure 6 It is a schematic diagram of multiple traffic traced back according to the first strategy provided in an embodiment of the present application.

[0045] Figure 7 It is a flow chart of tracing back traffic according to the second strategy provided in an embodiment of the present application.

[0046] Figure 8 This is a schematic diagram of traffic before and after generalization provided in an embodiment of the present application.

[0047] Fig. 9 It is a flow chart of tracing back traffic according to the third strategy provided in an embodiment of the present application.

[0048] Fig.10 It is a flowchart of a method for obtaining first characteristic information provided in an embodiment of the present application.

[0049] Fig.11 This embodiment of the present application provides a method based on E n is the independent variable, a n is the function graph of the dependent variable.

[0050] Fig.12 It is a flowchart of determining the type of malicious traffic provided by an embodiment of the present application.

[0051] Fig.13 It is a structural diagram of a malicious traffic identification device provided in an embodiment of the present application.

[0052] Fig.14It is a structural diagram of another malicious traffic identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0054] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c or a, b and c, where a, b and c can be single or multiple.

[0056] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0058] First, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0059] (1) HyperText Transfer Protocol (HTTP) is an application layer protocol for distributed, collaborative, and hypermedia information systems. It is the basis of data communication on the World Wide Web and the most widely used network transmission protocol on the Internet. HTTP was originally designed to provide a method for publishing and receiving HTML pages.

[0060] (2) Command and Control Server (CC / C2): A remote command and control server. The target machine can receive commands from the server, thereby achieving the purpose of the server controlling the target machine. This method is often used by viruses and trojans to control infected machines.

[0061] (3) Internet Relay Chat (IRC), an application layer protocol mainly used for group chat. IRC users use specific client chat software to connect to the IRC server and communicate with other users connected to the server through the server relay, so the Chinese name of IRC is "Internet Relay Chat".

[0062] (4) N-gram, n-gram, refers to n words that appear consecutively in a text. The n-gram model is a probabilistic language model based on a (n-1)-order Markov chain, which infers the structure of a sentence by the probability of the occurrence of n words.

[0063] (5) Content-Type, content type, generally refers to the Content-Type in a web page, which is used to define the type of network files and the encoding of the web page, and determines the form and encoding in which the browser will read the file. This is why you often see that the result of clicking on some web pages is a downloaded file or a picture. The ContentType attribute specifies the HTTP content type of the response. If ContentType is not specified, the default is TEXT / HTML.

[0064] (6) Representation Learning, also known as learning representation. In the field of deep learning, representation refers to the form and method used to represent the model's input observation sample X through the model's parameters. Representation learning refers to learning an effective representation for the observation sample X. The low-dimensional vector representation obtained by representation learning is a distributed representation. It is named so because each dimension in the vector has no clear corresponding meaning when viewed in isolation; however, when all dimensions are combined to form a vector, the semantic information of the object can be represented.

[0065] (7) Decision Tree is a decision analysis method that uses a decision tree to determine the probability that the expected value of the net present value is greater than or equal to zero, evaluate project risks, and determine its feasibility based on the known probabilities of various situations. It is a graphical method that intuitively uses probability analysis. Since this decision branch looks like a tree branch when drawn, it is called a decision tree. In machine learning, a decision tree is a prediction model that represents a mapping relationship between object attributes and object values. Classification tree (decision tree) is a very commonly used classification method. It is a supervised learning method. The so-called supervised learning is to give a bunch of samples, each sample has a set of attributes and a category, and these categories are determined in advance. Then, a classifier is obtained through learning, and this classifier can give the correct classification for the newly appeared object. This kind of machine learning is called supervised learning.

[0066] (8) User Agent (UA) refers to the browser and also includes search engines. Its information includes hardware platform, system software, application software and user personal preferences.

[0067] (9) Uniform Resource Locator (URL), also known as web page address, is the address of a standard resource on the Internet. Every file on the Internet has a unique URL, which contains information indicating the location of the file and how the browser should handle it. URL was originally invented by Tim Berners-Lee as the address of the World Wide Web.

[0068] (10) Content type, Content-Type, generally refers to the Content-Type in the web page, which is used to define the type of network files and the encoding of the web page, and determines the form and encoding in which the browser will read the file. This is why you often see that the result of clicking on some Asp web pages is a downloaded file or a picture. The ContentType attribute specifies the HTTP content type of the response. If ContentType is not specified, the default is TEXT / HTML.

[0069] (11) TF-IDF (Term Frequency–Inverse Document Frequency) is a commonly used weighting technique for information retrieval and data mining to assess the importance of a term to a document set or a document in a corpus. The importance of a term increases in direct proportion to the number of times it appears in a document, but decreases in inverse proportion to the frequency of its occurrence in the corpus. Various forms of TF-IDF weighting are often used by search engines as a measure or rating of the relevance of documents to user queries. In addition to TF-IDF, search engines on the Internet also use ranking methods based on link analysis to determine the order in which documents appear in search results.

[0070] (12) Bag-of-words (BOW) model is a commonly used document representation method in the field of information retrieval. In information retrieval, the BOW model assumes that for a document, its word order, grammar, syntax and other elements are ignored, and it is only regarded as a collection of several words. The appearance of each word in the document is independent and does not depend on whether other words appear. (It is not related to order.) In other words, any word that appears at any position in the document is independently selected without being affected by the semantics of the document.

[0071] (13) ROC (Receiver Operating Characteristic Curve): Receiver Operating Characteristic Curve. The ROC curve and AUC coefficient are mainly used to test the model's ability to correctly sort customers. The ROC curve describes the cumulative proportion of bad customers under a certain cumulative proportion of good customers. The stronger the model's ability to distinguish, the closer the ROC curve is to the upper left corner. The AUC coefficient represents the area under the ROC curve. The higher the AUC coefficient, the stronger the model's ability to distinguish risks.

[0072] (14) KS (Kolmogorov-Smirnov) test: The KS test is mainly used to verify the model's ability to distinguish default objects. Usually, after the model predicts the credit scores of all samples, all samples are divided into two parts according to default and non-default, and then the KS statistic is used to test whether the distribution of the credit scores of the two groups of samples is significantly different.

[0073] Based on the above technical issues and to facilitate understanding of the embodiments of the present application, the following first describes one of the malicious traffic identification system architectures based on the embodiments of the present application. Figure 1 , Figure 1 This is a schematic diagram of a malicious traffic identification system architecture provided by an embodiment of the present application. The client in this application may include Figure 1 The first service device 001, the second service device 002 and the third service device 003 in the embodiment, wherein the first service device 001, the second service device 002 and the third service device 003 can be connected to each other by wired or wireless means, and the second service device 002 and the third service device 003 can both send a HyperText Transfer Protocol (HTTP) request to the first service device.

[0074] The first service device 001 may include but is not limited to a background server, a component server, a data processing server, etc., and may be a device that provides various local service programs for customers. In addition, the first service device 001 may receive or respond to a HyperText Transfer Protocol (HTTP) request sent by one or more service devices, so as to provide corresponding application services for other service devices. However, the first service device 001 needs to identify whether the HTTP request sent by other service devices is a malicious traffic. If it is a malicious traffic, it cannot be responded to in order to ensure network security. Therefore, the first service device 001 is configured with a local service for malicious traffic identification, wherein the local service may include but is not limited to: determining the receiving time of the first alarm traffic; obtaining multiple second alarm traffic corresponding to the first alarm traffic within a target time period according to a preset strategy; the target time period is a time period determined based on the receiving time; the similarity between each of the multiple second alarm traffic and the first alarm traffic is greater than a preset threshold; extracting features from the multiple second alarm traffic to obtain first feature information; and judging whether the first alarm traffic is a malicious traffic based on the first feature information.

[0075] The second service device 002 may also include but is not limited to a backend server, a component server, a data processing server, etc., and may provide various local service programs to clients. Related applications may be installed and run, and HTTP requests may be sent to the first service device so that the first service device may respond and obtain corresponding services.

[0076] The third service device 003 can be a command and control server (CC / C2) of the Trojan horse. Other service devices can receive commands from the third service device 003 (CC server), so as to achieve the purpose of the third service device 003 controlling the above service devices, which is often used for virus Trojans to control infected service devices. For example: In the embodiment of the present application, the third service device 003 can send an HTTP request to the first service device, so that the first service device receives an HTTP stream, which is malicious traffic and can be identified by the first service device.

[0077] Understandably, Figure 1 The network architecture in is only an exemplary implementation in the embodiments of the present application. The malicious traffic identification system architecture in the embodiments of the present application includes but is not limited to the above malicious traffic identification system architecture.

[0078] based on Figure 1 The malicious traffic identification system architecture provided, combined with the malicious traffic identification method provided in this application, specifically analyzes and solves the technical problems raised in this application.

[0079] See also Figure 2 , Figure 2 is a flow chart of a malicious traffic identification method provided in an embodiment of the present application, which can be applied to the above Figure 1 In the malicious traffic identification system architecture described in the above, the first service device 001 can be used to support and execute Figure 2 The method flow steps S201 to S209 shown in FIG. Figure 2 The method may include the following steps S201 to S209 .

[0080] Step S201: receiving a plurality of fourth HTTP streams.

[0081] Specifically, the malicious traffic identification device receives multiple fourth Hypertext Transfer Protocol (HTTP) streams, wherein the fourth HTTP stream may be a Hypertext Transfer Protocol (HTTP) stream received by the first service device and sent from one or more second service devices and / or the third service device.

[0082] Step S202: extracting features from each of the plurality of fourth HTTP flows to obtain a second feature set.

[0083] Specifically, the malicious traffic identification device performs feature extraction on each of the multiple fourth HTTP streams to obtain a second feature set, wherein the second feature set includes: second feature information corresponding to the multiple fourth HTTP streams. It can be understood that for each fourth HTTP stream, the first service device 001 can perform feature extraction according to a preset feature extraction rule, and then obtain the corresponding non-numeric feature vectors and other digital feature vectors, and splice these non-numeric feature vectors and other digital feature vectors according to a unified rule to obtain the final single-stream feature vector, that is, the second feature information corresponding to the fourth HTTP stream. Please refer to the attached Figure 3 , Figure 3 Schematic diagram of a framework for malicious traffic identification provided by an embodiment of the present application. Figure 3 As shown, the embodiment of the present application first passes the existing network traffic through a single-stream classifier (feature processing and single-stream classification (single-stream filtering) are performed through a model trained with multiple black flows and white flows) to obtain the first alarm flow of suspected malicious traffic. For example: based on the single-stream data traffic, feature extraction is performed through a feature processor, and the single-stream features in the traffic are extracted to form a feature vector. Such feature vectors are input into the classifier to make a preliminary judgment on whether the traffic is CC communication traffic of malware (i.e., the first alarm traffic); multi-stream feature extraction is performed based on the first alarm traffic to obtain a multi-stream feature representation (multi-stream backtracking); finally, based on the multi-stream feature representation, the backtracking model and the baseline model are used to determine whether the first alarm traffic is malicious traffic. In addition, the features extracted by the above-mentioned model can be further used through a malicious family classifier to finally determine the type of the first alarm traffic. Among them, the specific implementation method can refer to the following steps, and the embodiment of the present application is not described here.

[0084] Optionally, the second feature information includes manual feature information and / or representation learning feature information; wherein the manual feature information includes: one or more of domain name readability features, uniform resource locator URL structure features, behavior indication features, and HTTP Header features corresponding to the fourth HTTP stream; the representation learning feature information includes high-dimensional features corresponding to the fourth HTTP stream.

[0085] For example: Please refer to the attached Figure 4 , Figure 4 Schematic diagram of a feature extraction provided by an embodiment of the present application. Figure 4As shown, for the received multiple fourth HTTP streams, feature engineering methods can be used to extract manual features and representation learning methods to extract features. Wherein, (1) the manual feature information includes one or more of the following features: domain name readability features, uniform resource locator URL structure features, behavior indication features, HTTP Header features (HTTP response features) corresponding to the fourth HTTP stream; wherein the URL statistical features include one or more of the following features: length, vowel ratio, consonant ratio, special character ratio, uppercase letter ratio, lowercase letter ratio, number ratio, domain level, domain name character distribution, top-level domain name, path length, path layer number, file suffix, number of parameters, average parameter value length, whether base64 exists, whether it follows the common pattern; HTTP Header features include one or more of the following features: content type Content Type, user agent UA, HTTP return status code, and N-gram of the Header sequence. (2) The representation learning feature information is assisted by representation learning. Before the output layer of the neural network, the high-dimensional features of the fourth HTTP stream are extracted to maximize the feature extraction of the existing data set and to perform association at a higher dimension. in, Figure 4 The white traffic shown refers to normal traffic, and the black traffic refers to malicious traffic. Feature extraction (manual feature extraction and representation learning feature extraction) is performed on each of the multiple fourth HTTP flows, and the extracted features are preprocessed (such as digital feature processing and non-digital feature conversion), and then the features are combined and screened to obtain a second feature set.

[0086] Optionally, the method of performing feature extraction on each of the multiple fourth HTTP streams to obtain a second feature set includes: performing feature extraction on each of the multiple fourth HTTP streams to obtain an initial feature set; and performing text processing on non-numeric features in the initial feature set to obtain the second feature set. It should be noted that, since the classification model generally processes digital input, text-to-digital conversion is required for the text features or non-numeric features in the features to convert them into digital vectors that can be processed by the classification model. The method of performing feature extraction on each of the multiple fourth HTTP streams may be through manual feature extraction and / or representation learning feature extraction.

[0087] Optionally, the text features involved above include but are not limited to: top-level domain name, file suffix, Content Type, UA, etc. It is understandable that since the inputs of these four field features are all strings, and the machine learning classifier cannot process strings, it is necessary to convert the strings and convert them into digital vectors that can be processed by the classification model. The method used in the above text processing process is: TF-IDF. Among them, the "term frequency" (Term Frequency, abbreviated as TF) in TF-IDF, TF = the number of times a word appears in an article, reflects the frequency of a word appearing in a document, and the "inverse document frequency" (Inverse Document Frequency, abbreviated as IDF), IDF = the number of times a word appears in an article / the total number of words in the article, reflects the inverse proportion of the commonness of a word, and can effectively solve some words that appear frequently but do not have great significance. In an embodiment of the present application, the method of TF-IDF = TF*IDF can be used to effectively reflect the frequency of a string in a field in the traffic. For example: first perform TF-IDF conversion on these features, and calculate their vector expression based on term frequency and document order. It should be noted that in the process of classification and identification, the data processed by TF-IDF needs to be compared with the basic TF-IDF library of the detection model to find anomalies. The basic TF-IDF library can be obtained by statistics of white traffic (which can refer to normal data traffic that can be confirmed and identified by certain technical means, used for model training or correctness verification) during the training process, and can be generated using specific white traffic in a specific detection scenario.

[0088] In a possible implementation, text processing is performed on the non-numeric features in the initial feature set to obtain the second feature set, including: text processing is performed on the non-numeric features in the initial feature set to obtain a digital feature vector set; and the above-mentioned digital feature vector set is subjected to dimensionality reduction processing to obtain the second feature set. It can be understood that after the extracted initial feature set is subjected to TF-IDF processing, the vector dimension obtained is relatively large. Such a high-dimensional vector consumes resources for the classification model and subsequent processing, and the processing efficiency is not high. Therefore, dimensionality reduction processing can be performed to convert such a high-dimensional vector into a low-dimensional vector space. Among them, the dimensionality reduction processing method may include but is not limited to singular value decomposition (SVD), principal component analysis (PCA), etc. For example, in an embodiment of the present application, since the vector dimension after TF-IDF calculation is too large, the problem of dimensional explosion is prone to occur, so a dimensionality reduction operation is performed to reduce the vector after TF-IDF processing from a high-dimensional space to a ten-dimensional space.

[0089] Optionally, the features extracted by different methods for each fourth HTTP stream are combined and screened to obtain the second feature information corresponding to each fourth HTTP stream. For example, feature engineering features and representation learning features are combined, and feature selection algorithms such as minimum redundancy maximum relevance (mRMR) are used to screen out the best feature set corresponding to each fourth HTTP stream. Figure 4 As shown, in an embodiment of the present application, traffic features of a single flow are extracted from existing network traffic, non-digital features are subjected to text processing, and the traffic features are combined and screened to obtain a second feature set.

[0090] Step S203: Based on the second feature set and using the first classification model, the first alarm flow is screened out from the plurality of fourth HTTP flows.

[0091] Specifically, the malicious traffic identification device can filter out the first alarm traffic from the multiple fourth HTTP flows through the first classification model based on the second feature set. Among them, the first alarm traffic is the traffic suspected of malicious traffic filtered out from the multiple fourth HTTP flows through the first classification model. For example: the traffic feature vector (i.e., the second feature information) of each fourth HTTP flow obtained above is input into the first classification model. The first classification model can adopt a stack mode, train different classifiers based on different features for judgment, and the first classification model can be used to finally obtain the first layer detection result based on the HTTP session using the decision tree mechanism based on the judgment result of each classifier. In addition, the first classification model can be a model trained using a labeled black and white traffic training data set. This preprocessing of data realizes the initial screening of normal traffic. On this basis, the data traffic for a single flow is extracted and selected based on artificial experience and representation learning methods to form a single flow feature vector, and then such feature vectors are input into the classifier to perform the first step of judging whether the traffic is suspected of CC communication traffic of malware. If so, the next step of judgment is further performed, which greatly improves the efficiency of judging whether the traffic is malicious traffic.

[0092] Step S204: Determine the reception time of the first alarm flow.

[0093] Specifically, the malicious traffic identification device determines the reception time of the first alarm traffic. After the first alarm traffic is screened out, the reception time of the first alarm traffic can be determined to trace back multiple traffics.

[0094] Step S205: Acquire multiple second alarm flows corresponding to the first alarm flows within the target time period according to a preset strategy.

[0095] Specifically, the malicious traffic identification device obtains multiple second alarm flows corresponding to the first alarm flow within a target time period according to a preset strategy, and the target time period is a time period determined based on the receiving time; the similarity between each second alarm flow and the first alarm flow in the multiple second alarm flows is greater than a preset threshold. In the detection process, since the traffic situation in the existing network is relatively complex, the detection of a single HTTP flow has a certain degree of randomness. If the communication behavior of malicious samples can be observed from a multi-stream perspective, multiple requests can be traced back to different clusters based on different methods, and the statistical feature combination of each alarm flow to different clusters is used to determine the positive and negative, thereby eliminating accidental errors. That is, by observing the overall communication behavior of malicious samples within a certain period of time, malicious samples can be judged more accurately from a behavioral perspective, making the final multi-stream results more robust and behaviorally explainable.

[0096] Optionally, the target time period is a time period determined based on the receiving time, for example, the target time period is a time period with a preset time period from the receiving time as the starting point, or a time period with a preset time period from the receiving time as the end point. For another example, the target time period may also be a time period including the receiving time. Acquiring the second alarm flow near the receiving first alarm flow can ensure that as many second alarm flows as possible similar to the first alarm flow are obtained.

[0097] Optionally, the preset strategy includes: one or more of the first strategy, the second strategy, and the third strategy, wherein the first strategy is a strategy for obtaining the multiple second alarm flows based on the Internet Protocol IP address and the User Agent UA information of the first alarm flow; the second strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and a preset generalization rule; the third strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and the Hypertext Transfer Protocol HTTP Header information of the first alarm flow. Wherein, on the basis of the original detection method, after the first classification model reports the result, a traffic backtracking method is used to collect CC communication flows for a period of time forward and / or backward based on the first alarm flow, and then multi-flow feature extraction is performed. It can be understood that the first strategy is to trace back based on the IP address and UA information of the first alarm traffic, and can trace back to multiple traffic sent by the same software, the same service device or the same application; the second strategy is to trace back multiple traffic based on the IP address of the first alarm traffic, and then generalize the traced traffic according to the preset generalization rules, so as to filter out multiple traffic sent by different applications of the same software as the first alarm traffic; the third strategy is to trace back based on the IP address and HTTP Header information of the first alarm traffic, and can trace back to multiple traffic sent by different applications in the same software. Multiple traffic tracing methods can accurately trace back to multiple traffics with the same source as the first alarm traffic, so as to identify whether the first alarm traffic is malicious traffic based on the behavioral characteristics of the multiple traffics, thereby improving the accuracy of identifying malicious traffic.

[0098] Optionally, the preset strategy includes the first strategy; the collecting of multiple second alarm flows corresponding to the first alarm flow in the target time period according to the preset strategy includes: obtaining the IP address and UA information of the first alarm flow; collecting multiple HTTP flows sent by the IP address in the target time period, and the HTTP flow with the same UA information as the first alarm flow is the second alarm flow. Please refer to the attached Figure 5 , Figure 5 FIG. 1 is a flow chart of tracing back traffic according to a first strategy provided by an embodiment of the present application. Figure 5 As shown, if the preset strategy includes the first strategy, the UA information and source IP address information of the first alarm traffic can be used as a unique index to trace back the traffic, and the application traffic can be identified through the UA Header information, and all HTTP flows with the same UA information sent N minutes before or after the source IP address (src-ip) are extracted as the second alarm traffic for traceback analysis. In this way, multiple flows sent by the same software, the same service device, or the same application can be traced back, which improves the accuracy of identifying malicious traffic. Please refer to the attached Figure 6 , Figure 6 Schematic diagram of multiple traffic traced back according to the first strategy provided by the embodiment of the present application. Figure 6 As shown, according to the three first alarm flows, ten HTTP requests were traced back according to the first strategy, among which the ten HTTP requests were divided into three groups according to the IP address information and UA information, corresponding to the three first alarm flows respectively. Multi-stream grouping 1 corresponds to the first alarm flow with IP address IP and UA information UA1; multi-stream grouping 2 corresponds to the first alarm flow with IP address IP and UA information UA2; multi-stream grouping 3 corresponds to the first alarm flow with IP address IP and UA information UA3. In addition, HTTP request 1-HTTP request 4: corresponds to the typical site polling + URL change mode; HTTP request 5-HTTP request 7: corresponds to the typical stable heartbeat mode; HTTP request 8-HTTP request 10: corresponds to some specific sample communication behaviors.

[0099] Optionally, the preset strategy includes the second strategy, and the collecting of multiple second alarm flows corresponding to the first alarm flow within the target time period according to the preset strategy includes: obtaining the IP address of the first alarm flow; collecting multiple first HTTP flows sent by the IP address within the target time period; generalizing the multiple first HTTP flows according to the preset generalization rule to obtain multiple second HTTP flows, and the preset generalization rule is to uniformly replace the target string corresponding to each first HTTP flow in the multiple first HTTP flows with a preset standard; and screening out from the multiple second HTTP flows the target second HTTP flow whose similarity with the first alarm flow is greater than the preset threshold as the second alarm flow. Please refer to the attached Figure 7 , Figure 7 FIG. 1 is a flow chart of tracing back traffic according to the second strategy provided by an embodiment of the present application. Figure 7 As shown, by generalizing the formed traffic, replacing the changed fields with characters, generalizing the traffic from the same source IP, and calculating the string similarity between templates, all the most similar HTTP flows are matched in the historical traffic of the source IP (such as: the traffic data of the same source IP in the target time period), which is the second alarm traffic. The so-called generalization is to replace the changed string positions in the traffic using the same standard (for example, in the embodiment of the present application, all lowercase letters can be replaced with x, special characters can be replaced with T, and uppercase letters can be replaced with X). Please refer to the attached Figure 8 , Figure 8 is a schematic diagram of a flow before and after generalization provided by an embodiment of the present application. Figure 8As shown, after generalizing multiple first HTTP flows according to a unified generalization rule, the corresponding second HTTP flows are obtained. Further, the similarity between the multiple second HTTP flows and the first alarm flow can be calculated. This method of calculating the similarity between flows after generalization can further determine multiple flows (sent by the same software but different applications) in the same cluster as the first alarm flow (the similarity exceeds a preset threshold), and then determine whether the first alarm flow is malicious flow based on the behavioral characteristics of the multiple flows, thereby improving the accuracy of identifying malicious flow.

[0100] Optionally, from the multiple second HTTP streams, a target second HTTP stream whose similarity with the first alarm traffic is greater than the preset threshold is selected as the second alarm traffic, including: vectorizing the multiple second HTTP streams, and then calculating the similarity between the vectorized multiple second HTTP streams and the first alarm traffic. Among them, the malicious traffic identification device can first use the bag of words model (BOW) for vectorization, and then use the cosine similarity in the vector space model (VSM) to calculate the similarity between the multiple second HTTP streams and the first alarm traffic. When performing string similarity measurement, the vector representation of the two generalized requests under the same BOW is obtained, and the cosine distance (similarity) is calculated. Among them, the similarity calculation between flows can use the cosine similarity in the vector space model (VSM): the vector space model is an algebraic model that represents text files as identifier (such as index) vectors. It is applied to information filtering, information retrieval, indexing and related sorting. Among them, A is the template vector of the alarm flow, and B is the vector of the backtracking flow.

[0101] Optionally, the preset strategy includes the third strategy, and the collecting of multiple second alarm flows corresponding to the first alarm flow within the target time period according to the preset strategy includes: obtaining the IP address and the HTTP Header information of the first alarm flow; collecting multiple third HTTP flows sent by the IP address within the target time period; performing N-gram processing on the HTTP Header corresponding to each of the multiple third HTTP flows to obtain a first matrix, wherein the first matrix includes HTTP Header sequence information corresponding to each third HTTP flow; performing dimensionality reduction processing on the first matrix to extract target HTTP Header sequence information that matches the HTTP Header information of the first alarm flow in the first matrix after dimensionality reduction processing; based on the target HTTP Header sequence information, obtaining the third HTTP flow corresponding to the target HTTP Header sequence information as the second alarm flow. Please refer to the attached Fig. 9 , Fig. 9FIG. 1 is a flow chart of tracing back traffic according to the third strategy provided by an embodiment of the present application. Fig. 9 As shown, the HTTP Header of the HTTP request of the source IP is processed by N-gram, that is, the HTTP Header sequence information in the traffic is extracted, and different values ​​of N are taken (depending on performance considerations) to form the sample-header combination matrix (HTTP header sequence N-gram matrix) shown in Table 1 below. Hash Trick is used to reduce the dimension and extract the HTTP flow with the same sequence after the dimension reduction.

[0102] Table 1. HTTP header sequence N-gram matrix

[0103]

[0104] Among them, Fig. 9 As shown, since the dimension of the combination matrix is ​​high, the hash trick method can be used to reduce the dimension of the matrix to obtain the matrix after the N-gram matrix is ​​reduced in dimension. For example: perform a random transformation on the feature vector x and perform a MinHash to obtain the hash result, and take the last b bits of the hash result (which can be represented in binary). This is the b-bit Min Hash process. The process is repeated k times, and each sample can be represented by k*b bits, which greatly reduces the processing time and space requirements. This method of backtracking by extracting HTTP Header sequence information in traffic (such as traffic data of the same source IP in the target time period) can trace back to multiple traffic sent by different applications in the same software, and then determine whether the first alarm traffic is malicious traffic based on the behavioral characteristics of the multiple traffic, thereby improving the accuracy of identifying malicious traffic.

[0105] Step S206: extracting features from the plurality of second alarm flows to obtain first feature information.

[0106] Specifically, the malicious traffic identification device extracts features from multiple second alarm flows to obtain first feature information. It is understandable that the HTTP flows obtained according to the above one or more policy backtracking methods are respectively input into the next stage for feature extraction. The representation vectors corresponding to the multiple HTTP flows obtained by one or more of the three backtracking methods are respectively obtained, and they are connected into a vector, which is the first feature information. Please refer to the attached Fig.10 , Fig.10 : is a flow chart of a method for obtaining first characteristic information provided by an embodiment of the present application. Fig.10As shown, the first alarm traffic, that is, the pre-classification result, is obtained through a single-stream classifier; according to the multiple second alarm traffic (multi-stream data) traced back after the first alarm traffic passes through the first strategy (i.e., UA aggregation), the second strategy (i.e., traffic template similarity clustering) and / or the third strategy (HTTP header N-gram), features are extracted from the multiple second alarm traffic to obtain the feature representation vector (Vector-traceback) corresponding to each strategy, and the multi-stream feature representation vector combined by the plurality of second alarm traffic is the first feature information.

[0107] Optionally, the first characteristic information is a characteristic representation vector; the feature extraction of the multiple second alarm flows to obtain the first characteristic information includes: feature extraction of the multiple second alarm flows to obtain behavior characteristic information corresponding to the multiple second alarm flows, the behavior characteristic information including: connection behavior characteristics, request difference characteristics, request response characteristics or one or more; according to the behavior characteristic information, the feature representation vector is obtained. This allows the malicious traffic identification device to fully consider the characteristics of the multi-stream network behavior of malicious CC communication traffic when identifying traffic, and can more accurately detect and distinguish malicious traffic in the existing network. It should be noted that please refer to the following Table 2, which is a multi-stream behavior characteristic information table provided in an embodiment of the present application.

[0108] Table 2. Description of multi-stream model features

[0109]

[0110]

[0111] Step S207: Based on the first characteristic information, determine whether the first alarm traffic is malicious traffic.

[0112] Specifically, the malicious traffic identification device can determine whether the first alarm traffic is malicious traffic based on the first characteristic information. The first characteristic information can be used to characterize the behavioral characteristic information of the multi-stream traffic corresponding to the first alarm traffic. Based on the behavioral characteristic information, the backtracking model is used for detection to determine whether the first alarm traffic is malicious traffic. For example, the obtained multi-stream behavioral characteristic information, such as: vector representation, is input into the above-mentioned backtracking model (multi-stream classifier). In addition, in order to make full use of the vector features, stacking can be used to perform multiple training to extract the behavioral features of the vector, that is, multiple detection results based on the backtracking model can be obtained. Among them, the backtracking model can be a pre-trained classification model for identifying whether the traffic is malicious traffic.

[0113] Optionally, judging whether the first alarm flow is malicious flow according to the first characteristic information includes: performing detection through a backtracking model based on the first characteristic information to obtain a first detection result; performing detection through a baseline model based on the plurality of second alarm flows to obtain a second detection result, wherein the baseline model is a detection model pre-trained based on historical flow; judging whether the first alarm flow is malicious flow based on the first detection result and the second detection result. Among them, a period of flow data accumulation is performed for the existing network production environment, and on this basis, multi-flow features of the existing network flow are extracted, and this is used as training data to construct a single classification model of the existing network historical data (i.e., the baseline model), so that this model can represent the behavior baseline of the existing network, so that traffic different from conventional behavior can be distinguished from the perspective of the baseline. In addition, the backtracking model can be a pre-trained multi-flow classifier for identifying whether the flow is malicious flow. Then, the first detection result y1(x) of the backtracking model is smoothly integrated with the second detection result y2(x) of the single classification baseline anomaly detection model pre-trained by historical flow to obtain the final discrimination result Y(x). The final integrated value is obtained through the decision formula, and finally, according to the judgment result Y(x), it is determined whether the first alarm traffic is malicious traffic. The specific decision formula is: in, Please refer to the attached Fig.11 , Fig.11 This embodiment of the present application provides a method based on E n is the independent variable, a n is the function graph of the dependent variable, where E n ∈(0,1). Fig.11 As shown, when the error rate E n The larger the value, the n It extends in the decreasing direction, resulting in a decrease in the judgment weight of the corresponding model. After the weight value and the output value of different models are arithmetic averaged, the smooth symbol function sigmoid is input to calculate the final mapping value, thereby obtaining the output results of 0 (white sample label or normal traffic label) and 1 (black sample label or malicious traffic label). In addition, by comprehensively considering the first detection result detected by the backtracking model and the second detection result detected by the baseline model, it is finally determined whether the first alarm traffic is malicious traffic, which greatly improves the accuracy of malicious traffic identification.

[0114] Step S208: If the first alarm traffic is malicious traffic, perform preset generalization processing on the first alarm traffic to obtain the generalized first alarm traffic.

[0115] Specifically, if the first alarm flow is malicious flow, the malicious flow identification device performs preset generalization processing on the first alarm flow to obtain the generalized first alarm flow. It is understandable that if the first alarm flow is indeed determined to be malicious flow, it is also possible to identify which type of malicious flow the malicious flow belongs to.

[0116] Step S209: classify the generalized first alarm traffic to determine the malicious traffic type matched by the first alarm traffic.

[0117] Specifically, the malicious traffic identification device classifies the generalized first alarm traffic and determines the malicious traffic type that matches the first alarm traffic. The malicious traffic identification device classifies the generalized first alarm traffic through a trained category classification model. The classification model used to identify the category to which the malicious traffic belongs is a multi-family classification model that is completed by training the features extracted from the model (backtracking model) involved in the above step S207 after generalizing the communication traffic of the known malicious samples. It is used to determine the family to which the malicious traffic belongs. Therefore, please refer to the attached Fig.12 , Fig.12 FIG. 1 is a flow chart of determining the type of malicious traffic provided by an embodiment of the present application. Fig.12 As shown, the malicious traffic sample can determine the type of malicious traffic that the alarm traffic matches after generalization processing, extraction of traffic templates, representation learning, feature extraction, feature identification and multiple classifiers. That is, in the embodiment of the present application, the malicious traffic identification device performs preset generalization processing on the first alarm traffic to obtain the generalized first alarm traffic, performs feature extraction (equivalent to feature extraction) on the generalized first alarm traffic, and obtains the corresponding feature representation vector; finally, the feature representation vector is input into the above-mentioned multi-family classification model to identify the type of malicious traffic.

[0118] In implementing the embodiment of the first aspect, the malicious traffic identification device can trace back multiple flows (i.e., multiple second alarm flows) that match the single flow within the target time period according to a preset strategy from the time of receiving the single flow (i.e., the first alarm flow). Then, feature extraction is performed on the multiple flows traced back to obtain feature information, so that the malicious traffic identification device can classify the above-mentioned single flow according to the feature information, thereby determining whether the single flow is malicious traffic. Among them, the similarity between the multiple second alarm flows and the first alarm flow is greater than the preset threshold. This method of classifying a single flow according to the feature information of multiple flows that are similar to a single flow allows the malicious traffic identification device to fully consider the characteristics of the multi-flow network behavior of the malicious CC communication flow when identifying the flow, thereby more accurately detecting and distinguishing the malicious traffic in the existing network. This avoids the randomness of the detection of a single HTTP flow in the existing technology during the detection process due to the relatively complex traffic conditions in the existing network. In addition, the embodiment of the present application observes the communication behavior of the traffic from a multi-stream perspective, traces back multiple alarm flows to different clusters based on one or more methods, uses the statistical feature information of the different clusters to which each alarm flow belongs, and judges the positive and negative (i.e., whether the alarm flow is malicious traffic) based on the feature information, thereby eliminating accidental errors. This observation of the overall communication behavior of malicious traffic within a certain period of time can judge malicious samples from a behavioral perspective, making the final multi-stream judgment result more robust and also behaviorally interpretable. Moreover, the embodiment of the present application detects multi-stream traffic from the characteristics corresponding to the traffic layer or from the characteristics corresponding to the host behavior. The richness of basic information is sufficient for the malicious traffic identification device to effectively and accurately identify whether the traffic is malicious traffic. Therefore, the communication traffic of rogue software and the communication traffic of malicious software can be distinguished from the characteristics of multiple streams, thereby improving the accuracy of malicious traffic identification.

[0119] In addition, in the application scenario of network data identification using the prior art and the embodiments of the present application, the following experimental data were obtained for 16 million pieces of normal live network data and more than 10,000 pieces of malicious traffic sample data collected from a certain X campus network.

[0120] 1. Only use the single-stream detection model in the existing technology:

[0121] Please refer to the following Table 3, which is a single-flow model performance data table provided in the embodiment of the present application. However, based on threat intelligence and other side confirmations, the accuracy of the detection algorithm in actual network operation is estimated to be around 80%. (The above-mentioned X campus network confirmed more than 40 flow alarms)

[0122] Table 3, single stream model performance data table

[0123] Accuracy 0.9999664730928924 F1 0.9999831782138391 Precision 0.9999728493273421 Recall rate 0.999993507313716

[0124] Among them, Table 3 shows that for all HTTP communications, the ACC value in the experimental environment (test set) reaches more than 99.99%, and the ROC value is close to 1 (0.99999). Among them, the ROC value is generally between 0.5-1.0. The larger the value, the higher the model judgment accuracy, that is, the closer to 1, the better. ROC = 0.5 means that the prediction ability of the model is no different from the random result. The KS value represents the ability of the model to distinguish between addition and subtraction. The larger the KS value, the better the prediction accuracy of the model. Generally, KS>0.2 can be considered that the model has a relatively good prediction accuracy.

[0125] 2. Use the malicious traffic identification method described in the embodiment of this application

[0126] Based on the single-layer detection model, multi-flow judgment was performed, and the clustered infection behavior of IP in the current network was successfully discovered in the experimental environment. Among all the final alarm samples, the recognition accuracy of the backtracking model of a certain X campus network reached 100%. Please refer to the malicious traffic samples detected in Table 4 below, such as the two clusters of malicious HTTP flows with IP addresses 166.***.**.111 and 166.***.***.191 detected in Table 4 below.

[0127] Table 4, Malicious traffic sample data

[0128] http: / / arimaexim.com / logo.gif?f5da****=-119****187 158****498 166.***.**.111 http: / / arimaexim.com / logo.gif?faa7****=-89****66 158****025 166.***.**.111 http: / / arimaexim.com / logo.gif?f69c****=-110****150 158****218 166.***.**.111 http: / / www.arimaexim.com / logo.gif?faa7****=-89****66 158****026 166.***.**.111 … … … http: / / ampyazilim.com.tr / images / xs2.jpg?cdd****=21****164 158****717 166.***.***.191 http: / / ahmediye.net / xs.jpg?857****=559****96 158****826 166.***.***.191 … … …

[0129] In summary, by implementing the embodiments of the present application, the traffic separation method based on multi-stream backtracing can first accurately separate the HTTP traffic of the same malware / application communication in continuous time periods; secondly, the multi-level detection framework based on backtracing (first single-stream filtering and then multi-stream backtracing) can effectively reduce the storage and detection of a large number of irrelevant data flows during the detection process (the backtracing traffic only requires the suspicious traffic detected at the first layer, which accounts for a very small proportion), improve the analysis efficiency, and is more suitable for application in enterprise network environments. In addition, the traffic separation method based on multi-stream backtracing distinguishes the communication traffic of rogue software and the communication traffic of malware from the multi-stream behavior characteristics.

[0130] The method of the embodiment of the present application is described in detail above, and the related device of the embodiment of the present application is provided below.

[0131] See also Fig.13 , Fig.13 1 is a schematic diagram of the structure of a malicious traffic identification device provided in an embodiment of the present application. The malicious traffic identification device 10 may include a determination unit 101, a backtracking unit 102, an extraction unit 103, and a judgment unit 104. It may also include: a generalization unit 105, a classification unit 106, and an alarm traffic unit 107. The detailed description of each unit is as follows.

[0132] The determining unit 101 is used to determine the receiving time of the first alarm flow;

[0133] The backtracking unit 102 is used to obtain multiple second alarm flows corresponding to the first alarm flow in a target time period according to a preset strategy; the target time period is a time period determined based on the receiving time; the similarity between each of the multiple second alarm flows and the first alarm flow is greater than a preset threshold;

[0134] The extraction unit 103 is used to extract features from the plurality of second alarm flows to obtain first feature information;

[0135] The judging unit 104 is configured to judge whether the first alarm traffic is malicious traffic based on the first characteristic information.

[0136] In one possible implementation, the preset strategy includes: one or more of a first strategy, a second strategy, and a third strategy, wherein the first strategy is a strategy for obtaining the multiple second alarm flows based on the Internet Protocol IP address and user agent UA information of the first alarm flow; the second strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and a preset generalization rule; the third strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and the Hypertext Transfer Protocol HTTP Header information of the first alarm flow.

[0137] In one possible implementation, the preset strategy includes the first strategy; the backtracking unit 102 is specifically used to: obtain the IP address and UA information of the first alarm traffic; collect multiple HTTP streams sent by the IP address within the target time period, and the HTTP stream with the same UA information as the first alarm traffic is the second alarm traffic.

[0138] In one possible implementation, the preset strategy includes the second strategy, and the backtracking unit 102 is specifically used to: obtain the IP address of the first alarm traffic; collect multiple first HTTP streams sent by the IP address within the target time period; generalize the multiple first HTTP streams according to the preset generalization rule to obtain multiple second HTTP streams, and the preset generalization rule is to uniformly replace the target string corresponding to each first HTTP stream in the multiple first HTTP streams using a preset standard; from the multiple second HTTP streams, filter out the target second HTTP stream whose similarity with the first alarm traffic is greater than a preset threshold as the second alarm traffic.

[0139] In one possible implementation, the preset strategy includes the third strategy, and the backtracking unit 102 is specifically used to: obtain the IP address and the HTTP Header information of the first alarm traffic; collect multiple third HTTP streams sent by the IP address within the target time period; perform N-gram processing on the HTTP Header corresponding to each of the multiple third HTTP streams to obtain a first matrix, wherein the first matrix includes HTTP Header sequence information corresponding to each third HTTP stream; perform dimensionality reduction processing on the first matrix to extract the target HTTP Header sequence information that matches the HTTP Header information of the first alarm traffic in the first matrix after dimensionality reduction processing; based on the target HTTP Header sequence information, obtain the third HTTP stream corresponding to the target HTTP Header sequence information as the second alarm traffic.

[0140] In one possible implementation, the first feature information is a feature representation vector; the extraction unit 103 is specifically used to: perform feature extraction on the multiple second alarm flows to obtain behavior feature information corresponding to the multiple second alarm flows, the behavior feature information including: one or more of connection behavior features, request difference features, and request response features; and obtain the feature representation vector based on the behavior feature information.

[0141] In one possible implementation, the judgment unit 104 is specifically used to: perform detection through a backtracking model based on the first feature information to obtain a first detection result; perform detection through a baseline model based on the multiple second alarm flows to obtain a second detection result, wherein the baseline model is a detection model pre-trained based on historical flows; and determine whether the first alarm flow is malicious flow based on the first detection result and the second detection result.

[0142] In one possible implementation, the device also includes: a generalization unit 105, which is used to perform preset generalization processing on the first alarm traffic to obtain a generalized first alarm traffic if the first alarm traffic is malicious traffic; and a classification unit 106, which is used to classify the generalized first alarm traffic to determine the malicious traffic type that matches the first alarm traffic.

[0143] In one possible implementation, the device also includes an alarm traffic unit 107, and the alarm traffic unit 107 is used to: receive multiple fourth HTTP streams before determining the receiving time of the first alarm traffic; perform feature extraction on each of the multiple fourth HTTP streams according to a preset feature extraction rule to obtain a second feature set, and the second feature set includes: second feature information corresponding to the multiple fourth HTTP streams respectively; based on the second feature set, filter out the first alarm traffic from the multiple fourth HTTP streams through a first classification model.

[0144] In one possible implementation, the second feature information includes manual feature information and / or representation learning feature information; wherein the manual feature information includes: one or more of domain name readability features, uniform resource locator URL structure features, behavior indication features, and HTTP Header features corresponding to the fourth HTTP stream; and the representation learning feature information includes high-dimensional features corresponding to the fourth HTTP stream.

[0145] It should be noted that the functions of the various functional units in the malicious traffic identification device 10 described in the embodiment of the present application can be referred to in the above Figure 2 The descriptions of steps S201 to S209 in the method embodiment are not repeated here.

[0146] like Fig.14 As shown, Fig.14 2 is a schematic diagram of the structure of another malicious traffic identification device provided in an embodiment of the present application, wherein the device 20 includes at least one processor 201, at least one memory 202, and at least one communication interface 203. In addition, the device may also include common components such as antennas, which will not be described in detail here.

[0147] The processor 201 may be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program.

[0148] The communication interface 203 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), core network, Wireless Local Area Networks (WLAN), etc.

[0149] The memory 202 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0150] The memory 202 is used to store application code for executing the above solution, and the execution is controlled by the processor 201. The processor 201 is used to execute the application code stored in the memory 202.

[0151] The code stored in the memory 202 can execute the above Figure 2 The provided network traffic identification method, for example, determines the receiving time of the first alarm traffic; obtains multiple second alarm traffic corresponding to the first alarm traffic within a target time period according to a preset strategy; the target time period is a time period determined based on the receiving time; the similarity between each of the multiple second alarm traffic and the first alarm traffic is greater than a preset threshold; performs feature extraction on the multiple second alarm traffic to obtain first feature information; and determines whether the first alarm traffic is malicious traffic based on the first feature information.

[0152] It should be noted that the functions of the various functional units in the malicious traffic identification device 20 described in the embodiment of the present application can be referred to in the above Figure 2 The descriptions of steps S201 to S209 in the method embodiment are not repeated here.

[0153] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0154] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0156] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0159] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying malicious traffic, characterized in that: include: Determine the reception time of the first alarm flow; Acquire multiple second alarm flows corresponding to the first alarm flow within a target time period based on a preset strategy; The target time period is a time period determined based on the receiving time; the similarity between each of the plurality of second alarm flows and the first alarm flow is greater than a preset threshold; Extracting features of the plurality of second alarm flows to obtain first feature information; Based on the first characteristic information, determine whether the first alarm traffic is malicious traffic.

2. The method according to claim 1, characterized in that: The preset strategy includes: one or more of a first strategy, a second strategy, and a third strategy, wherein the first strategy is a strategy for obtaining the multiple second alarm flows based on the Internet Protocol IP address and user agent UA information of the first alarm flow; the second strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and a preset generalization rule; the third strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and the Hypertext Transfer Protocol HTTP Header information of the first alarm flow.

3. The method according to claim 2, characterized in that: The preset strategy is the first strategy; and obtaining multiple second alarm flows corresponding to the first alarm flow within a target time period based on the preset strategy includes: Obtaining the IP address and UA information of the first alarm flow; Among the multiple HTTP flows sent by the IP address within the target time period, an HTTP flow having the same UA information as the first alarm flow is collected as the second alarm flow.

4. The method according to claim 2, characterized in that: The preset strategy is the second strategy, and the acquiring a plurality of second alarm flows corresponding to the first alarm flow within a target time period based on the preset strategy includes: Obtaining the IP address of the first alarm flow; Collecting a plurality of first HTTP flows sent by the IP address within the target time period; Performing generalization processing on the plurality of first HTTP streams according to the preset generalization rule to obtain a plurality of second HTTP streams, wherein the preset generalization rule is to uniformly replace the target string corresponding to each of the plurality of first HTTP streams using a preset standard; From the multiple second HTTP flows, a target second HTTP flow having a similarity with the first alarm flow greater than the preset threshold is screened out as the second alarm flow.

5. The method according to claim 2, characterized in that: The preset strategy is the third strategy, and the acquiring a plurality of second alarm flows corresponding to the first alarm flow within a target time period based on the preset strategy includes: Obtaining the IP address and HTTP Header information of the first alarm traffic; Collecting multiple third HTTP flows sent by the IP address within the target time period; Performing N-gram processing on the HTTP Header corresponding to each of the plurality of third HTTP streams to obtain a first matrix, wherein the first matrix includes HTTP Header sequence information corresponding to each of the third HTTP streams; Performing dimensionality reduction processing on the first matrix, and extracting target HTTP Header sequence information matching the HTTP Header information of the first alarm traffic in the first matrix after the dimensionality reduction processing; Based on the target HTTP Header sequence information, a third HTTP flow corresponding to the target HTTP Header sequence information is obtained as the second alarm flow.

6. The method according to any one of claims 1 to 5, characterized in that: The first characteristic information is a characteristic representation vector; and extracting the characteristics of the plurality of second alarm flows to obtain the first characteristic information includes: Extracting features of the plurality of second alarm flows to obtain behavior feature information corresponding to the plurality of second alarm flows, the behavior feature information including: one or more of connection behavior features, request difference features, and request response features; The feature representation vector is obtained according to the behavior feature information.

7. The method according to claim 6, characterized in that: The determining, based on the first characteristic information, whether the first alarm traffic is malicious traffic includes: Performing detection through a backtracking model based on the first feature information to obtain a first detection result; Perform detection based on the plurality of second alarm flows through a baseline model to obtain a second detection result, wherein the baseline model is a detection model pre-trained based on historical flows; Based on the first detection result and the second detection result, determine whether the first alarm traffic is malicious traffic.

8. The method according to claim 6, characterized in that: The method further comprises: If the first alarm traffic is malicious traffic, performing a preset generalization process on the first alarm traffic to obtain a generalized first alarm traffic; The generalized first alarm traffic is classified to determine a malicious traffic type matched by the first alarm traffic.

9. The method according to claim 1, characterized in that: Before determining the receiving time of the first alarm flow, the method further includes: receiving a plurality of fourth HTTP streams; Performing feature extraction on each of the plurality of fourth HTTP flows to obtain a second feature set, where the second feature set includes second feature information respectively corresponding to the plurality of fourth HTTP flows; Based on the second feature set, the first alarm traffic is filtered out from the multiple fourth HTTP flows through a first classification model.

10. The method according to claim 9, characterized in that: The second feature information includes manual feature information and / or representation learning feature information; wherein, the manual feature information includes: one or more of domain name readability features, uniform resource locator URL structure features, behavior indication features, and HTTP Header features corresponding to the fourth HTTP stream; the representation learning feature information includes high-dimensional features corresponding to the fourth HTTP stream.

11. A malicious traffic identification device, characterized in that: include: A determination unit, used to determine a reception time of the first alarm flow; A tracing unit, configured to obtain a plurality of second alarm flows corresponding to the first alarm flow within a target time period based on a preset strategy; The target time period is a time period determined based on the receiving time; the similarity between each of the plurality of second alarm flows and the first alarm flow is greater than a preset threshold; An extraction unit, configured to extract features from the plurality of second alarm flows to obtain first feature information; A judging unit is used to judge whether the first alarm traffic is malicious traffic based on the first characteristic information.

12. The device according to claim 11, characterized in that: The preset strategy includes: one or more of a first strategy, a second strategy, and a third strategy, wherein the first strategy is a strategy for obtaining the multiple second alarm flows based on the Internet Protocol IP address and user agent UA information of the first alarm flow; the second strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and a preset generalization rule; the third strategy is a strategy for obtaining the multiple second alarm flows based on the IP address of the first alarm flow and the Hypertext Transfer Protocol HTTP Header information of the first alarm flow.

13. The device according to claim 12, characterized in that: The preset strategy is the first strategy; the backtracking unit is specifically used to: Obtaining the IP address and UA information of the first alarm flow; Among the multiple HTTP flows sent by the IP address within the target time period, an HTTP flow having the same UA information as the first alarm flow is collected as the second alarm flow.

14. The device according to claim 12, characterized in that: The preset strategy is the second strategy, and the backtracking unit is specifically used to: Obtaining the IP address of the first alarm flow; Collecting a plurality of first HTTP flows sent by the IP address within the target time period; Performing generalization processing on the plurality of first HTTP streams according to the preset generalization rule to obtain a plurality of second HTTP streams, wherein the preset generalization rule is to uniformly replace the target string corresponding to each of the plurality of first HTTP streams using a preset standard; From the multiple second HTTP flows, a target second HTTP flow having a similarity with the first alarm flow greater than a preset threshold is screened out as the second alarm flow.

15. The device according to claim 12, characterized in that: The preset strategy is the third strategy, and the backtracking unit is specifically used to: Obtaining the IP address and HTTP Header information of the first alarm traffic; Collecting multiple third HTTP flows sent by the IP address within the target time period; Performing N-gram processing on the HTTP Header corresponding to each of the plurality of third HTTP streams to obtain a first matrix, wherein the first matrix includes HTTP Header sequence information corresponding to each of the third HTTP streams; Performing dimensionality reduction processing on the first matrix, and extracting target HTTP Header sequence information matching the HTTP Header information of the first alarm traffic in the first matrix after the dimensionality reduction processing; Based on the target HTTP Header sequence information, a third HTTP flow corresponding to the target HTTP Header sequence information is obtained as the second alarm flow.

16. The device according to any one of claims 11 to 15, characterized in that: The first feature information is a feature representation vector; the extraction unit is specifically used to: Extracting features of the plurality of second alarm flows to obtain behavior feature information corresponding to the plurality of second alarm flows, the behavior feature information including: one or more of connection behavior features, request difference features, and request response features; The feature representation vector is obtained according to the behavior feature information.

17. The device according to claim 16, characterized in that: The judging unit is specifically used for: Performing detection through a backtracking model based on the first feature information to obtain a first detection result; Perform detection based on the plurality of second alarm flows through a baseline model to obtain a second detection result, wherein the baseline model is a detection model pre-trained based on historical flows; Based on the first detection result and the second detection result, determine whether the first alarm traffic is malicious traffic.

18. The device according to claim 16, characterized in that: The device also includes: A generalization unit, configured to perform a preset generalization process on the first alarm flow to obtain a generalized first alarm flow if the first alarm flow is malicious flow; The classification unit is used to classify the generalized first alarm traffic and determine the malicious traffic type matched by the first alarm traffic.

19. The device according to claim 11, characterized in that: The device further comprises an alarm flow unit, wherein the alarm flow unit is configured to: Before determining the receiving time of the first alarm flow, receiving a plurality of fourth HTTP flows; Performing feature extraction on each of the plurality of fourth HTTP flows to obtain a second feature set, where the second feature set includes second feature information respectively corresponding to the plurality of fourth HTTP flows; Based on the second feature set, the first alarm traffic is filtered out from the multiple fourth HTTP flows through a first classification model.

20. The device according to claim 19, characterized in that: The second feature information includes manual feature information and / or representation learning feature information; wherein, the manual feature information includes: one or more of domain name readability features, uniform resource locator URL structure features, behavior indication features, and HTTP Header features corresponding to the fourth HTTP stream; the representation learning feature information includes high-dimensional features corresponding to the fourth HTTP stream.

21. A service device, characterized in that: It comprises a processor and a memory, wherein the memory is used to store malicious traffic identification program code, and the processor is used to call the malicious traffic identification program code to execute the method described in any one of claims 1-10.

22. A chip system, characterized in that: The chip system includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected by lines. Instructions are stored in at least one memory. When the instructions are executed by the processor, the method described in any one of claims 1 to 10 is implemented.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 10.

Images