Real-time Risk Detection Method for Identifying Group Attacks Based on Graph Neural Networks

Abstract

The present invention provides a real-time risk detection method for identifying group attacks based on graph neural networks. The detection method includes marking historical data to construct a resource black library; preprocessing each request data and putting the preprocessed data into a data set to be constructed into a graph; when the data in the data set to be constructed into a graph reaches a preset data volume, performing a graph structure extraction once to obtain a pre-constructed graph; preprocessing real-time data and incrementally adding nodes of the real-time data to the pre-constructed graph to obtain a fine-tuned graph structure; then using a GCN model to extract features from the fine-tuned graph structure and performing a clustering risk assessment, and then summarizing the risks in real time and outputting. Through the above method, the present invention can fuse data features within a short time period on the basis of combining historical markings and effectively shorten the data processing time to ensure the real-time nature of risk detection.

Description

Technical Field

[0001] The present invention relates to the field of Internet security technology, and in particular to a real-time risk detection method for identifying group attacks based on graph neural networks. Background Art

[0002] In the field of Internet security, black production attack behaviors not only show consistency in behavior patterns, but also have similarity and relevance in resource usage. For example: device farms with the same hardware resources, IP pools with the same number segment, account reuse, etc. Graph neural networks use graphs as data structures and have natural advantages in representing node and edge relationships. Especially the GCN model can integrate the topological structure of the graph and node features, and effectively extract the correlation characteristics between resources. Therefore, it has good applications in identifying group attacks in the risk control field.

[0003] Due to the time-consuming composition and graph query process of graph neural networks, the current related implementation methods of graph neural networks are mostly offline solutions. For example, when applying graph neural networks in the field of payment risk control, the existing technology specifically includes an offline batch processing part and a real-time risk judgment part: in the offline batch processing part, according to the transaction data of the past few months, graphs are composed using accounts, email addresses, devices, delivery addresses, IPs, etc., entity embedding inferences are made, and they are saved in a key-value database; in the real-time risk judgment part, when predicting the risk of new online transactions, related entities are used to directly query from the key-value database to obtain relevant embedding information, and the information is input into the last layer of the GNN to obtain the corresponding evaluation score, thereby completing the online inference.

[0004] However, the above method can only query from the historical data of offline batch processing during real-time risk judgment. However, in the long-term confrontation process, black production will continuously change its own resources and characteristics. For a new group or a group with an updated resource pool that appears at a certain moment, since the embedding information cannot be queried from the entity library of the existing solution, the attack characteristics of these newly emerging groups in the adjacent period cannot be effectively identified. In addition, the entity types used in the existing graph composition are single, only having identification nodes, and the robustness is poor when detecting group attacks such as those of device farms and those using IP pools.

[0005] In view of this, it is necessary to design an improved real-time risk detection method for identifying group attacks based on graph neural networks to solve the above problems. Summary of the Invention

[0006] Aiming at the defects of the above-mentioned existing technologies, the purpose of the present invention is to provide a real-time risk detection method for identifying group attacks based on a graph neural network. By adopting incremental graph construction of the graph neural network, the correlation features of real-time data are added to the nearly real-time pre-constructed graph for fine-tuning, so as to fuse the data features within a short period on the basis of combining historical markings and effectively shorten the data processing time to ensure the real-time nature of risk detection.

[0007] To achieve the above purpose, the present invention provides a real-time risk detection method for identifying group attacks based on a graph neural network, including the following steps:

[0008] 1) Historical resource marking: Mark historical data to construct a resource blacklist.

[0009] 2) Pre-constructed graph: Preprocess each request data and put the preprocessed data into the data set to be graph-constructed; when the data in the data set to be graph-constructed reaches a preset data volume, perform a graph structure extraction once to obtain a pre-constructed graph.

[0010] 3) Real-time risk assessment of data: After preprocessing the real-time data, incrementally add the nodes of the real-time data to the pre-constructed graph to obtain a fine-tuned graph structure; then use the GCN model to extract features from the fine-tuned graph structure, perform clustering risk assessment, and then summarize the risks in real time and output.

[0011] As a further improvement of the present invention, in step 3), before performing the clustering risk assessment, prepare a predetermined amount of white samples in advance. After using the GCN model to extract features, sort the data of each feature dimension and divide it into several equal parts to obtain the cut-off points of each feature dimension of the white samples.

[0012] As a further improvement of the present invention, in step 3), when performing the clustering risk assessment, for each feature dimension of the data to be evaluated, perform bucketing according to the cut-off points of each feature dimension of the white samples, and then calculate the data volume in each bucket and the proportion of the data with risks in the resources in each bucket to the total risk data; if the data volume or the proportion in a certain bucket is greater than the preset condition, then the data volume of the bucket is involved in the summation. If the total sum of the data volumes involved in the summation in all feature dimensions is greater than the preset threshold, then it is judged as structural risk data.

[0013] As a further improvement of the present invention, the preprocessing method is: Mark the resource risks of each request data according to the resource blacklist and extract the required fields and put them into the data set to be graph-constructed.

[0014] As a further improvement of the present invention, the required fields include a resource identification field, a resource attribute field, and a resource risk marker; preferably, the resource identification field includes one or more of a session ID, an account, a device fingerprint, and an IP; the resource attribute field includes one or more of a system version, a model, a desktop system, a sound card, an IP geographical location, and an IP number segment.

[0015] As a further improvement of the present invention, in step 3), the risks summarized and output in real time include resource risks and structural risks. The resource risks are judged according to the resource risk marker, and the structural risks are judged according to the clustering risk assessment.

[0016] As a further improvement of the present invention, in step 2), when the data in the data set to be graph-constructed reaches the preset data volume, after the pre-graph is obtained through graph structure extraction, the data set to be graph-constructed is cleared, and when the data continues to accumulate to reach the preset data volume, the graph structure is extracted again and the pre-graph is updated; preferably, the preset data volume is 1000 pieces.

[0017] As a further improvement of the present invention, in step 2), the process of graph structure extraction is as follows: converting the structured log into a graph structure, processing the resource identification field and the resource attribute field into nodes, and connecting two nodes that appear in the same request with an edge.

[0018] As a further improvement of the present invention, in step 1), the historical data is marked through a rule stack and a model library, and the resource blacklist is updated regularly; preferably, the update time is daily update.

[0019] As a further improvement of the present invention, the resource blacklist includes an IP library, a device library, and an account library.

[0020] The beneficial effects of the present invention are:

[0021] (1) The real-time risk detection method for identifying group attacks based on a graph neural network provided by the present invention decouples the graph construction process, adopts a graph neural network incremental graph construction scheme, adds the correlation features of real-time data to the near-real-time pre-graph for fine-tuning, so that on the basis of combining historical marks, the data features in a short time period are also fused, and the risks of new groups or groups that have changed resources can be judged in real time, especially for the identification of short-term group attacks, which has a better effect.

[0022] (2) Based on incremental composition, the present invention further uses a GCN model to extract features from the fine-tuned graph structure. Compared with the traditional method of using a GCN model for real-time marking, it avoids reconstructing the graph for each request, effectively solving the problem of time-consuming graph composition due to large amounts of data. The computational time for each request is only within 20 ms, ensuring the real-time nature of risk detection. On this basis, the present invention further optimizes the clustering risk assessment method. By referring to the feature distribution of white samples, it assesses risks for the associated features extracted by the graph neural network according to the feature distribution differences, reducing the dependence on labels and data volume while ensuring the recognition effect. At the same time, this risk assessment process can further combine labeled data to increase the types of detected risk groups, with high practical application value.

[0023] (3) When setting up composition nodes, the present invention simultaneously adds resource identification nodes and resource attribute nodes, enabling the identification of more types of gang attacks. On this basis, by further extracting the associated information of the nodes, it can effectively enhance the relevance of similar resource nodes and has stronger recognition ability for device resources with the same or similar attributes. Description of the Drawings

[0024] Figure 1 It is a schematic flowchart of the real-time risk detection method for identifying gang attacks based on a graph neural network provided by the present invention. Detailed Embodiments

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail below with reference to the drawings and specific embodiments.

[0026] Here, it should also be noted that to avoid obscuring the present invention with unnecessary details, only the structures and / or processing steps closely related to the solution of the present invention are shown in the drawings, while other details less relevant to the present invention are omitted.

[0027] In addition, it should be noted that the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article, or device.

[0028] To achieve the above objectives, the present invention provides a real-time risk detection method for identifying gang attacks based on a graph neural network, including the following steps:

[0029] 1) Historical resource marking: Mark historical data to construct a resource blacklist.

[0030] 2) Pre-constructed graph: Preprocess each piece of request data and put the preprocessed data into the data set to be graph-constructed. When the data in the data set to be graph-constructed reaches the preset data volume, perform a graph structure extraction once to obtain a pre-constructed graph;

[0031] 3) Real-time data risk assessment: After performing the preprocessing on the real-time data, incrementally add the nodes of the real-time data to the pre-constructed graph to obtain a fine-tuned graph structure. Then use the GCN model to extract features from the fine-tuned graph structure and perform clustering risk assessment, and then summarize the risks in real-time and output.

[0032] Through the above method, the present invention can incrementally construct a graph using a graph neural network, add the correlation features of real-time data to the near-real-time pre-constructed graph for fine-tuning, thereby fusing the data features within a short period on the basis of combining historical markings, and effectively shortening the data processing time to ensure the real-time nature of risk detection.

[0033] More specifically, in some embodiments of the present invention, the schematic flow diagram of the real-time risk detection method for identifying group attacks based on a graph neural network is as Figure 1 shown, and specifically includes the following steps:

[0034] 1. Historical resource marking

[0035] Regularly mark the historical data through a rule stack and a model library, and regularly maintain and update the resource blacklist.

[0036] The resource blacklist includes an IP library, a device library, and an account library, which are used to mark the IPs, devices, and accounts with security risks in the historical data; the update time is daily update to meet the requirements of actual applications.

[0037] 2. Pre-constructed graph

[0038] 2.1. Preprocess each piece of request data and put the preprocessed data into the data set S1 to be graph-constructed.

[0039] Among them, the preprocessing method is: Mark the resource risk of each piece of request data according to the resource blacklist to form a resource risk mark, and extract the required fields and put them into the data set S1 to be graph-constructed. The required fields include resource identification fields, resource attribute fields, and the resource risk mark; more specifically, in an embodiment of the present invention, the resource identification fields include session id, account, device fingerprint, IP; the resource attribute fields include system version, model, desktop system, sound card, IP geographical location, IP number segment.

[0040] Through the above method, it is possible to extract resource identification fields and resource attribute fields, so as to add resource identification nodes and resource attribute nodes simultaneously in the subsequent graph construction process, which is convenient for identifying more types of gang attacks. For example, gang attacks in the device farm category are mostly a large number of devices with similar software and hardware. Although gang attacks using IP pools rarely reuse IPs, their number segments, types, and geographical locations may show clustering. The above method provided by the present invention adds resource attribute nodes such as system version, model, desktop system, sound card, IP geographical location, and IP number segment while setting resource identification nodes, and can effectively identify the above types of gang attacks.

[0041] At the same time, the resource risk markers formed in the above process can be further combined with subsequent clustering risk assessment methods to increase the types of risky gangs detected.

[0042] 2.2. For the data set S1 to be graph-constructed, when the data in the set S1 reaches the preset data volume, a graph structure extraction is performed once to obtain a preliminary graph G, and at the same time, the set S1 is cleared. When the preset data volume is reached again after continuous accumulation, the graph structure is re-extracted and the preliminary graph G is updated.

[0043] For example, in an embodiment of the present invention, the preset data volume is 1000 entries; with such a setting, when the set S1 reaches 1000 entries of data each time, a graph structure extraction is performed once to obtain a preliminary graph G, and at the same time, the set S1 is cleared. Then, the request data is continuously accumulated. When 1000 entries of data are reached again, the graph structure is re-extracted and the preliminary graph G is updated.

[0044] More specifically, the process of graph structure extraction is as follows: convert the structured log into a graph structure, process the resource identification fields and the resource attribute fields into nodes, and connect two nodes that appear in the same request with an edge. Preferably, the feature of the node is the one-hot encoding of the node type, and the initial feature values of the same type of nodes are the same, enabling the model to pay more attention to extracting associated information, thereby effectively enhancing the relevance of similar resource nodes and having a stronger recognition ability for some device resources with the same or similar attributes.

[0045] 3. Real-time data risk assessment

[0046] 3.1. After preprocessing the real-time data in the manner of step 2.1, incrementally add the nodes of the real-time data to the preliminary graph G to obtain a fine-tuned graph structure.

[0047] 3.2. Use the GCN model to extract features from the fine-tuned graph structure.

[0048] Compared with the method of using the GCN model for real-time marking in the prior art, the present invention performs feature extraction on the fine-tuned graph structure using the GCN model on the basis of incremental graph construction, avoiding re-graph construction for each request, effectively solving the problem of time-consuming graph construction due to a large amount of data, and the computing time for each request is only within 20 ms, ensuring the real-time nature of risk detection.

[0049] 3.3. Conduct clustering risk assessment

[0050] Prepare a predetermined amount of white samples in advance. After using the GCN model with the same parameters as in step 3.2 for feature extraction, sort the data of each feature dimension and divide it equally into 10 parts to obtain the cut-off points of each feature dimension of the white samples.

[0051] When the preset data volume in step 2.2 is 1000 pieces, after pre-constructing a graph for each newly added request, there are a total of 1001 pieces of data to be predicted. Divide each feature dimension of the 1001 pieces of data to be predicted this time according to the cut-off points of each feature dimension of the white samples, and then calculate the data volume cnt in each bucket and the proportion black_rate of the data with risks in the resources in each bucket to the total risk data; if the data volume cnt in a certain bucket is greater than 1.2 times the average data volume of all buckets, or the proportion black_rate is greater than one-third, then the data volume cnt of this bucket participates in the summation; if the total sum dense_sum of the data volumes cnt participating in the summation in all feature dimensions is greater than the preset threshold, then it is judged as structural risk data.

[0052] At the same time, combining the resource risk marks in the data set S1 to be graph-constructed, the resource risk is newly added as a type of risk gang.

[0053] Through the above method, the present invention can further optimize the clustering risk assessment method, refer to the feature distribution of the white samples, and evaluate the risks of the associated features extracted by the graph neural network according to the feature distribution differences, reducing the dependence on labels and data volume while ensuring the recognition effect; at the same time, combining the marked data to increase the detected risk gang types, which has high practical application value.

[0054] 3.4. Risk summary

[0055] After summarizing the structural risk data, resource risk data and the resource risk data in the current real-time data obtained in step 3.3, output risk marks in real time to achieve real-time risk detection of gang attacks.

[0056] In summary, the present invention provides a real-time risk detection method for identifying group attacks based on graph neural networks. The detection method includes marking historical data to construct a resource blacklist; preprocessing each request data and putting the preprocessed data into a data set to be graph-constructed; when the data in the data set to be graph-constructed reaches a preset data volume, performing a graph structure extraction once to obtain a pre-constructed graph; after preprocessing the real-time data, incrementally adding nodes of the real-time data to the pre-constructed graph to obtain a fine-tuned graph structure; then using a GCN model to extract features from the fine-tuned graph structure and performing a clustering risk assessment, and then summarizing the risks in real time and outputting. Through the above method, the present invention can fuse data features within a short time period on the basis of combining historical marks and effectively shorten the data processing time to ensure the real-time nature of risk detection.

[0057] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A real-time risk detection method for identifying gang attacks based on graph neural networks, characterized in that, It includes the following steps: 1) Historical resource marking: Mark historical data to build a resource black library; 2) Pre-constructed graph: Preprocess each request data and put the preprocessed data into the data set to be graph-constructed; When the data in the data set to be graph-constructed reaches the preset data volume, perform a graph structure extraction once to obtain a pre-constructed graph; 3) Real-time data risk assessment: After performing the preprocessing on the real-time data, incrementally add the nodes of the real-time data to the pre-constructed graph to obtain a fine-tuned graph structure; Then use the GCN model to extract features from the fine-tuned graph structure, perform clustering risk assessment, and then summarize and output the risks in real time; Among them, before performing the clustering risk assessment, prepare a predetermined amount of white samples in advance. After using the GCN model to extract features, sort the data of each feature dimension and divide it into several equal parts to obtain the cut-off points of each feature dimension of the white samples; When performing the clustering risk assessment, for each feature dimension of the data to be evaluated, perform bucketing according to the cut-off points of each feature dimension of the white samples, and then calculate the data volume in each bucket and the proportion of the data with risks in the resources in each bucket to the total risk data; if the data volume or the proportion in a certain bucket is greater than the preset condition, then include the data volume of this bucket in the summation; if the total sum of the data volumes included in the summation in all feature dimensions is greater than the preset threshold, then judge it as structural risk data.

2. The real-time risk detection method for identifying gang attacks based on graph neural network according to claim 1, wherein: The method of the preprocessing is: Mark the resource risk for each request data according to the resource black library, and extract the required fields and put them into the data set to be graph-constructed.

3. The real-time risk detection method for identifying gang attacks based on a graph neural network according to claim 2, wherein: The required fields include resource identification fields, resource attribute fields, and resource risk marks; the resource identification fields include one or more of session id, account, device fingerprint, and ip; the resource attribute fields include one or more of system version, model, desktop system, sound card, ip geographical location, and ip number segment.

4. The real-time risk detection method for identifying group attacks based on a graph neural network according to claim 3, wherein: In step 3), the risks summarized and output in real time include resource risks and structural risks. The resource risks are judged according to the resource risk marks, and the structural risks are judged according to the clustering risk assessment.

5. The real-time risk detection method for identifying group attacks based on graph neural network according to claim 1, characterized in that: In step 2), when the data in the data set to be graph-constructed reaches the preset data volume, after obtaining the pre-constructed graph through the graph structure extraction, empty the data set to be graph-constructed, and wait until it continues to accumulate to reach the preset data volume and then re-extract the graph structure and update the pre-constructed graph; the preset data volume is 1000 pieces.

6. The real-time risk detection method for identifying gang attacks based on graph neural network according to claim 3, characterized in that: In step 2), the process of the graph structure extraction is: Convert the structured log into a graph structure, process the resource identification fields and the resource attribute fields into nodes, and connect two nodes that appear in the same request with an edge.

7. The real-time risk detection method for identifying gang attacks based on a graph neural network according to claim 1, wherein: In step 1), mark the historical data through a rule stack and a model library, and regularly update the resource black library; the update time is daily update.

8. The real-time risk detection method for identifying group attacks based on graph neural network according to claim 7, characterized in that: The resource black library includes an ip library, a device library, and an account library.

Images