An internal threat detection method based on log behavior and user characteristics
By encoding log behavior and constructing user association graphs, combined with graph convolutional neural networks, the problem of low accuracy and recall in existing technologies for internal threat detection is solved, and efficient identification and detection of malicious users is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-23
- Publication Date
- 2026-04-07
AI Technical Summary
Existing insider threat detection methods struggle to accurately capture behavioral differences between malicious and legitimate users and fail to effectively leverage the relationships between users, resulting in low detection accuracy and recall rates.
By encoding log behaviors to construct user behavior sequences, using recurrent neural networks to extract behavioral features, and constructing user relationship graphs based on user social relationships, threat behavior detection is performed using graph convolutional neural networks, assigning different weights to different types of relationships.
It improves the accuracy and recall rate of internal threat detection, enabling more accurate identification of malicious users and reducing operational workload.
Smart Images
Figure CN116455644B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of network security and deep learning technology, and specifically relates to an internal threat detection method based on log behavior and user characteristics. Background Technology
[0002] As various sectors become increasingly intelligent, organizations are adopting more efficient digital systems, leading to a greater likelihood of internal attacks by employees exploiting privileges. Examples include the infamous data breaches involving Edward Snowden and Chelsea Manning. These incidents demonstrate the higher demands placed on internal system security by modern systems. According to a survey conducted by Vormetric, only 11% of respondents believed their organizations were not vulnerable to internal attacks, while 89% believed they were at least somewhat vulnerable. Furthermore, nearly half of the respondents indicated that attacks exploiting internal privileges were more threatening than external attacks.
[0003] Therefore, among numerous cybersecurity issues, insider threats remain the most challenging. The unique nature of insider threats has attracted widespread attention in related academic fields. Compared to external attacks, insider threats are more covert; attackers utilize known information such as network topology and protection systems to evade security equipment monitoring. Insider threats are also more destructive. Because attackers are internal personnel, the entire network configuration is exposed, resulting in greater harm to the entire system. Preventing both insider threats and external attacks is like the two wings of system security, requiring equal attention and emphasis. Furthermore, with increasingly abundant information sharing, attackers and attack methods are becoming more diverse, and traditional external attack detection methods cannot be applied to insider threat detection, making the issue of insider threats increasingly crucial.
[0004] Most existing internal threat detection systems based on user and entity behavior analysis focus on learning user behavior patterns by comparing the behavior of different users or the behavior of the same user at different time periods. These methods rely too heavily on feature engineering, making it difficult to accurately capture the behavioral differences between malicious and normal users, and they do not consider the social relationships between users. Rule-based and signature-based methods perform detection by matching existing databases. These methods heavily depend on database characteristics. Therefore, there is a need for an efficient detection method that can both monitor user behavior on a large scale and quickly identify internal threats by analyzing the relationships between users, thereby reducing the workload of operations and maintenance. Summary of the Invention
[0005] The purpose of this invention is to provide an internal threat detection method based on log behavior and user characteristics, thereby improving the accuracy and recall rate of internal threat detection.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] An internal threat detection method based on log behavior and user characteristics, the method comprising:
[0008] User behavior sequences are constructed by encoding log behaviors in the log data;
[0009] Behavioral features are obtained by representing the user behavior sequence using a recurrent neural network;
[0010] A user relationship graph is built based on users' social relationships to serve as user characteristics;
[0011] Based on the behavioral and user characteristics, a threat behavior detection model based on graph convolutional neural networks outputs information about malicious users with internal threats.
[0012] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.
[0013] Preferably, the step of encoding log behaviors in the log data to construct a user behavior sequence includes:
[0014] Based on a preset behavior dictionary, the log behaviors of a specific user in the log data are transformed, and the transformed code is converted into a vector as user behavior using multi-hot encoding.
[0015] Multiple user behaviors within a preset time interval are combined to form a user behavior sequence;
[0016] Multiple user behavior sequences of the same user are combined to form a behavior sequence set.
[0017] Preferably, the construction process of the behavior dictionary is as follows:
[0018] Based on the log behaviors involved in the log data, select the behavioral attributes used to form user behavior;
[0019] Assign values to the attributes of each behavior to form a behavior dictionary.
[0020] Preferably, the behavioral attributes include user behavior category, computer ownership, behavior time, file type, disk to which the file belongs, file operation, whether the file was moved to a USB device, whether the file came from a USB device, HTTP access website type, HTTP operation type, whether a file was sent, whether a file was received, whether an email was sent to an external organization, and whether the email was copied.
[0021] Preferably, the step of representing the user behavior sequence using a recurrent neural network to obtain behavioral features includes:
[0022] The user behavior sequence of user u within a time interval n starting from time t. Inputting the data into a recurrent neural network yields the behavioral characteristics of user u at time t. Represents a sequence of user behaviors The first user action in the process.
[0023] Preferably, the step of establishing a user relationship graph based on user social relationships as user features includes:
[0024] Extracting user social relationships based on email correspondence: Traversing all user behavior sequences for a user based on user ID from log data, if user u i With user u j If there is email correspondence between them, then at node u i With node u j Create an edge between them;
[0025] Extracting user social relationships based on department affiliation: Iterate through the department IDs in the log data, forming an array containing all user IDs for each department ID. If user u i With user u j If they belong to the same department, then at node u i With node u j Create an edge between them;
[0026] Extracting user social relationships based on project group affiliation: Iterate through the project group IDs in the log data, forming an array containing all user IDs of that project group for each project group ID. If user u i With user u j If they belong to the same project team, then at node u i With node u j Create an edge between them;
[0027] User association graph representation: Using users as nodes and their behavioral characteristics as node features, a user association graph TG is generated based on user social relationships. The user association graph TG is represented by an N*N dimensional adjacency matrix TM, where N is the number of users. If node u... i With node uj If an edge exists, set TM[i][j] = 1 * weight; otherwise, set TM[i][j] = 0.
[0028] Preferably, the weight of email correspondence in the user association graph is greater than the weight of project group affiliation, which is greater than the weight of department affiliation.
[0029] Preferably, the structure of the threat behavior detection model is as follows:
[0030] The input layer is used to receive network input and pass it to the convolutional layer;
[0031] A convolutional layer is used to perform convolution processing on the network input and pass the classification features of multiple nodes to the output layer. It is configured as a graph convolutional neural network. The graph convolutional neural network has two convolutional layers. The neighborhood sample size in the first convolutional layer and the second convolutional layer is set to 10 and 15, respectively.
[0032] The output layer computes the classification features for each node and inputs them into the softmax function. The softmax function outputs an N-dimensional vector Y, where Y[i] represents the user u. i The classification results are as follows, where N is the number of users, and the classification results are malicious users and normal users.
[0033] This invention provides an insider threat detection method based on log behavior and user characteristics. Addressing the issue that current insider threat detection methods do not consider the relationships between users, this invention treats user behavior features as nodes and direct relationships as edges, proposing the use of a graph convolutional neural network for insider threat detection. Furthermore, pre-setting weights for relationships enhances the mutual influence between users with closer connections. The method provided by this invention improves the accuracy and recall of insider threat detection methods. Attached Figure Description
[0034] Figure 1 This is a flowchart of the internal threat detection method based on log behavior and user characteristics according to the present invention. Detailed Implementation
[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention.
[0037] To overcome the weaknesses in existing technologies for detecting insider threats, such as Figure 1 As shown in the figure, this embodiment provides an internal threat detection method based on log behavior and user characteristics, including the following steps:
[0038] (1) Construction of user behavior sequence based on log behavior: After encoding the log behavior in the log data, a user behavior sequence is constructed.
[0039] (1-1) Definition of Behavioral Attributes: Based on the log behaviors involved in the log data, select behavioral attributes to form user behavior. The behavioral attributes selected in this embodiment include, but are not limited to, user behavior category, computer ownership, behavior time, file type, disk to which the file belongs, file operation, whether the file was moved to a USB device, whether the file came from a USB device, HTTP access website type, HTTP operation type, whether a file was sent, whether a file was received, whether an email was sent to an external organization, and whether the email was copied.
[0040] (1-2) Set up the behavior dictionary (behavior encoding table): Assign values to the attribute values of each behavior attribute to form a behavior dictionary. Taking file operation as an example, its attribute values include open file, copy file, write file, delete file and empty field, which are assigned values with numbers 1-5 (which can be adjusted as needed, for example, 0-4). Each behavior attribute is assigned values in the same way. See Table 1 for the specific encoding format.
[0041] Table 1 Behavior Dictionary
[0042]
[0043]
[0044] (1-3) Single user behavior transformation: Based on the preset behavior dictionary, the log behavior of a specific user in the log data is transformed, and the transformed code is converted into a vector as user behavior through multi-hot encoding.
[0045] Each data entry in the log represents a behavioral operation. All attributes of this behavioral operation are mapped one-to-one with the behavioral attribute values in the behavioral dictionary (Table 1). For example, the behavioral operation is: "Opened a job search website on my computer during work hours." According to the behavioral dictionary, this is converted to [5,1,1,6,3,5,3,3,3,1,3,3,3,3]. Then, it is converted into a vector using multi-hot encoding as follows:
[0046] [[0,0,0,0,1,0,0],[1,0,0],[1,0,0],[1,0,0,0],[0,0,0,0,0,1],[0,0,1],[0,0,0,0,1],[0,0,1],[0,0,1],[0,0,1],[0,0,1],[0,0,1],[1,0,0,0],[0,0,1],[0,0,1]]. This vector is used as a user action.
[0047] (1-4) User behavior sequence construction: Combine multiple user behaviors within a preset time interval to form a user behavior sequence, and combine multiple user behavior sequences of the same user to form a behavior sequence set.
[0048] Assuming a certain time is t, and a time interval n is set (the default time interval is the time when 20 user behaviors occur), all user behaviors of user u within the time interval [t, t+n] are formed into a user behavior sequence. Represents a sequence of user behaviors The first user action in the time interval [t, t+n] is the first action of user u. Assuming user u has m user action sequences, then all user action sequences of user u combine to form the action sequence set I. u .
[0049] (2) User behavior sequence representation: User behavior sequence is represented by a recurrent neural network to obtain behavioral features.
[0050] The user behavior sequence of user u within a time interval n starting from time t. Inputting a recurrent neural network (LSTM) to model and obtain the behavioral features of user u at time t. Behavioral characteristics User behavior sequence A corresponding eigenvalue.
[0051] (3) User relationship graph construction: A user relationship graph is built based on the user's social relationship as user characteristics.
[0052] (3-1) User Social Relationship Extraction Based on Email Correspondence: Define a user relationship graph with users as nodes and user behavior characteristics as node features. Traverse the set of corresponding behavior sequences based on user ID from log data. If user u... i With user u j There is email correspondence between user u and user u (represented as user u in the user behavior sequence). i The email service contains an Email field for user u j Email address, or user u j The email service contains an Email field for user u i (email address), then in node u i With node u j Create an edge between them.
[0053] (3-2) Extraction of user social relationships based on department affiliation: Traverse the department IDs in the log data, forming an array containing all user IDs of that department for each department ID. If user u i With user u j Belonging to the same department (represented by user u) i With user u j If the department ID field is equal (i.e., they belong to the same array), then in node u i With node u j Create an edge between them.
[0054] (3-3) Extraction of user social relationships based on project group affiliation: Iterate through the project group IDs in the log data, forming an array containing all user IDs of that project group for each project group ID. If user u i With user u j Belonging to the same project group (represented by user u) i With user u j If the project group ID field is equal (i.e., they belong to the same array), then in node u i With node u j Create an edge between them.
[0055] (3-4) User Association Graph Representation: Based on the user social relationships extracted in steps (3-1), (3-2), and (3-3), a user association graph TG is generated. An N*N dimensional adjacency matrix TM represents the user association graph TG, where N is the number of users. If node u... i With node u j If an edge exists, set TM[i][j] = 1 * weight; otherwise, set TM[i][j] = 0.
[0056] Since there are three types of edges in the user association graph TG, to more closely relate the mutual influence between users, for the three social relationships of project association, email correspondence, and same department, the weight of email correspondence is set to be greater than the weight of project group affiliation, which is greater than the weight of department affiliation. In this embodiment, the weight of project group affiliation is set to 0.9, the weight of email correspondence is set to 1, and the weight of department affiliation is set to 0.7. These can be adjusted according to actual needs.
[0057] (4) Construction and application of threat behavior detection model based on graph convolutional network (GCN): Based on behavioral features and user features, the threat behavior detection model based on graph convolutional network outputs malicious users with internal threats.
[0058] (4-1) The structure of the threat behavior detection model is as follows:
[0059] The input layer is used to receive network input and pass it to the convolutional layer.
[0060] The convolutional layer is used to perform convolution processing on the network input and pass the classification features of multiple nodes to the output layer. It is configured as a graph convolutional neural network. The graph convolutional neural network has two convolutional layers. The neighborhood sample size in the first convolutional layer and the second convolutional layer is set to 10 and 15, respectively.
[0061] The output layer (fully connected layer) is used to calculate the classification features of each node and input them into the softmax function. The softmax function outputs an N-dimensional vector Y, where Y[i] represents user u. i The classification results are categorized as follows: N is the number of users, and the classification results are categorized as malicious users and normal users.
[0062] (4-2) Construction of training and test sample sets: Take a preset proportion (e.g., 80%) of the data in the dataset (which can be a public dataset or a custom dataset) as training data, and take the remaining data as test data.
[0063] For the training data and test data, construct the user behavior sequence of all users according to the method in step (1), characterize the behavior features of all users according to the method in step (2), and obtain the adjacency matrix TM according to the method in step (3).
[0064] Let S be the training sample at time t. t =(WD t ,X t ), of which WD t Let X be the behavioral features of all users in the training data at time t, and let X be a set containing the behavioral features of all users. tUser behavior labels (e.g., labels with values of 0 and 1, representing malicious and normal users respectively) are a set of user behavior labels corresponding to the behavioral features of all users. Then, during model training, the input to the threat behavior detection model at time t is the training sample S. t And the adjacency matrix TM constructed based on the user social relationships of all users in the training data.
[0065] Let S be the test sample at time t′. t′ =(WD t′ ,X t′ ), of which WD t′ Let X be the behavioral characteristics of all users in the test data at time t′, and let X be a set containing the behavioral characteristics of all users. t′ Let S be the user behavior labels, a set containing user behavior labels corresponding to the behavioral characteristics of all users. Then, in the model test, the input to the threat behavior detection model at time t′ is the test sample S. t′ And an adjacency matrix TM' constructed based on the social relationships of all users in the test data.
[0066] (4-3) Model Training and Testing: The threat behavior detection model is trained using training samples until it reaches the accuracy threshold when tested using test samples. It should be noted that the training method (e.g., gradient descent) and loss function (e.g., mean squared error loss) used in this embodiment are conventional methods and functions, and will not be elaborated upon in this embodiment.
[0067] (4-4) Application of the model: Take log data within a specified time period, construct the user behavior sequence of all users according to the method in (1), characterize the behavior features of all users according to the method in step (2), and obtain the adjacency matrix TM according to the method in step (3). Input the behavior features of all users at a specific time within the specified time period and the adjacency matrix TM into the threat behavior detection model to obtain the classification results of all users as normal users or malicious users output by the threat behavior detection model.
[0068] This invention fully considers the social relationships between users, assigns different weights to different types of relationships, more closely links the mutual influence between users, and uses graph convolutional neural networks for insider threat detection, thereby improving the accuracy and recall of the insider threat detection method.
[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0070] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.
Claims
1. An internal threat detection method based on log behavior and user characteristics, characterized in that, The internal threat detection method based on log behavior and user characteristics includes: User behavior sequences are constructed by encoding log behaviors in the log data; Behavioral features are obtained by representing the user behavior sequence using a recurrent neural network; A user relationship graph is built based on users' social relationships to serve as user characteristics; Based on the behavioral characteristics and user characteristics, a threat behavior detection model based on graph convolutional neural networks outputs malicious users with internal threats. The step of establishing a user relationship graph based on user social relationships as user features includes: Extracting user social relationships based on email correspondence: Traversing all user behavior sequences for a user based on their user ID from log data, if the user... With users If there is email correspondence between them, then at the node With nodes Create an edge between them; Extracting user social relationships based on department affiliation: Iterate through the department IDs in the log data, creating an array for each department ID containing all user IDs within that department. If a user... With users If they belong to the same department, then at the node With nodes Create an edge between them; Extracting user social relationships based on project group affiliation: Iterate through the project group IDs in the log data, forming an array containing all user IDs of that project group for each ID. If a user... With users If they belong to the same project team, then at the node With nodes Create an edge between them; User Relationship Graph Representation: A user relationship graph (TG) is generated based on user social relationships, using users as nodes and their behavioral characteristics as node features. A 3D adjacency matrix TM represents the user association graph TG. For the number of users, if the node With nodes If there is an edge, then set ;on the contrary, .
2. The internal threat detection method based on log behavior and user characteristics as described in claim 1, characterized in that, The process of encoding log behaviors in log data to construct user behavior sequences includes: Based on a preset behavior dictionary, the log behaviors of a specific user in the log data are transformed, and the transformed code is converted into a vector as user behavior using multi-hot encoding. Multiple user behaviors within a preset time interval are combined to form a user behavior sequence; Multiple user behavior sequences of the same user are combined to form a behavior sequence set.
3. The internal threat detection method based on log behavior and user characteristics as described in claim 2, characterized in that, The process of constructing the behavior dictionary is as follows: Based on the log behaviors involved in the log data, select the behavioral attributes used to form user behavior; Assign values to the attributes of each behavior to form a behavior dictionary.
4. The internal threat detection method based on log behavior and user characteristics as described in claim 3, characterized in that, The behavioral attributes include user behavior category, computer ownership, behavior time, file type, disk to which the file belongs, file operation, whether the file was moved to a USB device, whether the file came from a USB device, HTTP access website type, HTTP operation type, whether a file was sent, whether a file was received, whether an email was sent to an external organization, and whether the email was copied.
5. The internal threat detection method based on log behavior and user characteristics as described in claim 1, characterized in that, The process of representing the user behavior sequence using a recurrent neural network to obtain behavioral features includes: Will Time interval from time to time Users within User behavior sequence Input into a recurrent neural network, and obtain Time users behavioral characteristics , Represents a sequence of user behaviors The first user action in the process.
6. The internal threat detection method based on log behavior and user characteristics as described in claim 1, characterized in that, In the user association graph, the weight of email correspondence is greater than the weight of project team affiliation, which in turn is greater than the weight of department affiliation.
7. The internal threat detection method based on log behavior and user characteristics as described in claim 1, characterized in that, The structure of the threat behavior detection model is as follows: The input layer is used to receive network input and pass it to the convolutional layer; A convolutional layer is used to perform convolution processing on the network input and pass the classification features of multiple nodes to the output layer. It is configured as a graph convolutional neural network. The graph convolutional neural network has two convolutional layers. The neighborhood sample size in the first convolutional layer and the second convolutional layer is set to 10 and 15, respectively. The output layer is used to calculate the classification features of each node and input them into the softmax function, which outputs a... A dimensional vector Y, where Y[i] represents the user. The classification results, the The number of users is used to classify users into malicious users and normal users.