Multi-dimensional internal threat detection method based on deep learning
By employing a deep learning-based multi-dimensional internal threat detection method, which utilizes LSTM, GCN, and autoencoders for feature extraction and fusion, this approach addresses the shortcomings of traditional methods in identifying complex behavioral patterns and achieves more efficient internal threat detection.
Patent Information
- Application Number
- CN202411016138.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-27
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional insider threat detection methods struggle to accurately identify complex behavioral patterns. They are limited by the need for large amounts of high-quality historical data, difficulty in capturing dynamic changes in behavior, subjective threshold selection, and high computational resource consumption, resulting in poor performance when facing covert and dynamically changing modern threats.
A deep learning-based multi-dimensional internal threat detection method is adopted. Through multi-dimensional feature extraction and feature fusion, including temporal pattern features, graph structure behavior pattern features and sequence comprehensive features, combined with statistical features, deep learning is performed using LSTM, GCN and autoencoder, and feature weights are dynamically adjusted to detect internal threats.
It enables multi-dimensional analysis of user behavior, reveals the essential characteristics of user behavior, improves the accuracy and efficiency of internal threat detection, enhances the complexity and expressive power of the model, and reduces the risk of overfitting.
Smart Images

Figure CN118972113A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of threat detection technology, and in particular to a multi-dimensional internal threat detection method based on deep learning. Background Art
[0002] In the digital age, internal threats have become a major issue that cannot be ignored in corporate network security. Internal personnel may use their legitimate access rights to damage the information assets of an organization for various reasons, from financial fraud to data leakage. These incidents not only cause huge economic losses to the company, but also seriously damage its reputation. Traditional internal threat detection relies on rule-based auditing and monitoring, but this method is powerless in the face of modern threats that are hidden and dynamically changing. Traditional statistical theory methods such as HMM and GMM can accurately describe user behavior patterns, but they are limited by the need for a large amount of high-quality historical data, difficulty in capturing dynamic changes in behavior, subjectivity in threshold selection, and model computing resource consumption, making it difficult to accurately identify complex behavior patterns.
[0003] With the popularization of big data and cloud computing, traditional security defense measures have become difficult to cope with increasingly complex internal threats. Therefore, the present invention proposes a multi-dimensional internal threat detection method based on deep learning to solve the above problems. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a multi-dimensional internal threat detection method based on deep learning, which can analyze and deeply explore user behavior patterns from multiple angles, reveal the essential characteristics of user behavior, identify potential security risks, and effectively improve the accuracy and efficiency of internal threat detection.
[0005] To achieve the above object, the present invention provides the following solution: a multi-dimensional internal threat detection method based on deep learning, comprising the following steps:
[0006] Collecting a multi-source data set consisting of normal behaviors and malicious behaviors, and then preprocessing the multi-source data set to obtain a user behavior data file;
[0007] Based on the user behavior data file, using statistical methods and statistical coding rules to extract key behavior features and mark potential threat behaviors, to obtain statistical features for representing user behavior patterns;
[0008] Based on the user behavior data file, the user operation behavior is serialized in chronological order using the behavior coding rules to obtain a behavior sequence feature;
[0009] The behavior sequence features are subjected to in-depth feature extraction using three dimensions: time dependency, graph structure relevance, and data dimensionality reduction, to obtain multi-dimensional features; the multi-dimensional features include time series pattern features, graph structure behavior pattern features, and sequence comprehensive features;
[0010] Using deep learning technology to fuse the multi-dimensional features to obtain multi-dimensional fusion features;
[0011] The multi-dimensional fusion feature is integrated with the statistical feature to obtain a comprehensive total feature;
[0012] The comprehensive overall feature is input into the threat detection module to perform internal threat detection.
[0013] Optionally, the multi-source data set includes login and logout log files, device interaction record log files, browsing history log files, email record log files, file operation log files, and user directory log files.
[0014] Optionally, the multi-source data set is preprocessed to obtain a user behavior data file, including:
[0015] Integrate the login and logout log files, the device interaction record log files, the browsing history log files, the email record log files, and the file operation log files based on the user ID and time, and fill in the non-existent null values with 0 to obtain integrated data;
[0016] The integrated data is grouped based on user IDs to generate the user behavior data file.
[0017] Optionally, the process of extracting key behavior features using statistical methods and statistical coding rules is as follows:
[0018] Based on the login and logout log files, extract the first login time and the last logout time of the user in a day, and calculate the ratio of the first login time and the last logout time to 24 hours, so as to obtain the first login time ratio and the last logout time ratio;
[0019] Based on the device interaction record log file, extract the first and last time the user uses the external device in a day, and calculate the ratio of the first and last time the external device is used to 24 hours, to obtain the first device use time ratio and the last device use time ratio;
[0020] The device interaction log file is used to record statistics and obtain the number of times the external device is used during non-working hours.
[0021] Optionally, based on the user behavior data file, the user operation behavior is serialized in chronological order using behavior coding rules to obtain behavior sequence features, including:
[0022] By defining 13 behavioral activities and dividing each behavioral activity into two groups according to the time of commuting to and from get off work, 26 behavioral codes were obtained;
[0023] Based on 26 types of behavior codes, the user's daily behavior data is encoded into a daily behavior sequence to obtain the behavior sequence features.
[0024] Optionally, the behavior sequence features are subjected to in-depth feature extraction using three dimensions: time dependency, graph structure relevance, and data dimensionality reduction, to obtain multi-dimensional features, including:
[0025] The behavior sequence features are input into LstmAutoEncoder, the behavior sequence is compressed by the encoder, and then the behavior sequence is reconstructed by the decoder to extract the time series pattern features containing time series pattern information.
[0026] Optionally, the behavior sequence features are subjected to in-depth feature extraction using three dimensions of time dependency, graph structure relevance, and data dimensionality reduction to obtain multi-dimensional features, further comprising:
[0027] Extracting and analyzing the relationship information between users in the user directory log file to construct a user relationship network graph, and inputting the behavior sequence and the user relationship network graph into a graph convolutional network;
[0028] In the graph convolutional network, the first GCNConv layer is used to extract and output preliminary graph structure features, and then the ReLU activation function is used to adjust the preliminary graph structure features into nonlinear features to input the second GCNConv layer. Finally, the output of the second GCNConv layer is normalized using the softmax function to obtain graph structure behavior pattern features containing user behavior and key relationship network information.
[0029] Optionally, the behavior sequence features are subjected to in-depth feature extraction using three dimensions of time dependency, graph structure relevance, and data dimensionality reduction to obtain multi-dimensional features, further comprising:
[0030] Build an autoencoder based on fully connected layers;
[0031] The autoencoder is used to extract low-dimensional features from the behavior sequence to obtain comprehensive sequence features.
[0032] Optionally, the multi-dimensional fusion feature is fused with the statistical feature to obtain a comprehensive total feature, including:
[0033] The temporal pattern features, the graph structure behavior pattern features and the sequence comprehensive features are fused by using an attention mechanism, and weights are dynamically assigned according to different features during the fusion process to obtain the multi-dimensional fusion features;
[0034] The multi-dimensional fusion features and the statistical features are fused using a concatenation strategy and a bitwise addition strategy to obtain the comprehensive total features.
[0035] The present invention discloses the following technical effects by providing a multi-dimensional internal threat detection method based on deep learning:
[0036] 1. Data coding: This paper uses statistical methods to encode user behaviors, focusing on analyzing the frequency of occurrence of behavioral patterns. This method helps us identify the mainstream trends and potential anomalies of user activities, and provides a quantitative perspective to understand user behavior habits. On the other hand, we capture the user's daily behavior sequence by defining a set of behavior coding rules. This encoding method retains the continuity and structure of time series data, provides the necessary data format for subsequent time series analysis, and ensures the accuracy and depth of the analysis process.
[0037] 2. Multi-dimensional feature extraction: In the field of internal threat detection, by extracting time series pattern features, we can obtain pattern information rich in time series, which is extremely critical for subsequent threat detection; by extracting graph structure behavior pattern features, we can extract deep features from the organization's social network and the interaction patterns between users, which are crucial for identifying abnormal behaviors and potential threats; by extracting sequence comprehensive features, we can obtain compact and meaningful feature representations. The three features work together to achieve a comprehensive analysis of user behavior, reveal the essential characteristics of user behavior from multiple angles, and effectively enhance the performance of the internal threat detection system.
[0038] 3. Feature fusion: Through the splicing strategy, the complete information of all features is retained, ensuring that the model can independently decide which feature combination is most helpful for the final prediction, increasing the complexity and expressiveness of the model; through the bitwise addition strategy, the feature combination is simplified, the risk of overfitting is reduced, and the feature weights are adjusted at the same time, which is conducive to highlighting the importance of common features.
[0039] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0041] Figure 1 A flow chart of a method provided by an embodiment of the present invention;
[0042] Figure 2 A schematic diagram of a framework for internal threat detection provided by an embodiment of the present invention;
[0043] Figure 3 A schematic diagram of a process for extracting time series pattern features using LSTM provided in an embodiment of the present invention;
[0044] Figure 4 A schematic diagram of the process of GCN feature extraction provided by an embodiment of the present invention;
[0045] Figure 5 A schematic diagram of the feature fusion process provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0048] like Figure 1-2 As shown, the present invention provides a multi-dimensional internal threat detection method based on deep learning, which is specifically described by taking the public dataset Cert dataset maintained by Carnegie Mellon University (CMU) as an example. It includes the following steps:
[0049] 1. Collect a multi-source data set consisting of normal behaviors and malicious behaviors, and then pre-process the multi-source data set to obtain a user behavior data file.
[0050] 1.1 Dataset Introduction
[0051] The CERT dataset r4.2 is a synthetic dataset consisting of normal and malicious insider behaviors. The dataset covers 1,000 virtual users, 70 of which are simulated malicious insiders, providing a rich research sample.
[0052] The multi-source data set of CERT dataset r4.2 version includes:
[0053] Login and logout log file logon.csv: records the user's login and logout activities on the computer system;
[0054] Device interaction log file device.csv: details the interaction between the user and external devices (such as USB);
[0055] Browsing history log file http.csv: collects the user's web browsing history;
[0056] Email record log file email.csv: saves the sending and receiving records of emails;
[0057] File operation log file file.csv: records the user's operations on files, including creation, editing, and deletion;
[0058] User Directory Log File LDAP: Describes information about all users and their job roles, and is a detailed user directory. The total number of records is as high as 32,770,220.
[0059] As shown in Table 1, the main reason for choosing version r4.2 is that this version has more threat records in terms of the number of threat users and threat behavior logs than version r6.2, and the data of version r4.2 covers almost every instance of threat scenario, which means that each scenario has multiple examples, thus providing a comprehensive and diverse case analysis opportunity for our research.
[0060] Table 1 Cert4.2, 6.2 version data information
[0061] Version Number of users Number of threatened users Number of threat behavior logs Total number of behavior logs r4.2 1000 70 7323 32,770,227 r6.2 2500 5 466 135,117,169
[0062] 1.2 Data Preprocessing
[0063] During data preprocessing, we process multi-source user behavior data into separate user behavior data files. We integrate the five log files of email, file, device, logon, and http by user ID and time, and fill in the non-existent null values with 0. We group the integrated data by user ID to generate user behavior data files. The processed user behavior data files are used as input for subsequent feature encoding to construct statistical features and user behavior sequence features.
[0064] 2. Data coding: Based on the user behavior data file, statistical methods and statistical coding rules are used to extract key behavior features and mark potential threat behaviors to obtain statistical features for representing user behavior patterns.
[0065] In the statistical feature encoding section, we extracted a series of key user behavior features from the device.csv and logon.csv log files to characterize the user's daily login and device interaction patterns. In addition, in order to mark potential internal threat behaviors, if a user performs at least one threat activity in a day, we mark that day as an internal threat behavior event. The following are the specific statistical encoding rules:
[0066] First login time ratio: Extract the first login time of the user in a day from the logon.csv file and convert it into a ratio of the whole 24 hours.
[0067] Last logout time percentage: From the logon.csv file, record the last logout time and calculate its percentage of 24 hours in a day.
[0068] First device usage time ratio: Search the device.csv file for the first use of external devices in a day and calculate the ratio of this time point to 24 hours.
[0069] Last device usage time percentage: Get the last time the device was used from the device.csv file and convert it into a percentage of the entire day.
[0070] Number of times the device is used during non-working hours: The total number of times the user uses the device during non-working hours is counted from the device.csv file.
[0071] Based on the above encoding rules, we extracted dynamic information about users’ login, logout, and device usage, so as to fully describe the users’ daily behavior patterns. These encoding features will be further used in model construction to identify possible internal threat behaviors.
[0072] 3. Data coding: Based on the user behavior data file, the user operation behavior is serialized in chronological order using the behavior coding rules to obtain the behavior sequence features;
[0073] The goal of user behavior sequence feature coding is to serialize user operation behaviors into a data format with a time sequence, code the preprocessed data, and add them to the sequence according to the order in which each behavior is executed. After analyzing emali.csv, logon.csv, file.csv, device.csv, and http.csv, the coding rules shown in Table 2 are defined. A total of 13 behavior activities are defined, and each behavior activity is divided into those performed during working hours and non-working hours, and finally 26 behavior codes are obtained.
[0074] The last type of behavior activity is the behavior activity of visiting low-frequency URLs. We extract the substrings that can be uniquely identified from the URLs visited by users. Then we count the substrings of the URLs and divide the calculated frequencies according to the box plot calculation formulas 1, 2, 3, and 4. We extract the substrings below the lower whisker line LowerWhisker as abnormal URL substrings. We define these abnormal URL substrings as low-frequency URL information. Finally, if the substring of the URL visited by the user exists in the low-frequency URL information, this behavior is defined as a behavior activity of visiting a low-frequency URL.
[0075]
[0076] IQR=Q3-Q1#(3)
[0077] Lower Whisker=Q1-1.5×IQR#(4)
[0078] Table 2 Behavior coding rules
[0079]
[0080] According to the behavior coding defined in Table 2, the user's daily behavior data is encoded into a daily behavior sequence S d =[A1,A2,…,A i ], each item A in the sequence i It’s a behavioral activity.
[0081] In summary, after statistical feature encoding and behavior sequence feature encoding, we finally get two data with different meanings. One is the statistical features of statistical user behavior operations, and the other is the user behavior sequence features based on the operation order.
[0082] 4. Multi-dimensional feature extraction: The behavior sequence features are extracted in depth using the three dimensions of time dependency, graph structure relevance, and data dimensionality reduction to obtain multi-dimensional features.
[0083] 4.1 Temporal pattern feature extraction
[0084] In the field of internal threat detection, it is crucial to capture the temporal patterns of user behavior. As an advanced recurrent neural network, LSTM stands out for its unique ability to process time series data, especially when faced with complex sequences with long-term dependencies. The cell state and gating mechanism of LSTM enable it to remember important historical information while filtering out irrelevant content, which is particularly critical when identifying abnormal patterns in user behavior.
[0085] Figure 3 The process of extracting temporal pattern features for LSTM, such as Figure 3 As shown, by training LstmAutoEncoder, our goal is to capture the deep structure in time series data. User behavior sequence X b is input into the network, where the encoder compresses the information and the decoder attempts to reconstruct the sequence, thereby learning the inherent properties of the sequence. b Transformed into a more compact representation X by the encoder lstm-encoded Then, we get X through the linear layer. linear-encoded , which helps the fusion of features. In the decoding stage, from Y a First, we gradually restore the feature dimensions through the reverse linear layer to generate X linear-decoded , and then sent back to LSTM for sequence reconstruction to get X' b .
[0086] Through the trained feature extraction model, we successfully extracted the temporal pattern feature Y a ,These features are rich in time series pattern information and are extremely ,critical for subsequent threat detection.
[0087] 4.2 Graph structure behavior pattern feature extraction
[0088] In the part of graph structure behavior pattern extraction, the graph convolutional network (GCN) technology is used. GCN is good at processing graph structure data, can effectively transmit information between nodes, and learn the embedded representation of nodes. In the application of internal threat detection, GCN uses the relationship network between users, including organizational structure and collaboration mode, to extract the graph structure features of user behavior. Through GCN, we can extract deep features from the organization's social network and the interaction patterns between users. These features are crucial for identifying abnormal behavior and potential threats.
[0089] In order to construct the relationship graph network between users, we first parse the relationship information between users from the LDAP log file. Table 3 shows the LDAP log file information:
[0090] Table 3 LDAP log file information
[0091]
[0092] LDAP logs record the specific details of each user's department and team. By deeply analyzing this information, we constructed a user relationship network diagram. In this diagram, each user's daily behavior sequence is regarded as an independent node, and the connection between nodes, that is, the edge, follows the following definition:
[0093] If user B is the supervisor of user A, connect user A to all nodes of user B;
[0094] If user C is a colleague of user A, connect all nodes of user A and user C;
[0095] All nodes of user A also establish connections with themselves.
[0096] Figure 4 This is a schematic diagram of the GCN feature extraction process, as shown Figure 4 As shown in the figure, after constructing the relationship graph between users, the data is then subjected to feature extraction through the graph convolutional network (GCN). The user's behavior sequence and their relationship network are jointly used as the input of the GCN module. First, the data is processed by the GCNConv layer, which extracts preliminary graph structure features. Next, after adjusting the activation function, the obtained features become more nonlinear and more suitable as the input of the next layer of GCNConv. In the last step, the output result is normalized by the Softmax function to ensure that the value of each element in the feature vector can intuitively reflect the relative importance. From this, we obtain the graph structure behavior pattern feature Y extracted by GCN b ,This feature set integrates key information of user behaviors and their ,relationship networks.
[0097] 4.3 Sequence comprehensive feature extraction
[0098] In the sequence comprehensive feature extraction, our core is to extract low-dimensional features from high-dimensional data, so we use the autoencoder (AE) based on the fully connected layer for feature extraction. b As the input of the feature extraction module, after multiple fully connected layers, the output X' is finally obtained b After multiple rounds of training, we select the optimal model for feature extraction and use the Encoder layer of the constructed autoencoder to extract features, and finally obtain the sequence comprehensive feature Y extracted based on AE c .
[0099] The multi-dimensional feature extraction module integrates time series pattern feature extraction, graph structure behavior pattern feature extraction and sequence comprehensive feature extraction. LSTM is used to capture the time series pattern of user behavior, GCN mines the graph structure features of the relationship network between users, and the autoencoder is responsible for extracting compact and meaningful feature representations from high-dimensional data. The three work together to achieve a comprehensive analysis of user behavior and effectively enhance the performance of the internal threat detection system.
[0100] 5. Using deep learning technology to fuse the multi-dimensional features to obtain multi-dimensional fusion features, and then fusing the multi-dimensional fusion features with the statistical features to obtain comprehensive total features;
[0101] Figure 5 It is a schematic diagram of the feature fusion process. Figure 5 As shown in the figure, in this link, we adopted an innovative feature fusion strategy, combining the features discovered by the deep learning model with the intuitive statistical features, aiming to fully utilize the complementary advantages of the two types of features. Specifically, we use the Attention mechanism to fuse the features extracted by LSTM, GCN, and AE to dynamically assign weights to different features, ensuring that the model can automatically adjust which features are most important in a specific situation according to the context to obtain multi-dimensional fused features. The multi-dimensional fused features are then fused with the statistically encoded features to obtain the comprehensive total features.
[0102] Comprehensive total feature fusion method: The fusion process first uses the Attention mechanism to obtain the fused features, and then further combines these features with the features obtained in the statistical feature encoding module. The combination methods include concatenation and element-wise addition strategies.
[0103] Splicing: Through splicing, we retain the complete information of all features, ensuring that the model can independently decide which feature combinations are most helpful for the final prediction, increasing the complexity and expressiveness of the model.
[0104] Bitwise addition: By bitwise addition, we simplify the feature combination, which may reduce the risk of overfitting. At the same time, we adjust the feature weights to highlight the importance of common features.
[0105] 6. Input the comprehensive total features into the threat detection module for internal threat detection. The threat detection module is composed of CNNs and FNNs.
[0106] 7. Experiment
[0107] In this section, we will evaluate the proposed MITD insider threat detection model from the perspective of accuracy, precision, recall, and F1 score. First, we discussed the limitations of a single feature, and then we discussed the importance of features based on statistical coding to our proposed model. On this basis, we discussed the impact of fusion methods in feature fusion on the model, and finally compared our model with existing baseline models and neural network models. Our experimental environment uses torch2.2.2+cu121, where the experimental parameters are Epoch=200, batch size=64, lr=0.001.
[0108] 7.1 Limitations of Single Feature Detection
[0109] In our proposed MITD internal threat detection framework, we conducted a key experiment, namely, exploring the limitations of single feature detection. In the experiment, we compared the effects of using only statistical coding features and only using sequence features fused by the Attention mechanism. As shown in Table 4, the data in the first two rows reflect the performance of using statistical coding features and sequence coding features alone without fusing other features. The last two rows show the effect of the fusion of these two features, in which we used two fusion strategies, bitwise addition and concatenation, for comparison.
[0110] Through a detailed analysis of the data in the experimental comparison of single features and fused features in Table 4, we observed that in the MITD framework, the fused features showed significant advantages over single features. Whether using bitwise addition or concatenation, the fused features showed better performance in accuracy (Acc), precision (Pre), recall (Rec), and F1 score. This strongly proves that in the task of internal threat detection, by fusing features from different sources, the detection efficiency of the model can be significantly improved, thereby better identifying potential internal threats.
[0111] Table 4 Experimental comparison of single feature and fusion feature
[0112]
[0113]
[0114] 7.2 Discussion of the Importance of Statistical Coding Features
[0115] To further explore the indispensability of statistical features within the MITD framework, we designed a series of experiments that combined intuitive statistical coding features with sequence features extracted through three deep learning models: GCN (graph convolutional network), LSTM (long short-term memory network), and AE (autoencoder). We randomly selected three of the four features and intelligently fused them through the Attention mechanism, aiming to dynamically adjust and assign appropriate weights to different features. Subsequently, we further explored the fusion strategy, namely bitwise addition and concatenation, to combine the fused features with the statistical coding features for a second time.
[0116] Table 5 clearly shows that the fusion of statistical encoding features significantly improves the performance of internal threat detection. Almost all fusion configurations show better performance than single features or partially fused features in terms of accuracy, precision, recall, and F1 score. However, it is worth noting that when using GCN+AE+Statistical for feature fusion, the model performance did not meet expectations, but instead showed performance degradation. The reason for this anomaly may be that the features extracted by GCN and AE are redundant or conflicting with the statistical features to some extent, resulting in the Attention mechanism failing to effectively balance the importance of these features, thereby affecting the overall performance of the model.
[0117] Table 5 compares the effects of statistical coding features
[0118]
[0119]
[0120] 7.3 Discussion on the impact of fusion methods on the model
[0121] In order to determine the most effective feature fusion strategy, we conducted detailed comparative experiments, focusing on analyzing how different fusion techniques affect the performance of the final model. Specifically, we selected three of the four features, GCN (graph convolutional network), LSTM (long short-term memory network), AE (autoencoder), and statistical encoding features, and first performed deep feature fusion through the Attention mechanism. Next, we used two different fusion methods - element-wise addition and concatenation - to further integrate statistical encoding features to examine the impact of these two strategies on model performance.
[0122] As shown in Table 6 below, it is clearly revealed that the concatenation method has advantages over bitwise addition fusion in various evaluation indicators, including accuracy, precision, recall, and F1 score. The concatenation method gives the model greater flexibility by maintaining the integrity of all features, allowing it to independently decide which feature combinations are most critical for prediction, thereby improving the expressiveness and complexity of the model. On the contrary, although bitwise addition simplifies feature combinations and helps reduce the risk of overfitting, it may mask the unique contributions of different features in some cases, limiting the performance of the model.
[0123] Table 6 Effects of bitwise addition and concatenation
[0124]
[0125]
[0126] Therefore, our experimental results strongly support the use of splicing as the preferred strategy for feature fusion, especially when dealing with multiple heterogeneous features, which can more effectively capture and exploit the interactions between features, thereby significantly enhancing the performance of the MITD insider threat detection model.
[0127] 7.4 Performance comparison with other models
[0128] Given that our proposed internal threat detection model uses a unique architecture that integrates multiple sub-models and fully exploits the advantages of each sub-model, we conducted a comprehensive performance comparison with a series of baseline models and neural network models. In this comparative experiment, we strictly controlled the consistency of experimental conditions to ensure that all models competed fairly on the same parameter settings and datasets, and feature fusion was uniformly spliced.
[0129] As shown in Table 7 below, the comparison results are summarized, which intuitively demonstrates the superiority of our proposed model in the field of internal threat detection. Compared with traditional machine learning models such as logistic regression (LR), K-nearest neighbor algorithm (KNN), support vector machine (SVM) and single neural network such as convolutional neural network (CNN), our model shows significant performance improvement. It is particularly worth mentioning that even compared with those complex combination neural network models such as AE+CNN, LSTM+CNN and GCN+CNN, our model still maintains a leading position.
[0130] Table 7 Comparison results with other baseline models and neural network models
[0131]
[0132]
[0133] Specifically, in terms of the two key evaluation indicators, accuracy (Acc) and recall (Rec), our proposed model achieved results of 93.2% and 93.5% respectively. These results strongly prove that by cleverly integrating GCN, LSTM, AE and statistical features, supplemented by an intelligent Attention fusion mechanism, our model can more accurately identify internal threats.
[0134] In summary, this study proposed a multi-model integrated internal threat detection framework MITD, which demonstrated the significant advantages of integrating multiple feature extraction methods and intelligent fusion mechanisms in the field of internal threat detection. Experimental comparisons confirmed that MITD not only surpassed traditional machine learning models, but also showed excellent performance in the competition with complex neural network models, especially in terms of accuracy and recall rate indicators, reaching high levels of 93.2% and 93.5% respectively. Looking forward, we plan to continue to deepen the exploration of feature extraction models, especially by introducing more advanced sequence modeling techniques, to further improve the detection accuracy and generalization ability of the MITD framework. At the same time, we will work to optimize the fusion mechanism to make it more adaptable to the evolving internal threat environment and ensure that MITD becomes a cutting-edge tool in the field of internal threat detection.
[0135] Therefore, the present invention provides a multi-dimensional internal threat detection method based on deep learning, which can analyze and deeply explore user behavior patterns from multiple angles, reveal the essential characteristics of user behavior, identify potential security risks, and effectively improve the accuracy and efficiency of internal threat detection.
[0136] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0137] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A multi-dimensional internal threat detection method based on deep learning, It is characterized in that The following steps are involved: Collecting a multi-source data set consisting of normal behaviors and malicious behaviors, and then preprocessing the multi-source data set to obtain a user behavior data file; Based on the user behavior data file, using statistical methods and statistical coding rules to extract key behavior features and mark potential threat behaviors, to obtain statistical features for representing user behavior patterns; Based on the user behavior data file, the user operation behavior is serialized in chronological order using the behavior coding rules to obtain a behavior sequence feature; The behavior sequence features are subjected to in-depth feature extraction using three dimensions: time dependency, graph structure relevance, and data dimensionality reduction, to obtain multi-dimensional features; the multi-dimensional features include time series pattern features, graph structure behavior pattern features, and sequence comprehensive features; Using deep learning technology to fuse the multi-dimensional features to obtain multi-dimensional fusion features; The multi-dimensional fusion feature is integrated with the statistical feature to obtain a comprehensive total feature; The comprehensive overall feature is input into the threat detection module to perform internal threat detection.
2. According to the multi-dimensional internal threat detection method based on deep learning according to claim 1, It is characterized in that The multi-source data set includes login and logout log files, device interaction record log files, browsing history log files, email record log files, file operation log files, and user directory log files.
3. According to claim 2, a multi-dimensional internal threat detection method based on deep learning, It is characterized in that Preprocess the multi-source data set to obtain a user behavior data file, including: Integrate the login and logout log files, the device interaction record log files, the browsing history log files, the email record log files, and the file operation log files based on the user ID and time, and fill in the non-existent null values with 0 to obtain integrated data; The integrated data is grouped based on user IDs to generate the user behavior data file.
4. According to claim 3, a multi-dimensional internal threat detection method based on deep learning, It is characterized in that The process of extracting key behavioral features using statistical methods and statistical coding rules is as follows: Based on the login and logout log files, extract the first login time and the last logout time of the user in a day, and calculate the ratio of the first login time and the last logout time to 24 hours, so as to obtain the first login time ratio and the last logout time ratio; Based on the device interaction record log file, extract the first and last time the user uses the external device in a day, and calculate the ratio of the first and last time the external device is used to 24 hours, to obtain the first device use time ratio and the last device use time ratio; The device interaction log file is used to record statistics and obtain the number of times the external device is used during non-working hours.
5. According to claim 4, a multi-dimensional internal threat detection method based on deep learning, It is characterized in that Based on the user behavior data file, the user operation behavior is serialized in chronological order using the behavior coding rules to obtain behavior sequence features, including: By defining 13 behavioral activities and dividing each behavioral activity into two groups according to the time of commuting to and from get off work, 26 behavioral codes were obtained; Based on 26 types of behavior codes, the user's daily behavior data is encoded into a daily behavior sequence to obtain the behavior sequence features.
6. According to claim 5, a multi-dimensional internal threat detection method based on deep learning, It is characterized in that The behavior sequence features are further extracted using the three dimensions of time dependency, graph structure relevance, and data dimensionality reduction to obtain multi-dimensional features, including: The behavior sequence features are input into LstmAutoEncoder, the behavior sequence is compressed by the encoder, and then the behavior sequence is reconstructed by the decoder to extract the time series pattern features containing time series pattern information.
7. The multi-dimensional internal threat detection method based on deep learning according to claim 6, It is characterized in that The behavior sequence features are further extracted using the three dimensions of time dependency, graph structure relevance, and data dimensionality reduction to obtain multi-dimensional features, including: Extracting and analyzing the relationship information between users in the user directory log file to construct a user relationship network graph, and inputting the behavior sequence and the user relationship network graph into a graph convolutional network; In the graph convolutional network, the first GCNConv layer is used to extract and output preliminary graph structure features, and then the ReLU activation function is used to adjust the preliminary graph structure features into nonlinear features to input the second GCNConv layer. Finally, the output of the second GCNConv layer is normalized using the softmax function to obtain graph structure behavior pattern features containing user behavior and key relationship network information.
8. The multi-dimensional internal threat detection method based on deep learning according to claim 7, It is characterized in that The behavior sequence features are further extracted using the three dimensions of time dependency, graph structure relevance, and data dimensionality reduction to obtain multi-dimensional features, including: Build an autoencoder based on fully connected layers; The autoencoder is used to extract low-dimensional features from the behavior sequence to obtain comprehensive sequence features.
9. The multi-dimensional internal threat detection method based on deep learning according to claim 8, It is characterized in that The multi-dimensional fusion feature is integrated with the statistical feature to obtain a comprehensive total feature, including: The temporal pattern features, the graph structure behavior pattern features and the sequence comprehensive features are fused by using an attention mechanism, and weights are dynamically assigned according to different features during the fusion process to obtain the multi-dimensional fusion features; The multi-dimensional fusion features and the statistical features are fused using a concatenation strategy and a bitwise addition strategy to obtain the comprehensive total features.
Citation Information
Patent Citations
Internal threat detection method based on feature fusion
CN114553497A
Abnormal behavior detection method and system based on behavior and attention mechanism
CN114565002A
Internal security threat detection method, system and device and storage medium
CN114598545A
Internal threat detection method and system based on multiple modes
CN115034292A
Internal threat detection method based on log behaviors and user characteristics
CN116455644A
Cited By
Multi-level dynamic threat monitoring system based on deep learning
CN120074883A
Deep Learning-Based Multi-Level Dynamic Threat Detection System
CN120074883B
