A database insider attack detection method, system, device and medium based on ensemble learning

Abstract

A database insider attack detection method, system, device and medium based on ensemble learning, the method comprises the following steps: obtaining logs and human resource forms in a database, and preprocessing the obtained data; dividing a training set and a validation set; splitting and recombining to construct a training set and a validation set of query syntax data set, query result data set and insider information data set; training respective sub-models using the training set, optimizing the sub-models through the validation set, and obtaining optimal sub-models; fusing the three sub-models to form a trained integrated model, calculating an alarm threshold, and confirming data stealing behavior; obtaining newly generated logs and human resource forms from the database, preprocessing the new data; reconstructing a new data set; inputting the new data set into the optimal sub-model and the integrated model to determine data stealing behavior; the present application reduces the false detection rate and the missed detection rate, improves the detection accuracy, and effectively improves the comprehensive performance of the model.

Description

Technical Field

[0001] This invention relates to the field of data theft detection technology, specifically to a method, system, device, and medium for detecting internal attacks on databases based on ensemble learning. Background Technology

[0002] Data has become a crucial strategic resource for nations and a new type of production factor driving economic and social development. As databases are the storage medium for data, database security is of paramount importance for organizations (such as companies and government departments) to protect data and defend against various external and internal threats. Common external threats include unauthorized access, hacking, viruses, ransomware, distributed denial-of-service (DDoS) attacks, and phishing attacks. These threats can often be effectively protected against using access control mechanisms such as network firewalls, antivirus software, and user authentication. Common internal threats are categorized into imposters and insiders: imposters are hackers who steal usernames, passwords, and other credentials to impersonate insiders and access the database. Depending on whether the attack is intentional, insiders can be further divided into malicious and careless insiders. Attacks by imposters and malicious insiders to steal confidential data (such as user information records, trade secrets, military secrets, and intellectual property) are termed insider attacks. Because insider attackers gain access credentials to the database, traditional access control mechanisms often become ineffective. Furthermore, insiders know the exact location of sensitive data and the network security measures deployed on the system, making insider attack detection extremely challenging.

[0003] Existing insider attack detection methods mainly utilize machine learning methods such as Bayesian networks, clustering, and Hidden Markov Models to analyze whether there are anomalies in the interaction patterns between users and databases. Based on the input feature vectors of the machine learning models used, existing insider attack detection methods can be divided into three main categories: (1) syntax detection methods; (2) semantic detection methods; and (3) hybrid detection methods. Among them, the basic idea of ​​syntax detection methods is to use machine learning models to analyze the syntax features of the SQL statements entered by users when performing a series of operations such as adding, deleting, modifying, and querying in the database. Commonly used syntax features include SQL command type, range table (i.e., the table referenced by the query), and projection list (including attributes appearing in the query result set and attributes appearing in the WHERE clause). Since syntax features only require parsing the SQL statement string and traversing the generated syntax parse tree, they can be extracted quickly. Since imposters usually do not know the access patterns of the real account owner, they are unlikely to perform operations consistent with the typical behavior of the account owner, which also leads to significant differences in the query structure of the SQL statements of imposters and normal users. Therefore, syntax-based detection methods are often very effective against imposter attacks. The basic idea behind semantic detection methods is to use machine learning to analyze the data in the query result set. Commonly used semantic features include relevant statistical information of the result set (such as the number of rows, columns, and the percentage of data in the result set relative to the original tables), the size of the result set, and the original tuples corresponding to the result set. Because it requires executing the query, parsing the query results, and matching the query results with the original tuples in the monitored database, extracting semantic features can be very costly. However, compared to syntax detection, semantic detection can handle more complex insider attacks. This is because insider attacks on databases often involve extracting data that exceeds normal sizes or viewing data records beyond the scope of an insider's job responsibilities. Hybrid detection methods typically utilize existing machine learning models to analyze both the aforementioned syntax and semantic features simultaneously.

[0004] Therefore, existing methods only consider the "results" (the syntax and semantics of SQL statements) of attacks by internal database personnel, but do not consider the "motivation" (the root cause) of employees launching internal attacks, resulting in low detection accuracy and high false positive and false negative rates.

[0005] Patent application CN115277113A discloses a power grid network intrusion event detection and identification method based on ensemble learning. This method utilizes random forests and XGBoost, employing a multi-classifier ensemble learning approach to continuously enhance the learning of anomalous events in small samples. However, due to the complexity and numerous features of attacks from internal database attackers, this method suffers from low detection accuracy and high false positive and false negative rates.

[0006] The patent application with publication number CN113014529B discloses a method, apparatus, medium, and device for identifying network attacks. It identifies vulnerable targets by acquiring log data within a preset time period, the vulnerable targets including specified paths and / or URL types; based on the vulnerable targets, it constructs user behavior features and establishes a dataset; based on the dataset, it trains a machine learning model and extracts decision logic, constructs algorithm rules, and verifies them; the verified algorithm rules are used to identify network attacks. However, because the features considered are relatively singular and do not take into account the "motivation" of employees launching internal attacks, it suffers from low detection accuracy and high false positive and false negative rates. Summary of the Invention

[0007] To overcome the shortcomings of the existing technology, the present invention aims to provide a database insider attack detection method, system, device, and medium based on ensemble learning. This method not only fully considers the various syntactic and semantic features of insider interactions with the database, but also the insider's own profile information. Random forests are used to analyze these three factors, thereby fully exploring the results and motivations of insider attacks, resulting in high detection accuracy. The ensemble learning method comprehensively considers the detection results of various random forests, leading to a high accuracy rate and low false positive and false negative rates in detecting database insider attacks. Furthermore, the ensemble learning approach effectively improves the generalization ability of the detection model. In addition, an enumeration optimization strategy is used to determine the number of decision trees in each sub-model (i.e., random forest), i.e., the size of each sub-model, optimizing the size of each sub-model and largely avoiding local optima. A sub-model fusion threshold is calculated based on a binomial distribution model, ensuring that the false positive rate is as low as possible while maintaining a certain false positive rate (i.e., not exceeding a pre-specified value).

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A database insider attack detection method based on ensemble learning includes the following steps:

[0010] Step S1: Obtain logs and human resources forms from the database. Extract the SQL statements, role information, time information, space information, and query results from the logs. Also, obtain the user profile data of the users who performed the operation. Perform preprocessing on the obtained data to obtain the preprocessed access and user dataset.

[0011] Step S2: Divide the preprocessed access and user dataset obtained in step S1 into a training set and a validation set;

[0012] Step S3: Based on the training set and validation set divided in step S2, split and recombine both the training set and validation set to construct the training set and validation set for the query syntax dataset, query result dataset, and internal personnel information dataset.

[0013] Step S4: Use the training set obtained in step S3 to train their respective sub-models, and optimize the trained sub-models using the validation set to obtain the optimal sub-model;

[0014] Step S5: The three sub-models optimized in step S4 are fused using the fusion constraint to form a trained ensemble model. An alarm threshold is calculated using the validation set in step S3, the optimal sub-model obtained in step S4, and the fusion constraint. When the number of alarms for a user exceeds the threshold, an alarm is issued to confirm that the user has engaged in data theft.

[0015] Step S6: Retrieve newly generated logs and human resources forms from the database, and retrieve the SQL statements, role information, time information, space information and query results from the newly generated logs. Also retrieve user profile data of the users who performed the operation. Preprocess the newly retrieved data to obtain the preprocessed access and user dataset.

[0016] Step S7: Split and recombine the access and user datasets preprocessed in Step S6 to construct a query syntax dataset, a query result dataset, and an internal personnel information dataset.

[0017] Step S8: Input the query syntax dataset, query result dataset, and internal personnel information dataset constructed in Step S7 into the optimal sub-model optimized in Step S4, and input the results of the sub-model into the ensemble model trained in Step S5. Then, output the final result through the alarm module of the ensemble model to determine whether each user accessing the database has engaged in data theft.

[0018] Furthermore, step S1 includes:

[0019] Step S11: Obtain the logs and human resources forms of the database with n tables. Extract the SQL statements from the logs of the database with n tables and mark the operators in the SQL statements as O. C The table being accessed is characterized as A. T The attribute feature of the access table is marked as A. A Mark f i Indicates whether table i has been accessed, marked m i Let A represent the total number of attributes accessed in table i. T ={f1, f2, ..., f n},A A ={m1, m2, ..., m n};

[0020] Step S12: Extract the user role marker U from the log of the database with n tables obtained in step S11. R The access time is marked as T A Accessing a geographic location marked as S A and U R and S A The process is performed using tag encoding, where T... A The process uses only the hourly time;

[0021] Step S13: Extract the query results from the log of the database with n tables obtained in step S11, and mark the number of query results as R. N The number of attributes contained in the query results is denoted as R. A The size of the query results is denoted as R. S The distribution characteristics of the query results R T R T p represents the proportion of data retrieved from each table in the query results for that table. i R represents the ratio of the number of rows retrieved from table i to the total number of rows in table i in the query results; T ={p1, p2, ..., p n};

[0022] Step S14: Extract the monthly salary of the internal personnel corresponding to each visit from the human resources form obtained in step S11, and mark it as S. M Internal staff performance is marked as P W The length of service for internal staff is marked as Y. W The monthly working hours of internal staff are marked as H. W Whether an internal staff member is promoted within five years is marked as P5, and S M and H WThe data is divided into w levels using digits of 1 / w, 2 / w, ..., 1 / w-w, for data processing.

[0023] Further, step S3 includes:

[0024] Step S31: Split the training set and validation set obtained in step S2 according to features;

[0025] Step S32: Reassemble the dataset split in step S31 to construct a query syntax dataset. The query syntax dataset is constructed by using U R O C A T and A A The dataset D1 is composed of the training set and the validation set.

[0026] Step S33: Reassemble the dataset split in step S31 to construct the query result dataset. The query result dataset is constructed by using U R O C A T R N R A R S and R T The dataset D2 is composed of the training set and the validation set.

[0027] Step S34: Reassemble the dataset split in step S31 to construct an internal personnel information dataset. The internal personnel information dataset is constructed by using U R O C A T T A S A S M P W Y W H W The dataset D3 is formed by combining P5 and P5. Dataset D3 includes a training set and a validation set.

[0028] Further, step S4 includes:

[0029] Step S41: Construct three random forest sub-models h respectively. i (x), each h i In (x), the decision tree selection K = log2|D i | Attributes are trained, where D i Includes datasets D1, D2, and D3, |D i |Represents D i The total number of attributes in the tree is used to select the best feature for tree splitting based on the Gini coefficient minimization criterion;

[0030] Step S42: Use the training sets from dataset D1 in step S32, dataset D2 in step S33, and dataset D3 in step S34 to test h respectively. i (x) is trained, and each h... i (x) Using the corresponding FNR value as the objective function value, through D i The validation set for each h i Optimize the decision trees in (x) to find the h-th tree that minimizes the FNR. i The number of decision trees in h1(x) is calculated, where the initial range of the number of decision trees in h2(x) is 5-100, the initial range of the number of decision trees in h3(x) is 5-400, and the initial range of the number of decision trees in h3(x) is 5-100. This yields the optimal sub-model. in include

[0031] Furthermore, step S5 includes:

[0032] Step S51: Set the fusion constraint Where T0 is the constraint value, let T0 = 1, and use the validation sets of dataset D1 in step S32, dataset D2 in step S33, and dataset D3 in step S34, respectively, and the optimal sub-model obtained in step S42. And the fusion constraint H(x) is used to calculate the TPR value of the ensemble model, where TPR represents the True Positive Rate, also known as sensitivity, which represents the percentage of detected abnormal samples out of all abnormal samples;

[0033] Step S52: Using the TPR value from step S51, calculate the T value according to the binomial distribution model to obtain the alarm threshold T. That is, when the user's cumulative alarm count exceeds the threshold T, an alarm will be issued to confirm that the user has engaged in data theft.

[0034] Furthermore, step S6 includes:

[0035] Step S61: Retrieve newly generated logs and human resources forms from the database, and extract SQL statements from the newly generated logs in the database, marking the operators in the SQL statements as O. C The table being accessed is characterized as A. T The attribute feature of the access table is marked as A. A Mark f i Indicates whether table i has been accessed, marked m i Let A represent the total number of attributes accessed in table i; then A T={f1, f2, ..., f n},A A ={m1, m2, ..., m n};

[0036] Step S62: Extract the user role marker U from the newly generated logs obtained from the database in step S61. R The access time is marked as T A Accessing a geographic location marked as S A and U R and S A The process is performed using tag encoding, where T... A The process uses only the hourly time;

[0037] Step S63: Extract the query results from the newly generated logs obtained from the database in step S61, and mark the number of query results as R. N The number of attributes contained in the query results is denoted as R. A The size of the query results is denoted as R. S The distribution characteristics of the query results R T R T p represents the proportion of data retrieved from each table in the query results for that table. i R represents the ratio of the number of rows retrieved from table i in the query results to the total number of rows in table i. T ={p1, p2, ..., p n};

[0038] Step S64: Extract the monthly salary of the internal personnel corresponding to each visit from the human resources form obtained in step S61, and mark it as S. M Internal staff performance is marked as P W The length of service for internal staff is marked as Y. W The monthly working hours of internal staff are marked as H. W Whether an internal staff member is promoted within five years is marked as P5, and S M and H W The data is divided into w levels using digits of 1 / w, 2 / w, ..., 1 / w-w, for data processing.

[0039] Further, step S7 includes:

[0040] Step S71: Split the access and user datasets after preprocessing in step S6;

[0041] Step S72: Construct a query syntax dataset from the access and user datasets split in step S71. The query syntax dataset is constructed by using U R O C AT and A A The dataset D1′ is formed by combining the data together;

[0042] Step S73: Construct a query result dataset from the access and user datasets split in step S71. The query result dataset is constructed by using U R O C A T R N R A R S , and R T The dataset D′2 is formed by combining the data.

[0043] Step S74: Construct an internal personnel information dataset from the access and user datasets split in step S71. The internal personnel information dataset is constructed by using U R O C A T T A S A S M P W Y W H W The dataset D′3 is formed by combining P5 with it;

[0044] Further, step S8 includes:

[0045] Step S81: Input the dataset D1′ from step S72, the dataset D′2 from step S73, and the dataset D′3 from step S74 into the optimal sub-model optimized in step S42. The detection is performed in the middle, and the detection results of each sub-model are obtained respectively;

[0046] Step S82: Input the detection results of each sub-model obtained in step S81 into the fusion constraint H(x) set in step S51 to obtain the final detection result;

[0047] Step S83: If the detection result obtained in step S82 is "abnormal", then check whether the number of alarms from the internal personnel accessing this access exceeds the T value calculated in step S52. If it exceeds, the model issues an alarm and determines that the user has engaged in data theft.

[0048] This invention also provides a database insider attack detection system based on ensemble learning, comprising:

[0049] Data preprocessing module: Used to obtain logs and human resources forms from the database, extract SQL statements, role information, time information, spatial information and query results from the logs, and obtain user profile data of the users who operate. The obtained data are preprocessed to obtain the preprocessed access and user dataset.

[0050] Dataset partitioning module: used to divide the preprocessed access and user datasets into training and validation sets;

[0051] Dataset Reconstruction Module: This module is used to split and recombine each dataset based on the pre-defined training and validation sets to construct training and validation sets for the query syntax dataset, query result dataset, and internal personnel information dataset.

[0052] Sub-model optimization module: Used to train each sub-model separately using the training set, and optimize each trained sub-model using the validation set to obtain the optimal sub-model;

[0053] Alarm threshold calculation and alarm module: It is used to fuse the three optimized sub-models through the fusion constraint to form a trained ensemble model, and calculate an alarm threshold through the validation set and the optimal sub-model. When the number of alarms for a user exceeds the threshold, an alarm is issued to confirm that the user has engaged in data theft.

[0054] The new data preprocessing module is used to retrieve newly generated logs and human resources forms from the database, and to retrieve SQL statements, role information, time information, spatial information and query results from the newly generated logs. It also retrieves user profile data of the users who operate the data, and preprocesses the new data to obtain the preprocessed access and user dataset.

[0055] New Dataset Partitioning Module: Used to split and recombine the preprocessed access and user datasets to construct query syntax datasets, query result datasets, and internal personnel information datasets;

[0056] The detection and alerting module is used to input the constructed query syntax dataset, query result dataset, and internal personnel information dataset into the optimized sub-model, input the results of the sub-model into the trained ensemble model, and output the final result through the alerting module of the ensemble model to determine whether each user accessing the database has engaged in data theft.

[0057] This invention also provides a database insider attack detection device based on ensemble learning, comprising:

[0058] Memory: A computer-readable device that stores the computer program of the above-mentioned database insider attack detection method based on ensemble learning;

[0059] Processor: Used to implement the database insider attack detection method based on ensemble learning when executing the computer program.

[0060] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned database insider attack detection method based on ensemble learning.

[0061] Compared with the prior art, the beneficial effects of the present invention are:

[0062] 1. This invention improves the detection capability against attacks by internal database personnel by extracting user profile information from human resources forms, thus compensating for the lack of syntactic and semantic information in constructing data theft behaviors and motivations.

[0063] 2. This invention improves model performance and more accurately analyzes data feature patterns by splitting and recombining data according to features.

[0064] 3. This invention, by pre-setting the optimization range, enumerating the size of the sub-models and using the FNR value as the target value for optimization, more accurately determines the optimal model and largely avoids the model getting trapped in local optima.

[0065] 4. By constructing a fusion constraint, this invention can improve the generalization ability of the model and reduce the false negative rate of the model.

[0066] 5. By establishing an alarm module and calculating alarm thresholds, this invention can reduce the false detection rate of the model.

[0067] In summary, this invention, by extracting user information, compensates for the lack of syntactic and semantic information in constructing data theft behaviors and motivations, greatly improving the detection capability against attacks by internal database personnel. Furthermore, by splitting and recombining different features, training them separately, and optimizing them within preset ranges, enumerating sub-model sizes, and using the FNR value as the target value, the optimal model is determined more accurately, largely avoiding the model getting trapped in local optima. Then, by fusing the models through a fusion constraint, the model's generalization ability is improved. Finally, by using an alarm module to set alarm thresholds, the model is further optimized. The resulting detection results reduce false positive and false negative rates, improve detection accuracy, and effectively enhance the overall performance of the model. Attached Figure Description

[0068] Figure 1A flowchart of a database insider attack detection method based on ensemble learning is provided for an embodiment of the present invention.

[0069] Figure 2 This is a schematic diagram of a data theft integrated detection model that integrates internal personnel profile features, provided as an embodiment of the present invention.

[0070] Figure 3 This diagram illustrates data extraction, data recombination, and data splitting in a database insider attack detection model based on ensemble learning, as provided in an embodiment of the present invention.

[0071] Figure 4 This is a graph showing the detection accuracy of a database insider attack detection model based on ensemble learning in an embodiment of the present invention on datasets of different data sizes. Detailed Implementation

[0072] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0073] like Figure 1 As shown, a database insider attack detection method based on ensemble learning includes the following steps:

[0074] Training phase:

[0075] Step S1: As Figure 3 As shown, logs and human resources forms in the database are retrieved. SQL statements, role information, time information, and location information, as well as query results, are obtained from the logs. User profile data of the users who performed the operations are also obtained. The retrieved data are preprocessed to obtain the preprocessed access and user dataset.

[0076] Step S11: In this embodiment, SQL statements are extracted from the logs of the database with n tables, and operators in the SQL statements are marked as O. C The table being accessed is characterized as A. T The attribute feature of the access table is marked as A. A Mark f i Indicates whether table i has been accessed, where f i The value is 0 or 1, marking m i Let A represent the total number of attributes accessed in table i. T ={f1, f2, ..., f n},A A ={m1, m2, ..., m n};

[0077] Step S12: Extract the user role marker U from the log of the database with n tables obtained in step S11. RThe access time is marked as T A Accessing a geographic location marked as S A and U R and S A The process is performed using tag encoding, where T... A Data processing is performed using only the hourly time; specifically, tag encoding involves converting U... R and S A The category string in the text is converted into the form 1, 2, 3, etc., for example: in U R There are 10 categories in total, so the strings for the 10 categories correspond to numbers 1 to 10 respectively; in T A The method for rounding up to the nearest whole number is to remove the minutes and seconds after the hour when processing the data after obtaining the specific time. For example, if the time obtained is "14:30:29", the value obtained after rounding up to the nearest whole number is "14".

[0078] Step S13: Extract the query results from the log of the database with n tables obtained in step S11, and mark the number of query results as R. N The number of attributes contained in the query results is denoted as R. A The size of the query results is denoted as R. S The distribution characteristics of the query results R T R T p represents the proportion of data retrieved from each table in the query results for that table. i R represents the ratio of the number of rows retrieved from table i to the total number of rows in table i in the query results; T ={p1, p2, ..., p n};

[0079] Step S14: Extract the monthly salary of the internal personnel corresponding to each visit from the human resources form obtained in step S11, and mark it as S. M Internal staff performance is marked as P W The length of service for internal staff is marked as Y. W The monthly working hours of internal staff are marked as H. W Whether an internal staff member is promoted within five years is marked as P5, and S M and H W The data is divided into w levels using digits of 1 / w, 2 / w, ..., 1 / w-w, for data processing. In this embodiment, w = 3 is selected, which is equivalent to dividing S... M and H W It is divided into three levels. Specifically, the monthly salary of internal staff is divided into three levels: high, medium and low, and the monthly working hours of internal staff are divided into three levels: long, medium and short.

[0080] Step S2: Divide the preprocessed access and user dataset obtained in step S1 into a training set and a validation set. The access and user dataset is divided into the training set and the validation set in an 8:2 ratio.

[0081] Step S3: As Figure 3 As shown, based on the training set and validation set divided in step S2, both the training set and validation set are split and recombined to construct the training set and validation set of query syntax dataset D1, query result dataset D2 and internal personnel information dataset D3.

[0082] Step S31: Split the training set and validation set obtained in step S2 according to features;

[0083] Step S32: Reassemble the dataset split in step S31 to construct a query syntax dataset. The query syntax dataset is constructed by using U R O C A T and A A The dataset D1 is composed of the training set and the validation set.

[0084] Step S33: Reassemble the dataset split in step S31 to construct the query result dataset. The query result dataset is constructed by using U R O C A T R N R A R S and R T The dataset D2 is composed of the training set and the validation set.

[0085] Step S34: Reassemble the dataset split in step S31 to construct an internal personnel information dataset. The internal personnel information dataset is constructed by using U R O C A T T A S A S M P W Y W H W The dataset D3 is formed by combining P5 and P5. Dataset D3 includes a training set and a validation set.

[0086] Step S4: Use the training set obtained in step S3 to train their respective sub-models, and optimize the trained sub-models using the validation set to obtain the optimal sub-model;

[0087] Step S41: As Figure 2 As shown, three random forest sub-models h are constructed respectively.i (x), each h i In (x), the decision tree selection K = log2|D i | Attributes are trained, where D i Includes datasets D1, D2, and D3, |D i |Represents D i The total number of attributes in the tree is used to select the best feature for tree splitting based on the Gini coefficient minimization criterion;

[0088] Step S42: Use the training sets from dataset D1 in step S32, dataset D2 in step S33, and dataset D3 in step S34 to test h respectively. i (x) is trained, and each h... i (x) Using the corresponding FNR value as the objective function value, through D i The validation set for each h i Optimize the decision trees in (x) to find the h-th tree that minimizes the FNR. i The number of decision trees in h1(x) is calculated, where the initial range of the number of decision trees in h2(x) is 5-100, the initial range of the number of decision trees in h3(x) is 5-400, and the initial range of the number of decision trees in h3(x) is 5-100. This yields the optimal sub-model. in include

[0089] Specifically, each value t in the initial range of the decision tree is assigned to h. i (x), using its corresponding training set for h i Training (x) and using the corresponding validation set with the trained h i (x) Calculate the FNR at the corresponding t value, construct F = {t → FNR}, and then find the t value corresponding to the minimum FNR (denoted as t). * The calculation formula is as follows:

[0090]

[0091] In the above formula, F.FNR represents a list of all FNR values ​​in F, and Ft represents the h corresponding to all FNR values ​​in F.FNR. i The number of trees t in (x).

[0092] Step S5: The three sub-models optimized in step S4 are fused using the fusion constraint to form a trained ensemble model. An alarm threshold is calculated using the validation set in step S3, the optimal sub-model obtained in step S4, and the fusion constraint. When the number of alarms for a user exceeds the threshold, an alarm is issued to confirm that the user has engaged in data theft.

[0093] Step S51: Set the fusion constraint Where T0 is the constraint value, let T0 = 1, and use the validation sets of dataset D1 in step S32, dataset D2 in step S33, and dataset D3 in step S34, respectively, and the optimal sub-model obtained in step S42. And the fusion constraint H(x) is used to calculate the TPR value of the ensemble model, where TPR represents the True Positive Rate, also known as sensitivity, which represents the percentage of detected abnormal samples out of all abnormal samples;

[0094] Step S52: Using the TPR value from step S51, calculate the T value according to the binomial distribution model to obtain the alarm threshold T. An alarm will only be issued when the user's cumulative alarm count exceeds the threshold T, confirming that the user has engaged in data theft. Figure 2 The alarm module shown;

[0095] Specifically, construct a binomial distribution X ~ B(n, p), where p is the TPR (True Positive Rate) calculated after the ensemble model training in step S51. Suppose the model labels a user with T anomalies, and the probability of an anomaly being true is p. Let event A be an event where at least one of the T anomalies is true. Then... To ensure that P(A) ≥ 1 - ε, we need to use 1 - (1 - p). T ≥1-ε, i.e. (1-p) T ≤ε, after transformation is T≥log 1-p ε. Then we can calculate a threshold T, which is the probability that at least one of the T alarms is not a false positive alarm is greater than 1-ε, where ε is a small constant. ε is generally taken as a small constant between 0.01 and 0.1.

[0096] Testing phase:

[0097] Step S6: Retrieve newly generated logs and human resources forms from the database, and extract the SQL statements, role information, time information, space information, and query results from the newly generated logs. Also, retrieve user profile data of the users who performed the operations. Preprocess the newly acquired data to obtain the preprocessed access and user dataset.

[0098] Step S61: Retrieve newly generated logs and human resources forms from the database, and extract SQL statements from the newly generated logs in the database, marking the operators in the SQL statements as O. C The table being accessed is characterized as A. T The attribute feature of the access table is marked as A. A Mark f i Indicates whether table i has been accessed, marked m i Let A represent the total number of attributes accessed in table i. T ={f1, f2, ..., f n},A A ={m1, m2, ..., m n};

[0099] Step S62: Extract the user role marker U from the newly generated logs obtained from the database in step S61. R The access time is marked as T A Accessing a geographic location marked as S A And U R and S A The process is performed using tag encoding, where T... A The process uses only the hourly time;

[0100] Step S63: Extract the query results from the newly generated logs obtained from the database in step S61, and mark the number of query results as R. N The number of attributes contained in the query results is denoted as R. A The size of the query results is denoted as R. S The distribution characteristics of the query results R T R T p represents the proportion of data retrieved from each table in the query results for that table. i This represents the ratio of the number of rows retrieved from table i in the query results to the total number of rows in table i. Then R... T ={p1, p2, ..., p n};

[0101] Step S64: Extract the monthly salary of the internal personnel corresponding to each visit from the human resources form obtained in step S61, and mark it as S. M Internal staff performance is marked as P W The length of service for internal staff is marked as Y. W The monthly working hours of internal staff are marked as H. W Internal staff are categorized as P5 based on whether they have been promoted within five years. And S... M and H WThe data is divided into w levels using digits of 1 / w, 2 / w, ..., 1 / w-w, for data processing. In this embodiment, w is the same as in step S14, and w = 3.

[0102] Step S7: Split and recombine the access and user datasets preprocessed in Step S6 to construct a query syntax dataset, a query result dataset, and an internal personnel information dataset.

[0103] Step S71: Split the access and user datasets after preprocessing in step S6;

[0104] Step S72: Construct a query syntax dataset from the access and user datasets split in step S71. The query syntax dataset is constructed by using U R O C A T and A A The dataset D1′ is formed by combining the data together;

[0105] Step S73: Construct a query result dataset from the access and user datasets split in step S71. The query result dataset is constructed by using U R O C A T R N R A R S and R T The dataset D′2 is formed by combining the data.

[0106] Step S73: Construct an internal personnel information dataset from the access and user datasets split in step S71. The internal personnel information dataset is constructed by using U R O C A T T A S A S M P W Y W H W The dataset D′3 is formed by combining P5 with P5.

[0107] Step S8: Input the query syntax dataset, query result dataset, and internal personnel information dataset constructed in Step S7 into the optimal sub-model optimized in Step S4, and input the results of the sub-model into the ensemble model trained in Step S5, and then... Figure 2The alarm module of the integrated model shown outputs the final result, determining whether each user accessing the database has engaged in data theft. Specifically, the alarm module records the number of times each user is suspected of data theft. When this recorded number exceeds the threshold T calculated by the model, the model will output an alarm indicating that the user has engaged in data theft.

[0108] Step S81: Input the dataset D1′ from step S72, the dataset D′2 from step S73, and the dataset D′3 from step S74 into the optimal sub-model optimized in step S42. The detection is performed in the middle, and the detection results of each sub-model are obtained respectively;

[0109] Step S82: Input the detection results of each sub-model obtained in step S81 into the fusion constraint H(x) set in step S51 to obtain the final detection result;

[0110] Step S83: If the detection result obtained in step S82 is "abnormal", then check whether the number of alarms from the internal personnel accessing this access exceeds the T value calculated in step S52. If it exceeds, the model issues an alarm and determines that the user has engaged in data theft.

[0111] This invention addresses the problems of low detection accuracy, high false positive and false negative rates in existing database insider attack detection methods. It proposes an ensemble learning-based method for detecting database insider attacks. This method not only fully considers the syntactic and semantic features of SQL statements used by users to interact with the database, but also the spatiotemporal features of internal employee database access (such as the time and geographical location of access) and the employee's profile features (such as monthly salary level, performance, and monthly working hours). The method uses random forests to analyze these features separately, and then uses ensemble learning to comprehensively evaluate the results, achieving high accuracy and low false positive and false negative rates in detecting database insider attacks. By progressively adjusting the size of sub-models and training them enumerating them, the optimization objective is to minimize the FNR value. The optimal sub-model is then selected based on the size corresponding to the best sub-model, largely avoiding the model getting trapped in local optima. The ensemble learning-based fusion model improves the model's generalization ability and effectively enhances its overall performance. Furthermore, by setting a fusion threshold, the false negative rate is significantly reduced.

[0112] like Figure 2The diagram shows the architecture of the integrated model. The data is split and recombined by the data splitting and recombining module, and then trained and optimized separately. After obtaining the optimal sub-model, the sub-models are fused and integrated with constraints. Then, a threshold alarm mechanism is set, which greatly improves the model's ability to detect attacks from inside the database and its generalization ability, and avoids the model getting stuck in local optima.

[0113] like Figure 3 The diagram shows the architecture of the data splitting and reorganization module. By extracting features from the acquired database logs and human resources forms, reorganizing them, and then splitting them into query syntax datasets, query result datasets, and internal personnel information datasets according to specific rules, the module can improve model performance and more accurately analyze data feature patterns.

[0114] Experimental data analysis

[0115] like Figure 4 As shown, the simulation experiment dataset of this invention consists of six sets of data, with data volumes of 10,000, 20,000, 30,000, 40,000, 50,000, and 60,000 records respectively; training and testing were performed diligently on each dataset, and the results were obtained through... Figure 4 It can be seen that the accuracy of the dataset insider attack detection model based on ensemble learning is above 92% under different data scales, indicating that the model has high accuracy and performs well under different data scales, indicating that the model has high comprehensive ability and generalization ability.

[0116] This invention also provides a database insider attack detection system based on ensemble learning, comprising:

[0117] The data preprocessing module is used to obtain logs and human resources forms from the database in step 1, extract SQL statements, role information, time information, space information and query results from the logs, and obtain user profile data of the users who operate. The module preprocesses the obtained data to obtain the preprocessed access and user dataset.

[0118] Dataset partitioning module: This is used to partition the preprocessed access and user datasets into training and validation sets in step 2.

[0119] Dataset Reconstruction Module: This module is used to implement step 3, which involves splitting and recombining each dataset based on the pre-defined training and validation sets to construct training and validation sets for the query syntax dataset, query result dataset, and internal personnel information dataset.

[0120] Sub-model optimization module: This module is used to train the respective sub-models using the training set in step 4, and then optimize the trained sub-models using the validation set to obtain the optimal sub-model.

[0121] Alarm threshold calculation and alarm module: It is used to merge the three optimized sub-models through the fusion constraint in step 5 to form a trained ensemble model, and calculate an alarm threshold through the validation set and the optimal sub-model. When the number of alarms for a user exceeds the threshold, an alarm is issued to confirm that the user has data theft behavior.

[0122] New data preprocessing module: used to retrieve newly generated logs and human resources forms from the database in step 6, and to retrieve SQL statements, role information, time information, space information and query results from the newly generated logs, and to retrieve user profile data of the operating users. The new data is preprocessed to obtain the preprocessed access and user dataset.

[0123] New dataset partitioning module: used to split and recombine the preprocessed access and user datasets in step 7 to construct query syntax dataset, query result dataset, and internal personnel information dataset;

[0124] The detection and alarm module is used to input the constructed query syntax dataset, query result dataset, and internal personnel information dataset into the optimized sub-model in step 8, input the results of the sub-model into the trained ensemble model, and output the final result through the alarm module of the ensemble model to determine whether each user accessing the database has engaged in data theft.

[0125] This invention also provides a database insider attack detection device based on ensemble learning, comprising:

[0126] Memory: A computer-readable device that stores the computer program of the above-mentioned database insider attack detection method based on ensemble learning;

[0127] Processor: Used to implement the database insider attack detection method based on ensemble learning when executing the computer program.

[0128] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned database insider attack detection method based on ensemble learning.

Claims

1. A database insider attack detection method based on ensemble learning, characterized in that: Includes the following steps: Step S1: Obtain logs and human resources forms from the database. Extract the SQL statements, role information, time information, space information, and query results from the logs. Also, obtain the user profile data of the users who performed the operation. Perform preprocessing on the obtained data to obtain the preprocessed access and user dataset. Step S2: Divide the preprocessed access and user dataset obtained in step S1 into a training set and a validation set; Step S3: Based on the training set and validation set divided in step S2, split and recombine both the training set and validation set to construct the training set and validation set for the query syntax dataset, query result dataset, and internal personnel information dataset. Step S4: Use the training set obtained in step S3 to train their respective sub-models, optimize their respective sub-models using the validation set, enumerate the sub-model size using the preset tuning range and tune them with the FNR value as the target value to obtain the optimal sub-model. Step S5: The three sub-models optimized in step S4 are fused using the fusion constraint to form a trained ensemble model. An alarm threshold is calculated using the validation set in step S3, the optimal sub-model obtained in step S4, and the fusion constraint. When the number of alarms for a user exceeds the threshold, an alarm is issued to confirm that the user has engaged in data theft. Step S6: Retrieve newly generated logs and human resources forms from the database, and retrieve the SQL statements, role information, time information, space information and query results from the newly generated logs. Also retrieve user profile data of the users who performed the operation. Preprocess the newly retrieved data to obtain the preprocessed access and user dataset. Step S7: Split and recombine the access and user datasets preprocessed in Step S6 to construct a query syntax dataset, a query result dataset, and an internal personnel information dataset. Step S8: Input the query syntax dataset, query result dataset, and internal personnel information dataset constructed in Step S7 into the optimal sub-model optimized in Step S4, and input the results of the sub-model into the ensemble model trained in Step S5. Output the final result through the alarm module of the ensemble model to determine whether each user accessing the database has engaged in data theft. Step S1 includes: Step S11: Obtain the logs and human resources forms of the database with n tables. Extract the SQL statements from the logs of the database with n tables and mark the operators in the SQL statements as O. C The table being accessed is characterized as A. T The attribute feature of the access table is marked as A. A Mark f i Indicates whether table i has been accessed, marked m i Let A represent the total number of attributes accessed in table i. T ={f1, f2, ..., f n }, A A ={m1, m2, ..., m n }; Step S12: Extract the user role marker U from the log of the database with n tables obtained in step S11. R The access time is marked as T A Accessing a geographic location marked as S A and U R and S A The process is performed using tag encoding, where T... A The process uses only the hourly time; Step S13: Extract the query results from the log of the database with n tables obtained in step S11, and mark the number of query results as R. N The number of attributes contained in the query results is denoted as R. A The size of the query results is denoted as R. S The distribution characteristics of the query results R T R T p represents the proportion of data retrieved from each table in the query results for that table. i R represents the ratio of the number of rows retrieved from table i to the total number of rows in table i in the query results; T ={p1, p2, ..., p n }; Step S14: Extract the monthly salary of the internal personnel corresponding to each visit from the human resources form obtained in step S11, and mark it as S. M Internal staff performance is marked as P W The length of service for internal staff is marked as Y. W The monthly working hours of internal staff are marked as H. W Whether an internal staff member is promoted within five years is marked as P5, and S M and H W The data is divided into w levels using 1 / w, 2 / w, ..., w-1 / w digits for data processing. Step S3 includes: Step S31: Split the training set and validation set obtained in step S2 according to features; Step S32: Reassemble the dataset split in step S31 to construct a query syntax dataset. The query syntax dataset is constructed by using U R O C A T and A A The dataset D1 is composed of the training set and the validation set. Step S33: Reassemble the dataset split in step S31 to construct the query result dataset. The query result dataset is constructed by using U R O C A T R N R A R S and R T The dataset D2 is composed of the training set and the validation set. Step S34: Reassemble the dataset split in step S31 to construct an internal personnel information dataset. The internal personnel information dataset is constructed by using U R O C A T T A S A S M P W Y W H W The dataset D3 is formed by combining P5 and P5. Dataset D3 includes a training set and a validation set. Step S4 includes: Step S41: Construct three random forest sub-models h respectively. i (x), each h i In (x), the decision tree selection K= Training is performed on several attributes, where D i Includes datasets D1, D2, and D3. Represents D i The total number of attributes in the tree is used to select the best feature for tree splitting based on the Gini coefficient minimization criterion; Step S42: Use the training sets from dataset D1 in step S32, dataset D2 in step S33, and dataset D3 in step S34 to test h respectively. i (x) is trained, and each h... i (x) Using the corresponding FNR value as the objective function value, through D i The validation set for each h i Optimize the decision trees in (x) to find the h-th tree that minimizes the FNR. i The number of decision trees in h1(x) is calculated, where the initial range of the number of decision trees in h2(x) is 5-100, the initial range of the number of decision trees in h3(x) is 5-400, and the initial range of the number of decision trees in h3(x) is 5-100. This yields the optimal sub-model. ,in include , , .

2. The database insider attack detection method based on ensemble learning according to claim 1, characterized in that: Step S5 includes: Step S51: Set the fusion constraint H(x) = Where T0 is a constraint value, let T0=1, and use the validation sets of dataset D1 in step S32, dataset D2 in step S33, and dataset D3 in step S34, respectively, and the optimal sub-model obtained in step S42. , , And the fusion constraint H(x) is used to calculate the TPR value of the ensemble model, where TPR represents the True Positive Rate, also known as sensitivity, which represents the percentage of detected anomalous samples out of all anomalous samples; Step S52: Using the TPR value from step S51, calculate the T value according to the binomial distribution model to obtain the alarm threshold T. That is, when the user's cumulative alarm count exceeds the threshold T, an alarm will be issued to confirm that the user has engaged in data theft.

3. The database insider attack detection method based on ensemble learning according to claim 1, characterized in that: Step S6 includes: Step S61: Retrieve newly generated logs and human resources forms from the database, and extract SQL statements from the newly generated logs in the database, marking the operators in the SQL statements as O. C The table being accessed is characterized as A. T The attribute feature of the access table is marked as A. A Mark f i Indicates whether table i has been accessed, marked m i Let A represent the total number of attributes accessed in table i; then A T ={f1, f2, ..., f n }, A A ={m1, m2, ..., m n }; Step S62: Extract the user role marker U from the newly generated logs obtained from the database in step S61. R The access time is marked as T A Accessing a geographic location marked as S A and U R and S A The process is performed using tag encoding, where T... A The process uses only the hourly time; Step S63: Extract the query results from the newly generated logs obtained from the database in step S61, and mark the number of query results as R. N The number of attributes contained in the query results is denoted as R. A The size of the query results is denoted as R. S The distribution characteristics of the query results R T R T p represents the proportion of data retrieved from each table in the query results for that table. i R represents the ratio of the number of rows retrieved from table i in the query results to the total number of rows in table i. T ={p1, p2, ..., p n }; Step S64: Extract the monthly salary of the internal personnel corresponding to each visit from the human resources form obtained in step S61, and mark it as S. M Internal staff performance is marked as P W The length of service for internal staff is marked as Y. W The monthly working hours of internal staff are marked as H. W Whether an internal staff member is promoted within five years is marked as P5, and S M and H W The data is divided into w levels using digits of 1 / w, 2 / w, ..., 1 / w-w, for data processing.

4. The database insider attack detection method based on ensemble learning according to claim 1, characterized in that: Step S7 includes: Step S71: Split the access and user datasets after preprocessing in step S6; Step S72: Construct a query syntax dataset from the access and user datasets split in step S71. The query syntax dataset is constructed by using U R O C A T and A A The dataset formed by combining them ; Step S73: Construct a query result dataset from the access and user datasets split in step S71. The query result dataset is constructed by using U R O C A T R N R A R S , and R T The dataset formed by combining them ; Step S74: Construct an internal personnel information dataset from the access and user datasets split in step S71. The internal personnel information dataset is constructed by using U R O C A T T A S A S M P W Y W H W The dataset formed by combining it with P5 ; Step S8 includes: Step S81: Use the dataset from step S72 respectively The dataset in step S73 and the dataset in step S74 Put the optimized sub-model from step S42 into the input. , , The detection is performed in the middle, and the detection results of each sub-model are obtained respectively; Step S82: Input the detection results of each sub-model obtained in step S81 into the fusion constraint H(x) set in step S51 to obtain the final detection result; Step S83: If the detection result obtained in step S82 is "abnormal", then check whether the number of alarms from the internal personnel accessing this access exceeds the T value calculated in step S52. If it exceeds, the model issues an alarm and determines that the user has engaged in data theft.

5. A database insider attack detection system based on ensemble learning, used to implement the method of claim 1, characterized in that: include: Data preprocessing module: Used to obtain logs and human resources forms from the database, extract SQL statements, role information, time information, spatial information and query results from the logs, and obtain user profile data of the users who operate. The obtained data are preprocessed to obtain the preprocessed access and user dataset. Dataset partitioning module: used to divide the preprocessed access and user datasets into training and validation sets; Dataset Reconstruction Module: This module is used to split and recombine each dataset based on the pre-defined training and validation sets to construct training and validation sets for the query syntax dataset, query result dataset, and internal personnel information dataset. Sub-model optimization module: Used to train each sub-model separately using the training set, and optimize each trained sub-model using the validation set to obtain the optimal sub-model; Alarm threshold calculation and alarm module: It is used to fuse the three optimized sub-models through the fusion constraint to form a trained ensemble model, and calculate an alarm threshold through the validation set and the optimal sub-model. When the number of alarms for a user exceeds the threshold, an alarm is issued to confirm that the user has engaged in data theft. The new data preprocessing module is used to retrieve newly generated logs and human resources forms from the database, and to retrieve SQL statements, role information, time information, spatial information and query results from the newly generated logs. It also retrieves user profile data of the users who operate the data, and preprocesses the new data to obtain the preprocessed access and user dataset. New Dataset Partitioning Module: Used to split and recombine the preprocessed access and user datasets to construct query syntax datasets, query result datasets, and internal personnel information datasets; The detection and alerting module is used to input the constructed query syntax dataset, query result dataset, and internal personnel information dataset into the optimized sub-model, input the results of the sub-model into the trained ensemble model, and output the final result through the alerting module of the ensemble model to determine whether each user accessing the database has engaged in data theft.

6. A database insider attack detection device based on ensemble learning, characterized in that: include: Memory: A computer program for detecting database insider attacks based on ensemble learning as described in any one of claims 1-4, which is a computer-readable device; Processor: Used to implement the database insider attack detection method based on ensemble learning as described in any one of claims 1-4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the database insider attack detection method based on ensemble learning as described in any one of claims 1-4.

Images