Threat detection model training method, threat detection method, device and medium

Abstract

The embodiment of the invention provides a training method of a threat detection model, a threat detection method, equipment and a medium. The method comprises the steps that multiple pieces of threat information data are acquired, multiple threat information data features are extracted, a training sample set is determined, the training sample set comprises multiple samples, and each sample comprises multiple threat information feature vectors; constructing a plurality of basic classifiers based on the plurality of threat information feature vectors of the training sample set; determining a scale factor of a compensation false alarm rate based on the plurality of basic classifiers, and determining an optimal initial weight of the sample based on the scale factor of the compensation false alarm rate; and constructing a threat detection model based on the optimal initial weight, the training sample set and the plurality of basic classifiers. According to the method, the sample weight is optimized, the model performance is improved, and the model false alarm rate and the calculation complexity are reduced.

Description

Technical Field

[0001] The present application relates to the field of network security, and in particular to a threat detection model training method, threat detection method, device and medium. Background Art

[0002] With the rapid development of the Internet and digital technology, the means of network attacks are also constantly evolving, showing diversified and covert characteristics. The key to protecting network security lies in threat detection. Therefore, technicians have begun to explore the use of machine learning and artificial intelligence technologies to improve the effectiveness of threat detection.

[0003] Ensemble learning is currently an effective technical means to achieve security threat detection. Ensemble learning integrates multiple learners to conduct comprehensive detection of security attack behaviors, and then conducts comprehensive evaluation of threat information through cross-validation, voting and other mechanisms among multiple learners.

[0004] In the prior art, when training a threat detection model, the initial sample weights are usually set to the same value. However, in actual applications, most samples are usually normal. Therefore, the trained threat detection model may have a high false alarm rate, and threat analysis needs to be performed for each alarm, resulting in a waste of a lot of resources and time. Summary of the invention

[0005] The threat detection model training method, threat detection method, device and medium provided in the embodiments of the present application optimize sample weights, improve model performance, and reduce model false alarm rate and computational complexity.

[0006] In a first aspect, the embodiments of the present application provide a threat detection model training method, a threat detection method, a device, and a medium method, including:

[0007] Acquire multiple threat information data, extract multiple threat information data features, and determine a training sample set, where the training sample set includes multiple samples, and each sample includes multiple threat information feature vectors;

[0008] Constructing multiple basic classifiers based on multiple threat information feature vectors of the training sample set;

[0009] Determining a proportional factor for compensating for a false alarm rate based on the multiple basic classifiers, and determining an optimal initial weight for the sample based on the proportional factor for compensating for the false alarm rate;

[0010] A threat detection model is constructed based on the optimal initial weights, the training sample set and the multiple basic classifiers.

[0011] In a possible implementation, each threat information feature vector includes multiple features, and multiple basic classifiers are constructed based on the multiple threat information feature vectors of the training sample set, including:

[0012] Based on any one of the multiple features, construct a basic classifier;

[0013] Based on the basic classifier and the training sample set, a plurality of error classification rates are determined, and the basic classifier is optimized by minimizing the sum of the error classification rates, and the optimized basic classifier is used as the basic classifier corresponding to each feature.

[0014] In a possible implementation, the training sample set includes a plurality of normal samples, a plurality of abnormal samples, and a label corresponding to each feature; the method of determining a proportional factor for compensating a false alarm rate based on the plurality of basic classifiers and determining an optimal initial weight of the sample includes:

[0015] Determine a feature average of multiple features in the multiple normal samples, and determine a discrete value between the multiple features in each normal sample and the feature average;

[0016] Determine a basic classifier set according to the multiple basic classifiers, and determine a false alarm rate and a detection rate based on the basic classifier set, multiple normal samples, multiple abnormal samples and labels corresponding to each feature, wherein the false alarm rate is used to indicate the proportion of normal samples classified as abnormal samples by the basic classifier set, and the detection rate is used to indicate the proportion of all samples correctly classified by the basic classifier set;

[0017] Based on the false alarm rate and the detection rate, a proportional factor compensating for the false alarm rate is determined, and based on the discrete value and the proportional factor compensating for the false alarm rate, an optimal initial weight is determined.

[0018] In a possible implementation manner, determining a proportional factor for compensating the false alarm rate based on the false alarm rate and the detection rate, and determining an optimal initial weight based on the discrete value and the proportional factor for compensating the false alarm rate, includes:

[0019] determining initial weights of the plurality of samples;

[0020] Determine a weighted classification rate according to the false alarm rate and the detection rate, wherein the weighted classification rate is used to weigh the false alarm rate and the detection rate and determine a proportional factor for compensating the false alarm rate;

[0021] Based on the discrete value and the proportional factor compensating the false alarm rate, the initial weight is adjusted, and the adjusted initial weight is used as the optimal initial weight.

[0022] In a possible implementation, the training sample set further includes a label corresponding to each threat information feature vector, where the label is used to indicate whether the corresponding threat information feature vector is normal. The threat detection model is constructed based on the optimal initial weight, the training sample set, and the plurality of basic classifiers, including:

[0023] Taking the optimal initial weight as the sample weight, and determining the sum of weighted classification errors corresponding to each basic classifier according to the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers, wherein the weighted classification error is used to indicate the weighted classification error rate of each sample;

[0024] According to the multiple basic classifiers, a basic classifier with the smallest sum of weighted classification errors is used as a first target classifier, and a classifier weight corresponding to the first target classifier is determined;

[0025] constructing a destination classifier based on the first target classifier and the weight corresponding to the first target classifier;

[0026] Determine whether the target classifier meets the preset requirements;

[0027] If yes, then the target classifier is used as a threat detection model;

[0028] If not, the sample weights are updated, and a second target classifier and a corresponding classifier weight are further determined based on the updated sample weights.

[0029] In a possible implementation, the updating of the sample weights and further determining the second target classifier based on the updated sample weights include:

[0030] Determine whether the sum of weighted classification errors of the first target classifier is less than a preset threshold, where the preset threshold is used to indicate the classification effect of the classifier;

[0031] When the sum of weighted classification errors of the first target classifiers is less than a preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from a plurality of basic classifiers.

[0032] In a second aspect, an embodiment of the present application provides a threat detection method, comprising: obtaining a detection result based on threat information to be detected and a threat detection model, wherein the detection result is used to indicate whether the threat information to be detected is normal, and the threat detection model is obtained by training according to the first aspect and / or various possible implementation methods of the first aspect above.

[0033] In a third aspect, an embodiment of the present application provides a training device for a threat detection model, including:

[0034] An acquisition module is used to acquire a plurality of threat information data, extract a plurality of threat information data features, and determine a training sample set, wherein the training sample set includes a plurality of samples, and each sample includes a plurality of threat information feature vectors;

[0035] A processing module, configured to construct a plurality of basic classifiers based on a plurality of threat information feature vectors of the training sample set;

[0036] The processing module is further configured to determine a proportional factor for compensating for a false alarm rate based on the multiple basic classifiers, and determine an optimal initial weight of the sample based on the proportional factor for compensating for the false alarm rate;

[0037] A construction module is used to construct a threat detection model based on the optimal initial weights, a training sample set and a plurality of basic classifiers.

[0038] In a possible implementation, each threat information feature vector includes multiple features, and the above-mentioned processing module is also used to construct a basic classifier based on any one of the multiple features; based on the basic classifier and the training sample set, multiple error classification rates are determined, and the basic classifier is optimized by minimizing the sum of the error classification rates, and the optimized basic classifier is used as the basic classifier corresponding to each feature.

[0039] In a possible implementation, the training sample set includes multiple normal samples, multiple abnormal samples and a label corresponding to each feature; the processing module is also used to determine the feature average of multiple features in the multiple normal samples, and determine the discrete value between the multiple features in each normal sample and the feature average; a basic classifier set is determined according to the multiple basic classifiers, and a false alarm rate and a detection rate are determined based on the basic classifier set, multiple normal samples, multiple abnormal samples and the label corresponding to each feature, the false alarm rate is used to indicate the proportion of normal samples classified as abnormal samples by the basic classifier set, and the detection rate is used to indicate the proportion of all samples correctly classified by the basic classifier set; based on the false alarm rate and the detection rate, a proportional factor to compensate for the false alarm rate is determined, and based on the discrete value and the proportional factor to compensate for the false alarm rate, an optimal initial weight is determined.

[0040] In a possible implementation, the processing module is also used to determine the initial weights of the multiple samples; determine a weighted classification rate based on the false alarm rate and the detection rate, and the weighted classification rate is used to weigh the false alarm rate and the detection rate, and determine a proportional factor for compensating the false alarm rate; based on the discrete value and the proportional factor for compensating the false alarm rate, adjust the initial weight, and use the adjusted initial weight as the optimal initial weight.

[0041] In a possible implementation, the training sample set also includes a label corresponding to each threat information feature vector, and the label is used to indicate whether the corresponding threat information feature vector is normal. The construction module is also used to use the optimal initial weight as the sample weight, and determine the sum of weighted classification errors corresponding to each basic classifier according to the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers, wherein the weighted classification error is used to indicate the weighted classification error rate of each sample; according to the multiple basic classifiers, the basic classifier with the smallest sum of weighted classification errors is used as the first target classifier, and the classifier weight corresponding to the first target classifier is determined; based on the first target classifier and the weight corresponding to the first target classifier, a target classifier is constructed; it is determined whether the target classifier meets the preset requirements; if so, the target classifier is used as a threat detection model; if not, the sample weight is updated, and the second target classifier and the corresponding classifier weight are further determined based on the updated sample weight.

[0042] In one possible implementation, the construction module is also used to determine whether the sum of the weighted classification errors of the first target classifiers is less than a preset threshold, and the preset threshold is used to indicate the classification effect of the classifier; when the sum of the weighted classification errors of the first target classifiers is less than the preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple basic classifiers.

[0043] In a fourth aspect, an embodiment of the present application provides a training device for a threat detection model, including: a memory, a processor;

[0044] The memory stores computer-executable instructions;

[0045] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the training method of the threat detection model of the first aspect and / or various possible implementation methods of the first aspect or the threat detection method of the second aspect.

[0047] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the threat detection model of the first aspect and / or various possible implementation methods of the first aspect or the threat detection method of the second aspect.

[0048] The training method, threat detection method, device and medium of the threat detection model provided in the embodiments of the present application determine a proportional factor for compensating the false alarm rate through multiple basic classifiers determined based on multiple threat information data, and then determine the optimal sample weight through the proportional factor for compensating the false alarm rate. Based on the optimal sample weight, the training sample set and the multiple basic classifiers, the threat detection model is constructed, the sample weight is optimized, the model performance is improved, the model false alarm rate and the computational complexity are reduced, the relationship between the two classification features and the continuous features is effectively processed, and the needs of threat detection in modern society can be better met. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1 Schematic diagram of the training method of the threat detection model provided in the embodiment of the present application Figure 1 ;

[0051] Figure 2 Schematic diagram of the training method of the threat detection model provided in the embodiment of the present application Figure 2 ;

[0052] Figure 3 Schematic diagram of the training method of the threat detection model provided in the embodiment of the present application Figure 3 ;

[0053] Figure 4 A schematic diagram of the structure of a training device for a threat detection model provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of a training device for a threat detection model provided in an embodiment of the present application.

[0055] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0057] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, products or devices.

[0058] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0059] "Multiple" means two or more, and other quantifiers are similar. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0060] With the rapid development of the Internet and digital technology, the means of network attacks are also constantly evolving, showing diversified and covert characteristics. The key to protecting network security lies in threat detection. Therefore, technicians have begun to explore the use of machine learning and artificial intelligence technologies to improve the effectiveness of threat detection.

[0061] Ensemble learning is currently an effective technical means to achieve security threat detection. Ensemble learning integrates multiple learners to conduct comprehensive detection of security attack behaviors, and then conducts comprehensive evaluation of threat information through cross-validation, voting and other mechanisms among multiple learners.

[0062] The core idea of ​​the AdaBoost algorithm is to iteratively train multiple basic classifiers and combine them into a target classifier. In each round of iteration, the algorithm adjusts the weights of the training samples so that those samples that were misclassified in the previous round receive higher weights in the next round, while the weights of correctly classified samples decrease. In this way, each round of training focuses on those samples that are difficult to classify, gradually improving the accuracy of the model.

[0063] In practical applications, most samples are usually normal samples. Therefore, when training a threat detection model, the false alarm rate of the threat detection model obtained by setting the initial sample weights to the same value is too high, and each alarm requires threat analysis, resulting in a waste of a lot of resources and time. At the same time, normal samples usually follow a certain distribution, but there are great differences between abnormal samples. Therefore, when samples show similar characteristics to normal samples in the training set, it is more important to pay attention to whether these samples are classified normally.

[0064] The training method of the threat detection model provided in the present application solves the technical problem of excessively high false alarm rate in the prior art by adjusting the initial weights and optimizing the weight distribution of normal samples.

[0065] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] Figure 1 Schematic diagram of the training method of the threat detection model provided in the embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes:

[0067] S101, acquiring a plurality of threat information data, extracting a plurality of threat information data features, and determining a training sample set, wherein the training sample set includes a plurality of samples, and each sample includes a plurality of threat information feature vectors;

[0068] The threat information data is information data of potential threats, which is used to indicate various information points that may cause harm to the system or data.

[0069] Obtain multiple threat information data, such as network traffic information, log information, file change information, etc., pre-process the multiple threat information data through operations such as one-hot encoding and principal component analysis (PCA), and extract multiple threat information data features to obtain a training sample set. The classification data in the threat information data is processed through one-hot encoding to improve the processing ability of classification data. The principal component analysis method uses the idea of ​​dimensionality reduction to improve data processing efficiency.

[0070] S102, constructing multiple basic classifiers based on multiple threat information feature vectors of the training sample set;

[0071] Among them, the basic classifier is used to classify the input features into different categories or labels.

[0072] In a possible implementation, in this embodiment, the training sample set includes multiple normal samples and multiple abnormal samples. The error classification rates of the basic classifier for normal samples and error samples are calculated, and the basic classifier is optimized by minimizing the sum of the error classification rates. The specific process is as follows: based on any one of the multiple features, a basic classifier is constructed; based on the basic classifier and the training sample set, multiple error classification rates are determined, and the basic classifier is optimized by minimizing the sum of the error classification rates, and the optimized basic classifier is used as the basic classifier corresponding to each feature.

[0073] Among the multiple threat information data features extracted, the feature types include classification features and continuous features, and according to the feature type, a basic classifier corresponding to the feature is constructed using a corresponding method.

[0074] 1) Classification features:

[0075] The classification feature v can only take a limited number of discrete values. The base classifier divides the range of v into two non-overlapping subsets and . Let X be the threat information feature vector, X v is the characteristic component of X, which corresponds to the feature v.

[0076] The basic classifier is determined using the following formula:

[0077]

[0078] Among them, +1 is used to indicate that the sample is normal; -1 is used to indicate that the sample is abnormal.

[0079] According to the classification results of the basic classifier and the sample training set, the misclassification rates of normal samples and abnormal samples are determined.

[0080] set up and Represent the basic classifiers The misclassification rate of normal samples and abnormal samples. The classification feature is determined by minimizing the sum of the misclassification rates of normal samples and abnormal samples. The corresponding optimized basic classifier The optimal subset of and .

[0081] The following formula is used to optimize the basic classifier to determine the optimized basic classifier:

[0082]

[0083] 2) Continuous features:

[0084] Given a split value .

[0085] The basic classifier is determined using the following formula:

[0086]

[0087] The following formula is used to optimize the basic classifier to determine the optimized basic classifier:

[0088]

[0089] in, is the optimal split value.

[0090] The optimized basic classifier is used as the basic classifier corresponding to each feature.

[0091] S103, determining a proportional factor for compensating for a false alarm rate based on the multiple basic classifiers, and determining an optimal initial weight of a sample based on the proportional factor for compensating for the false alarm rate;

[0092] The proportional factor for compensating the false alarm rate is used to compensate for the false alarm rate.

[0093] According to the classification effects of multiple basic classifiers on the sample training set, the proportional factor for compensating the false alarm rate is determined to compensate for the false alarm rate, and the optimal initial weight of the sample is determined according to the proportional factor for compensating the false alarm rate, thereby improving the algorithm performance and reducing the computational complexity.

[0094] S104: construct a threat detection model based on the optimal initial weight, the training sample set and multiple basic classifiers.

[0095] Based on the optimal initial weights and training sample sets, since multiple basic classifiers are determined based on categorical features and continuous features, classification rules are provided for categorical features and continuous features. Therefore, by training multiple basic classifiers, the threat detection model constructed can effectively handle the relationship between two different types of features without forced conversion.

[0096] In one possible implementation, noise and outliers are common problems when processing training sample sets for threat detection. The following steps are taken to improve the robustness of the AdaBoost algorithm in threat detection: use the training sample set containing noise and outliers to train the initial classifier; remove the samples identified as noise or outliers from the training data set, and retrain the classifier using the training data set after removing noise and outliers. This step reduces the emphasis on samples that may be noise or outliers, helping the model focus more on samples that can represent the normal data distribution, thereby improving its performance in practical applications.

[0097] The training method of the threat detection model provided in the embodiment of the present application determines a proportional factor for compensating the false alarm rate through multiple basic classifiers determined based on multiple threat information data, and then determines the optimal sample weight through the proportional factor for compensating the false alarm rate. Based on the optimal sample weight, the training sample set and the multiple basic classifiers, the threat detection model is constructed, the sample weight is optimized, the model performance is improved, the model false alarm rate and the computational complexity are reduced, the relationship between the classification features and the continuous features is effectively processed, and the needs of threat detection in modern society can be better met.

[0098] The AdaBoost algorithm optimizes the classifier by iteratively adjusting the sample weights so that samples that were misclassified in the previous iteration receive higher weights in the next iteration. In this way, the classifier can pay more attention to samples that are difficult to classify. The setting of the initial weights has a significant impact on the average value of the classification error. Therefore, the setting of the initial weights has a significant impact on the average value of the classification error.

[0099] Figure 2 Schematic diagram of the training method of the threat detection model provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, in this embodiment, the training sample set includes multiple normal samples, multiple abnormal samples and labels corresponding to each feature. Figure 1 Based on the embodiment, determining a proportional factor for compensating the false alarm rate based on the multiple basic classifiers, and determining the optimal initial weight of the sample based on the proportional factor for compensating the false alarm rate are described in detail. The method includes:

[0100] S201, determining a feature average value of multiple features in multiple normal samples, and determining a discrete value between the multiple features in each normal sample and the feature average value;

[0101] Among them, the characteristic mean value is determined by the following formula:

[0102]

[0103] in, is the characteristic mean value; is the number of features in multiple normal samples; For the characteristics; Used to indicate a normal sample.

[0104] The discrete value of each feature relative to the feature mean is defined using the following formula:

[0105]

[0106] in, Used to indicate the dispersion of each feature relative to the feature mean.

[0107] And the following formula is used to normalize the discrete values:

[0108]

[0109] in, is the normalized value of the discrete value.

[0110] S202, determining a basic classifier set according to the multiple basic classifiers, and determining a false alarm rate and a detection rate based on the basic classifier set, multiple normal samples, multiple abnormal samples, and labels corresponding to each feature;

[0111] Among them, the false alarm rate is used to indicate the proportion of normal samples classified as abnormal samples by the basic classifier set, and the detection rate is used to indicate the proportion of all samples correctly classified by the basic classifier set; the label corresponding to each feature is used to indicate whether the feature represents normal behavior.

[0112] Multiple basic classifiers built based on threat information features , determine the basic classifier set , based on the basic classifier set and multiple normal samples, multiple abnormal samples in the training sample set and the label corresponding to each feature, the false alarm rate and the detection rate are determined.

[0113] S203: Determine a proportional factor for compensating the false alarm rate based on the false alarm rate and the detection rate, and determine an optimal initial weight based on the discrete value and the proportional factor for compensating the false alarm rate.

[0114] In one possible implementation, the specific process of determining the optimal initial weight is as follows: determine the initial weights of the multiple samples; determine the weighted classification rate based on the false alarm rate and the detection rate, and determine the proportional factor for compensating the false alarm rate; adjust the initial weight based on the discrete value and the proportional factor for compensating the false alarm rate, and use the adjusted initial weight as the optimal initial weight.

[0115] Among them, the weighted classification rate is used to weigh the false alarm rate and the detection rate.

[0116] Determine the initial weights of multiple samples, where the initial weights

[0117] Satisfy the following formula:

[0118]

[0119] Where n is the number of samples.

[0120] For example, the initial weight of each sample can be set to 1 / n.

[0121] set up and is the number of normal samples and abnormal samples in the training set, and

[0122] are the false alarm rate and misclassification rate of normal samples and abnormal samples respectively. The misclassification rate is used to indicate the proportion of all samples misclassified by the basic classifier set. The sum of the misclassification rate and the detection rate is equal to "1".

[0123] The weighted mean r is determined using the following formula:

[0124]

[0125]

[0126] Among them, the weighted mean r is used to indicate the classification error; It is used to indicate whether the classification result of the basic classifier is consistent with the label of the training sample set.

[0127] The scaling factor to compensate for the false alarm rate is determined using the following formula:

[0128] in, is the scaling factor that compensates for the false alarm rate, The larger it is, the greater the tendency is to reduce the false positive rate during the learning process.

[0129] Use the following formula to adjust the initial weight:

[0130]

[0131] The training method of the threat detection model provided in the embodiment of the present application determines the similar features of normal samples by determining the feature average value, calculates the proportional factor for compensating the false alarm rate by determining the false alarm rate and the detection rate, and then determines the optimal initial weight through the proportional factor for compensating the false alarm rate and the discrete value, so that the weight distribution of normal samples that are closer to the mean of the overall distribution is higher, the weight distribution of normal samples is optimized, the complexity of model training is reduced, the false alarm rate of the model is reduced, and the detection accuracy is improved.

[0132] Figure 3 Schematic diagram of the training method of the threat detection model provided in the embodiment of the present application Figure 3 ,like Figure 3 As shown, in this embodiment, the training sample set also includes a label corresponding to each threat information feature vector, and the label is used to indicate whether the corresponding threat information feature vector is normal; in this embodiment, Figure 1 or Figure 2 Based on the embodiment, a threat detection model is constructed based on the optimal initial weight, the training sample set and multiple basic classifiers. The method includes:

[0133] S301, taking the optimal initial weight as the sample weight, and determining the sum of the weighted classification errors corresponding to each basic classifier according to the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers;

[0134] Among them, the weighted classification error is used to indicate the weighted classification error rate of each sample.

[0135] The sum of the weighted classification errors corresponding to each base classifier is determined using the following formula:

[0136]

[0137] in,

[0138] in, is the weighted classification error, is the sample weight.

[0139] S302: According to the multiple basic classifiers, a basic classifier with the smallest sum of weighted classification errors is used as a first target classifier, and a classifier weight corresponding to the first target classifier is determined;

[0140] From the constructed base classifiers, select the base classifier that minimizes the sum of weighted classification errors , the basic classifier as the first target classifier.

[0141]

[0142] The classifier weight is calculated using the following formula:

[0143]

[0144] in, is the classifier weight; is the sum of the weighted classification errors of the determined target classifiers.

[0145] S303: construct a destination classifier based on the first target classifier and the weight corresponding to the first target classifier;

[0146] The target classifier is constructed using the following formula:

[0147]

[0148]

[0149] in, For the purpose classifier, is a normalized vector.

[0150] S304, determining whether the target classifier meets the preset requirements; if so, executing step S305, using the target classifier as a threat detection model; if not, executing step S306, updating the sample weights, and further determining a second target classifier and a corresponding classifier weight based on the updated sample weights;

[0151] The preset requirement is used to indicate that the classification effect of the target classifier meets the threat detection requirement.

[0152] Determine whether the classification effect of the target classifier meets the threat detection requirements. If so, use the target classifier as a threat detection model; if not, update the sample weights.

[0153] The sample weights are updated using the following formula:

[0154]

[0155] The second target classifier and the corresponding classifier weight are further determined based on the updated sample weight, and then the destination classifier is constructed according to the first target classifier and the corresponding classifier weight, the second target classifier and the corresponding classifier weight.

[0156] S305, using the target classifier as a threat detection model;

[0157] S306: Update the sample weights, and further determine a second target classifier and a corresponding classifier weight based on the updated sample weights.

[0158] In one possible implementation, a threshold can be set to prevent the constructed threat detection model from overfitting. The specific process is as follows: determine whether the sum of the weighted classification errors of the first target classifiers is less than a preset threshold. If the sum of the weighted classification errors of the first target classifiers is less than the preset threshold, determine a second target classifier whose sum of weighted classification errors is not less than the preset threshold from the multiple basic classifiers after optimization processing.

[0159] The preset threshold is used to indicate the classification effect of the classifier.

[0160] In the first iteration, if any of the sum of weighted errors of multiple base classifiers is less than the preset threshold, it indicates that the base classifier has overfitted the sample, that is, it may be too complex, so that it performs very well on the training data, but may perform poorly on new, unseen data. Since the base classifier that minimizes the sum of weighted classification errors is selected as the first target classifier when selecting the first target classifier, when any of the sum of weighted errors is less than the preset threshold, it indicates that the determined first target classifier has overfitted the sample. Therefore, in subsequent iterations, the second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple base classifiers, that is, the overfitting correction process is no longer performed. The overall classification performance is improved.

[0161] The training method of the threat detection model provided in the embodiment of the present application calculates the sum of the weighted classification errors of each optimized base classifier, compares the optimized base classifiers, and uses the base classifier with the smallest sum of weighted errors as the first target classifier, and calculates the weight of the first target classifier, and then constructs a destination classifier based on the first target classifier and its weight, and based on the destination classifier, continuously updates the sample weights to determine the second classifier until the constructed destination classifier meets the preset requirements, thereby reducing the computational complexity, improving the computational efficiency of the threat detection model, and improving the efficiency and accuracy of threat detection.

[0162] The threat detection method provided in the embodiment of the present application obtains a detection result based on the threat information to be detected and the threat detection model.

[0163] The detection result is used to indicate whether the threat information to be detected is normal, and the threat detection model is trained by the above-mentioned model training method.

[0164] The embodiment of the present application obtains a threat detection result by extracting the features of the threat information to be detected and inputting the features of the threat information to be detected into the threat detection model trained by the above-mentioned model training method. The result provides a threat analysis reference for the staff, improves the threat detection efficiency, and saves a lot of resources and time.

[0165] Figure 4 A structural diagram of a training device for a threat detection model provided in an embodiment of the present application, such as Figure 4 As shown, the training device 40 of the threat detection model provided in this embodiment includes:

[0166] The acquisition module 401 is used to acquire multiple threat information data, extract multiple threat information data features, and determine a training sample set, where the training sample set includes multiple samples, and each sample includes multiple threat information feature vectors;

[0167] A processing module 402 is used to construct multiple basic classifiers based on multiple threat information feature vectors of the training sample set;

[0168] The processing module 402 is further used to determine a proportional factor for compensating for the false alarm rate based on the multiple basic classifiers, and determine the optimal initial weight of the sample based on the proportional factor for compensating for the false alarm rate;

[0169] The construction module 403 is used to construct a threat detection model based on the optimal initial weight, the training sample set and multiple basic classifiers.

[0170] In a possible implementation, each threat information feature vector includes multiple features, and the above-mentioned processing module 402 is also used to construct a basic classifier based on any one of the multiple features; based on the basic classifier and the training sample set, multiple error classification rates are determined, and the basic classifier is optimized by minimizing the sum of the error classification rates, and the optimized basic classifier is used as the basic classifier corresponding to each feature.

[0171] In a possible implementation, the training sample set includes multiple normal samples, multiple abnormal samples, and a label corresponding to each feature; the processing module 402 is also used to determine the feature average of multiple features in the multiple normal samples, and determine the discrete value between the multiple features in each normal sample and the feature average; determine a basic classifier set based on multiple basic classifiers, and determine the false alarm rate and the detection rate based on the basic classifier set, multiple normal samples, multiple abnormal samples, and the label corresponding to each feature, the false alarm rate is used to indicate the proportion of normal samples classified as abnormal samples by the basic classifier set, and the detection rate is used to indicate the proportion of all samples correctly classified by the basic classifier set; based on the false alarm rate and the detection rate, determine the proportional factor to compensate for the false alarm rate, and determine the optimal initial weight based on the discrete value and the proportional factor to compensate for the false alarm rate.

[0172] In a possible implementation, the processing module 402 is also used to determine the initial weights of multiple samples; determine the weighted classification rate based on the false alarm rate and the detection rate, and the weighted classification rate is used to weigh the false alarm rate and the detection rate, and determine the proportional factor that compensates for the false alarm rate; adjust the initial weight based on the discrete value and the proportional factor that compensates for the false alarm rate, and use the adjusted initial weight as the optimal initial weight.

[0173] In a possible implementation, the training sample set also includes a label corresponding to each threat information feature vector, and the label is used to indicate whether the corresponding threat information feature vector is normal. The construction module 403 is also used to use the optimal initial weight as the sample weight, and determine the sum of the weighted classification errors corresponding to each basic classifier according to the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers, wherein the weighted classification error is used to indicate the weighted classification error rate of each sample; according to multiple basic classifiers, the basic classifier with the smallest sum of weighted classification errors is used as the first target classifier, and the classifier weight corresponding to the first target classifier is determined; based on the first target classifier and the weight corresponding to the first target classifier, a target classifier is constructed; it is determined whether the target classifier meets the preset requirements; if so, the target classifier is used as a threat detection model; if not, the sample weight is updated, and the second target classifier and the corresponding classifier weight are further determined based on the updated sample weight.

[0174] In one possible implementation, the construction module 403 is also used to determine whether the sum of the weighted classification errors of the first target classifier is less than a preset threshold, and the preset threshold is used to indicate the classification effect of the classifier; when the sum of the weighted classification errors of the first target classifier is less than the preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple basic classifiers.

[0175] The training device for the threat detection model provided in this embodiment can execute the method provided in the above-mentioned training method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0176] Figure 5 A schematic diagram of the structure of a training device for a threat detection model provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0177] In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above-mentioned threat detection model training method or threat detection method.

[0178] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0179] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0180] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0181] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0182] The present application also provides a computer program product, including a computer program, which implements the above-mentioned threat detection model training method or threat detection method when executed by a processor.

[0183] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned threat detection model training method or threat detection method is implemented.

[0184] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0185] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0186] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0189] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0190] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0191] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for training a threat detection model, characterized in that: include: Acquire multiple threat information data, extract multiple threat information data features, and determine a training sample set, where the training sample set includes multiple samples, and each sample includes multiple threat information feature vectors; Constructing multiple basic classifiers based on multiple threat information feature vectors of the training sample set; Determining a proportional factor for compensating for a false alarm rate based on the multiple basic classifiers, and determining an optimal initial weight for the sample based on the proportional factor for compensating for the false alarm rate; A threat detection model is constructed based on the optimal initial weights, the training sample set and the multiple basic classifiers.

2. The method according to claim 1, characterized in that Each threat information feature vector includes multiple features, and the multiple threat information feature vectors based on the training sample set are used to construct multiple basic classifiers, including: Based on any one of the multiple features, construct a basic classifier; Based on the basic classifier and the training sample set, a plurality of error classification rates are determined, and the basic classifier is optimized by minimizing the sum of the error classification rates, and the optimized basic classifier is used as the basic classifier corresponding to each feature.

3. The method according to claim 2, characterized in that The training sample set includes a plurality of normal samples, a plurality of abnormal samples and a label corresponding to each feature; The step of determining a proportional factor for compensating a false alarm rate based on the multiple basic classifiers and determining an optimal initial weight of the sample comprises: Determine a feature average of multiple features in the multiple normal samples, and determine a discrete value between the multiple features in each normal sample and the feature average; Determine a basic classifier set according to the multiple basic classifiers, and determine a false alarm rate and a detection rate based on the basic classifier set, multiple normal samples, multiple abnormal samples and labels corresponding to each feature, wherein the false alarm rate is used to indicate the proportion of normal samples classified as abnormal samples by the basic classifier set, and the detection rate is used to indicate the proportion of all samples correctly classified by the basic classifier set; Based on the false alarm rate and the detection rate, a proportional factor compensating for the false alarm rate is determined, and based on the discrete value and the proportional factor compensating for the false alarm rate, an optimal initial weight is determined.

4. The method according to claim 3, characterized in that The determining, based on the false alarm rate and the detection rate, a proportional factor for compensating the false alarm rate, and determining, based on the discrete value and the proportional factor for compensating the false alarm rate, an optimal initial weight, comprises: determining initial weights of the plurality of samples; Determine a weighted classification rate according to the false alarm rate and the detection rate, wherein the weighted classification rate is used to weigh the false alarm rate and the detection rate and determine a proportional factor for compensating the false alarm rate; Based on the discrete value and the proportional factor compensating the false alarm rate, the initial weight is adjusted, and the adjusted initial weight is used as the optimal initial weight.

5. The method according to claim 2, characterized in that: The training sample set includes a label corresponding to each threat information feature vector, and the label is used to indicate whether the corresponding threat information feature vector is normal. The threat detection model is constructed based on the optimal initial weight, the training sample set and the multiple basic classifiers, including: Taking the optimal initial weight as the sample weight, and determining the sum of weighted classification errors corresponding to each basic classifier according to the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers, wherein the weighted classification error is used to indicate the weighted classification error rate of each sample; According to the multiple basic classifiers, a basic classifier with the smallest sum of weighted classification errors is used as a first target classifier, and a classifier weight corresponding to the first target classifier is determined; constructing a destination classifier based on the first target classifier and the weight corresponding to the first target classifier; Determine whether the target classifier meets the preset requirements; If yes, then the target classifier is used as a threat detection model; If not, the sample weights are updated, and a second target classifier and a corresponding classifier weight are further determined based on the updated sample weights.

6. The method according to claim 5, characterized in that The updating of the sample weights and further determining a second target classifier based on the updated sample weights include: Determine whether the sum of weighted classification errors of the first target classifier is less than a preset threshold, where the preset threshold is used to indicate the classification effect of the classifier; When the sum of weighted classification errors of the first target classifiers is less than a preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from a plurality of basic classifiers.

7. A threat detection method, characterized in that: include: A detection result is obtained according to the threat information to be detected and a threat detection model, wherein the detection result is used to indicate whether the threat information to be detected is normal, and the threat detection model is obtained by training through the method according to any one of claims 1 to 6.

8. A training device for a threat detection model, characterized in that: include: An acquisition module is used to acquire a plurality of threat information data, extract a plurality of threat information data features, and determine a training sample set, wherein the training sample set includes a plurality of samples, and each sample includes a plurality of threat information feature vectors; A processing module, configured to construct a plurality of basic classifiers based on a plurality of threat information feature vectors of the training sample set; The processing module is further used to determine a proportional factor for compensating for a false alarm rate based on the multiple basic classifiers, and determine an optimal initial weight of the sample based on the proportional factor for compensating for the false alarm rate; A construction module is used to construct a threat detection model based on the optimal initial weights, a training sample set and a plurality of basic classifiers.

9. A training device for a threat detection model, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the training method of the threat detection model as described in any one of claims 1 to 6 or the threat detection method as described in claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the training method of the threat detection model as described in any one of claims 1 to 6 or the threat detection method as described in claim 7.

Images