Threat detection model training method, threat detection method, device, and medium

Abstract

Embodiments of the present application provide a threat detection model training method, a threat detection method, equipment and a medium. The method comprises: obtaining a plurality of threat information data, extracting a plurality of threat information data features, determining a training sample set, the training sample set comprising a plurality of samples, each sample comprising a plurality of threat information feature vectors; based on the plurality of threat information feature vectors of the training sample set, constructing a plurality of basic classifiers; based on the plurality of basic classifiers, determining a compensation false positive rate scaling factor, and based on the compensation false positive rate scaling factor, determining an optimal initial weight of the sample; based on the optimal initial weight, the training sample set and the plurality of basic classifiers, constructing a threat detection model. The method optimizes the sample weight, improves the model performance, and reduces the model false positive rate and the computational complexity.

Description

Technical Field

[0001] This application relates to the field of cybersecurity, and in particular to a method for training a threat detection model, a threat detection method, an apparatus, and a medium. Background Technology

[0002] With the rapid development of the Internet and digital technologies, cyberattack methods are also constantly evolving, exhibiting diversified and covert characteristics. The key to safeguarding cybersecurity lies in threat detection. Therefore, technicians have begun to explore the use of machine learning and artificial intelligence technologies to improve the effectiveness of threat detection.

[0003] Ensemble learning is currently an effective technical means to detect security threats. Ensemble learning integrates multiple learners to comprehensively detect security attack behaviors, and then uses mechanisms such as cross-validation and voting among multiple learners to comprehensively evaluate threat information.

[0004] In existing technologies, when training a threat detection model, the initial sample weights are usually set to the same value. However, in practical applications, most samples are usually normal, so the trained threat detection model may have an excessively high false alarm rate. Each alarm requires threat analysis, resulting in a waste of a lot of resources and time. Summary of the Invention

[0005] The threat detection model training method, threat detection method, device, and medium provided in this application have optimized sample weights, improved model performance, and reduced false alarm rate and computational complexity.

[0006] In a first aspect, embodiments of this application provide a method for training a threat detection model, a threat detection method, an apparatus, and a medium method, comprising:

[0007] Acquire multiple threat information data and extract features from multiple threat information data to determine a training sample set, wherein the training sample set includes multiple samples, and each sample includes multiple threat information feature vectors;

[0008] Based on the multiple threat information feature vectors of the training sample set, multiple basic classifiers are constructed.

[0009] Based on the multiple base classifiers, a scaling factor for compensating for false positives is determined, and based on the scaling factor for compensating for false positives, the optimal initial weights of the samples are determined.

[0010] Based on the optimal initial weights, training sample set, and multiple basic classifiers, a threat detection model is constructed.

[0011] In one possible implementation, each threat information feature vector includes multiple features, and multiple base classifiers are constructed based on the multiple threat information feature vectors of the training sample set, including:

[0012] A basic classifier is constructed based on any one of the multiple features;

[0013] Based on the base classifier and the training sample set, multiple misclassification rates are determined, and the base classifier is optimized by minimizing the sum of the misclassification rates. The optimized base classifier is then used as the base classifier for each feature.

[0014] In one possible implementation, the training sample set includes multiple normal samples, multiple abnormal samples, and a label corresponding to each feature; the step of determining a scaling factor to compensate for false positive rates based on the multiple base classifiers, and determining the optimal initial weights for the samples, includes:

[0015] Determine the average value of multiple features in the plurality of normal samples, and determine the discrete value between the multiple features in each normal sample and the average value of the features;

[0016] Based on the multiple base classifiers, a set of base classifiers is determined, and based on the set of base classifiers, multiple normal samples, multiple abnormal samples, and the label corresponding to each feature, the false alarm rate and the detection rate are determined. The false alarm rate is used to indicate the proportion of normal samples that the set of base classifiers classifies as abnormal samples, and the detection rate is used to indicate the proportion of all samples that the set of base classifiers correctly classifies.

[0017] Based on the false alarm rate and detection rate, a scaling factor for compensating the false alarm rate is determined, and based on the discrete values ​​and the scaling factor for compensating the false alarm rate, the optimal initial weights are determined.

[0018] In one possible implementation, the above-mentioned determination of a scaling factor to compensate for the false alarm rate based on the false alarm rate and the detection rate, and the determination of an optimal initial weight based on the discrete value and the scaling factor to compensate for the false alarm rate, includes:

[0019] Determine the initial weights of the plurality of samples;

[0020] Based on the false alarm rate and the detection rate, a weighted classification rate is determined. The weighted classification rate is used to balance the false alarm rate and the detection rate and to determine the proportional factor for compensating the false alarm rate.

[0021] Based on the discrete values ​​and the scaling factor for compensating for false alarms, the initial weights are adjusted, and the adjusted initial weights are taken as the optimal initial weights.

[0022] In one possible implementation, the training sample set further includes a label corresponding to each threat information feature vector, the label indicating whether the corresponding threat information feature vector is normal. The step of constructing a threat detection model based on the optimal initial weights, the training sample set, and multiple base classifiers includes:

[0023] The optimal initial weights are used as sample weights. Based on the sample weights, the labels corresponding to each threat information feature vector, and multiple base classifiers, the sum of weighted classification errors corresponding to each base classifier is determined. The weighted classification error is used to indicate the classification error rate of each sample after weighting.

[0024] Based on the multiple base classifiers, the base classifier with the smallest sum of weighted classification errors is selected as the first target classifier, and the classifier weights corresponding to the first target classifier are determined.

[0025] Based on the first target classifier and the weights corresponding to the first target classifier, a target classifier is constructed.

[0026] Determine whether the target classifier meets the preset requirements;

[0027] If so, the target classifier will be used as the threat detection model;

[0028] If not, the sample weights are updated, and the second target classifier and its corresponding classifier weights are further determined based on the updated sample weights.

[0029] In one possible implementation, the above-described update process for the sample weights, followed by further determining a second target classifier based on the updated sample weights, includes:

[0030] Determine whether the sum of the weighted classification errors of the first target classifier is less than a preset threshold, wherein the preset threshold is used to indicate the classification effect of the classifier;

[0031] If the sum of the weighted classification errors of the first target classifier is less than a preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple base classifiers.

[0032] Secondly, embodiments of this application provide a threat detection method, including: obtaining a detection result based on threat information to be detected and a threat detection model, wherein the detection result is used to indicate whether the threat information to be detected is normal, and the threat detection model is trained through the first aspect and / or various possible implementations of the first aspect.

[0033] Thirdly, embodiments of this application provide a training apparatus for a threat detection model, comprising:

[0034] The acquisition module is used to acquire multiple threat information data, extract features from the multiple threat information data, and determine a training sample set. The training sample set includes multiple samples, and each sample includes multiple threat information feature vectors.

[0035] The processing module is used to construct multiple basic classifiers based on multiple threat information feature vectors of the training sample set;

[0036] The aforementioned processing module is also used to determine a scaling factor for compensating for false positives based on the multiple base classifiers, and to determine the optimal initial weights of the samples based on the scaling factor for compensating for false positives.

[0037] The module is used to build a threat detection model based on the optimal initial weights, training sample set, and multiple base classifiers.

[0038] In one possible implementation, each threat information feature vector includes multiple features. The aforementioned processing module is further configured to construct a base classifier based on any one of the multiple features; determine multiple misclassification rates based on the base classifier and the training sample set; optimize the base classifier by minimizing the sum of the misclassification rates; and use the optimized base classifier as the base classifier corresponding to each feature.

[0039] In one possible implementation, the training sample set includes multiple normal samples, multiple abnormal samples, and a label corresponding to each feature; the processing module is further configured to determine the feature average of multiple features in the multiple normal samples, and determine the discrete value between the multiple features in each normal sample and the feature average; determine a set of basic classifiers based on the multiple basic classifiers, and determine the false positive rate and detection rate based on the set of basic classifiers, the multiple normal samples, the multiple abnormal samples, and the label corresponding to each feature, wherein the false positive rate indicates the proportion of normal samples classified as abnormal samples by the set of basic classifiers, and the detection rate indicates the proportion of all samples correctly classified by the set of basic classifiers; determine a scaling factor to compensate for the false positive rate based on the false positive rate and the detection rate, and determine the optimal initial weights based on the discrete value and the scaling factor to compensate for the false positive rate.

[0040] In one possible implementation, the above processing module is further configured to determine the initial weights of the plurality of samples; determine a weighted classification rate based on the false alarm rate and the detection rate, wherein the weighted classification rate is used to balance the false alarm rate and the detection rate, and determine a proportional factor to compensate for the false alarm rate; adjust the initial weights based on the discrete values ​​and the proportional factor to compensate for the false alarm rate, and use the adjusted initial weights as the optimal initial weights.

[0041] In one possible implementation, the training sample set further includes a label corresponding to each threat information feature vector. The label is used to indicate whether the corresponding threat information feature vector is normal. The construction module is further used to use the optimal initial weight as the sample weight, and to determine the sum of weighted classification errors corresponding to each basic classifier based on the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers. The weighted classification error is used to indicate the classification error rate of each sample after weighting. Based on the multiple basic classifiers, the basic classifier with the smallest sum of weighted classification errors is selected as the first target classifier, and the classifier weight corresponding to the first target classifier is determined. Based on the first target classifier and the weight corresponding to the first target classifier, a target classifier is constructed. It is determined whether the target classifier meets the preset requirements. If yes, the target classifier is used as the threat detection model. If not, the sample weights are updated, and a second target classifier and its corresponding classifier weights are further determined based on the updated sample weights.

[0042] In one possible implementation, the construction module is further configured to determine whether the sum of the weighted classification errors of the first target classifier is less than a preset threshold, the preset threshold being used to indicate the classification effect of the classifier; if the sum of the weighted classification errors of the first target classifier is less than the preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple base classifiers.

[0043] Fourthly, embodiments of this application provide a training device for a threat detection model, including: a memory and a processor;

[0044] The memory stores computer-executed instructions;

[0045] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0046] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement a training method for a threat detection model of the first aspect and / or various possible implementations of the first aspect, or a threat detection method of the second aspect.

[0047] Sixthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements a method for training a threat detection model as described in the first aspect and / or various possible implementations of the first aspect, or a threat detection method as described in the second aspect.

[0048] The threat detection model training method, threat detection method, device, and medium provided in this application embodiment determine a proportional factor to compensate for the false alarm rate by using multiple basic classifiers determined based on multiple threat information data. Then, the optimal sample weights are determined by using the proportional factor to compensate for the false alarm rate. Based on the optimal sample weights, the training sample set, and multiple basic classifiers, a threat detection model is constructed. This optimizes the sample weights, improves model performance, reduces the model's false alarm rate and computational complexity, and effectively handles the relationship between two types of classification features and continuous features, thus better meeting the needs of threat detection in modern society. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0050] Figure 1 A flowchart illustrating the training method for the threat detection model provided in this application embodiment. Figure 1 ;

[0051] Figure 2 A flowchart illustrating the training method for the threat detection model provided in this application embodiment. Figure 2 ;

[0052] Figure 3 A flowchart illustrating the training method for the threat detection model provided in this application embodiment. Figure 3 ;

[0053] Figure 4 A schematic diagram of the structure of the training device for the threat detection model provided in the embodiments of this application;

[0054] Figure 5 A schematic diagram of the structure of the training device for the threat detection model provided in the embodiments of this application.

[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0057] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, products, or apparatus.

[0058] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0059] "Multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0060] With the rapid development of the Internet and digital technologies, cyberattack methods are also constantly evolving, exhibiting diversified and covert characteristics. The key to safeguarding cybersecurity lies in threat detection. Therefore, technicians have begun to explore the use of machine learning and artificial intelligence technologies to improve the effectiveness of threat detection.

[0061] Ensemble learning is currently an effective technical means to detect security threats. Ensemble learning integrates multiple learners to comprehensively detect security attack behaviors, and then uses mechanisms such as cross-validation and voting among multiple learners to comprehensively evaluate threat information.

[0062] The core idea of ​​the AdaBoost algorithm is to iteratively train multiple base classifiers and combine them into a single target classifier. In each iteration, the algorithm adjusts the weights of the training samples, giving higher weights to samples misclassified in the previous round and lower weights to correctly classified samples. In this way, each training round focuses on the difficult-to-classify samples, gradually improving the model's accuracy.

[0063] In practical applications, most samples are typically normal. Therefore, when training threat detection models, setting the initial sample weights to the same value often results in a high false alarm rate. Each alert requires threat analysis, leading to a waste of resources and time. Furthermore, while normal samples usually follow a certain distribution, abnormal samples exhibit significant differences. Therefore, when samples display characteristics similar to normal samples in the training set, it is crucial to pay close attention to whether these samples have been correctly classified.

[0064] The training method for the threat detection model provided in this application solves the technical problem of excessively high false alarm rate in the prior art by adjusting the initial weights and optimizing the weight allocation of normal samples.

[0065] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0066] Figure 1 A flowchart illustrating the training method for the threat detection model provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes:

[0067] S101. Acquire multiple threat information data and extract features from multiple threat information data to determine a training sample set, wherein the training sample set includes multiple samples, and each sample includes multiple threat information feature vectors.

[0068] Among them, threat information data refers to information data on potential threats, which is used to indicate various information points that may cause harm to the system or data.

[0069] Multiple threat information data sets are acquired, such as network traffic information, log information, and file change information. These data sets are preprocessed using one-hot encoding and Principal Component Analysis (PCA), and features are extracted to obtain a training sample set. One-hot encoding improves the processing capability for categorical data, while PCA, utilizing dimensionality reduction, enhances data processing efficiency.

[0070] S102. Based on the multiple threat information feature vectors of the training sample set, construct multiple basic classifiers;

[0071] The base classifier is used to classify the input features into different categories or labels.

[0072] In one possible implementation, in this embodiment, the training sample set includes multiple normal samples and multiple abnormal samples. The base classifier is optimized by calculating the misclassification rate of the base classifier for normal samples and incorrect samples, and by minimizing the sum of the misclassification rates. The specific process is as follows: a base classifier is constructed based on any one of the multiple features; multiple misclassification rates are determined based on the base classifier and the training sample set, and the base classifier is optimized by minimizing the sum of the misclassification rates. The optimized base classifier is then used as the base classifier for each feature.

[0073] Among the extracted threat information data features, the feature types include categorical features and continuous features. Based on the feature type, the corresponding method is used to construct the basic classifier for that feature.

[0074] 1) Classification characteristics:

[0075] The categorical feature v can only take finite discrete values. The base classifier divides the range of v into two non-overlapping subsets. and Let X be the threat information feature vector, X v Let X be a feature component, which corresponds to feature v.

[0076] The base classifier is determined using the following formula:

[0077]

[0078] +1 indicates that the sample is normal; -1 indicates that the sample is abnormal.

[0079] Based on the classification results of the base classifier and the sample training set, determine the misclassification rates of normal and abnormal samples.

[0080] set up and These represent the basic classifiers. The misclassification rate for normal and abnormal samples. This classification feature is determined by minimizing the sum of the misclassification rates for normal and abnormal samples. The corresponding optimized base classifier optimal subset and .

[0081] The base classifier is optimized using the following formula to determine the optimized base classifier:

[0082]

[0083] 2) Continuous characteristics:

[0084] Given a segment value .

[0085] The base classifier is determined using the following formula:

[0086]

[0087] The base classifier is optimized using the following formula to determine the optimized base classifier:

[0088]

[0089] in, This is the optimal segmentation value.

[0090] The optimized base classifier is used as the base classifier for each feature.

[0091] S103. Based on the multiple base classifiers, determine the scaling factor for compensating the false alarm rate, and based on the scaling factor for compensating the false alarm rate, determine the optimal initial weight of the sample.

[0092] The proportional factor for compensating for false alarms is used to compensate for the false alarm rate.

[0093] Based on the classification performance of multiple base classifiers on the training set, a scaling factor for compensating the false alarm rate is determined to compensate for the false alarm rate. Based on this scaling factor, the optimal initial weights of the samples are determined, thereby improving algorithm performance and reducing computational complexity.

[0094] S104. Based on the optimal initial weights, the training sample set, and multiple basic classifiers, construct a threat detection model.

[0095] Based on the optimal initial weights and training sample set, and since multiple base classifiers are determined based on categorical and continuous features, providing classification rules for categorical and continuous features, the threat detection model constructed by training multiple base classifiers can effectively handle the relationship between two different types of features without forced conversion.

[0096] In one possible implementation, the presence of noise and outliers is a common problem when processing training sample sets for threat detection. The following steps improve the robustness of the AdaBoost algorithm in threat detection: an initial classifier is trained using a training sample set containing noise and outliers; samples identified as noise or outliers are removed from the training dataset, and the classifier is retrained using the noise- and outlier-removed training dataset. This step reduces the emphasis on samples that may be noise or outliers, helping the model focus more on samples that represent the normal data distribution, thereby improving its performance in practical applications.

[0097] The threat detection model training method provided in this application determines a proportional factor to compensate for the false alarm rate by using multiple basic classifiers based on multiple threat information data. Then, it determines the optimal sample weights using the proportional factor to compensate for the false alarm rate. Based on the optimal sample weights, the training sample set, and the multiple basic classifiers, a threat detection model is constructed. This method optimizes the sample weights, improves model performance, reduces the false alarm rate and computational complexity, and effectively handles the relationship between categorical features and continuous features, thus better meeting the needs of threat detection in modern society.

[0098] The AdaBoost algorithm optimizes the classifier by iteratively adjusting sample weights, giving higher weights to samples misclassified in previous iterations. This allows the classifier to focus more on samples that are difficult to classify. The initial weight settings have a significant impact on the average classification error.

[0099] Figure 2 A flowchart illustrating the training method for the threat detection model provided in this application embodiment. Figure 2 ,like Figure 2 As shown, in this embodiment, the training sample set includes multiple normal samples, multiple abnormal samples, and a label corresponding to each feature. This embodiment... Figure 1 Based on the embodiments, a detailed explanation is provided on determining the scaling factor for compensating for false positive rates based on the multiple base classifiers, and determining the optimal initial weights for samples based on the scaling factor for compensating for false positive rates. This method includes:

[0100] S201. Determine the average value of multiple features in multiple normal samples, and determine the discrete value between the multiple features in each normal sample and the average value of the features;

[0101] The characteristic average value is determined using the following formula:

[0102]

[0103] in, The characteristic average; The number of features in multiple normal samples; As a characteristic; Used to indicate normal samples.

[0104] The discrete value of each feature relative to the feature mean is defined using the following formula:

[0105]

[0106] in, Used to indicate the discrete value of each feature relative to the feature mean.

[0107] The discrete values ​​are then normalized using the following formula:

[0108]

[0109] in, It is the normalized value of the discrete value.

[0110] S202. Based on the multiple basic classifiers, determine the basic classifier set, and based on the basic classifier set, multiple normal samples, multiple abnormal samples, and the label corresponding to each feature, determine the false alarm rate and detection rate.

[0111] The false positive rate indicates the proportion of normal samples that the base classifier set classifies as abnormal samples, the detection rate indicates the proportion of all samples that the base classifier set classifies correctly, and the label corresponding to each feature indicates whether the feature represents normal behavior.

[0112] Based on multiple basic classifiers constructed using threat information features Determine the set of basic classifiers Based on the basic classifier set and multiple normal samples, multiple abnormal samples, and the label corresponding to each feature in the training sample set, the false alarm rate and detection rate are determined.

[0113] S203. Based on the false alarm rate and detection rate, determine the scaling factor for compensating the false alarm rate, and based on the discrete value and the scaling factor for compensating the false alarm rate, determine the optimal initial weight.

[0114] In one possible implementation, the specific process for determining the optimal initial weights is as follows: determining the initial weights of the plurality of samples; determining the weighted classification rate and the compensation factor for the false alarm rate based on the false alarm rate and the detection rate; adjusting the initial weights based on the discrete values ​​and the compensation factor for the false alarm rate, and using the adjusted initial weights as the optimal initial weights.

[0115] The weighted classification rate is used to balance the false alarm rate and the detection rate.

[0116] Determine the initial weights for multiple samples, where the initial weights are...

[0117] Satisfy the following formula:

[0118]

[0119] Where n is the number of samples.

[0120] For example, the initial weight of each sample can be set to 1 / n.

[0121] set up and To determine the number of normal and abnormal samples in the training set, let... and

[0122] These are the false alarm rate and misclassification rate for normal samples and abnormal samples, respectively. The misclassification rate indicates the proportion of all samples that are misclassified by the base classifier set. The sum of the misclassification rate and the detection rate equals "1".

[0123] The weighted average r is determined using the following formula:

[0124]

[0125]

[0126] The weighted average value r is used to indicate the classification error. Used to indicate whether the classification result of the base classifier is consistent with the labels of the training sample set.

[0127] The scaling factor for compensating for false alarms is determined using the following formula:

[0128]

[0129] in, It is a proportional factor that compensates for false alarm rates. The larger the value, the greater the tendency to reduce the false alarm rate during the learning process.

[0130] The initial weights are adjusted using the following formula:

[0131]

[0132] The threat detection model training method provided in this application determines the similar features of normal samples by determining the average value of features, calculates the compensation factor for the false alarm rate by determining the false alarm rate and the detection rate, and then determines the optimal initial weights by using the compensation factor for the false alarm rate and the discrete values. This allows normal samples that are closer to the mean of the overall distribution to have higher weights, optimizes the weight allocation of normal samples, reduces the complexity of model training, reduces the false alarm rate of the model, and improves the detection accuracy.

[0133] Figure 3 A flowchart illustrating the training method for the threat detection model provided in this application embodiment. Figure 3 ,like Figure 3As shown, in this embodiment, the training sample set also includes a label corresponding to each threat information feature vector. The label is used to indicate whether the corresponding threat information feature vector is normal. This embodiment... Figure 1 or Figure 2 Based on the embodiments, a detailed explanation is provided on constructing a threat detection model based on the optimal initial weights, the training sample set, and multiple basic classifiers. This method includes:

[0134] S301. Using the optimal initial weights as sample weights, and based on the sample weights, the labels corresponding to each threat information feature vector, and multiple basic classifiers, determine the sum of weighted classification errors corresponding to each basic classifier.

[0135] The weighted classification error is used to indicate the classification error rate after weighting each sample.

[0136] The sum of weighted classification errors for each base classifier is determined using the following formula:

[0137]

[0138] in,

[0139] in, For weighted classification error, These are the sample weights.

[0140] S302. Based on the plurality of base classifiers, the base classifier with the smallest sum of weighted classification errors is selected as the first target classifier, and the classifier weights corresponding to the first target classifier are determined.

[0141] From the constructed base classifiers, select the base classifier that minimizes the sum of weighted classification errors. The base classifier As the first target classifier.

[0142]

[0143] The classifier weights are calculated using the following formula:

[0144]

[0145] in, These are the classifier weights; This is the sum of the weighted classification errors of the determined target classifier.

[0146] S303. Construct a target classifier based on the first target classifier and the weights corresponding to the first target classifier;

[0147] The target classifier is constructed using the following formula:

[0148]

[0149]

[0150] in, For the target classifier, It is a normalized vector.

[0151] S304. Determine whether the target classifier meets the preset requirements; if yes, proceed to step S305. Use the target classifier as a threat detection model; if no, proceed to step S306. Update the sample weights and further determine the second target classifier and its corresponding classifier weights based on the updated sample weights.

[0152] Among them, the preset requirements are used to indicate that the classification effect of the target classifier meets the threat detection requirements.

[0153] Determine whether the classification performance of the target classifier meets the threat detection requirements. If yes, use the target classifier as the threat detection model; otherwise, update the sample weights.

[0154] Update the sample weights using the following formula:

[0155]

[0156] Based on the updated sample weights, the second target classifier and its corresponding classifier weights are further determined. Then, based on the first target classifier and its corresponding classifier weights, the second target classifier and its corresponding classifier weights, the target classifier is constructed.

[0157] S305. Use the target classifier as a threat detection model;

[0158] S306. Update the sample weights, and further determine the second target classifier and its corresponding classifier weights based on the updated sample weights.

[0159] In one possible implementation, a threshold can be set to prevent the constructed threat detection model from overfitting. The specific process is as follows: determine whether the sum of the weighted classification errors of the first target classifier is less than a preset threshold. If the sum of the weighted classification errors of the first target classifier is less than the preset threshold, determine a second target classifier whose sum of weighted classification errors is not less than the preset threshold from the multiple optimized base classifiers.

[0160] The preset threshold is used to indicate the classification effect of the classifier.

[0161] In the first iteration, if the sum of the weighted errors of any one of the base classifiers is less than a preset threshold, it indicates that the base classifier is overfitting the samples. That is, it may be too complex, performing very well on the training data but poorly on new, unseen data. Since the base classifier that minimizes the sum of weighted classification errors is chosen as the first target classifier, the fact that any weighted error sum is less than the preset threshold indicates that the determined first target classifier is overfitting the samples. Therefore, in subsequent iterations, a second target classifier with a weighted classification error sum not less than the preset threshold is selected from the multiple base classifiers; that is, the overfitting correction process is no longer performed. This improves the overall classification performance.

[0162] The threat detection model training method provided in this application embodiment calculates the sum of weighted classification errors of each optimized base classifier, compares the optimized base classifiers, and selects the base classifier with the smallest sum of weighted errors as the first target classifier. The weight of the first target classifier is then calculated, and a target classifier is constructed based on the first target classifier and its weights. Based on the target classifier, the sample weights are continuously updated to determine the second classifier until the constructed target classifier meets the preset requirements. This reduces computational complexity, improves the computational efficiency of the threat detection model, and enhances the efficiency and accuracy of threat detection.

[0163] The threat detection method provided in this application obtains the detection result based on the threat information to be detected and the threat detection model.

[0164] The detection results are used to indicate whether the threat information to be detected is normal, and the threat detection model is trained using the model training method described above.

[0165] This application embodiment extracts features from the threat information to be detected and inputs these features into the threat detection model trained by the above-mentioned model training method to obtain the threat detection result. This result provides staff with a reference for threat analysis, improves threat detection efficiency, and saves a lot of resources and time.

[0166] Figure 4 A schematic diagram of the structure of the training device for the threat detection model provided in the embodiments of this application is shown below. Figure 4 As shown, the training device 40 for the threat detection model provided in this embodiment includes:

[0167] The acquisition module 401 is used to acquire multiple threat information data, extract features from multiple threat information data, and determine a training sample set. The training sample set includes multiple samples, and each sample includes multiple threat information feature vectors.

[0168] Processing module 402 is used to construct multiple basic classifiers based on multiple threat information feature vectors of the training sample set;

[0169] The aforementioned processing module 402 is also used to determine the scaling factor for compensating the false alarm rate based on multiple base classifiers, and to determine the optimal initial weight of the sample based on the scaling factor for compensating the false alarm rate.

[0170] Module 403 is used to build a threat detection model based on the optimal initial weights, training sample set and multiple base classifiers.

[0171] In one possible implementation, each threat information feature vector includes multiple features. The processing module 402 is further configured to construct a base classifier based on any one of the multiple features; determine multiple misclassification rates based on the base classifier and the training sample set; optimize the base classifier by minimizing the sum of the misclassification rates; and use the optimized base classifier as the base classifier corresponding to each feature.

[0172] In one possible implementation, the training sample set includes multiple normal samples, multiple abnormal samples, and a label corresponding to each feature; the processing module 402 is further configured to determine the feature average of multiple features in the multiple normal samples, and determine the discrete value between the multiple features in each normal sample and the feature average; determine a set of base classifiers based on multiple base classifiers, and determine the false alarm rate and detection rate based on the set of base classifiers, multiple normal samples, multiple abnormal samples, and the label corresponding to each feature, wherein the false alarm rate is used to indicate the proportion of normal samples classified as abnormal samples by the set of base classifiers, and the detection rate is used to indicate the proportion of all samples correctly classified by the set of base classifiers; determine a scaling factor to compensate for the false alarm rate based on the false alarm rate and the scaling factor to compensate for the false alarm rate, and determine the optimal initial weights based on the discrete value and the scaling factor to compensate for the false alarm rate.

[0173] In one possible implementation, the processing module 402 is further configured to determine the initial weights of multiple samples; determine the weighted classification rate based on the false alarm rate and the detection rate, the weighted classification rate being used to balance the false alarm rate and the detection rate, and determine the scaling factor for compensating the false alarm rate; adjust the initial weights based on the discrete values ​​and the scaling factor for compensating the false alarm rate, and use the adjusted initial weights as the optimal initial weights.

[0174] In one possible implementation, the training sample set also includes a label corresponding to each threat information feature vector. The label is used to indicate whether the corresponding threat information feature vector is normal. The construction module 403 is also used to take the optimal initial weight as the sample weight, and determine the sum of weighted classification errors corresponding to each basic classifier based on the sample weight, the label corresponding to each threat information feature vector, and multiple basic classifiers. The weighted classification error is used to indicate the classification error rate of each sample after weighting. Based on multiple basic classifiers, the basic classifier with the smallest sum of weighted classification errors is taken as the first target classifier, and the classifier weight corresponding to the first target classifier is determined. Based on the first target classifier and the weight corresponding to the first target classifier, a target classifier is constructed. It is determined whether the target classifier meets the preset requirements. If yes, the target classifier is used as the threat detection model. If not, the sample weights are updated, and the second target classifier and its corresponding classifier weights are further determined based on the updated sample weights.

[0175] In one possible implementation, the construction module 403 is further configured to determine whether the sum of the weighted classification errors of the first target classifier is less than a preset threshold, the preset threshold being used to indicate the classification effect of the classifier; if the sum of the weighted classification errors of the first target classifier is less than the preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple base classifiers.

[0176] The training device for the threat detection model provided in this embodiment can execute the method provided in the above training method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0177] Figure 5 This is a schematic diagram of the structure of the training device for the threat detection model provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0178] In the specific implementation process, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to execute the training method or threat detection method of the above-mentioned threat detection model.

[0179] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0180] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0181] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0182] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0183] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described threat detection model training method or threat detection method.

[0184] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described training method or threat detection method for the threat detection model.

[0185] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0186] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0187] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0188] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0189] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0190] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0192] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A training method for a threat detection model, characterized in that, include: Acquire multiple threat information data and extract features from multiple threat information data to determine a training sample set, wherein the training sample set includes multiple samples, and each sample includes multiple threat information feature vectors; Based on the multiple threat information feature vectors of the training sample set, multiple basic classifiers are constructed. The training sample set includes: multiple normal samples, multiple abnormal samples, and a label corresponding to each feature; Determine the average value of multiple features in the plurality of normal samples, and determine the discrete value between the multiple features in each normal sample and the average value of the features; Based on the multiple base classifiers, a set of base classifiers is determined, and based on the set of base classifiers, multiple normal samples, multiple abnormal samples, and the label corresponding to each feature, the false alarm rate and the detection rate are determined. The false alarm rate is used to indicate the proportion of normal samples that the set of base classifiers classifies as abnormal samples, and the detection rate is used to indicate the proportion of all samples that the set of base classifiers correctly classifies. Determine the initial weights of the plurality of samples; Based on the false alarm rate and the detection rate, a weighted classification rate is determined. The weighted classification rate is used to balance the false alarm rate and the detection rate and to determine the proportional factor for compensating the false alarm rate. Based on the discrete values ​​and the scaling factor for compensating for false alarms, the initial weights are adjusted, and the adjusted initial weights are taken as the optimal initial weights. Based on the optimal initial weights, training sample set, and multiple basic classifiers, a threat detection model is constructed.

2. The method according to claim 1, characterized in that, Each threat information feature vector includes multiple features. Based on the multiple threat information feature vectors of the training sample set, multiple basic classifiers are constructed, including: A basic classifier is constructed based on any one of the multiple features; Based on the base classifier and the training sample set, multiple misclassification rates are determined, and the base classifier is optimized by minimizing the sum of the misclassification rates. The optimized base classifier is then used as the base classifier for each feature.

3. The method according to claim 2, characterized in that, The training sample set includes a label corresponding to each threat information feature vector. The label is used to indicate whether the corresponding threat information feature vector is normal. The threat detection model is constructed based on the optimal initial weights, the training sample set, and multiple basic classifiers, including: The optimal initial weights are used as sample weights. Based on the sample weights, the labels corresponding to each threat information feature vector, and multiple base classifiers, the sum of weighted classification errors corresponding to each base classifier is determined. The weighted classification error is used to indicate the classification error rate of each sample after weighting. Based on the multiple base classifiers, the base classifier with the smallest sum of weighted classification errors is selected as the first target classifier, and the classifier weights corresponding to the first target classifier are determined. Based on the first target classifier and the weights corresponding to the first target classifier, a target classifier is constructed. Determine whether the target classifier meets the preset requirements; If so, the target classifier will be used as the threat detection model; If not, the sample weights are updated, and the second target classifier and its corresponding classifier weights are further determined based on the updated sample weights.

4. The method according to claim 3, characterized in that, The step of updating the sample weights and further determining the second target classifier based on the updated sample weights includes: Determine whether the sum of the weighted classification errors of the first target classifier is less than a preset threshold, wherein the preset threshold is used to indicate the classification effect of the classifier; If the sum of the weighted classification errors of the first target classifier is less than a preset threshold, a second target classifier whose sum of weighted classification errors is not less than the preset threshold is determined from multiple base classifiers.

5. A threat detection method, characterized in that, include: Based on the threat information to be detected and the threat detection model, a detection result is obtained. The detection result is used to indicate whether the threat information to be detected is normal. The threat detection model is trained by the method described in any one of claims 1-4.

6. A training device for a threat detection model, characterized in that, include: The acquisition module is used to acquire multiple threat information data, extract features from the multiple threat information data, and determine a training sample set. The training sample set includes multiple samples, and each sample includes multiple threat information feature vectors. The processing module is used to construct multiple basic classifiers based on multiple threat information feature vectors of the training sample set; The training sample set includes: multiple normal samples, multiple abnormal samples, and a label corresponding to each feature; The processing module is further configured to determine the feature average value of multiple features in the plurality of normal samples, and to determine the discrete value between the multiple features in each normal sample and the feature average value; The processing module is further configured to determine a set of basic classifiers based on the multiple basic classifiers, and to determine a false alarm rate and a detection rate based on the set of basic classifiers, multiple normal samples, multiple abnormal samples, and the label corresponding to each feature. The false alarm rate is used to indicate the proportion of normal samples that the set of basic classifiers classifies as abnormal samples, and the detection rate is used to indicate the proportion of all samples that the set of basic classifiers correctly classifies. The processing module is also used to determine the initial weights of the plurality of samples; The processing module is further configured to determine a weighted classification rate based on the false alarm rate and the detection rate, wherein the weighted classification rate is used to balance the false alarm rate and the detection rate and determine a proportional factor to compensate for the false alarm rate. The processing module is further configured to adjust the initial weights based on the discrete values ​​and the scaling factor for compensating for false alarms, and to use the adjusted initial weights as the optimal initial weights. The module is used to build a threat detection model based on the optimal initial weights, training sample set, and multiple base classifiers.

7. A training device for a threat detection model, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the training method of the threat detection model as described in any one of claims 1-4 or the threat detection method as described in claim 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the training method of the threat detection model as described in any one of claims 1-4 or the threat detection method as described in claim 5.

Images