Method, device, equipment, medium and program product for determining anomaly recognition model

Abstract

The present application relates to a method, apparatus, device, medium, and program product for determining an anomaly identification model. The method includes: determining multiple candidate anomaly identification models that match the indicator type of a network access device; when each candidate anomaly identification model is a predictive numerical model, calculating the weight value of each predictive numerical model based on the multi-class model evaluation indicator value corresponding to each predictive numerical model; for each predictive numerical model, calculating the weight ratio of the predictive numerical model based on each weight value; determining multiple predictive numerical models whose weight ratio is greater than a preset weight ratio as target anomaly identification models; and using the target anomaly identification model to perform anomaly identification processing on the network access device. This method can improve the accuracy of anomaly identification results.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and program product for determining an anomaly recognition model. Background Art

[0002] With the development of artificial intelligence technology, anomaly recognition models have emerged. By deploying anomaly recognition models on network access devices, such as base stations, anomalies can be identified and monitored.

[0003] However, since the anomaly recognition models used by network access devices are often fixed and can only identify a single anomaly category, the accuracy of the anomaly recognition results obtained is low. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for determining an abnormality identification model that can improve the accuracy of abnormality identification results in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for determining an anomaly recognition model, the method comprising:

[0006] determining a plurality of candidate anomaly recognition models that match the indicator type of the network access device;

[0007] When each candidate anomaly recognition model is a prediction numerical model, a weight value of each prediction numerical model is calculated according to the multi-class model evaluation index value corresponding to each prediction numerical model;

[0008] For each prediction numerical model, the weight ratio of the prediction numerical model is calculated based on each weight value;

[0009] Multiple prediction numerical models whose weight ratios are greater than preset weight ratios are determined as target anomaly recognition models; the target anomaly recognition models are used to perform anomaly recognition processing on network access devices.

[0010] In some embodiments, calculating the weight value of each prediction numerical model according to the multi-class model evaluation index value corresponding to each prediction numerical model includes:

[0011] For each prediction numerical model, the inverse of each type of model evaluation index value corresponding to the prediction numerical model is normalized to obtain the normalized inverse of each type of model evaluation index value;

[0012] For each prediction numerical model, the minimum prediction error of the prediction numerical model is calculated based on the difference between each type of standardized inverse corresponding to the prediction numerical model and the minimum standardized inverse among the standardized inverses of the same type;

[0013] For each prediction numerical model, based on the difference between each type of standardized inverse corresponding to the prediction numerical model and the maximum standardized inverse among the standardized inverses of the same type, the maximum prediction error of the prediction numerical model is calculated;

[0014] The weight value of the prediction numerical model is determined based on the minimum prediction error and the maximum prediction error.

[0015] In some embodiments, determining a weight value of a prediction numerical model based on a minimum prediction error and a maximum prediction error includes:

[0016] The minimum prediction error and the maximum prediction error are summed to obtain the prediction error sum;

[0017] The weight value of the prediction numerical model is determined according to the quotient between the minimum prediction error and the sum of the prediction errors.

[0018] In some embodiments, the method further comprises:

[0019] Predicting indicator prediction values ​​of network access equipment based on the determined multiple prediction numerical models;

[0020] Obtain the true values ​​of indicators of network access devices;

[0021] Calculate the difference between the predicted value of the indicator corresponding to each prediction numerical model and the actual value of the indicator, and obtain the abnormal prediction error of each prediction numerical model for the network access device;

[0022] When the average abnormal prediction error of the abnormal prediction errors is greater than a preset threshold, it is determined that the network access device is abnormal.

[0023] In some embodiments, the method further comprises:

[0024] In the case where each candidate anomaly recognition model is a classification model, a test sample data set of the network access device is divided into multiple categories of sub-test sample data sets;

[0025] For each type of sub-test sample data set, classify the test sample data in the sub-test sample data set through each classification model, and use the classification model with the highest classification accuracy among the classification models as the optimal classification model for the sub-test sample data set;

[0026] The optimal classification model corresponding to each type of sub-test sample data set is determined as the target anomaly recognition model.

[0027] In some embodiments, the step of training the classification model includes:

[0028] When the ratio of positive samples to negative samples in the training sample data set of the network access device is greater than a preset ratio, synthesizing new negative samples based on the negative samples to obtain sample-balanced training sample data;

[0029] When the number ratio is less than or equal to the preset ratio, the positive samples and negative samples are upsampled and downsampled to obtain balanced training sample data;

[0030] Each classification model is trained based on sample-balanced training sample data.

[0031] In a second aspect, the present application further provides a device for determining an abnormality recognition model, the device comprising:

[0032] A first determination module is used to determine multiple candidate anomaly recognition models that match the indicator type of the network access device;

[0033] a calculation module for calculating, when each candidate anomaly recognition model is a prediction numerical model, a weight value of each prediction numerical model according to the multi-class model evaluation index value corresponding to each prediction numerical model; and for each prediction numerical model, calculating a weight ratio of the prediction numerical model based on each weight value;

[0034] The second determination module is used to determine multiple prediction numerical models with weight ratios greater than a preset weight ratio as target anomaly recognition models; the target anomaly recognition model is used to perform anomaly recognition processing on the network access device.

[0035] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0037] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the above method when executed by a processor.

[0038] The above-mentioned method, apparatus, computer equipment, storage medium and computer program product for determining anomaly identification models determine multiple candidate anomaly identification models that match the indicator type of the network access device, and when each candidate anomaly identification model is a predictive numerical model, calculate the weight value of each predictive numerical model according to the multi-category model evaluation index value of each predictive numerical model, calculate the weight ratio of each predictive numerical model according to the weight value, and select a predictive numerical model with a weight ratio greater than a preset weight ratio from multiple predictive numerical models as the target anomaly identification model, that is, select a predictive numerical model with higher prediction accuracy as the target anomaly identification model, and jointly determine the anomaly identification result of the network access device based on each selected predictive numerical model. Compared with a single anomaly identification model, the network access device is subjected to anomaly identification processing by using multiple predictive numerical models with higher prediction accuracy, and multiple anomaly categories can be identified at the same time, thereby improving the accuracy of the anomaly identification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 1 is a flow chart of a method for determining an abnormality recognition model in one embodiment;

[0040] Figure 2 is a schematic diagram of classification model fusion in one embodiment;

[0041] Figure 3 is a flow chart of a method for determining an abnormality recognition model in another embodiment;

[0042] Figure 4 is a structural block diagram of a device for determining an abnormality identification model in one embodiment;

[0043] Figure 5 is a structural block diagram of a second determination module in one embodiment;

[0044] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0046] In one embodiment, Figure 1 As shown, a flow chart of a method for determining an anomaly identification model is provided. This embodiment takes the method applied to a network access device as an example for explanation. It is understandable that the method can also be applied to a server, and can also be applied to a system including a network access device and a server, and is implemented through the interaction between the network access device and the server. The method specifically includes the following steps:

[0047] Step 101: Determine multiple candidate anomaly recognition models that match the indicator type of the network access device.

[0048] The candidate anomaly recognition model is obtained by training the initial model using the training sample data of the network access device. Before using the training sample data to train the initial model, the training sample data can be preprocessed to match the category of the initial model.

[0049] In some embodiments, when the initial model is a predictive numerical model, since the predictive numerical model is sensitive to time series and data order, abnormal data in the training sample data is removed or backfilled. The backfilling process can use at least one of the K-nearest neighbor, regression, or mean method.

[0050] The indicator is a parameter used to measure whether a network access device has an abnormality, including but not limited to at least one of a static indicator, a periodic indicator, or an unstable indicator.

[0051] Static indicators are those that change little over a short period of time, including the number of cells using network access devices. Periodic indicators are those that change periodically, including the sleep duration of network access devices. Unstable indicators are those whose patterns of change are unpredictable, including the number of commands received by network access devices.

[0052] It is understood that different types of anomaly recognition models have different capabilities for identifying different types of indicators. Therefore, the network access device selects multiple indicators for anomaly monitoring and classifies these indicators into at least one of static, periodic, or unstable indicators. Multiple candidate anomaly recognition models are then determined to match the indicator types. In other words, different types of models are used to identify different types of indicators. Anomaly recognition for a network access device essentially involves identifying anomalies in its indicators and determining whether the device is abnormal based on whether the indicators are abnormal.

[0053] In some embodiments, the indicator types include periodic indicators and unstable indicators; the step of determining candidate anomaly recognition models that match the indicator types includes: determining multiple predicted numerical types as multiple candidate anomaly recognition models that match the periodic indicators; and determining multiple classification models as multiple candidate anomaly recognition models that match the unstable indicators.

[0054] In some embodiments, since the static indicator changes little in a short period of time, it is only necessary to compare the static indicator with a threshold to determine whether the static indicator is abnormal.

[0055] It should be noted that the multiple candidate anomaly recognition models can be different anomaly recognition models in the same category. For example, each candidate anomaly recognition model is a prediction value model, but each candidate anomaly recognition model is a different prediction value model. The multiple candidate anomaly recognition models can also be the same anomaly recognition model in the same category.

[0056] Step 102: When each candidate anomaly recognition model is a prediction numerical model, a weight value of each prediction numerical model is calculated according to a multi-class model evaluation index value corresponding to each prediction numerical model.

[0057] Among them, the predictive numerical model is a model for predicting continuous values, that is, a model that predicts the value of the dependent variable corresponding to a given independent variable by establishing a continuous value function.

[0058] Model evaluation metrics are used to evaluate the accuracy of a numerical model. These metrics include, but are not limited to, MSE (mean squared error), MAE (median absolute error), and RMSE (root mean squared error).

[0059] Exemplarily, when each candidate anomaly recognition model is a predictive numerical model, the multi-class model evaluation index value corresponding to each predictive numerical model is calculated, and the weight value of each predictive numerical model is calculated based on the multi-class model evaluation index value corresponding to each predictive numerical model to obtain the weight value corresponding to each predictive numerical model.

[0060] Step 103: For each prediction numerical model, calculate the weight ratio of the prediction numerical model based on each weight value.

[0061] Exemplarily, the network access device calculates, for each prediction numerical model, the weight proportion of the prediction numerical model according to the weight value corresponding to the prediction numerical model and the sum of the weight values ​​corresponding to all prediction numerical models.

[0062] Step 104: Determine multiple prediction numerical models with weights greater than a preset weight as target anomaly recognition models; the target anomaly recognition models are used to perform anomaly recognition processing on the network access device.

[0063] Exemplarily, the network access device compares the weight ratio of each prediction numerical model with the preset weight ratio, and determines multiple prediction numerical models whose weight ratio is greater than the preset weight ratio as target abnormality recognition models, that is, the prediction numerical model with higher prediction accuracy among the multiple prediction numerical models is determined as the target abnormality recognition model, and uses the target abnormality recognition model to perform abnormality recognition processing on the network access device, that is, the abnormality recognition result of the network access device is jointly predicted by the determined multiple prediction numerical models.

[0064] In the above-mentioned method for determining the abnormality identification model, multiple candidate abnormality identification models that match the indicator type of the network access device are determined, and when each candidate abnormality identification model is a predictive numerical model, the weight value of each predictive numerical model is calculated according to the multi-category model evaluation index value of each predictive numerical model, the weight ratio of each predictive numerical model is calculated according to the weight value, and a predictive numerical model with a weight ratio greater than a preset weight ratio is selected from multiple predictive numerical models as the target abnormality identification model, that is, a predictive numerical model with higher prediction accuracy is selected as the target abnormality identification model, and the abnormality identification result of the network access device is jointly determined based on each selected predictive numerical model. Compared with a single abnormality identification model, the network access device is subjected to abnormality identification processing by multiple predictive numerical models with higher prediction accuracy, and multiple abnormality categories can be identified at the same time, thereby improving the accuracy of the abnormality identification result of the network access device.

[0065] That is to say, the present application creatively proposes a weight evaluation mechanism for predictive numerical models, so as to select multiple predictive numerical models with higher prediction accuracy as target anomaly recognition models, and output the anomaly recognition results of the network access device based on the joint evaluation of each selected predictive numerical model, thereby improving the accuracy of the anomaly recognition results.

[0066] In some embodiments, the weight value of each prediction numerical model is calculated based on the multi-category model evaluation index values ​​corresponding to each prediction numerical model, including: for each prediction numerical model, standardizing the inverse of each category of model evaluation index value corresponding to the prediction numerical model to obtain the standardized inverse corresponding to each category of model evaluation index value; for each prediction numerical model, calculating the minimum prediction error of the prediction numerical model based on the difference between the standardized inverse of each category corresponding to the prediction numerical model and the minimum standardized inverse among the same category of standardized inverses; for each prediction numerical model, calculating the maximum prediction error of the prediction numerical model based on the difference between the standardized inverse of each category corresponding to the prediction numerical model and the maximum standardized inverse among the same category of standardized inverses; and determining the weight value of the prediction numerical model based on the minimum prediction error and the maximum prediction error.

[0067] In some embodiments, the weight value of the prediction numerical model is determined based on the minimum prediction error and the maximum prediction error, including: summing the minimum prediction error and the maximum prediction error to obtain the sum of the prediction errors; and determining the weight value of the prediction numerical model according to the quotient between the minimum prediction error and the sum of the prediction errors.

[0068] In some embodiments, the process of calculating the weight value is described by taking the number of prediction numerical models as N and the multi-class model evaluation index values ​​corresponding to each prediction numerical model as M as an example, where:

[0069] 1. Construct an evaluation matrix based on M*N model evaluation index values.

[0070] 2. For each prediction numerical model, perform row and column normalization on the inverses of the M model evaluation index values ​​corresponding to each prediction numerical model to obtain the standardized inverses of the M model evaluation index values ​​corresponding to each prediction numerical model. Since there are N prediction numerical models in total, a total of M*N standardized inverses of model evaluation index values ​​are obtained.

[0071] In some embodiments, the model evaluation index values ​​are normalized using the following formula:

[0072]

[0073] Among them, d i，j is the model evaluation index value; d k,j is with d i，j Model evaluation metric values ​​located in the same column of the evaluation matrix; is the column-normalized inverse of the model evaluation metric value;

[0074] The model evaluation index values ​​are normalized using the following formula:

[0075]

[0076] in, It is the standardized inverse of the model evaluation index value obtained by performing row standardization on the inverse of column standardization.

[0077] 3. For each prediction numerical model, based on the difference between each type of standardized inverse corresponding to the prediction numerical model and the minimum standardized inverse of the same type of standardized inverse, the squares of each difference are summed and the square root is taken to obtain the minimum prediction error of the prediction numerical model.

[0078] In some embodiments, the minimum prediction error is calculated by the following formula:

[0079]

[0080] Among them, Dis i，min is the minimum prediction error; It is the smallest normalized reciprocal among normalized reciprocals of the same type.

[0081] 4. For each prediction numerical model, based on the difference between each type of standardized inverse corresponding to the prediction numerical model and the maximum standardized inverse of the same type of standardized inverse, the squares of each difference are summed and the square root is taken to obtain the maximum prediction error of the prediction numerical model.

[0082] In some embodiments, the minimum prediction error is calculated by the following formula:

[0083]

[0084] Among them, Dis i，max is the maximum prediction error; ec j max It is the largest normalized reciprocal among similar normalized reciprocals.

[0085] 5. Sum the minimum prediction error and the maximum prediction error to obtain the sum of the prediction errors; determine the weight value of the prediction numerical model based on the quotient between the minimum prediction error and the sum of the prediction errors.

[0086] In some embodiments, the weight value of the prediction numerical model is calculated by the following formula;

[0087]

[0088] Among them, ω i is the weight value of the prediction numerical model.

[0089] In some embodiments, the weight ratio of the prediction numerical model is calculated by the following formula:

[0090]

[0091] Among them, ω k is the weight value of the prediction numerical model; is the weight ratio of the prediction numerical model.

[0092] In the above embodiment, the weight value of the prediction numerical model is determined based on the evaluation index value of each type of model corresponding to the prediction numerical model, and the prediction accuracy of the prediction numerical model can be accurately judged based on the weight value.

[0093] In some embodiments, the method also includes: predicting the index prediction value of the network access device based on the determined multiple prediction numerical models; obtaining the true value of the index of the network access device; respectively calculating the difference between the index prediction value corresponding to each prediction numerical model and the true value of the index, and obtaining the abnormal prediction error for the network access device corresponding to each prediction numerical model; when the average abnormal prediction error of each abnormal prediction error is greater than a preset threshold, judging the abnormality of the network access device.

[0094] It can be understood that the predicted index value of the network access device is predicted based on the determined multiple prediction numerical models, that is, the multiple prediction numerical models with weight ratios greater than the preset weight ratio jointly predict the predicted index value of the network access device. Since multiple prediction numerical models are used for prediction, each prediction numerical model has a corresponding index prediction value, that is, there are multiple index prediction values. Based on obtaining the difference between the true value of the index of the network access device and the predicted value of each indicator, that is, calculating the difference between the predicted value of the indicator corresponding to each prediction numerical model and the true value of the indicator, the abnormal prediction error of the indicator for the network access device corresponding to each prediction numerical model is obtained. When the average abnormal prediction error of each abnormal prediction error is greater than the preset threshold, that is, the error between the predicted value of the index of the network access device for each prediction numerical model and the true value of the index of the network access device is large, it indicates that the indicator matched with the prediction numerical model is abnormal, that is, the network access device is abnormal.

[0095] In the above embodiment, the accuracy of the abnormality identification result of the indicator of the network access device is improved by jointly predicting the abnormality identification result based on multiple prediction numerical models with weight ratios greater than the preset weight ratio.

[0096] In some embodiments, the method also includes: when each candidate anomaly recognition model is a classification model, dividing the test sample data set of the network access device into multiple categories of sub-test sample data sets; for each category of sub-test sample data sets, classifying the test sample data in the sub-test sample data sets through each classification model, and taking the classification model with the highest classification accuracy among the classification models as the optimal classification model of the sub-test sample data set; and determining the optimal classification model corresponding to each category of sub-test sample data sets as the target anomaly recognition model.

[0097] Among them, the classification model is a model that predicts categories, that is, when the classification model performs anomaly identification, it outputs the category of the anomaly.

[0098] It is understandable that since different classification models have different classification accuracy for different types of anomalies, after obtaining multiple classification models, it is necessary to select a classification model with high classification accuracy from the multiple classification models to jointly classify the anomalies.

[0099] Exemplarily, when each candidate anomaly recognition model is a classification model, the network access device divides the test sample data set into multiple categories of sub-test sample data sets, and for each category of sub-test sample data sets, classifies the test sample data in the sub-test sample data sets through each classification model, so as to confirm the classification accuracy of each classification model for different sub-test sample data sets, and takes the classification model with the highest classification accuracy among each classification model as the optimal classification model for the sub-test sample data set, that is, obtains the optimal classification model in the local space; determines the optimal classification model corresponding to each category of sub-test sample data sets as the target anomaly recognition model, and jointly predicts whether the indicators of the network access device matching the classification model are abnormal based on each optimal classification model, and then determines whether the network access device is abnormal.

[0100] In the above embodiment, the accuracy of the anomaly recognition result can be improved by determining the optimal classification model corresponding to each type of sub-test sample data set as the target anomaly recognition model.

[0101] In some embodiments, the training steps of the classification model include: when the ratio between the number of positive samples and negative samples in the training sample data set of the network access device is greater than a preset ratio, synthesizing new negative samples based on the negative samples to obtain sample-balanced training sample data; when the ratio is less than or equal to the preset ratio, performing up and down sampling on the positive samples and negative samples to obtain sample-balanced training sample data; and training each classification model based on the sample-balanced training sample data.

[0102] It can be understood that since the number of negative samples in the training sample data set is often less than the number of positive samples, resulting in an imbalance in the number of positive and negative samples, if the classification model is trained using training data with unbalanced samples, the classification model has a higher classification accuracy for positive samples, and a higher AUC (the area under the ROC curve and the coordinate axis) can be obtained in the model evaluation index of the classification model. However, the classification accuracy of the classification model for negative samples may be lower, and negative samples may be missed. Therefore, it is necessary to perform sample balancing on the positive and negative samples in the training sample data set.

[0103] Among them, the ROC curve (receiver operating characteristic curve).

[0104] Exemplarily, the network access device determines the number ratio between positive samples and negative samples in the training sample data set, and adopts a corresponding sample balancing processing method according to the number ratio; when the number ratio between positive samples and negative samples in the training sample data set is greater than a preset ratio, new negative samples are synthesized based on the negative samples to obtain sample-balanced training sample data; when the number ratio is less than or equal to the preset ratio, since the error of synthesizing new negative samples based on negative samples is large, the positive samples and negative samples are subjected to up- and down-sampling processing, that is, by copying negative samples or reducing positive samples, sample-balanced training sample data is obtained; and each classification model is trained based on the sample-balanced training sample data.

[0105] In some embodiments, if the difference in number between positive samples and negative samples after sample balancing is still large, the classification model with the largest α=(AUC+sensitivity) is used as the trained classification model during the iterative training of the classification model.

[0106] In the above embodiment, a corresponding sample balancing processing method is adopted based on the ratio of the number of positive samples to negative samples, so that the classification model obtained based on the sample-balanced training sample data has a high classification accuracy for both positive and negative samples.

[0107] In some embodiments, as Figure 2 As shown in the figure, a schematic diagram of classification model fusion is provided. Figure 2 The following description is given, in which:

[0108] N classification models 21 and 22 that match the network access device's indicators are selected as base classifiers. Dynamic ensemble selection is used to fuse the classification models. The classification models 22 are integrated using ensemble learning methods such as voting, bagging (bootstrap aggregating), and boosting. The integrated classification models 22 are then fused with the classification model 21 using dynamic ensemble selection to obtain the target anomaly recognition model 23.

[0109] It should be noted that, in the process of iteratively training each classification model based on the training sample data of the network access device, the classification model with the largest α=(AUC+sensitivity) among the classification models obtained in each round of iterative training is taken as the trained classification model.

[0110] In some embodiments, as Figure 3 As shown, a schematic diagram of a method for determining an anomaly identification model is provided. The method is described by taking the application of the method to a network access device as an example. The method specifically includes the following steps:

[0111] Step 301: Indicator screening. That is, through an objective indicator screening method (hypothesis testing), the relevant indicators of the network access device are more comprehensively included, and the indicator screening module automatically screens out the indicator set K2 = {KPI1, KPI i ,...}, maximize the abnormal monitoring of network access devices under the optimal indicator set.

[0112] Step 302: qualitative analysis of indicators. v , and the autocorrelation coefficient (ACF) detects the degree of fluctuation and periodicity of the indicators, and divides the indicators of network access devices into static indicators A, periodic indicators B, and unstable indicators C. In addition, different types of indicators use different types of models for anomaly identification.

[0113] If the indicator type is a static indicator, execute step 303. If the indicator type is a periodic indicator, the model matching the periodic indicator is a predictive numerical model, and execute steps 304 and 305. If the indicator type is an unstable indicator, the model matching the unstable indicator is a classification model, and execute steps 306 and 307.

[0114] Step 303: Threshold determination. Since static indicators change little over a short period of time, it is only necessary to compare the static indicators with the threshold to determine whether the static indicators are abnormal. If the static indicators are less than the threshold, the network access device is determined to be abnormal.

[0115] Step 304: Eliminate or backfill. That is, perform elimination or backfill processing on abnormal data in the training sample data of the prediction numerical model.

[0116] Step 305: Determine the target anomaly recognition model based on the model evaluation index values. Specifically, calculate the weight of each prediction numerical model based on the multi-class model evaluation index values ​​corresponding to each prediction numerical model; calculate the weight ratio of each prediction numerical model based on each weight value; and determine the prediction numerical models with weight ratios greater than the preset weight ratio as the target anomaly recognition model.

[0117] Step 306: Sample balancing. This involves calculating the ratio β between the number of positive and negative samples in the training sample data for the classification model. When the ratio β is greater than the preset ratio t1, the SMOTE (Synthetic Minority Oversampling Technique) algorithm is used to synthesize new negative samples. When the ratio β is less than or equal to the preset ratio t1, the positive and negative samples are subjected to upsampling and downsampling.

[0118] Step 307: Dynamic ensemble learning is used to determine the target anomaly recognition model. Specifically, if each candidate anomaly recognition model is a classification model, the test sample dataset of the network access device is divided into multiple sub-test sample datasets. For each sub-test sample dataset, the test sample data in the sub-test sample dataset is classified using each classification model. The classification model with the highest classification accuracy among the classification models is then determined as the optimal classification model for the sub-test sample dataset. The optimal classification model corresponding to each sub-test sample dataset is then determined as the target anomaly recognition model.

[0119] Step 308: Outputting the abnormality identification result. That is, performing abnormality identification processing on the network access device based on the comparison result between the static indicator and the threshold or based on the target abnormality identification model, and outputting the abnormality identification result of the network access device.

[0120] In the above-mentioned method for determining the abnormality identification model, multiple candidate abnormality identification models that match the indicator type of the network access device are determined, and when each candidate abnormality identification model is a predictive numerical model, the weight value of each predictive numerical model is calculated according to the multi-category model evaluation index value of each predictive numerical model, the weight ratio of each predictive numerical model is calculated according to the weight value, and a predictive numerical model with a weight ratio greater than the preset weight ratio is selected from multiple predictive numerical models as the target abnormality identification model, that is, a predictive numerical model with higher prediction accuracy is selected as the target abnormality identification model, and the abnormality identification result of the network access device is jointly determined based on each selected predictive numerical model. Compared with a single abnormality identification model, the network access device is subjected to abnormality identification processing by using multiple predictive numerical models with higher prediction accuracy, and multiple abnormality categories can be identified at the same time, thereby improving the accuracy of the abnormality identification result.

[0121] In addition, sample balancing or elimination and backfilling processing are adopted for different categories of models to improve the quality of training sample data, thereby improving the classification accuracy of the trained classification model or the prediction accuracy of the prediction numerical model, and then accurately identifying anomalies of network access equipment and reducing losses caused by misjudgment or missed anomalies.

[0122] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0123] Based on the same inventive concept, the embodiments of the present application also provide a device for determining an abnormality recognition model for implementing the above-mentioned method for determining an abnormality recognition model. The implementation solution provided by the device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations of the embodiments of the device for determining one or more abnormality recognition models provided below can be found in the above-mentioned limitations of the method for determining an abnormality recognition model, and will not be repeated here.

[0124] In one embodiment, Figure 4 As shown, a device for determining an abnormality recognition model is provided, comprising: a first determination module 401, a calculation module 402, and a second determination module 403, wherein:

[0125] The first determination module 401 is configured to determine a plurality of candidate anomaly recognition models that match the indicator type of the network access device.

[0126] The calculation module 402 is used to calculate the weight value of each prediction numerical model according to the multi-class model evaluation index value corresponding to each prediction numerical model when each candidate anomaly recognition model is a prediction numerical model; for each prediction numerical model, the weight ratio of the prediction numerical model is calculated based on each weight value.

[0127] The second determination module 402 is used to determine multiple prediction numerical models with weight ratios greater than a preset weight ratio as target anomaly recognition models; the target anomaly recognition models are used to perform anomaly recognition processing on the network access device.

[0128] In some embodiments, the calculation module 402 is used to standardize the inverse of each type of model evaluation index value corresponding to the prediction numerical model for each prediction numerical model to obtain the standardized inverse corresponding to each type of model evaluation index value; for each prediction numerical model, based on the difference between the standardized inverse of each type corresponding to the prediction numerical model and the minimum standardized inverse among the same type of standardized inverses, calculate the minimum prediction error of the prediction numerical model; for each prediction numerical model, based on the difference between the standardized inverse of each type corresponding to the prediction numerical model and the maximum standardized inverse among the same type of standardized inverses, calculate the maximum prediction error of the prediction numerical model; and determine the weight value of the prediction numerical model based on the minimum prediction error and the maximum prediction error.

[0129] In some embodiments, the calculation module 402 is specifically configured to sum the minimum prediction error and the maximum prediction error to obtain a sum of prediction errors; and determine a weight value of the prediction numerical model based on a quotient between the minimum prediction error and the sum of prediction errors.

[0130] In some embodiments, the second determination module 403 is also used to predict the index prediction value of the network access device based on the determined multiple prediction numerical models; obtain the true value of the index of the network access device; calculate the difference between the index prediction value corresponding to each prediction numerical model and the true value of the index, and obtain the abnormal prediction error for the network access device corresponding to each prediction numerical model; when the average abnormal prediction error of each abnormal prediction error is greater than a preset threshold, judge the abnormality of the network access device.

[0131] In some embodiments, as Figure 5 As shown, a structural block diagram of a second determination module is provided, including a first classification unit 403a, a second classification unit 403b and a determination unit 403c, wherein:

[0132] The first classification unit 403a divides the test sample data set of the network access device into multiple categories of sub-test sample data sets when each candidate anomaly recognition model is a classification model.

[0133] The second classification unit 403b classifies the test sample data in each sub-test sample data set using each classification model, and uses the classification model with the highest classification accuracy among the classification models as the optimal classification model for the sub-test sample data set.

[0134] The determining unit 403c determines the optimal classification model corresponding to each type of sub-test sample data set as the target anomaly recognition model.

[0135] In some embodiments, the second determination module 403 is also used to perform the following training steps of the classification model: when the ratio between the number of positive samples and negative samples in the training sample data set of the network access device is greater than a preset ratio, new negative samples are synthesized based on the negative samples to obtain sample-balanced training sample data; when the ratio is less than or equal to the preset ratio, positive samples and negative samples are upsampled and downsampled to obtain sample-balanced training sample data; and each classification model is trained based on the sample-balanced training sample data.

[0136] Each module in the above-mentioned device for determining anomaly identification models can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0137] In one embodiment, a computer device is provided. The computer device may be a network access device, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for determining an abnormality recognition model is implemented.

[0138] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0139] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0141] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0143] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0144] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for determining an anomaly recognition model, characterized in that: The method comprises: determining a plurality of candidate anomaly recognition models that match the indicator type of the network access device; In the case where each of the candidate anomaly recognition models is a prediction numerical model, calculating a weight value of each of the prediction numerical models according to a multi-class model evaluation index value corresponding to each of the prediction numerical models; For each of the prediction numerical models, calculating a weight ratio of the prediction numerical model based on each of the weight values; Determine the multiple prediction numerical models whose weight ratios are greater than the preset weight ratios as target anomaly recognition models; the target anomaly recognition models are used to perform anomaly recognition processing on the network access device; Calculating the weight value of each prediction numerical model according to the multi-class model evaluation index value corresponding to each prediction numerical model includes: For each of the predicted numerical models, normalizing the inverse of each type of model evaluation index value corresponding to the predicted numerical model to obtain a normalized inverse corresponding to each type of model evaluation index value; For each of the prediction numerical models, calculating the minimum prediction error of the prediction numerical model based on the difference between the standardized inverse of each type corresponding to the prediction numerical model and the minimum standardized inverse among the standardized inverses of the same type; For each of the prediction numerical models, calculating the maximum prediction error of the prediction numerical model based on the difference between the standardized inverse of each type corresponding to the prediction numerical model and the maximum standardized inverse of the same type; A weight value of the prediction numerical model is determined based on the minimum prediction error and the maximum prediction error.

2. The method according to claim 1, characterized in that The determining of the weight value of the prediction numerical model based on the minimum prediction error and the maximum prediction error includes: Summing the minimum prediction error and the maximum prediction error to obtain a prediction error sum; The weight value of the prediction numerical model is determined according to the quotient between the minimum prediction error and the sum of the prediction errors.

3. The method according to claim 1, characterized in that The indicator types include periodic indicators and unstable indicators; the multiple candidate anomaly recognition models that match the indicator types of the network access device are determined, including: Identifying multiple predicted value types as multiple candidate anomaly identification models that match the periodic indicators; A plurality of classification models are determined as a plurality of candidate anomaly identification models that match the instability indicator.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Predicting an indicator prediction value of the network access device based on the determined multiple prediction numerical models; Obtaining the true value of the indicator of the network access device; Calculating the difference between the predicted value of the indicator corresponding to each of the prediction numerical models and the true value of the indicator respectively, to obtain the abnormal prediction error for the network access device corresponding to each of the prediction numerical models; When the average abnormal prediction error of each abnormal prediction error is greater than a preset threshold, it is determined that the network access device is abnormal.

5. The method according to claim 1, characterized in that The method further comprises: In the case where each of the candidate anomaly recognition models is a classification model, dividing the test sample data set of the network access device into multiple categories of sub-test sample data sets; For each type of sub-test sample data set, classify the test sample data in the sub-test sample data set using each of the classification models, and use the classification model with the highest classification accuracy among the classification models as the optimal classification model for the sub-test sample data set; The optimal classification model corresponding to each type of sub-test sample data set is determined as the target anomaly recognition model.

6. The method according to claim 5, characterized in that The training steps of the classification model include: When the ratio of the number of positive samples to negative samples in the training sample data set of the network access device is greater than a preset ratio, synthesizing new negative samples based on the negative samples to obtain sample-balanced training sample data; When the quantity ratio is less than or equal to the preset ratio, performing upsampling and downsampling processing on the positive samples and negative samples to obtain sample-balanced training sample data; Each of the classification models is trained based on the sample-balanced training sample data.

7. A device for determining an abnormality recognition model, characterized in that: The device comprises: A first determination module is used to determine multiple candidate anomaly recognition models that match the indicator type of the network access device; a calculation module for, when each of the candidate abnormality recognition models is a prediction numerical model, performing, for each of the prediction numerical models, normalizing the inverse of each type of model evaluation index value corresponding to the prediction numerical model to obtain the standardized inverse corresponding to each type of model evaluation index value; for each of the prediction numerical models, calculating the minimum prediction error of the prediction numerical model based on the difference between the standardized inverse of each type corresponding to the prediction numerical model and the minimum standardized inverse among the standardized inverses of the same type; for each of the prediction numerical models, calculating the maximum prediction error of the prediction numerical model based on the difference between the standardized inverse of each type corresponding to the prediction numerical model and the maximum standardized inverse among the standardized inverses of the same type; determining the weight value of the prediction numerical model based on the minimum prediction error and the maximum prediction error; and calculating the weight proportion of the prediction numerical model based on each weight value for each of the prediction numerical models; The second determination module is used to determine the multiple prediction numerical models whose weight ratios are greater than the preset weight ratios as target anomaly recognition models; the target anomaly recognition model is used to perform anomaly recognition processing on the network access device.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images