A method, device and electronic device for detecting abnormal data

Abstract

The present application relates to a method, device and electronic device for detecting abnormal data, which is used to solve the problem of low detection efficiency in the prior art when detecting a large amount of flow data. The method includes calling a feature classification model to perform feature classification on the data to be detected, and obtaining a first feature of the data to be detected. Optionally, based on a feature extraction method, feature extraction is performed on at least a part of the data to be detected, and a second feature of at least a part of the data to be detected is obtained. Then, a security detection model is called to detect the first feature and / or the second feature, and the abnormal index corresponding to each of the first feature and / or the second feature is obtained. Then, the data in the data to be detected with an abnormal index greater than a preset threshold is used as abnormal data, and the detection result is obtained based on the abnormal data. Through the above method, large flow data can be automatically detected in real time, and the detection efficiency can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of information security technology, and in particular, to a method, device, and electronic device for detecting abnormal data. Background Art

[0002] With the development of 5G (5th Generation Mobile Communication Technology) technology, the network security issue of 5GC (5G Core) has inevitably become one of the important research fields.

[0003] Currently, the security detection for the 5G core network mainly adopts the analysis and classification method based on regular rules, that is, using regular rules to analyze and classify the traffic data on the 5G core network to detect abnormal traffic data. On the one hand, this detection based on regular rules requires professionals to define the rules, and the rules also rely on the regular maintenance and update by professionals. However, with the development of 5GC, a large amount of traffic data that does not apply to the established rules will be generated. Based on this, this method has the problem of low detection accuracy; on the other hand, this method needs to match one by one according to the established rules in actual application. Therefore, when facing a large amount of traffic data, there is also the problem of low detection efficiency. Summary of the Invention

[0004] This application provides a method, device, and electronic device for detecting abnormal data to solve the problem of low detection efficiency of the existing technology when detecting a large amount of traffic data.

[0005] In the first aspect, this application provides a method for detecting abnormal data, and the method includes:

[0006] Invoking a feature classification model to perform feature classification on the data to be detected to obtain the first feature of the data to be detected; and / or

[0007] Performing feature extraction on at least a part of the data to be detected based on a feature extraction method to obtain the second feature of at least a part of the data to be detected;

[0008] Invoking a security detection model to detect the first feature and / or the second feature to obtain the abnormal index corresponding to each of the first feature and / or the second feature;

[0009] Taking the data with an abnormal index greater than a preset threshold in the data to be detected as abnormal data.

[0010] Through the above method, the problem of low detection efficiency of the existing technology when detecting a large amount of traffic data can be solved, real-time automatic detection of large traffic data can be realized, the detection efficiency can be effectively improved, and it is especially suitable for the security detection of 5GC data.

[0011] In a possible design, the feature classification model is obtained based on the following operations:

[0012] Feature classification is performed on the training data based on the current feature classification model to obtain a first index and a second index of the feature classification result; wherein, the first index is the inaccurate probability of the feature classification result, and the second index is the accurate probability of the feature classification result;

[0013] Determine whether the first index is within at least one preset interval;

[0014] If not, obtain the analysis result of the training data corresponding to the first index, and update the current feature classification model based on the analysis result and the training data;

[0015] If so, obtain the training data corresponding to the second index as candidate training data; wherein, the candidate training data corresponds to the at least one preset interval;

[0016] Until the evaluation index of the current feature classification model meets the requirements of the preset evaluation index, update the current feature classification model based on the candidate training data to obtain the final feature classification model.

[0017] Through the above method, during the training and updating process of the feature classification model, two preconditions for the end of training are set, that is, the first index falls within the preset interval and the evaluation index of the feature classification model meets the preset conditions. The feature classification model trained in this way can help improve the efficiency and accuracy of extracting the first feature.

[0018] In a possible design, the obtaining the analysis result of the training data corresponding to the first index includes:

[0019] Send the training data corresponding to the first index to an external data analysis resource as query data;

[0020] Receive the analysis result of the query data from the external data analysis resource.

[0021] Through the above method, during the training and updating process of the feature classification model, an active learning strategy is proposed, that is, updating based on the analysis result sent externally.

[0022] More specifically, during the model training process of the feature classification model, as long as the current prediction result does not meet the training requirements, the analysis result of the sample data with incorrect predictions is obtained, and then based on the analysis result and the original sample data, the model training is carried out again, that is, the annotation information of the sample data is continuously optimized. Through this active learning strategy, a relatively accurate feature classification model can be trained based on a small amount of sample data. In practical applications, this strategy can be applied to the scenario of training a feature classification model with few sample data, which not only saves the time for annotating the data to be detected to generate sample data, but also improves the feature classification accuracy of the finally trained feature classification model.

[0023] In a possible design, until the evaluation index of the current feature classification model meets the preset evaluation index requirements, updating the current feature classification model based on the candidate training data to obtain the final feature classification model includes:

[0024] Until the evaluation index of the current feature classification model meets the preset evaluation index requirements, determine whether each preset interval in the at least one preset interval corresponds to candidate training data;

[0025] If so, perform weighted fusion on the candidate training data corresponding to each preset interval to obtain fused training data;

[0026] Based on the fused training data, update the current feature classification model to obtain the final feature classification model.

[0027] Through the above method, the final feature classification model will be obtained by weighted fusion of the candidate training data corresponding to each preset interval, and then the feature classification model is called to perform feature classification on the data to be detected to obtain the first feature of the data to be detected, which can effectively improve the efficiency of obtaining the first feature and the accuracy of the first feature.

[0028] In a possible design, the security detection model is obtained based on the following operations:

[0029] Obtain the multi-dimensional features of the training data; wherein, the multi-dimensional features include time series features, traffic features, keyword features, 5G protocol features, and 5G domain features;

[0030] Call the current security detection model to perform weighted fusion on the multi-dimensional features to obtain fused features;

[0031] Based on the fused features, update the current security detection model to obtain the final security detection model.

[0032] Optionally, a standard rule template can also be constructed to further retrieve and match the above abnormal data to obtain the final abnormal data, so as to improve the detection accuracy.

[0033] Optionally, in the training process of the security detection model, the early stopping method strategy is introduced, that is, after each iteration training is completed, the model evaluation index of the current security detection model is calculated. If the model evaluation index does not increase significantly for n consecutive iterations, the model training process is terminated. Based on this, the model training is prevented from falling into overfitting.

[0034] Through the above method, the finally trained security detection model takes into account the effective features such as the characteristics of the 5G field, and the security detection model is obtained based on the fusion features that fuse the effective features. Therefore, the security detection model trained by the above method can help improve the accuracy of finally detecting abnormal data.

[0035] In a possible design, after taking the data with an abnormal index greater than a preset threshold in the to-be-detected data as abnormal data, it further includes:

[0036] Determine the abnormal category of the abnormal data based on the abnormal category to which the abnormal index corresponding to the first feature and / or the second feature belongs;

[0037] Generate a security report for the to-be-detected data based on the abnormal data and the abnormal category of the abnormal data.

[0038] Through the above method, the abnormal category of the abnormal data is determined by the abnormal index corresponding to the feature. Based on this, a security report including the abnormal data and the abnormal category is generated.

[0039] In a possible design, after taking the data with an abnormal index greater than a preset threshold in the to-be-detected data as abnormal data, it further includes:

[0040] Save the abnormal data in the database;

[0041] In response to receiving a display instruction for displaying the abnormal data, extract the abnormal data from the database and send the abnormal data to the display end.

[0042] Through the above method, the storage and visualization of the detected abnormal data are realized.

[0043] In a second aspect, the present application provides a device for detecting abnormal data, and the device includes:

[0044] A feature classification module that calls a feature classification model to classify the features of the to-be-detected data to obtain the first feature of the to-be-detected data; and / or

[0045] A feature extraction module that extracts features from at least a part of the data to be detected based on a feature extraction method, and obtains second features of at least a part of the data to be detected;

[0046] A security detection module that calls a security detection model to detect the first feature and / or the second feature, and obtains an anomaly index corresponding to each of the first feature and / or the second feature;

[0047] An abnormal data determination module that uses the data with an anomaly index greater than a preset threshold in the data to be detected as abnormal data.

[0048] In a possible design, the feature classification model is obtained based on the following operations. The device is further configured to:

[0049] Perform feature classification on training data based on the current feature classification model to obtain a first index and a second index of the feature classification result; wherein, the first index is the inaccurate probability of the feature classification result, and the second index is the accurate probability of the feature classification result;

[0050] Determine whether the first index is within at least one preset interval;

[0051] If not, obtain the analysis result of the training data corresponding to the first index, and update the current feature classification model based on the analysis result and the training data;

[0052] If so, obtain the training data corresponding to the second index as candidate training data; wherein, the candidate training data corresponds to the at least one preset interval;

[0053] Until the evaluation index of the current feature classification model meets the requirements of the preset evaluation index, update the current feature classification model based on the candidate training data to obtain the final feature classification model.

[0054] In a possible design, for obtaining the analysis result of the training data corresponding to the first index, the device is further configured to:

[0055] Send the training data corresponding to the first index to an external data analysis resource as query data;

[0056] Receive the analysis result of the query data from the external data analysis resource.

[0057] In a possible design, the device is further configured to:

[0058] Until the evaluation index of the current feature classification model meets the requirements of the preset evaluation index, determine whether each preset interval in the at least one preset interval corresponds to candidate training data;

[0059] If so, perform weighted fusion on the candidate training data corresponding to each preset interval to obtain fused training data;

[0060] Update the current feature classification model based on the fused training data to obtain the final feature classification model.

[0061] In a possible design, until the evaluation index of the current feature classification model meets the requirements of the preset evaluation index, update the current feature classification model based on the candidate training data to obtain the final feature classification model. The apparatus is further configured to:

[0062] Obtain the multi-dimensional features of the training data; wherein, the multi-dimensional features include time series features, traffic features, keyword features, 5G protocol features, and 5G domain features;

[0063] Call the current security detection model to perform weighted fusion on the multi-dimensional features to obtain fused features;

[0064] Update the current security detection model based on the fused features to obtain the final security detection model.

[0065] In a possible design, after taking the data in the to-be-detected data with an anomaly index greater than the preset threshold as anomaly data, the anomaly data determination module is further configured to:

[0066] Determine the anomaly category of the anomaly data based on the anomaly category to which the anomaly index corresponding to the first feature and / or the second feature belongs;

[0067] Generate a security report for the to-be-detected data based on the anomaly data and the anomaly category of the anomaly data.

[0068] In a possible design, after taking the data in the to-be-detected data with an anomaly index greater than the preset threshold as anomaly data, the anomaly data determination module is further configured to:

[0069] Save the anomaly data in a database;

[0070] In response to receiving a display instruction for displaying the anomaly data, extract the anomaly data from the database and send the anomaly data to a display end.

[0071] In a third aspect, the present application provides an electronic device, and the electronic device includes:

[0072] A memory for storing computer programs;

[0073] A processor, when executing the computer programs stored on the memory, implements the method steps of a method for detecting abnormal data as described above.

[0074] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method steps of a method for detecting abnormal data as described above are implemented.

[0075] For the aspects in the second to fourth aspects above and the possible technical effects that each aspect may achieve, please refer to the description of the possible technical effects that can be achieved for the first aspect or various possible solutions in the first aspect above, and will not be repeated here. Description of the Drawings

[0076] Figure 1 The first schematic diagram of a method for detecting abnormal data provided by the present application;

[0077] Figure 2 The flowchart of a method for detecting abnormal data provided by the present application;

[0078] Figure 3 The flowchart of a method for training a feature classification model provided by the present application;

[0079] Figure 4 The first schematic diagram of training a feature classification model using an active learning strategy provided by the present application;

[0080] Figure 5 The schematic diagram of the training process of a feature classification model provided by the present application;

[0081] Figure 6 The schematic diagram of an active learning strategy provided by the present application;

[0082] Figure 7 The second schematic diagram of training a feature classification model using an active learning strategy provided by the present application;

[0083] Figure 8 The flowchart of a method for training a security detection model provided by the present application;

[0084] Figure 9 The schematic diagram of training a security detection model provided by the present application;

[0085] Figure 10 The schematic diagram of security report visualization provided by the present application;

[0086] Figure 11 The second schematic diagram of a method for detecting abnormal data provided by this application;

[0087] Figure 12 The schematic diagram of a device for transmitting intersection data provided by this application;

[0088] Figure 13 The schematic diagram of the structure of an electronic device provided by this application. Detailed implementation manners

[0089] In order to make the objectives, technical solutions and advantages of this application clearer, the following will further describe this application in detail with reference to the accompanying drawings. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments.

[0090] In the description of this application, "a plurality of" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The connection between A and B may represent: the direct connection between A and B and the connection between A and B through C. In addition, in the description of this application, terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0091] The embodiments of this application provide a method, a device and an electronic device for detecting abnormal data, which solve the problem of low detection efficiency in the prior art when detecting a large amount of traffic data.

[0092] It should be noted that the technical solutions provided by the embodiments of this application can be applied to the security detection of 5GC.

[0093] As Figure 1 shown, according to the method provided by the embodiments of this application, a feature classification model is called to classify the features of the data to be detected, and the first features of the data to be detected are obtained. Optionally, at least a part of the data to be detected is subjected to feature extraction based on a feature extraction method, and the second features of at least a part of the data to be detected are obtained. Then, a security detection model is called to detect the first features and / or the second features, and the abnormal indexes corresponding to the first features and / or the second features are obtained. Then, the data with an abnormal index greater than a preset threshold in the data to be detected is used as abnormal data, and a detection result is obtained based on the abnormal data. Through the above method, real-time automatic detection of 5GC large traffic data can be realized, and the detection efficiency can be effectively improved.

[0094] Specifically, the anomaly indices corresponding to the above-mentioned first feature and / or second feature may include: the anomaly index corresponding to the first feature, the feature index corresponding to the second feature, and the anomaly index corresponding to the fused first feature and second feature; the above-mentioned preset threshold may be a value set according to the actual situation, or may also be a value obtained based on the training process of the security detection model.

[0095] Furthermore, the technical features included in the embodiments of the present application can be combined and used arbitrarily. Those skilled in the art should understand that, starting from the actual application situation, the technical solutions obtained by reasonably combining the technical features in the embodiments of the present application can also solve the same technical problems or achieve the same technical effects.

[0096] The following further elaborates in detail on the method provided by the embodiments of the present application with reference to the accompanying drawings.

[0097] Refer to Figure 2 As shown, the embodiments of the present application provide a method for detecting abnormal data, and the specific process is as follows:

[0098] Step 201: Invoke a feature classification model to classify the features of the data to be detected, and obtain the first feature of the data to be detected;

[0099] In the embodiments of the present application, the feature classification model will be used to predict the first feature of the data to be detected. Based on this, the feature classification accuracy of the feature classification model will directly affect the accuracy of the finally obtained first feature. In order to improve the feature classification accuracy of the feature classification model, a training method for the feature classification model is proposed here, and the following further elaborates in detail with reference to the accompanying drawings.

[0100] As Figure 3 shown, it is a training method for a feature classification model, and the specific process is as follows:

[0101] Step 301: Based on the current feature classification model, classify the features of the training data, and obtain the first index and the second index of the feature classification result;

[0102] Among them, the first index is the inaccurate probability of the feature classification result, and the second index is the accurate probability of the feature classification result.

[0103] Step 302: Determine whether the first index is within at least one preset interval;

[0104] In the embodiments of the present application, if not, then execute step 303; if so, then execute step 304.

[0105] Step 303: Obtain the analysis result of the training data corresponding to the first index, and update the current feature classification model based on the analysis result and the training data;

[0106] Step 304: Obtain the training data corresponding to the second metric as candidate training data;

[0107] Among them, the candidate training data corresponds to at least one preset interval, and the preset interval can be set according to the actual application situation.

[0108] Step 305: Until the evaluation metric of the current feature classification model meets the requirements of the preset evaluation metric, update the current feature classification model based on the candidate training data to obtain the final feature classification model.

[0109] The feature classification model trained based on the above method can help improve the efficiency and accuracy of extracting the first feature.

[0110] In some possible implementation manners, an active learning strategy can be adopted to train the feature classification model, as specifically shown in Figure 4 shown.

[0111] In Figure 4 , first, select a part of the data to be detected from the data to be detected, and then label the part of the data to be detected based on the labeling method. The labeling method can be some manual labeling methods or some labeling methods based on labeling software, and save the labeled data to be detected as sample data in the sample database. Then, extract at least a part of the sample data from the sample database for model training of the feature classification model. After the training is completed, the prediction result of the feature classification model is obtained. By comparing the prediction result with the annotation information of the sample data, the first metric with inaccurate prediction and the second metric with accurate prediction can be obtained. If the first metric does not fall within the preset interval, it is considered that the current prediction result does not meet the training requirements, and the sample data with incorrect prediction is sent as query data to the external data analysis resource. Then, receive the analysis result of the query data from the external data analysis resource, add the analysis result to the sample database, and repeat the process of model training based on the analysis result and other sample data in the sample database. Until the first metric falls within the preset interval and the evaluation metric of the current feature classification model meets the requirements of the preset evaluation metric, it is considered that the current prediction result meets the training requirements, and the model training is completed to obtain the final feature classification model.

[0112] Furthermore, after obtaining the final feature classification model, input the data to be detected into the feature classification model for feature classification, and the first feature of the data to be detected can be obtained.

[0113] It should be noted that, in order to further improve the feature classification accuracy of the final feature classification model, the above preset interval can also be set to multiple. Taking the setting of 3 preset intervals as an example, as specifically shown in Figure 5 shown.

[0114] In Figure 5 In , the sample data in the sample database is labeled data. Model training is performed based on the sample database to obtain the first indicator of the training result, which is the probability of prediction error. If the first indicator is not within any preset interval, the model is retrained using an active learning strategy. If the first indicator is within the preset interval, the correctly predicted data is output as the candidate data set.

[0115] Specifically, the preset interval can be set according to the actual situation. Here, the first preset interval is set to The second preset interval is The third preset interval is Wherein, C is the marked data, a, b, c, d, e, f are constants between 0 and 1, and the initial values ​​of a, b, c, d, e, f are engineering experience parameters.

[0116] In Figure 5 , if the first indicator is within the first preset interval, then the candidate data set U is output 1 ; If the first indicator is within the second preset interval, then output the candidate data set U 2 ; If the first indicator is within the third preset interval, then output the candidate data set U 3 . After obtaining the candidate data set U 1 、U 2 、U 3 After that, to U 1 、U 2 、U 3 Perform weighted fusion to obtain the final candidate data set: aU1+bU2+cU3. Then, train the model based on the final candidate data set aU1+bU2+cU to obtain the final feature classification model.

[0117] In summary, during the model training process of the feature classification model, as long as the current prediction result does not meet the training requirements, the analysis results of the sample data with incorrect predictions are obtained, and then the model training is performed based on the analysis results and the original sample data, that is, the labeling information of the sample data is continuously optimized. Through this active learning strategy, a more accurate feature classification model can be trained based on a small amount of sample data. In practical applications, this strategy can be applied to scenarios where feature classification models are trained with a small amount of sample data, which not only saves the time of labeling the data to be tested to generate sample data, but also improves the feature classification accuracy of the feature classification model finally trained.

[0118] Optionally, the above feature classification model can specifically be a SOM (Self-organizing Maps) neural network model. The training process of the above feature classification model can be understood as the training process of the SOM neural network model using an active learning strategy.

[0119] As Figure 6 shown, some sample data in the original dataset W is manually labeled to obtain the labeled dataset T. Then, a model is trained based on the labeled dataset T to obtain a SOM neural network model. Next, through training and aggregation of the SOM neural network model, the trained dataset W-T is obtained. Sample selection is performed on the trained dataset W-T, and the analysis results of the selected samples are obtained from external data analysis resources. Then, the analysis results are added to the labeled dataset T. Then, the SOM neural network model is trained again based on the labeled dataset T with added information.

[0120] Taking the SOM neural network model as an example below, the training process of the above feature classification model is described as follows by way of example. Specifically, refer to Figure 7 shown.

[0121] In Figure 7 , at least a part of the data in the original dataset W is labeled as the labeled sample T. The current SOM neural network model is called to perform feature classification on the labeled sample T to obtain a first index of inaccurate prediction, and it is judged whether the first index falls within a preset interval. For example, if the first index falls within (1 / C - a, 1 / C - b), then the determination result of SOM(1 / C - a, 1 / C - b) is yes, and the labeled sample A with accurate prediction is output. Based on the same idea, the labeled sample A, the labeled sample B, and the labeled sample C are obtained. Then, it is judged whether the current SOM neural network model meets the evaluation index requirements, and the evaluation index requirements are determined based on the actual situation. The evaluation index can be precision, recall rate, accuracy, etc.

[0122] Furthermore, if it is judged that the evaluation index requirements are not met, the current SOM neural network model is called to perform feature classification on the labeled sample A, the labeled sample B, or the labeled sample C until it is judged that the evaluation index requirements are met, and the output samples U 1 , output sample U 2 and output sample U 3 are output. Then, output sample U 1 , U 2 and U 3Perform weighted fusion to obtain aU1+bU2+cU3, then call the current SOM neural network model to perform feature classification, obtain the first indicator of inaccurate prediction, and determine whether the first indicator falls into any preset interval. For example, if the first indicator falls into (1 / Ca, 1 / Cb), [(1 / Ca, 1 / c)u(1 / Cb, 1 / d)] or [(0, 1 / e)u(1 / f, 1)], then the determination result of SOM((1 / Ca, 1 / Cb), [(1 / Ca, 1 / c)u(1 / Cb, 1 / d)], [(0, 1 / e)u(1 / f, 1)]) is yes, and output the accurately predicted labeled sample D. Then determine whether the current SOM neural network model meets the evaluation index requirements: if not, call the current SOM neural network model to perform feature classification on the labeled sample D until it is determined that the evaluation index requirements are met, and output the output sample U that is accurately predicted every time, and the output sample U includes the feature classification result of the sample data. At this point, the model training process is completed, and the current SOM neural network model is used as the final feature classification model.

[0123] Based on an active learning strategy for the first indicator provided in the embodiment of the present application, combined with the SOM neural network model for model training, the final feature classification model is obtained, and the feature classification model can predict more accurate feature information.

[0124] In summary, the feature classification model is trained by the above method, and then the feature classification model is called to perform feature classification on the data to be detected to obtain the first feature of the data to be detected, which can effectively improve the efficiency of obtaining the first feature and improve the accuracy of the first feature.

[0125] Step 202: extracting features from at least a portion of the data to be detected based on a feature extraction method to obtain a second feature of at least a portion of the data to be detected;

[0126] In order to improve the efficiency of feature extraction in actual application scenarios, the embodiment of the present application can also use artificial feature extraction methods to extract some special features based on the use of feature classification models.

[0127] For example, in the application scenarios of 5GC, some 5G-related information will be involved, such as 5G protocol information, 5G domain information, etc. Taking 5G domain information as an example, the 5G domain involves industrial fields, vehicle networks and autonomous driving, energy fields, education fields, medical fields, cultural and tourism fields, smart city fields, information consumption fields, financial fields, etc. These 5G domain information are not necessarily structured information, and the update and iteration of 5G domain information are relatively fast. Although the feature classification model can also identify these 5G domain information, the inventor has found in the current actual application that if, on the basis of using the feature classification model, combined with the way of manual extraction, the efficiency of feature extraction can be further improved.

[0128] Therefore, by extracting at least a part of the second features in the data to be detected through the above feature extraction method, the efficiency of feature extraction can be further improved.

[0129] Step 203: Invoke the security detection model to detect the first feature and / or the second feature, and obtain the anomaly index corresponding to each of the first feature and / or the second feature;

[0130] In the embodiments of the present application, the security detection model will be used to detect abnormal data in the data to be detected. Specifically, the security detection model mainly predicts the anomaly index corresponding to each of the first feature and / or the second feature of the data to be detected, and then detects the abnormal data based on the anomaly index.

[0131] Optionally, a standard rule template can also be constructed to further retrieve and match the above abnormal data to obtain the final abnormal data, so as to improve the detection accuracy.

[0132] Furthermore, in order to improve the accuracy of detecting abnormal data, in the embodiments of the present application, a training method for the security detection model is also proposed, which will be further described in detail below with reference to the accompanying drawings.

[0133] As Figure 8 shown, it is a training method for a security detection model, and the specific process is as follows:

[0134] Step 801: Obtain the multi-dimensional features of the training data;

[0135] In the embodiments of the present application, if applied to the 5GC field, the training data may include the original data and labeled data of 5GC, and the multi-dimensional features may include the time series features, traffic features, keyword features, 5G protocol features, and 5G domain features of the original data.

[0136] Step 802: Invoke the current security detection model to perform weighted fusion on the multi-dimensional features to obtain the fusion features;

[0137] Step 803: Update the current security detection model based on the fusion features to obtain the final security detection model.

[0138] Regarding the security detection model trained by the above method, considering the effective features such as the characteristics of the 5G field, and the security detection model is obtained based on the fusion features that fuse the effective features. Therefore, the security detection model trained by the above method can help improve the accuracy of finally detecting abnormal data.

[0139] More specifically, the training process of the above security detection model can be referred to Figure 9 as shown.

[0140] In Figure 9 , the relevant features are the features extracted by the feature extraction method, and the classification features are the features predicted based on the feature classification model. Specifically, the relevant features may include time series features, traffic features, keyword features, domain features, and protocol features.

[0141] As Figure 9 shown, model training is performed based on the relevant features and classification features. During the model training process, feature fusion will also be performed on the time series features, traffic features, keyword features, domain features, protocol features, and classification features, and then a prediction result is obtained based on the fused features, and then it is judged whether the prediction result meets the training requirements: if not, the model training is iteratively performed; if so, the final security detection model is obtained.

[0142] In some embodiments, the above security detection model may be a Conditional Random Field (CRF) model, a Bidirectional Long Short Term Memory-Conditional Random Field (BLSTM-CRF) model (sequence annotation-entity recognition model), a Hidden Markov Model (HMM), a Maximum Entropy Markov Model (MEMM), an SVM, etc.

[0143] In some embodiments, an early stopping strategy is also introduced during the model training process, that is, every time an iterative training is completed, the model evaluation index of the current security detection model is calculated. If the model evaluation index does not increase significantly after n consecutive iterations, the model training process is ended.

[0144] It should be noted that the model evaluation index may include accuracy, recall rate, and F1 value. The application scenarios of the model evaluation index will be briefly described below in combination with the scenario of detecting 5GC logs.

[0145] Scenarios applicable to accuracy rate include: when it is determined that all core network logs are alarm logs and it is necessary to determine the corresponding alarm types, the accuracy rate can be used as a judgment metric.

[0146] Scenarios applicable to recall rate include: when there are traffic logs (i.e., non-alarm logs) accompanying the alarm logs, and at this time it is necessary to identify as many alarm types as possible. Considering network security, the recall rate can be used as a judgment metric.

[0147] Scenarios applicable to F1 value include: The F1 value comprehensively considers all scenarios and tries to identify more alarms to the greatest extent while improving the recognition accuracy rate, and can be applied to daily traffic monitoring and judgment.

[0148] More specifically, the F1 value can be calculated based on the accuracy rate and the recall rate. For details, please refer to the following formula.

[0149]

[0150] As shown in the above formula, where P is the accuracy rate; recall is the recall rate. Specifically, the accuracy rate and the recall rate can be calculated based on the following formula.

[0151]

[0152]

[0153] Furthermore, the basis for determining that the F1 value does not increase significantly after continuous iteration for n times is: the increase granularity of the F1 value does not exceed a preset threshold. Among them, n can be set according to actual engineering experience, and generally n is set to 10; the increase granularity represents the difference in the F1 value between this iteration and the previous iteration; the preset threshold can be set according to actual engineering experience, and usually the preset threshold is set to 0.01%.

[0154] In the embodiments of the present application, by introducing the earlyStoping strategy, it is possible to prevent the model training from falling into an overfitting state.

[0155] It should be noted that those skilled in the art should know that the earlyStoping strategy is a possible implementation manner provided by the embodiments of the present application. Other strategies can also be adopted in the above training process, for example, Adam strategy, RMSprop strategy, BatchNormalization strategy, Dropout strategy, etc.

[0156] In summary, by training the security detection model through the above method and then calling the security detection model to detect the first feature and / or the second feature, the anomaly index corresponding to each of the first feature and / or the second feature is obtained, which can effectively improve the accuracy of the anomaly index.

[0157] Step 204: Use the data in the data to be detected that has an anomaly index greater than the preset threshold as anomaly data.

[0158] In the embodiments of the present application, the anomaly index can be used to represent the probability of the anomaly category corresponding to the first feature and / or the second feature. The higher the anomaly index, the higher the probability that the corresponding data is an anomaly category. Here, the data with an anomaly index greater than the preset threshold is used as anomaly data. Taking a single feature as an example, the following is a specific description.

[0159] For a single feature, if the probability of this feature corresponding to the first anomaly category is P1, the probability of corresponding to the first anomaly category is P2, and the probability of the first anomaly category is P3, then compare the sizes of P1, P2, and P3 with the preset threshold respectively. If the comparison results show that P1 and P2 are greater than the preset threshold, then it can be considered that the data corresponding to this feature is anomaly data, and the anomaly categories of this anomaly data are the first anomaly category and the second anomaly category. If there is no probability greater than the preset threshold in the comparison results, then it can be considered that the data corresponding to this feature is safe data.

[0160] In some embodiments, for a single feature, if the probability of this feature corresponding to the first anomaly category is P1, the probability of corresponding to the first anomaly category is P2, and the probability of the first anomaly category is P3, and P2 > P1 > P3, then select P2 and compare the size relationship between P2 and the preset threshold. If P2 is greater than the preset threshold, then it can be considered that the data corresponding to this feature is anomaly data, and the anomaly category of this anomaly data is the second anomaly category.

[0161] Furthermore, after determining the anomaly data and the anomaly category of the anomaly data, a security report of the data to be detected can also be generated based on the anomaly data and the anomaly category of the anomaly data.

[0162] Exemplarily, the security report may include the following content:

[0163] public static final int RES_API_INVALID_URL = 2 URL is illegal;

[0164] public static final int RES_API_INVALID_URL_PARAM_TYPE = 6 Illegal request;

[0165] Among them, "public static final int RES_API_INVALID_URL = 2" and "public static final int RES_API_INVALID_URL_PARAM_TYPE = 6" are abnormal data, and "Illegal URL" and "Illegal request" are abnormal categories.

[0166] Furthermore, the security report can also be saved in a database. After receiving a display instruction for displaying the security report, the security report is extracted from the database and sent to the display end for display.

[0167] As Figure 10 shown, on the display end, the visual display of the security report can include abnormal data and abnormal categories.

[0168] In some embodiments, in response to receiving a display instruction for displaying abnormal data, the abnormal data is extracted from the database and sent to the display end for display.

[0169] In summary, referring to Figure 11 shown, on the one hand, the embodiment of the present application adopts an active learning strategy to train a feature classification model, which can be applicable to the scenario of training a feature classification model with few-sample data, not only saving the time for annotating the data to be detected to generate sample data, but also improving the feature classification accuracy of the finally trained feature classification model; on the other hand, a feature fusion method is adopted to train a security detection model, which helps to improve the accuracy of the security detection model in detecting abnormal data.

[0170] Based on the same inventive concept, the present application also provides a device for detecting abnormal data to improve the efficiency of detecting abnormal data and solve the problem of low detection efficiency in the prior art when detecting a large amount of traffic data. Refer to Figure 12 and the device includes:

[0171] A feature classification module 1201 that calls a feature classification model to perform feature classification on the data to be detected to obtain a first feature of the data to be detected; and / or

[0172] A feature extraction module 1202 that performs feature extraction on at least a part of the data to be detected based on a feature extraction method to obtain a second feature of at least a part of the data to be detected;

[0173] A security detection module 1203 that calls a security detection model to detect the first feature and / or the second feature to obtain an abnormal index corresponding to each of the first feature and / or the second feature;

[0174] The abnormal data determination module 1204 determines the data with an abnormal index greater than a preset threshold in the data to be detected as abnormal data.

[0175] In a possible design, the feature classification model is obtained based on the following operations. The apparatus is further configured to:

[0176] Perform feature classification on the training data based on the current feature classification model to obtain a first metric and a second metric of the feature classification result; wherein, the first metric is the inaccurate probability of the feature classification result, and the second metric is the accurate probability of the feature classification result;

[0177] Determine whether the first metric is within at least one preset interval;

[0178] If not, obtain the analysis result of the training data corresponding to the first metric, and update the current feature classification model based on the analysis result and the training data;

[0179] If so, obtain the training data corresponding to the second metric as candidate training data; wherein, the candidate training data corresponds to the at least one preset interval;

[0180] Until the evaluation metric of the current feature classification model meets the requirements of the preset evaluation metric, update the current feature classification model based on the candidate training data to obtain the final feature classification model.

[0181] In a possible design, for obtaining the analysis result of the training data corresponding to the first metric, the apparatus is further configured to:

[0182] Send the training data corresponding to the first metric to an external data analysis resource as query data;

[0183] Receive the analysis result of the query data from the external data analysis resource.

[0184] In a possible design, the apparatus is further configured to:

[0185] Until the evaluation metric of the current feature classification model meets the requirements of the preset evaluation metric, determine whether each of the at least one preset interval corresponds to candidate training data;

[0186] If so, perform weighted fusion on the candidate training data corresponding to each of the preset intervals to obtain fused training data;

[0187] Update the current feature classification model based on the fused training data to obtain the final feature classification model.

[0188] In a possible design, until the evaluation index of the current feature classification model meets the preset evaluation index requirements, the current feature classification model is updated based on the candidate training data to obtain a final feature classification model. The apparatus is further configured to:

[0189] Obtain multi-dimensional features of training data; wherein, the multi-dimensional features include time series features, traffic features, keyword features, 5G protocol features, and 5G domain features;

[0190] Call the current security detection model to perform weighted fusion on the multi-dimensional features to obtain fused features;

[0191] Update the current security detection model based on the fused features to obtain a final security detection model.

[0192] In a possible design, after taking the data in the to-be-detected data with an anomaly index greater than a preset threshold as anomaly data, the anomaly data determination module 1204 is further configured to:

[0193] Determine the anomaly category of the anomaly data based on the anomaly category to which the anomaly index corresponding to the first feature and / or the second feature belongs;

[0194] Generate a security report for the to-be-detected data based on the anomaly data and the anomaly category of the anomaly data.

[0195] In a possible design, after taking the data in the to-be-detected data with an anomaly index greater than a preset threshold as anomaly data, the anomaly data determination module 1204 is further configured to:

[0196] Save the anomaly data in a database;

[0197] In response to receiving a display instruction for displaying the anomaly data, extract the anomaly data from the database and send the anomaly data to a display end.

[0198] Based on the above apparatus, on the one hand, an active learning strategy is adopted to train a feature classification model, which can be applicable to the scenario of training a feature classification model with few-sample data, not only saving the time for labeling to-be-detected data to generate sample data, but also improving the feature classification accuracy of the finally trained feature classification model; on the other hand, a feature fusion method is adopted to train a security detection model, which helps to improve the accuracy of the security detection model in detecting anomaly data.

[0199] Based on the same inventive concept, an electronic device is further provided in an embodiment of the present application. The electronic device can implement the functions of the foregoing apparatus for detecting anomaly data. Refer to Figure 13 , the electronic device includes:

[0200] At least one processor 1301, and a memory 1302 connected to the at least one processor 1301. In the embodiments of the present application, the specific connection medium between the processor 1301 and the memory 1302 is not limited. Figure 13 In the example, the processor 1301 and the memory 1302 are connected through a bus 1300. The bus 1300 is Figure 13 represented by a thick line in the figure. The connection manners between other components are only for illustrative purposes and are not limiting. The bus 1300 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 13 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 1301 can also be referred to as a controller, and the name is not limited.

[0201] In the embodiments of the present application, the memory 1302 stores instructions executable by the at least one processor 1301. By executing the instructions stored in the memory 1302, the at least one processor 1301 can execute the method for detecting abnormal data discussed above. The processor 1301 can implement Figure 12 the functions of each module in the device shown.

[0202] Among them, the processor 1301 is the control center of the device / system, and can connect various parts of the entire control device through various interfaces and lines. By running or executing the instructions stored in the memory 1302 and calling the data stored in the memory 1302, various functions of the device / system and process data, so as to monitor the device / system as a whole.

[0203] In a possible design, the processor 1301 may include one or more processing units. The processor 1301 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above modem processor may not be integrated into the processor 1301. In some embodiments, the processor 1301 and the memory 1302 can be implemented on the same chip, and in some embodiments, they can also be separately implemented on independent chips.

[0204] The processor 1301 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method for detecting abnormal data disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0205] The memory 1302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 1302 may include at least one type of storage medium, for example, it may include flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disc, and so on. The memory 1302 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1302 in the embodiments of the present application may also be a circuit or any other device / system capable of implementing a storage function, for storing program instructions and / or data.

[0206] By designing and programming the processor 1301, the code corresponding to the method for detecting abnormal data introduced in the foregoing embodiments can be solidified into the chip, so that the chip can execute Figure 2 the steps of the method for detecting abnormal data in the embodiments shown. How to design and program the processor 1301 is a well-known technology to those skilled in the art and will not be elaborated here.

[0207] Based on the same inventive concept, the embodiments of the present application further provide a storage medium that stores computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the method for detecting abnormal data discussed above.

[0208] In some possible embodiments, various aspects of the method for detecting abnormal data provided by this application can also be implemented in the form of a program product, which includes program code. When the program product runs on a device, the program code is used to cause the control device to execute the steps in the method for detecting abnormal data according to various exemplary embodiments of this application described above in this specification.

[0209] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a device / system, or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0210] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0211] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0213] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to cover these changes and modifications.

Claims

1. A method for detecting abnormal data, characterized in that: The method comprises: Calling a feature classification model to perform feature classification on the data to be detected to obtain a first feature of the data to be detected; and / or Performing feature extraction on at least a portion of the data to be detected based on a feature extraction method to obtain a second feature of at least a portion of the data to be detected; Calling a security detection model to detect the first feature and / or the second feature, and obtaining an abnormality index corresponding to the first feature and / or the second feature respectively; The data to be detected having an abnormality index greater than a preset threshold is regarded as abnormal data; The feature classification model is obtained based on the following operations: Based on the current feature classification model, the training data is feature classified to obtain a first index and a second index of the feature classification result; wherein the first index is an inaccurate probability of the feature classification result, and the second index is an accurate probability of the feature classification result; Determining whether the first indicator is within at least one preset interval; If not, obtaining an analysis result of the training data corresponding to the first indicator, and updating the current feature classification model based on the analysis result and the training data; If yes, obtaining the training data corresponding to the second indicator as candidate training data; wherein the candidate training data corresponds to the at least one preset interval; Until the evaluation index of the current feature classification model meets the preset evaluation index requirement, the current feature classification model is updated based on the candidate training data to obtain the final feature classification model.

2. The method according to claim 1, characterized in that The obtaining the analysis result of the training data corresponding to the first indicator includes: Sending the training data corresponding to the first indicator to an external data analysis resource as query data; An analysis result of the query data is received from the external data analysis resource.

3. The method according to claim 1, characterized in that Until the evaluation index of the current feature classification model meets the preset evaluation index requirement, the current feature classification model is updated based on the candidate training data to obtain the final feature classification model, including: Until the evaluation index of the current feature classification model meets the preset evaluation index requirement, it is determined whether each preset interval in the at least one preset interval corresponds to candidate training data; If yes, weighted fusion is performed on the candidate training data corresponding to each of the preset intervals to obtain fused training data; Based on the fused training data, the current feature classification model is updated to obtain a final feature classification model.

4. The method according to any one of claims 1 to 3, characterized in that: The safety detection model is obtained based on the following operations: Acquire multidimensional features of training data; wherein the multidimensional features include time series features, traffic features, keyword features, 5G protocol features, and 5G domain features; Calling the current security detection model to perform weighted fusion on the multi-dimensional features to obtain fused features; The current security detection model is updated based on the fusion features to obtain a final security detection model.

5. The method according to claim 1, characterized in that After the data to be detected having an abnormality index greater than a preset threshold is taken as abnormal data, the method further includes: Determine the abnormal category of the abnormal data based on the abnormal category to which the abnormal index corresponding to each of the first feature and / or the second feature belongs; A safety report of the data to be detected is generated based on the abnormal data and the abnormal category of the abnormal data.

6. The method according to claim 1, characterized in that After the data to be detected having an abnormality index greater than a preset threshold is taken as abnormal data, the method further includes: Storing the abnormal data in a database; In response to receiving a display instruction for displaying the abnormal data, the abnormal data is extracted from the database, and the abnormal data is sent to a display terminal.

7. A device for detecting abnormal data, characterized in that: The device comprises: A feature classification module calls a feature classification model to perform feature classification on the data to be detected to obtain a first feature of the data to be detected; and / or A feature extraction module, which extracts features from at least a portion of the data to be detected based on a feature extraction method to obtain a second feature of at least a portion of the data to be detected; A security detection module calls a security detection model to detect the first feature and / or the second feature to obtain an abnormality index corresponding to the first feature and / or the second feature; An abnormal data determination module is used to determine data with an abnormal index greater than a preset threshold in the data to be detected as abnormal data; The feature classification model is obtained based on the following operations, and the device is also used for: Based on the current feature classification model, the training data is feature classified to obtain a first index and a second index of the feature classification result; wherein the first index is an inaccurate probability of the feature classification result, and the second index is an accurate probability of the feature classification result; Determining whether the first indicator is within at least one preset interval; If not, obtaining an analysis result of the training data corresponding to the first indicator, and updating the current feature classification model based on the analysis result and the training data; If yes, obtaining the training data corresponding to the second indicator as candidate training data; wherein the candidate training data corresponds to the at least one preset interval; Until the evaluation index of the current feature classification model meets the preset evaluation index requirement, the current feature classification model is updated based on the candidate training data to obtain the final feature classification model.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method steps of any one of claims 1 to 6 when executing the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.

Images