A data processing method, device and computer readable storage medium

Abstract

Embodiments of the present application disclose a data processing method, the method comprising: obtaining first training data; performing model training based on the first training data to obtain a target clustering model and a plurality of target classification models by using a first metric function and a second metric function; wherein the first metric function and the second metric function are both used to measure the distance between target training data; performing clustering processing on to-be-processed data based on the target clustering model to obtain a class cluster to which the to-be-processed data belongs; and performing classification processing on the to-be-processed data based on a target classification model corresponding to the class cluster to which the to-be-processed data belongs to obtain a classification result of the to-be-processed data. Embodiments of the present application also disclose a data processing device and a computer readable storage medium.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data processing method, apparatus and computer-readable storage medium. Background Technology

[0002] With the rapid development of computer network technology, effectively ensuring network security has become a pressing issue. Currently, the general approach is to input the data to be detected into a Support Vector Machine (SVM) model to determine whether the data is normal or abnormal. If the data is normal, it is output directly; if it is abnormal, it is input into a Backpropagation Neural Network (BPNN) model to determine the abnormality category. However, this method of determining whether data is abnormal suffers from low detection accuracy. Summary of the Invention

[0003] To address the aforementioned technical problems, this application aims to provide a data processing method, apparatus, and computer-readable storage medium, thereby resolving the issue of low detection accuracy in related technologies and improving the accuracy of data detection.

[0004] The technical solution of this application is implemented as follows:

[0005] A data processing method, the method comprising:

[0006] Obtain the first training data;

[0007] Using a first metric function and a second metric function, a model is trained based on the first training data to obtain a target clustering model and multiple target classification models; wherein, both the first metric function and the second metric function are used to measure the distance between the target training data.

[0008] Based on the target clustering model, the data to be processed is clustered to obtain the cluster to which the data to be processed belongs;

[0009] The data to be processed is classified based on the target classification model corresponding to the cluster to which the data to be processed belongs, and the classification result of the data to be processed is obtained.

[0010] In the above scheme, the step of using a first metric function and a second metric function to train the model based on the first training data to obtain a target clustering model and multiple target classification models includes:

[0011] Determine the neighborhood parameters, the initial clustering model, and multiple initial classification models;

[0012] Using the first metric function, the second metric function, and the neighborhood parameters, the initial clustering model and the plurality of initial classification models are trained based on the first training data to obtain the target clustering model and the plurality of target classification models.

[0013] In the above scheme, the step of using the first metric function, the second metric function, and the neighborhood parameters to train the initial clustering model and the plurality of initial classification models based on the first training data to obtain the target clustering model and the plurality of target classification models includes:

[0014] Using the first metric function and the neighborhood parameters, the initial clustering model is trained based on the first training data to obtain a first clustering model and multiple first cluster data;

[0015] Based on the data of the multiple first-class clusters, the multiple initial classification models are trained to obtain multiple first-class classification models;

[0016] Based on the model parameters of the multiple first classification models, the second metric function is updated to obtain the first target metric function;

[0017] Using the first target metric function and the neighborhood parameters, the first clustering model is trained on the first training data to obtain a second clustering model and multiple second cluster data. Then, the multiple first classification models are trained on the multiple second cluster data respectively until the target clustering model and the multiple target classification models are obtained. The metric function corresponding to the nth clustering model other than the initial clustering model is the second metric function. The second metric function corresponding to the nth clustering model is determined based on the model parameters of the multiple (n-1)th classification models, where n is a positive integer.

[0018] In the above scheme, the step of using the first metric function and the neighborhood parameters to train the initial clustering model based on the first training data to obtain the first clustering model and multiple first cluster data includes:

[0019] The first training data is divided into two parts to obtain the second training data.

[0020] Using the first metric function and the neighborhood parameters, the initial clustering model is trained based on the second training data to obtain the first clustering model and multiple target cluster data;

[0021] Clustering is performed on the edge training data in the multiple target cluster data to obtain the multiple first cluster data.

[0022] In the above scheme, the step of clustering based on the edge training data in the multiple target cluster data to obtain the multiple first cluster data includes:

[0023] Identify the edge training data in the plurality of target cluster data;

[0024] The distance between the edge training data is obtained using the first metric function;

[0025] The multiple target cluster data are clustered based on the distance between the edge data and the neighborhood radius to obtain the multiple first cluster data.

[0026] In the above scheme, updating the second metric function based on the model parameters of the plurality of first classification models to obtain the first target metric function includes:

[0027] Based on the accuracy of the plurality of first classification models, the classification model to be processed is determined from the plurality of first classification models;

[0028] Based on the model parameters of the classification model to be processed, the second metric function is updated to obtain the first target metric function.

[0029] In the above scheme, the step of training the multiple first classification models based on the multiple second-category cluster data until the target clustering model and the multiple target classification models are obtained includes:

[0030] Based on the data of the multiple second-class clusters, the multiple first-class classification models are trained to obtain multiple second-class classification models;

[0031] Based on the model parameters of the multiple second classification models, the first target metric function is updated to obtain the second target metric function;

[0032] Using the second target metric function and the neighborhood parameters, the second clustering model is trained on the first training data until the target conditions are met, so as to obtain the target clustering model and the multiple target classification models.

[0033] In the above scheme, the target clustering model includes a density-based spatial clustering DBSCAN model with noise application, and the multiple target classification models include multiple support vector machine (SVM) models.

[0034] A data processing device, the device comprising: a processor, a memory, and a communication bus;

[0035] The communication bus is used to realize the communication connection between the processor and the memory;

[0036] The processor is used to execute a data processing program in the memory to implement the steps of the data processing method described above.

[0037] A computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the data processing method described above.

[0038] The data processing method, apparatus, and computer-readable storage medium provided in the embodiments of this application can employ a first metric function and a second metric function to train a model based on the acquired first training data to obtain a target clustering model and multiple target classification models. The target clustering model is then used to cluster the data to be processed to obtain the cluster to which the data belongs. Subsequently, the target classification model corresponding to the cluster to which the data belongs classifies the data to obtain the classification result. Thus, the target clustering model and multiple target classification models trained through the first and second metric functions are not affected by noise when classifying and clustering data, resulting in higher accuracy of the classification results determined by the target clustering model and multiple target classification models. This solves the problem of low detection accuracy in related technologies and improves the detection accuracy of data. Attached Figure Description

[0039] Figure 1 A flowchart illustrating a data processing method provided for an embodiment of this application;

[0040] Figure 2 A flowchart illustrating another data processing method provided for an embodiment of this application;

[0041] Figure 3 A schematic flowchart illustrating yet another data processing method provided in an embodiment of this application;

[0042] Figure 4 A flowchart illustrating a data processing method provided for another embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the structure of a data processing device provided for an embodiment of this application. Detailed Implementation

[0044] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0045] It should be understood that the phrases "embodiments of this application" or "foreign embodiments" throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "embodiments of this application" or "in the foreign embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0046] Unless otherwise specified, any step in the embodiments of this application performed by the electronic device may be executed by the processor of the electronic device. It is also worth noting that the embodiments of this application do not limit the order in which the electronic device performs the following steps. Furthermore, the methods used to process data in different embodiments may be the same or different methods. It should also be noted that any step in the embodiments of this application can be executed independently by the electronic device; that is, when the electronic device performs any step in the following embodiments, it may not depend on the execution of other steps.

[0047] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0048] This application provides a data processing method, which can be applied to a data processing device, as described above. Figure 1 As shown, the method includes the following steps:

[0049] Step 101: Obtain the first training data.

[0050] In this embodiment, the first training data may include network data of multiple different attack categories. The first training data may also carry labels to enable supervised training of the model based on these labels and to adjust the model parameters. After acquiring the first training data, the data processing device may preprocess it to improve its quality and the accuracy of the model obtained based on it. For example, after acquiring the training data, the intrusion detection device may perform preprocessing such as data cleaning, data fusion, and data association to improve the quality of the training data.

[0051] In this embodiment, after acquiring the first training data, the data processing device can use a portion of the first training data to train the model, thereby obtaining a target clustering model and multiple target classification models. It can also use another portion of the first training data to test the accuracy of the target clustering model and the multiple target classification models. The first training data can be represented as X(X1, X2, ..., X...). j ), X j =(X 1j X 2j , ..., X 1j ), where i is the dimension of the first training data and j is the number of the first training data.

[0052] Step 102: Using the first and second metric functions, train the model based on the first training data to obtain the target clustering model and multiple target classification models.

[0053] The first and second metric functions are both used to measure the distance between the target training data.

[0054] In this embodiment, the data processing device uses a first metric function and a second metric function to measure the correlation between target training data, which serves as the basis for clustering. It should be noted that when the data processing device trains a model based on the first training data, it treats each training data point in the first training data as a separate data point. Therefore, using the first and second metric functions to measure the distance between the first training data points is equivalent to using the first and second metric functions to measure the distance between data points.

[0055] Step 103: Based on the target clustering model, perform clustering processing on the data to be processed to obtain the cluster to which the data to be processed belongs.

[0056] In this embodiment of the application, a cluster represents a category. After the data processing device acquires the data to be processed, it performs clustering processing on the data to be processed based on the target clustering model to obtain the cluster to which the data to be processed belongs, so as to know which category the data to be processed belongs to, thereby determining the classification model corresponding to the data to be processed, and then determining the classification result of the data to be processed.

[0057] Step 104: Classify the data to be processed based on the target classification model corresponding to the cluster to which the data to be processed belongs, and obtain the classification result of the data to be processed.

[0058] In this embodiment, the data processing device classifies the data to be processed based on a target classification model corresponding to the cluster to which the data belongs, thereby improving the accuracy of the classification results. For example, the target classification model is trained on first training data carrying labels (the labels can be normal data or abnormal data). Based on this target classification model, it can be determined whether the data to be processed is abnormal or normal data. Thus, abnormal data can be processed promptly based on the classification results, ensuring network security.

[0059] The data processing method provided in this application embodiment is not affected by noise when classifying and clustering data in the target clustering model and multiple target classification models trained by the first metric function and the second metric function. This makes the classification results of the data determined by the target clustering model and multiple target classification models more accurate, solves the problem of low detection accuracy in related technologies, and improves the detection accuracy of data.

[0060] Based on the foregoing embodiments, this application provides a data processing method, referring to... Figure 2 As shown, the method includes the following steps:

[0061] Step 201: The data processing device acquires the first training data.

[0062] Step 202: The data processing equipment determines the neighborhood parameters, the initial clustering model, and multiple initial classification models.

[0063] In this embodiment, the neighborhood parameters may include neighborhood radius, quantity threshold, iteration stopping condition, number of iterations t, prediction accuracy p, etc. The neighborhood radius is the basis for clustering, determining which first training data belong to the same category and which belong to different categories. The quantity threshold is used to determine which first training data are core data. The iteration stopping condition determines whether the target clustering model and multiple target classification models have been trained. In one feasible implementation, the iteration stopping condition can be set to stop iteration when the number of iterations is greater than or equal to 50 and the prediction accuracy reaches 90%, thus obtaining the target clustering model and multiple target classification models.

[0064] Step 203: The data processing device uses the first metric function, the second metric function, and the neighborhood parameters to train the initial clustering model and multiple initial classification models based on the first training data, thereby obtaining the target clustering model and multiple target classification models.

[0065] In this embodiment, the target clustering model is a density-based spatial clustering of applications with noise (DBSCAN) model, and the multiple target classification models are SVM models. The data processing device can use a first metric function, a second metric function, and neighborhood parameters to train the model based on the first training data to obtain the DBSCAN model and multiple SVM models.

[0066] Step 203 can be achieved through the following:

[0067] Step 203a: The data processing device uses the first metric function and neighborhood parameters to train the initial clustering model based on the first training data to obtain the first clustering model and multiple first cluster data.

[0068] In this embodiment of the application, the first metric function may be a metric formula capable of determining the distance between the first training data: Where X1 and X2 are two initial training data points, and d is the distance between the initial training data points X1 and X2. During the first model training, the data processing device uses the aforementioned first metric function and neighborhood parameters to train the initial clustering model based on the initial training data, thereby obtaining the first clustering model and multiple first cluster data.

[0069] In the embodiments of this application, the first cluster data is the cluster data obtained by training the initial clustering model based on the first training data. One first cluster data is one category. Training the initial clustering model based on the first training data means dividing the first training data into multiple first cluster data, which means dividing the first training data into different categories.

[0070] Step 203b: The data processing device trains multiple initial classification models based on multiple first-class cluster data to obtain multiple first-class classification models.

[0071] In this embodiment, the data processing device trains multiple initial classification models based on multiple first-category cluster data obtained from the first training, which can make the trained multiple target classification models unaffected by noise and improve the detection accuracy of the data.

[0072] Step 203c: The data processing device updates the second metric function based on the model parameters of multiple first classification models to obtain the first target metric function.

[0073] In this embodiment, the second metric function uses the distance between the first training data and the hyperplane as the clustering basis, and is a metric formula determined based on the model parameters of multiple first classification models: Where ω and b are the model parameters of the classification model to be processed. The first objective metric function is the metric function updated by applying the second metric function based on the model parameters of multiple first classification models.

[0074] Step 203d: The data processing device uses the first target metric function and neighborhood parameters to train the first clustering model based on the first training data to obtain the second clustering model and multiple second cluster data. Then, it trains multiple first classification models based on the multiple second cluster data until the target clustering model and multiple target classification models are obtained, where n is a positive integer.

[0075] Among them, the metric function corresponding to the nth clustering model other than the initial clustering model is the second metric function. The second metric function corresponding to the nth clustering model is determined based on the model parameters of multiple (n-1)th classification models.

[0076] In this embodiment, the second cluster data is obtained by training the first clustering model based on the first training data. When the data processing device performs model training for the second time, it uses the updated first target metric function and neighborhood parameters, obtained from the model parameters of multiple first classification models obtained from the first model training, to train the model. Similarly, when performing model training for the third time, it uses the updated second target metric function and neighborhood parameters, obtained from the model parameters of multiple second classification models obtained from the second model training, to train the model, until a target clustering model and multiple target classification models are obtained.

[0077] It's important to note that the DBSCAN model doesn't require determining the number of clusters, and the order of data input has little impact on the model training results. The decision boundary of the SVM model is the maximum margin hyperplane calculated from the training samples; its loss function is affected by noise points, which can bias the classification results. The DBSCAN model, however, can identify noise points. Therefore, training the SVM model with cluster data obtained from training the DBSCAN model using the initial training data can eliminate the influence of noise points on the SVM model's classification results. Having a corresponding target classification model for each cluster can improve the accuracy of the classification results.

[0078] Step 204: The data processing device performs clustering processing on the data to be processed based on the target clustering model to obtain the cluster to which the data to be processed belongs.

[0079] Step 205: The data processing device classifies the data to be processed based on the target classification model corresponding to the cluster to which the data to be processed belongs, and obtains the classification result of the data to be processed.

[0080] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0081] The data processing method provided in this application embodiment is not affected by noise when classifying and clustering data in the target clustering model and multiple target classification models trained by the first metric function and the second metric function. This makes the classification results of the data determined by the target clustering model and multiple target classification models more accurate, solves the problem of low detection accuracy in related technologies, and improves the detection accuracy of data.

[0082] Based on the foregoing embodiments, embodiments of the present invention provide a data processing method, referring to... Figure 3 As shown, the method includes the following steps:

[0083] Step 301: The data processing device acquires the first training data.

[0084] Step 302: The data processing equipment determines the neighborhood parameters, the initial clustering model, and multiple initial classification models.

[0085] Step 303: The data processing device divides the first training data to obtain the second training data.

[0086] In this embodiment, the DBSCAN model takes too long to run when the amount of data in the first training data is too large. Therefore, when the amount of data in the first training data is too large, the first training data can be divided into second training data with a smaller amount of data. The model can be trained based on the second training data to improve the data processing speed and save time.

[0087] Step 304: The data processing device uses the first metric function and neighborhood parameters to train the initial clustering model based on the second training data to obtain the first clustering model and multiple target cluster data.

[0088] In this embodiment, the multiple target cluster data are obtained by training the initial clustering model based on the second training data. During the first model training, the data processing device can use a first metric function and neighborhood parameters to train the initial clustering model based on the second training data, which has a smaller data volume, to obtain the first clustering model and multiple target cluster data.

[0089] Step 304 can be achieved through the following steps:

[0090] Step 304a: The data processing device uses a first metric function to obtain the distance between the second training data.

[0091] In this embodiment of the application, the data processing device may use a first metric formula when performing model training for the first time: To obtain the distance between the second training data.

[0092] Step 304b: The data processing device determines the neighborhood set of the second training data based on the distance and neighborhood radius between the second training data.

[0093] The neighborhood parameter includes the neighborhood radius.

[0094] In this embodiment of the application, the data processing device determines, based on the distance between the second training data and each second training data in the second training data, the second training data whose distance is less than or equal to the neighborhood radius as the neighborhood set of the second training data.

[0095] Step 304c: The data processing device determines the core data based on the number of second training data in the neighborhood set and the number threshold.

[0096] The neighborhood parameter also includes a quantity threshold.

[0097] In this embodiment of the application, the data processing device determines the second training data corresponding to each neighborhood set as core data if the number of second training data in the neighborhood set is greater than or equal to a quantity threshold.

[0098] Step 304d: The data processing device obtains the first clustering model and multiple target cluster data based on the core data and the neighborhood set of the core data.

[0099] In this embodiment, for each core data, if the core data is not identified as target cluster data, the data processing device determines the second training data in the neighborhood set of the core data that is not identified as target cluster data as the first target training data; the second training data identified as core data in the first target training data is identified as target core data; if the first target training data does not include target core data, the first target training data is identified as target cluster data; if the first target training data includes target core data, the second training data in the neighborhood set of the target core data that is not identified as target cluster data is identified as the second target training data, and the first and second target training data are identified as target cluster data until all core data has been traversed.

[0100] In this embodiment of the application, when the first target training data includes target core data, the data processing device determines the second training data in the neighborhood set of the target core data that is not determined as target cluster data as the second target training data, and determines the first target training data and the second target training data together as target cluster data. That is, the second training data in the neighborhood set of the core object that is not determined as target cluster data and the second training data in the neighborhood sets of the other core objects in the neighborhood set of the core object that are not determined as target cluster data are determined as target cluster data.

[0101] In this embodiment, for each target core data, second training data that is not identified as target cluster data in the neighborhood set of the target core data is determined. If the determined second training data includes other core data that is not identified as target cluster data besides this target core data, the second training data that is not identified as target cluster data among the other core objects that are not identified as target cluster data besides this target core data is further determined. If the determined second training data also includes other core objects that are not identified as target cluster data besides this target core data and the other core data, the second training data that is not identified as target cluster data besides this core data and the other core data is further determined, until the second training data that is not identified as target cluster data among all the embedded core data is determined. All the determined second training data that is not identified as target cluster data is used as the second target training data.

[0102] Step 305: The data processing device performs clustering based on the edge training data in multiple target cluster data to obtain multiple first cluster data.

[0103] Step 305 can be achieved through the following steps:

[0104] Step 305a: The data processing device determines the edge training data in multiple target cluster data.

[0105] In this embodiment of the application, if the number of second training data in the neighborhood set of a second training data is less than a threshold and is included in the neighborhood set of the core data, the second training data is determined as edge training data.

[0106] Step 305b: The data processing device uses the first metric function to obtain the distance between edge training data.

[0107] In this embodiment of the application, the data processing device uses a first metric function as the basis for clustering when performing model training for the first time.

[0108] Step 305c: The data processing device clusters multiple target cluster data based on the distance and neighborhood radius between edge data to obtain multiple first cluster data.

[0109] In this embodiment of the application, for multiple target cluster data, the data processing device clusters the target cluster data whose distance between edge training data is less than or equal to the neighborhood radius based on the distance between edge data, and obtains multiple first cluster data.

[0110] Step 306: The data processing device trains multiple initial classification models based on multiple first-class cluster data to obtain multiple first-class classification models.

[0111] Step 307: The data processing device determines the classification model to be processed from multiple first classification models based on the accuracy of multiple first classification models.

[0112] In this embodiment of the application, the data processing device can determine the classification model to be processed based on the accuracy of multiple first classification models. For example, the first classification model with the highest accuracy among multiple first classification models can be determined as the classification model to be processed.

[0113] Step 308: The data processing device updates the second metric function based on the model parameters of the classification model to be processed to obtain the first target metric function.

[0114] In this embodiment, the data processing device updates the second metric function based on ω and b in the model parameters of the classification model to be processed. ω and b are used to obtain the first objective metric function.

[0115] Step 309: The data processing device uses the first objective metric function and neighborhood parameters to train the first clustering model based on the first training data to obtain the second clustering model and multiple second cluster data.

[0116] In this embodiment, the first target metric function is a metric function updated by the model parameters of multiple first classification models based on the second metric function.

[0117] Step 310: The data processing device obtains multiple second classification models based on multiple second-category cluster data and multiple first classification models.

[0118] Step 311: The data processing device updates the first objective metric function based on the model parameters of multiple second classification models to obtain the second objective metric function.

[0119] In this embodiment of the application, the operation of the data processing device in step 311 to obtain the second target metric function based on the model parameters of multiple second classification models is similar to the operation of the data processing device in steps 307-308 to obtain the first target metric function based on the parameters of multiple first classification models. For example, the first target metric function can be updated based on the model parameters of the second classification model with the highest accuracy among multiple second classification models to obtain the second target metric function.

[0120] Step 312: The data processing device uses the second objective metric function and neighborhood parameters to train the second clustering model based on the first training data until the objective conditions are met, so as to obtain the objective clustering model and multiple objective classification models.

[0121] In this embodiment, the target condition is the iteration stopping condition in step 202 above. The data processing device can use a second target metric function and neighborhood parameters to train a second clustering model based on the first training data to obtain a third clustering model and multiple third cluster data. Based on the multiple third cluster data, it can train multiple second classification models to obtain multiple third classification models. Based on the model parameters of the multiple third classification models, it can update the second target metric function to obtain a third metric function, and so on, until the iteration stopping condition is reached to obtain the target clustering model and multiple target classification models.

[0122] In this embodiment, the data processing device in step 312 uses a second objective metric function and neighborhood parameters to train the second clustering model based on the first training data. This is similar to the step in step 304 where the data processing device uses a first metric function and neighborhood parameters to train the first clustering model based on the first training data. Therefore, this embodiment will not be described again here.

[0123] Step 313: The data processing device performs clustering processing on the data to be processed based on the target clustering model to obtain the cluster to which the data to be processed belongs.

[0124] Step 314: The data processing device classifies the data to be processed based on the target classification model corresponding to the cluster to which the data to be processed belongs, and obtains the classification result of the data to be processed.

[0125] The data processing method provided in this application embodiment is not affected by noise when classifying and clustering data in the target clustering model and multiple target classification models trained by the first metric function and the second metric function. This makes the classification results of the data determined by the target clustering model and multiple target classification models more accurate, solves the problem of low detection accuracy in related technologies, and improves the detection accuracy of data.

[0126] Based on the foregoing embodiments, referring to Figure 4 As shown, the overall implementation logic of the data processing method provided in this application embodiment can be as follows: obtain first training data, then preprocess the first training data to improve its quality, then determine neighborhood parameters, an initial clustering model, and multiple initial classification models, use a first metric function and neighborhood parameters to train the initial clustering model based on the first training data to obtain a first clustering model and multiple first cluster data, use the multiple first cluster data to train the initial classification model to obtain multiple first classification models, then update the second metric function based on the model parameters of the multiple first classification models, use the second metric function and neighborhood parameters to train the first clustering model based on the first training data to obtain a second clustering model and multiple second cluster data, and use the multiple second cluster data to train the multiple first classification models to obtain multiple second classification models, and so on, until the iteration stopping condition is reached to obtain a target clustering model and multiple target classification models.

[0127] Based on the foregoing embodiments, embodiments of this application provide a data processing device that can be applied to... Figures 1-3 In the data processing method provided in the corresponding embodiment, refer to Figure 5 As shown, the data processing device 4 may include: a processor 41, a memory 42, and a communication bus 43, wherein:

[0128] Communication bus 43 is used to realize the communication connection between processor 41 and memory 42;

[0129] The processor 41 is used to execute the data processing program in the memory 42 to perform the following steps:

[0130] Obtain the first training data;

[0131] Using a first metric function and a second metric function, a model is trained based on the first training data to obtain a target clustering model and multiple target classification models; wherein, both the first metric function and the second metric function are used to measure the distance between target training data;

[0132] The data to be processed is clustered based on the target clustering model to obtain the cluster to which the data to be processed belongs;

[0133] The data to be processed is classified based on the target classification model corresponding to the cluster to which the data to be processed belongs, and the classification result of the data to be processed is obtained.

[0134] In other embodiments of this application, the processor 41 is used to execute the data processing program in the memory 42, employing a first metric function and a second metric function, to train a model based on the first training data to obtain a target clustering model and multiple target classification models, in order to implement the following steps:

[0135] Determine the neighborhood parameters, the initial clustering model, and multiple initial classification models;

[0136] Using a first metric function, a second metric function, and neighborhood parameters, the initial clustering model and multiple initial classification models are trained based on the first training data to obtain the target clustering model and multiple target classification models.

[0137] In other embodiments of this application, the processor 41 is used to execute the data processing program in the memory 42, employing a first metric function, a second metric function, and neighborhood parameters to train an initial clustering model and multiple initial classification models based on the first training data, thereby obtaining a target clustering model and multiple target classification models to achieve the following steps:

[0138] Using the first metric function and neighborhood parameters, the initial clustering model is trained based on the first training data to obtain the first clustering model and multiple first cluster data;

[0139] Multiple first-class classification models are obtained by training multiple initial classification models based on multiple first-class cluster data;

[0140] Based on the model parameters of multiple first classification models, the second metric function is updated to obtain the first target metric function;

[0141] Using a first target metric function and neighborhood parameters, a first clustering model is trained on the first training data to obtain a second clustering model and multiple second-class cluster data. Then, multiple first classification models are trained on the multiple second-class cluster data respectively until a target clustering model and multiple target classification models are obtained. Among them, the metric function corresponding to the nth clustering model other than the initial clustering model is the second metric function. The second metric function corresponding to the nth clustering model is determined based on the model parameters of multiple (n-1)th classification models, where n is a positive integer.

[0142] In other embodiments of this application, the processor 41 is used to execute the data processing program in the memory 42, which uses a first metric function and neighborhood parameters to train the initial clustering model based on the first training data, and divides the first training data to obtain the second training data.

[0143] Using the first metric function and neighborhood parameters, the initial clustering model is trained based on the second training data to obtain the first clustering model and multiple target cluster data;

[0144] Clustering is performed on the edge training data from multiple target clusters to obtain multiple first-class clusters.

[0145] In other embodiments of this application, the processor 41 is used to execute a data processing program in the memory 42 to cluster based on edge training data in multiple target cluster data to obtain multiple first cluster data, in order to implement the following steps:

[0146] Identify edge training data in multiple target clusters of data;

[0147] The first metric function is used to obtain the distance between edge training data;

[0148] Multiple target cluster data are clustered based on the distance between edge data and the neighborhood radius to obtain multiple first-class cluster data.

[0149] In other embodiments of this application, the processor 41 is used to execute the model parameters of the data processing program in the memory 42 based on multiple second classification models, update the second metric function to obtain the first target metric function, and implement the following steps:

[0150] Based on the accuracy of multiple first-class classification models, the classification model to be processed is determined from the multiple first-class classification models;

[0151] Based on the model parameters of the classification model to be processed, the second metric function is updated to obtain the first target metric function.

[0152] In other embodiments of this application, the processor 41 is used to execute the data processing program in the memory 42 to train multiple first classification models based on multiple second cluster data until a target clustering model and multiple target classification models are obtained, in order to achieve the following steps:

[0153] Multiple first classification models are trained based on multiple second-class cluster data to obtain multiple second-class classification models;

[0154] Based on the model parameters of multiple second classification models, the first objective metric function is updated to obtain the second objective metric function;

[0155] Using a second objective metric function and neighborhood parameters, the second clustering model is trained on the first training data until the objective conditions are met, so as to obtain the objective clustering model and multiple objective classification models.

[0156] In other embodiments of this application, the target clustering model includes a density-based spatial clustering DBSCAN model with noise application, and the multiple target classification models include multiple support vector machine (SVM) models.

[0157] It should be noted that a detailed description of the steps performed by the processor can be found in [reference needed]. Figures 1-3 The data processing methods provided in the corresponding embodiments will not be described in detail here.

[0158] The data processing device provided in this application embodiment is not affected by noise when classifying and clustering data in the target clustering model and multiple target classification models trained by the first metric function and the second metric function. This makes the classification results of the data determined by the target clustering model and multiple target classification models more accurate, solves the problem of low detection accuracy in related technologies, and improves the detection accuracy of data.

[0159] Based on the foregoing embodiments, embodiments of this application provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement... Figures 1-3 The corresponding embodiments provide the steps of the data processing method.

[0160] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0161] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the first training data; the first training data includes network data of multiple different attack categories, as well as labels; Using a first metric function and a second metric function, a target clustering model and multiple target classification models are obtained by training the model based on the first training data. Both the first and second metric functions are used to measure the distance between the first training data. The first metric function is a formula for determining the distance between the first training data and is used to train the initial clustering model. The second metric function is a formula that uses the distance between the first training data and the hyperplane as the clustering basis and is continuously updated based on the model parameters of the classification model during training. It is used to train other clustering models besides the initial clustering model. Based on the target clustering model, the data to be processed is clustered to obtain the cluster to which the data to be processed belongs; The data to be processed is classified based on the target classification model corresponding to the cluster to which the data to be processed belongs, and the classification result of the data to be processed is obtained; the classification result indicates whether the data to be processed is abnormal.

2. The method according to claim 1, characterized in that, The process involves using a first metric function and a second metric function to train a model based on the first training data to obtain a target clustering model and multiple target classification models, including: Determine the neighborhood parameters, the initial clustering model, and multiple initial classification models; Using the first metric function, the second metric function, and the neighborhood parameters, the initial clustering model and the plurality of initial classification models are trained based on the first training data to obtain the target clustering model and the plurality of target classification models.

3. The method according to claim 2, characterized in that, The step of training the initial clustering model and the plurality of initial classification models based on the first training data using the first metric function, the second metric function, and the neighborhood parameters to obtain the target clustering model and the plurality of target classification models includes: Using the first metric function and the neighborhood parameters, the initial clustering model is trained based on the first training data to obtain a first clustering model and multiple first cluster data; Based on the data of the multiple first-class clusters, the multiple initial classification models are trained to obtain multiple first-class classification models; Based on the model parameters of the multiple first classification models, the second metric function is updated to obtain the first target metric function; Using the first target metric function and the neighborhood parameters, the first clustering model is trained on the first training data to obtain a second clustering model and multiple second cluster data. Then, the multiple first classification models are trained on the multiple second cluster data respectively until the target clustering model and the multiple target classification models are obtained. The metric function corresponding to the nth clustering model other than the initial clustering model is the second metric function. The second metric function corresponding to the nth clustering model is determined based on the model parameters of the multiple (n-1)th classification models, where n is a positive integer.

4. The method according to claim 3, characterized in that, The step of using the first metric function and the neighborhood parameters to train the initial clustering model based on the first training data to obtain the first clustering model and multiple first cluster data includes: The first training data is divided into two parts to obtain the second training data. Using the first metric function and the neighborhood parameters, the initial clustering model is trained based on the second training data to obtain the first clustering model and multiple target cluster data; Clustering is performed on the edge training data in the multiple target cluster data to obtain the multiple first cluster data.

5. The method according to claim 4, characterized in that, The clustering based on the edge training data in the multiple target cluster data to obtain the multiple first cluster data includes: Identify the edge training data in the plurality of target cluster data; The distance between the edge training data is obtained using the first metric function; Based on the distance and neighborhood radius between the edge training data, the edge training data in the multiple target cluster data are clustered to obtain the multiple first cluster data.

6. The method according to claim 3, characterized in that, The step of updating the second metric function based on the model parameters of the plurality of first classification models to obtain the first target metric function includes: Based on the accuracy of the plurality of first classification models, the classification model to be processed is determined from the plurality of first classification models; Based on the model parameters of the classification model to be processed, the second metric function is updated to obtain the first target metric function.

7. The method according to claim 3, characterized in that, The step of training the multiple first classification models based on the multiple second-category cluster data until the target clustering model and the multiple target classification models are obtained includes: Based on the data of the multiple second-class clusters, the multiple first-class classification models are trained to obtain multiple second-class classification models; Based on the model parameters of the multiple second classification models, the first target metric function is updated to obtain the second target metric function; Using the second target metric function and the neighborhood parameters, the second clustering model is trained on the first training data until the target conditions are met, so as to obtain the target clustering model and the multiple target classification models.

8. The method according to claim 1, characterized in that, The target clustering model includes a density-based spatial clustering DBSCAN model with noise application, and the multiple target classification models include multiple support vector machine (SVM) models.

9. A data processing device, characterized in that, The device includes: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute a data processing program in the memory to implement the steps of the data processing method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which are executed by one or more processors to implement the steps of the data processing method as described in any one of claims 1 to 8.

Images