Data detection method, device and storage medium

Abstract

The present invention provides a data detection method, device and storage medium, the method comprising: processing a plurality of current data to be inspected to obtain a plurality of feature vectors; using the relevant data of the plurality of features corresponding to the plurality of feature vectors to determine the number of targets and the information of the plurality of target center points corresponding to the plurality of intermediate clustering result sets, classifying the plurality of feature vectors to obtain a plurality of intermediate clustering result sets, and then using a clustering model optimized by the number of targets and the information of the plurality of target center points, and a three-branch decision clustering algorithm to obtain a plurality of target clustering result sets; matching the plurality of target clustering result sets with the plurality of matching result sets to determine the detection results. Since this scheme optimizes the uncertain data to be inspected, each data to be inspected can be accurately classified, and the plurality of target clustering result sets calculated by the relevant data of the plurality of features are more closely matched with the plurality of current data to be inspected, thereby improving the detection accuracy of the plurality of current data to be inspected.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of Internet and data detection technology, and in particular to a data detection method, device and storage medium. Background Art

[0002] With the development of the Internet, the openness and sharing of World Wide Web (Web) services have brought convenience to people, but also brought network security issues such as various Web attacks. Therefore, Web attack detection technology has become a more important means to solve network security issues.

[0003] Cluster analysis is an important analytical method in data mining. It can classify data with similar characteristics into the same category by analyzing the attribute characteristics of the data. It does not require artificial division criteria to be given in advance and is an unsupervised learning method.

[0004] Traditional clustering methods often assume that the relationship between the studied objects and classes is one of belonging and not belonging. The clustering results contain clear decomposition lines, but these resulting clusters may contain some uncertain objects. If they are forced to be divided into a certain cluster, it is very likely to result in high decision-making risks, which will reduce the accuracy of Web attack detection.

[0005] Before running the clustering method, the number of clusters and the initial cluster centers must be specified as input parameters to the algorithm model function. If the number of clusters obtained does not match the data, or if the initial cluster centers differ significantly from the final initial cluster centers, the number of clustering algorithm iterations will increase, ultimately leading to a local optimum in the clustering results and reduced accuracy in web attack detection. Summary of the Invention

[0006] The embodiments of the present invention provide a data detection method, device, and storage medium, which can improve the accuracy of data detection.

[0007] The technical solution of the present invention is achieved as follows:

[0008] An embodiment of the present invention provides a data detection method, including:

[0009] Processing the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected;

[0010] Using the relevant data of multiple features corresponding to multiple feature vectors, the number of targets and multiple target center point information corresponding to multiple intermediate clustering result sets are determined, so as to classify the multiple feature vectors and obtain multiple intermediate clustering result sets;

[0011] The multiple intermediate clustering result sets are processed by the clustering model optimized by the number of targets and multiple target center point information, and the three-branch decision clustering algorithm to obtain multiple target clustering result sets;

[0012] The multiple target clustering result sets are matched with the preset multiple matching result sets to determine the detection results of multiple current data to be detected; the multiple matching result sets are obtained by processing multiple historical data in the same way as the multiple target clustering result sets.

[0013] In the above solution, the method of using the relevant data of the multiple features corresponding to the multiple feature vectors to determine the number of targets and multiple target center point information corresponding to the multiple intermediate clustering result sets, so as to classify the multiple feature vectors to obtain multiple intermediate clustering result sets, includes:

[0014] Calculating a mean value of each feature corresponding to the multiple feature vectors, and calculating multiple initial weights of the multiple features based on the mean value combined with the feature value of the corresponding feature;

[0015] Classifying the multiple feature vectors according to the multiple initial weights, the number Z of clustering result sets of the obtained initial clustering model, and the corresponding Z first center point information to obtain Z clustering result sets, and calculating the effectiveness index information of the number Z; Z is a positive integer greater than or equal to 2;

[0016] Calculating a plurality of Z-level weights corresponding to a plurality of features in the Z clustering result sets;

[0017] Classifying the plurality of feature vectors according to the plurality of Z-level weights and the obtained number Z+1 and the corresponding Z+1 first center point information to obtain Z+1 clustering result sets, and calculating the validity index information of the number Z+1 and the plurality of Z+1-level weights, until the plurality of feature vectors are divided into K clustering result sets, obtaining K clustering result sets and validity index information corresponding to the number K, where K is equal to a preset threshold;

[0018] According to the validity index information corresponding to the numbers Z to K, the target number of multiple intermediate clustering result sets and their corresponding multiple first intermediate clustering result sets are determined, and the multiple feature vectors are iteratively classified according to the multiple first intermediate clustering result sets until the center point information meets the preset conditions, and the multiple target center point information and the multiple intermediate clustering result sets are determined.

[0019] In the above scheme, after determining the target number and the corresponding target center point information of multiple intermediate clustering result sets by using the relevant data of the multiple feature vectors corresponding to the multiple features to classify the multiple feature vectors to obtain multiple intermediate clustering result sets, and before processing the multiple intermediate clustering result sets using the clustering model optimized by the target number and the multiple target center point information and the three-branch decision clustering algorithm to obtain multiple target clustering result sets, the method further includes:

[0020] Initializing the initial clustering model using the number of targets and the information of the multiple target center points to obtain the optimized clustering model;

[0021] The plurality of intermediate clustering result sets are processed by a clustering model optimized by the number of targets and the information of the plurality of target center points, and a three-branch decision clustering algorithm to obtain a plurality of target clustering result sets, including:

[0022] Determining, from the plurality of feature vectors, a neighborhood including a predetermined proportion of feature vectors;

[0023] Traversing each feature vector in the plurality of intermediate clustering result sets, and combining the neighborhood, dividing the plurality of feature vectors into core domain clusters and intermediate boundary domain clusters; the core domain clusters include: result sets corresponding to the number of the plurality of intermediate clustering result sets; the intermediate boundary domain clusters include: a certain number of feature vectors at the boundaries of the plurality of intermediate clustering result sets;

[0024] Inputting the intermediate boundary domain clusters into the optimized clustering model to obtain boundary domain clusters; the boundary domain clusters include: result sets corresponding to the number of the multiple intermediate clustering result sets;

[0025] The result set in the boundary domain cluster is combined with the corresponding result set in the core domain cluster to obtain the multiple target clustering result sets.

[0026] In the above solution, calculating the mean of each feature corresponding to the multiple feature vectors, and calculating multiple initial weights of the multiple features based on the mean combined with the feature value of the corresponding feature, includes:

[0027] Calculating the mean of each feature of the multiple feature vectors, and calculating the variance of each feature by combining the eigenvalue corresponding to each feature with the mean;

[0028] The variance of each feature is compared with the sum of the variances of each feature to obtain an initial weight of each feature.

[0029] In the above solution, the multiple feature vectors are classified according to the multiple initial weights, the number Z of clustering result sets of the obtained initial clustering model, and the corresponding Z first center point information to obtain Z clustering result sets, including:

[0030] Calculate, in combination with the multiple initial weights and the multiple eigenvalues ​​corresponding to the multiple eigenvectors, Z feature distances corresponding to each eigenvector and the Z first center point information;

[0031] Among the Z feature distances corresponding to each feature vector, the center point information corresponding to the minimum feature distance is determined, and each feature vector is divided into a clustering result set of the corresponding center point information, thereby obtaining the Z clustering result sets.

[0032] In the above solution, the validity indicator information of the calculation number Z includes:

[0033] Calculating Z intra-class distances corresponding to the Z clustering result sets;

[0034] Calculate the sum of the inter-class distances between each of the Z clustering result sets;

[0035] The ratio of the Z intra-class distances to the sum of the inter-class distances is calculated to obtain the effectiveness index information of the number Z.

[0036] In the above solution, the step of calculating the multiple Z-level weights corresponding to the multiple features in the Z clustering result sets includes:

[0037] Calculating a first mean of each feature in each clustering result set, and calculating a first variance of each feature in each clustering result set by combining a feature value corresponding to each feature with the first mean;

[0038] Subtract the first mean of each feature of the pairwise clustering result sets in the Z clustering result sets, and then calculate the square of the difference to obtain the cluster center point distance corresponding to each feature of the pairwise clustering result sets in the Z clustering result sets;

[0039] Sum the first variance of each feature of each pair of clustering result sets in the Z clustering result sets, and then calculate the square of the sum to obtain the intra-class dispersion degree of each feature;

[0040] The distance between the cluster centers of each feature and the ratio of the intra-class dispersion of each feature are calculated to obtain the Z-level weight of each feature, and then the multiple Z-level weights are obtained.

[0041] In the above scheme, the target number of multiple intermediate clustering result sets and their corresponding multiple first intermediate clustering result sets are determined based on the effectiveness indicator information corresponding to the numbers Z to K, and the multiple feature vectors are iteratively classified based on the multiple first intermediate clustering result sets to determine the multiple target center point information and the multiple intermediate clustering result sets, including:

[0042] Determine the target number corresponding to the minimum effectiveness index information among the effectiveness index information corresponding to the numbers Z to K, and a plurality of first intermediate clustering result sets corresponding to the target number;

[0043] According to the calculation of the characteristic vectors in the plurality of first intermediate clustering result sets, new means of the plurality of characteristic vectors are calculated to obtain a plurality of new center point information;

[0044] If the multiple new center point information does not match the corresponding multiple first center point information, the multiple feature vectors are classified according to the target number and the multiple new center point information to obtain new multiple first intermediate clustering result sets, so as to iterate the multiple new center point information until the multiple new center point information matches the previous multiple center point information, and the multiple target center point information and the multiple intermediate clustering result sets are obtained.

[0045] In the above solution, the K satisfies a preset condition, including: K is greater than or equal to 2 and is an integer less than the square root of the number of the multiple current data to be inspected.

[0046] In the above solution, traversing each feature vector in the plurality of intermediate clustering result sets and combining the neighborhood to divide the plurality of feature vectors into core domain clusters and intermediate boundary domain clusters includes:

[0047] Traverse the i-th eigenvector in the j-th intermediate clustering result set; i is a positive integer greater than or equal to 1 and less than n; j is a positive integer greater than or equal to 1 and less than k; n is the number of eigenvectors in the corresponding intermediate clustering result set; k is the number of the multiple intermediate clustering result sets;

[0048] If the i-th eigenvector belongs to the j-th intermediate clustering result set, and the j-th intermediate clustering result set contains the neighborhood; or, the i-th eigenvector does not belong to the j-th intermediate clustering result set, and the intersection of the j-th intermediate clustering result set and the neighborhood is not an empty set, then the i-th eigenvector is classified into the intermediate boundary domain cluster, otherwise the i-th eigenvector is classified into the core domain cluster;

[0049] The process stops when the nth eigenvector in the kth intermediate clustering result set is traversed, and the core domain cluster and the intermediate boundary domain cluster are obtained.

[0050] In the above solution, the processing of the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected includes:

[0051] Get multiple current data to be inspected at the current moment;

[0052] The plurality of current data to be inspected are subjected to impurity and duplication removal processing to obtain a plurality of intermediate data; each intermediate data includes: keywords of a plurality of features;

[0053] Structural processing is performed on keywords of multiple features of the multiple intermediate data to obtain multiple feature vectors corresponding to the multiple current data to be inspected.

[0054] In the above solution, the multiple matching result sets include two categories: normal behavior matching result sets and attack behavior matching result sets;

[0055] The matching of the plurality of target clustering result sets with the plurality of preset matching result sets to determine the detection results of the plurality of current to-be-detected data includes:

[0056] Matching multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set with multiple result keywords corresponding to each matching result set in the normal behavior matching result set;

[0057] If the matching degree of the multiple keywords corresponding to any one of the matching result sets in the normal behavior matching result set reaches a threshold, then the current data to be inspected corresponding to the first clustering result set is determined to be normal data;

[0058] Otherwise, matching multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set with multiple result keywords corresponding to each matching result set in the attack behavior matching result set;

[0059] If the matching degree of the multiple keywords corresponding to any one of the attack behavior matching result sets reaches a threshold, the current data to be tested corresponding to the first clustering result set is determined to be attack data, until the multiple target clustering result sets are matched, and the detection results of the multiple current data to be tested are obtained.

[0060] In the above solution, before processing the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected, the method further includes:

[0061] Process the acquired multiple historical data to obtain multiple historical feature vectors corresponding to the multiple historical data

[0062] Determining the number of historical targets and central point information of multiple historical targets corresponding to multiple historical intermediate clustering result sets using relevant data of multiple features corresponding to the multiple historical feature vectors, so as to classify the multiple historical feature vectors to obtain multiple historical intermediate clustering result sets;

[0063] Processing the multiple historical intermediate clustering result sets with the clustering model optimized by the number of historical targets and the information of the center points of the multiple historical targets, and the three-branch decision clustering algorithm to obtain multiple historical target clustering result sets;

[0064] The multiple historical target result sets are tested to obtain multiple test results corresponding to the multiple historical target result sets, and the multiple historical target result sets are mapped with the multiple test results to form the multiple matching result sets; the multiple matching result sets are divided into two categories: normal behavior matching result sets and attack behavior matching result sets according to the test results.

[0065] An embodiment of the present invention further provides a data detection device, comprising:

[0066] a processing unit, configured to process the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected;

[0067] a determining unit, configured to determine the number of targets and information of multiple target center points corresponding to multiple intermediate clustering result sets using relevant data of multiple features corresponding to the multiple feature vectors, so as to classify the multiple feature vectors to obtain the multiple intermediate clustering result sets;

[0068] The processing unit is further configured to process the multiple intermediate clustering result sets using the clustering model optimized by the number of targets and the multiple target center point information and a three-branch decision clustering algorithm to obtain multiple target clustering result sets;

[0069] A matching unit is used to match the multiple target clustering result sets with multiple preset matching result sets to determine the detection results of the multiple current data to be tested; the multiple matching result sets are obtained from multiple historical data through the same processing method as the multiple target clustering result sets.

[0070] An embodiment of the present invention further provides a data detection device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the program.

[0071] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0072] In an embodiment of the present invention, a plurality of acquired current data to be inspected are processed to obtain a plurality of feature vectors corresponding to the plurality of current data to be inspected; the relevant data of the plurality of feature vectors corresponding to the plurality of features are used to determine the number of targets and the information of the plurality of target center points corresponding to the plurality of intermediate clustering result sets, so as to classify the plurality of feature vectors and obtain a plurality of intermediate clustering result sets; the plurality of intermediate clustering result sets are processed by a clustering model optimized by the number of targets and the information of the plurality of target center points, and a three-branch decision clustering algorithm to obtain a plurality of target clustering result sets; the plurality of target clustering result sets are matched with a plurality of preset matching result sets to determine the detection results of the plurality of current data to be inspected; the plurality of matching result sets are obtained from a plurality of historical data by the same processing method as the plurality of target clustering result sets. Since the present solution optimizes the uncertain data to be inspected by the three-branch decision clustering algorithm, so that each data to be inspected can be accurately classified, and the number of targets and the information of the plurality of target center points are calculated by the relevant data of the plurality of features, the plurality of target clustering result sets are more matched with the plurality of current data to be inspected, thereby improving the detection accuracy of the plurality of current data to be inspected. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 An optional flowchart of a data detection method provided in an embodiment of the present invention;

[0074] Figure 2 An optional flowchart of a data detection method provided in an embodiment of the present invention;

[0075] Figure 3 An optional flowchart of a data detection method provided in an embodiment of the present invention;

[0076] Figure 4 An optional flowchart of a data detection method provided in an embodiment of the present invention;

[0077] Figure 5 An optional flowchart of a data detection method provided in an embodiment of the present invention;

[0078] Figure 6 An optional flowchart of a data detection method provided in an embodiment of the present invention;

[0079] Figure 7 A schematic structural diagram of a data detection device provided in an embodiment of the present invention;

[0080] Figure 8 A schematic diagram of a hardware entity of a data detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention are further elaborated in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0082] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0083] If similar descriptions of "first / second" appear in the invention document, the following explanation is added. In the following description, the terms "first\second\third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with the specific order or sequence where permitted, so that the embodiments of the invention described herein can be implemented in an order other than that illustrated or described herein.

[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0085] Figure 1 An optional flow chart of the data detection method provided in the embodiment of the present invention is combined with Figure 1 The steps shown are explained.

[0086] S101 : Processing a plurality of acquired current data to be inspected to obtain a plurality of feature vectors corresponding to the plurality of current data to be inspected.

[0087] In the embodiment of the present invention, the server processes the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected.

[0088] In the embodiment of the present invention, the server obtains a plurality of current to-be-checked data from the plurality of clients via communication lines pre-established with the plurality of clients.

[0089] In the embodiment of the present invention, the server may also extract a plurality of current data to be inspected from the corresponding access log record library.

[0090] In an embodiment of the present invention, each current data to be inspected includes keywords corresponding to multiple features. For example, the multiple features may include an Internet Protocol (IP) address, a Uniform Resource Locator (URL), an access date and time, an access result, a user's website, and configuration information.

[0091] In an embodiment of the present invention, the server can perform deduplication and de-cluttering processing on multiple current data to be inspected, remove audio and video image data and other unnecessary data in the multiple current data to be inspected, unify multiple features of the multiple current data to be inspected, and then perform structured processing on the keywords corresponding to the multiple features of each current data to be inspected, convert the current data to be inspected into a feature vector with a fixed dimension equal to the number of features, and then obtain multiple feature vectors.

[0092] S102 , using relevant data of multiple features corresponding to multiple feature vectors, determine the number of targets and multiple target center point information corresponding to multiple intermediate clustering result sets, so as to classify the multiple feature vectors to obtain multiple intermediate clustering result sets.

[0093] In an embodiment of the present invention, the server uses relevant data of multiple features corresponding to multiple feature vectors to determine the number of targets and multiple target center point information corresponding to multiple intermediate clustering result sets, so as to classify multiple feature vectors to obtain multiple intermediate clustering result sets.

[0094] In an embodiment of the present invention, the server uses the eigenvalues ​​of multiple features corresponding to multiple feature vectors to calculate the mean corresponding to each feature. The server uses the mean to calculate the initial weight corresponding to each feature. The server uses the number of clustering result sets of the obtained initial clustering model and the corresponding multiple first center point information to perform a first classification on the multiple feature vectors to obtain a clustering result set, and calculates the validity index information corresponding to the number of clustering result sets this time. The server then calculates the multiple weights corresponding to the multiple features after the first classification, and then uses the number of clustering result sets obtained and the corresponding multiple first center point information to classify the multiple feature vectors again to obtain the clustering result set of this classification, and calculates the validity index information corresponding to the number of clustering result sets this time, until the number of clustering result sets equals the preset threshold value. The server determines the minimum validity index information among the multiple validity index information, and then uses the minimum validity index information to determine the target number corresponding to the multiple intermediate clustering result sets. The server determines multiple first intermediate clustering result sets based on the number of targets, and then uses the multiple first intermediate clustering result sets to iteratively classify multiple feature vectors until the center point information meets the preset threshold, thereby determining multiple target center point information and multiple intermediate clustering result sets.

[0095] In an embodiment of the present invention, the minimum effectiveness indicator information can be obtained by the ratio of the sum of the intra-class distances of multiple intermediate clustering result sets to the sum of the inter-class distances of each pair of intermediate clustering result sets. The intra-class distance represents the distance between multiple vectors within the corresponding intermediate clustering result set, while the inter-class distance represents the distance between the center points of each pair of intermediate clustering result sets. The center point is the mean of the eigenvectors contained in the corresponding intermediate clustering result set.

[0096] S103 , processing the multiple intermediate clustering result sets using a clustering model optimized by the number of targets and multiple target center point information and a three-branch decision clustering algorithm to obtain multiple target clustering result sets.

[0097] In the embodiment of the present invention, the server processes the multiple intermediate clustering result sets using a clustering model optimized by the number of targets and multiple target center point information and a three-branch decision clustering algorithm to obtain multiple target clustering result sets.

[0098] In an embodiment of the present invention, the server initializes the initial clustering model using the number of targets and multiple target center point information to obtain an optimized clustering model. Each feature vector in the multiple intermediate clustering result sets is traversed, and the multiple feature vectors are divided into core domain clusters and boundary domain clusters in combination with the pre-determined neighborhood. The result set in the boundary domain cluster is combined with the corresponding result set in the core domain cluster to obtain multiple target clustering result sets. The core domain cluster includes: a result set corresponding to the number of multiple intermediate clustering result sets; the intermediate boundary domain cluster includes: a certain number of feature vectors at the boundaries of the multiple intermediate clustering result sets.

[0099] S104: Match the multiple target clustering result sets with the multiple preset matching result sets to determine multiple detection results of the current data to be detected.

[0100] In the embodiment of the present invention, the server matches the multiple target clustering result sets with the multiple preset matching result sets to determine the detection results of the multiple current data to be detected.

[0101] In an embodiment of the present invention, the multiple matching result sets include two categories: normal behavior matching result sets and attack behavior matching result sets. The server matches the multiple keywords of the current data to be inspected corresponding to the feature vectors in each clustering result set with the multiple result keywords corresponding to each matching result set in the normal behavior matching result set. If the matching degree of the multiple keywords with the multiple result keywords corresponding to any matching result set in the normal behavior matching result set reaches a threshold, the current data to be inspected corresponding to the first clustering result set is determined to be normal data.

[0102] Otherwise, multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set are matched with multiple result keywords corresponding to each matching result set in the attack behavior matching result set;

[0103] If the matching degree of multiple keywords with any matching result set in the attack behavior matching result set reaches the threshold, the current data to be inspected corresponding to the first clustering result set is determined to be the attack data, until multiple target clustering result sets are matched, and the detection results of multiple current data to be inspected are obtained.

[0104] In the embodiment of the present invention, the multiple matching result sets are obtained by processing the multiple historical data using the same processing method as the multiple target clustering result sets.

[0105] In some embodiments, see Figure 2 , Figure 2 An optional flowchart of a data processing method provided by an embodiment of the present invention is shown. Figure 1 S101 to S104 shown can be implemented through S201 to S209 , which will be described in conjunction with each step.

[0106] S201. Web log data.

[0107] S201: Web log data preprocessing.

[0108] S201. Cluster analysis model.

[0109] S201. Normal behavior feature library.

[0110] S201, attack behavior feature library.

[0111] S201: Anomaly detection.

[0112] S201. Misuse detection.

[0113] S201. Real-time response.

[0114] S201. Normal behavior.

[0115] In one embodiment of the present invention, a hybrid detection model is provided, comprising an anomaly detection module and a misuse detection module. The server first passes web log data (which may include historical and current data) through the anomaly detection module to distinguish between normal and attacking data. The attacking data is then tested using the misuse detection model to determine the specific attack type. These two detection models are combined in a cascaded fashion, leveraging their respective strengths while simultaneously compensating for their respective shortcomings.

[0116] like Figure 2As shown, the server uses historical web log data for training. The pre-processed web log data is fed into the cluster analysis algorithm model for processing and analysis, ultimately constructing a normal behavior signature library and an attack behavior signature library, completing the training objective. The server then feeds the current log data into the cluster analysis module for anomaly detection, matching it with the records in the normal behavior signature library. If a match is successful, it is considered normal behavior. If a match fails, it is considered an attack and enters the misuse detection module for matching with the records in the attack behavior signature library. If a match fails, the test data record is considered normal behavior; if a match is successful, it is considered an attack and a real-time response is issued.

[0117] In an embodiment of the present invention, a plurality of acquired current data to be inspected are processed to obtain a plurality of feature vectors corresponding to the plurality of current data to be inspected; the relevant data of the plurality of feature vectors corresponding to the plurality of features are used to determine the number of targets and the information of the plurality of target center points corresponding to the plurality of intermediate clustering result sets, so as to classify the plurality of feature vectors and obtain a plurality of intermediate clustering result sets; the plurality of intermediate clustering result sets are processed by a clustering model optimized by the number of targets and the information of the plurality of target center points, and a three-branch decision clustering algorithm to obtain a plurality of target clustering result sets; the plurality of target clustering result sets are matched with a plurality of preset matching result sets to determine the detection results of the plurality of current data to be inspected; the plurality of matching result sets are obtained from a plurality of historical data by the same processing method as the plurality of target clustering result sets. Since the present solution optimizes the uncertain data to be inspected by the three-branch decision clustering algorithm, so that each data to be inspected can be accurately classified, and the number of targets and the information of the plurality of target center points are calculated by the relevant data of the plurality of features, the plurality of target clustering result sets are more matched with the plurality of current data to be inspected, thereby improving the detection accuracy of the plurality of current data to be inspected.

[0118] In some embodiments, Figure 1 S102 to S103 shown can be implemented through S105 to S114, which will be described in conjunction with each step.

[0119] S105: Calculate the mean of each feature corresponding to the multiple feature vectors, and calculate multiple initial weights of the multiple features based on the mean and the feature value of the corresponding feature.

[0120] In the embodiment of the present invention, the server calculates the mean of each feature corresponding to the multiple feature vectors, and calculates multiple initial weights of the multiple features based on the mean combined with the feature value of the corresponding feature.

[0121] In this embodiment of the present invention, the server calculates the mean of each feature of the multiple feature vectors and combines the eigenvalue corresponding to each feature with the mean to calculate the variance of each feature. The server compares the variance of each feature with the sum of the variances of all features to obtain an initial weight for each feature.

[0122] S106. Classify multiple feature vectors based on multiple initial weights, the number Z of clustering result sets of the obtained initial clustering model and the corresponding Z first center point information, obtain Z clustering result sets, and calculate the effectiveness index information of the number Z; Z is a positive integer greater than or equal to 2.

[0123] In an embodiment of the present invention, the server classifies the multiple feature vectors based on the multiple initial weights, the number Z of clustering result sets obtained from the initial clustering model, and the corresponding Z first center point information, to obtain Z clustering result sets, and calculates effectiveness indicator information for the number Z. Here, Z is a positive integer greater than or equal to 2.

[0124] In the embodiment of the present invention, the initial clustering model is configured with the number Z of clustering result sets and the corresponding Z first center point information. The server obtains the number Z and the corresponding Z first center point information in the configuration of the initial clustering model.

[0125] In this embodiment of the present invention, the server calculates the characteristic distance between each feature vector and the Z first center point information based on multiple initial weights. The server then groups each feature vector into the cluster result set containing the first center point information corresponding to the minimum characteristic distance, thereby obtaining Z cluster result sets.

[0126] S107: Calculate multiple Z-level weights corresponding to multiple features in the Z clustering result sets.

[0127] In the embodiment of the present invention, the server calculates multiple Z-level weights corresponding to multiple features in the Z clustering result sets.

[0128] In the embodiment of the present invention, each time the server classifies multiple feature vectors, it calculates the weights of the multiple features.

[0129] In this embodiment of the present invention, after grouping multiple feature vectors into Z cluster result sets, the server calculates the mean of each feature in each cluster result set. Using this mean, the distance and dispersion of the center point of each cluster result set on the corresponding feature are calculated. Using the ratio of the distance and dispersion corresponding to each feature, the ratio corresponding to each feature is compared with the sum of the ratios of multiple features to obtain the Z-level weight corresponding to each feature.

[0130] S108. According to multiple Z-level weights and the obtained number Z+1 and the corresponding Z+1 first center point information, multiple feature vectors are classified to obtain Z+1 clustering result sets, and the effectiveness index information of the number Z+1 and multiple Z+1-level weights are calculated until the multiple feature vectors are divided into K clustering result sets, and K clustering result sets and effectiveness index information corresponding to the number K are obtained, where K is equal to the preset threshold.

[0131] In an embodiment of the present invention, the server classifies multiple feature vectors according to multiple Z-level weights and the obtained number Z+1 and its corresponding Z+1 first center point information, obtains Z+1 clustering result sets, and calculates the effectiveness index information of the number Z+1 and multiple Z+1-level weights until the multiple feature vectors are divided into K clustering result sets, obtaining K clustering result sets and effectiveness index information corresponding to the number K, where K is equal to the preset threshold.

[0132] In an embodiment of the present invention, after the server divides the multiple feature vectors into Z clustering result sets, it calculates the Z-level weights corresponding to the multiple features, and then selects Z+ feature vectors from the multiple feature vectors as Z+1 first center point information. The server divides the multiple feature vectors into Z+1 clustering result sets according to the feature distances between the multiple feature vectors and the Z+1 first center point information. The server then calculates the validity index information of the number Z+1 and the Z+1-level weights corresponding to the multiple features. Each time the server classifies the multiple feature vectors, it adds one more clustering result set until the multiple feature vectors are divided into K categories. K clustering result sets and validity index information corresponding to the number K are obtained.

[0133] In the embodiment of the present invention, K is an integer smaller than the square root of the number of the current data to be inspected.

[0134] S109. Based on the validity index information corresponding to the numbers Z to K, determine the target number of multiple intermediate clustering result sets and their corresponding multiple first intermediate clustering result sets, and iteratively classify multiple feature vectors based on the multiple first intermediate clustering result sets until the center point information meets the preset conditions, and determine multiple target center point information and multiple intermediate clustering result sets.

[0135] In an embodiment of the present invention, the server determines the target number of multiple intermediate clustering result sets and their corresponding multiple first intermediate clustering result sets based on the effectiveness index information corresponding to the numbers Z to K, and iteratively classifies multiple feature vectors based on the multiple first intermediate clustering result sets until the center point information meets the preset conditions, thereby determining multiple target center point information and multiple intermediate clustering result sets.

[0136] In this embodiment of the present invention, the server determines the target number corresponding to the minimum validity indicator information from the validity indicator information corresponding to the numbers Z to K, and the corresponding multiple first intermediate clustering result sets. The server iteratively classifies the multiple feature vectors based on the multiple first intermediate clustering result sets. After each classification, the server calculates the center point information of each clustering result set until the center point information of each clustering result set no longer changes, thereby determining multiple target center point information and multiple intermediate clustering result sets.

[0137] The target center point information is the average value of the contained feature vectors calculated after the last iterative classification of multiple intermediate clustering result sets.

[0138] In some embodiments, see Figure 3 , Figure 3 This is an optional flowchart of a data processing method provided by an embodiment of the present invention. S105 to S109 can be implemented through S210 to S215, which will be described in conjunction with each step.

[0139] S210 , initializing the clustering model (k=2, randomly selecting the initial cluster center point).

[0140] S211. Calculate feature weighted distance and divide samples.

[0141] S212. Calculate the K value validity index.

[0142] S213, k=k+1.

[0143] S214, Judgment

[0144] If K is less than Then, the process returns to S211 to continue calculating the feature weighted distance and classifying multiple feature vectors.

[0145] S215. Find the K value corresponding to the minimum K value validity index.

[0146] In the embodiment of the present invention, the server first initializes the clustering model, sets the initial number of clusters k=2, randomly selects the initial cluster center point, calculates the distance between each feature vector in a class and each cluster center point, and assigns different weights to each feature according to its different clustering effects on the entire sample data. This weight comprehensively considers the similarity relationship between data objects within and between classes, that is, obtains the feature weighted distance, and is calculated based on the empirical interval of the number of clusters. (n is the number of multiple current data to be tested) for clustering. During the clustering process, the K value validity index information is calculated, and the number of targets and multiple target center point information corresponding to the minimum validity index information are found. At this time, K should be the optimal value.

[0147] S110 , using the number of targets and information about multiple target center points, initializing the initial clustering model to obtain an optimized clustering model.

[0148] In the embodiment of the present invention, the server initializes the initial clustering model using the number of targets and multiple target center point information to obtain an optimized clustering model.

[0149] S111. Determine, from a plurality of eigenvectors, a neighborhood including a predetermined proportion of eigenvectors.

[0150] In the embodiment of the present invention, the server determines, from a plurality of feature vectors, a neighborhood including a predetermined proportion of feature vectors.

[0151] In the embodiment of the present invention, the server takes one tenth of the mean of all feature vectors as the neighborhood size, that is, Among them, q is the neighborhood size, K is the number of targets, and C j is the jth intermediate clustering result set. The server defines the neighborhood q as Neig q (x i ), that is, the distance from a data point x i The set of the most recent q data points.

[0152] S112. Traverse each feature vector in the multiple intermediate clustering result sets, and combine the neighborhood to divide the multiple feature vectors into core domain clusters and intermediate boundary domain clusters; the core domain clusters include: a result set corresponding to the number of multiple intermediate clustering result sets; the intermediate boundary domain clusters include: a certain number of feature vectors at the boundaries of the multiple intermediate clustering result sets.

[0153] In this embodiment of the present invention, the server traverses each feature vector in multiple intermediate clustering result sets and, based on the neighborhood, divides the multiple feature vectors into core domain clusters and intermediate boundary domain clusters. The core domain clusters include a result set corresponding to the number of intermediate clustering result sets, while the intermediate boundary domain clusters include a certain number of feature vectors at the boundaries of the multiple intermediate clustering result sets.

[0154] S113 , inputting the intermediate boundary domain clusters into the optimized clustering model to obtain boundary domain clusters; the boundary domain clusters include: result sets corresponding to the number of the multiple intermediate clustering result sets.

[0155] In the embodiment of the present invention, the server inputs the intermediate boundary domain clusters into the optimized clustering model to obtain boundary domain clusters; the boundary domain clusters include: result sets corresponding to the number of the plurality of intermediate clustering result sets.

[0156] S114 , combining the result set in the boundary domain cluster with the corresponding result set in the core domain cluster to obtain multiple target clustering result sets.

[0157] In the embodiment of the present invention, the server combines the result set in the boundary domain cluster with the corresponding result set in the core domain cluster to obtain multiple target clustering result sets.

[0158] Exemplarily, the boundary domain cluster includes three clustering result sets, and the core domain cluster also includes three clustering result sets. The server combines the three clustering result sets included in the boundary domain cluster with the corresponding clustering result sets in the core domain cluster to obtain three target clustering result sets.

[0159] Since this scheme optimizes the uncertain data to be inspected through the three-branch decision clustering algorithm, each data to be inspected can be accurately classified, and the number of targets and the information of multiple target center points are determined by the weights calculated by the relevant data of multiple features, the multiple target clustering result sets and the multiple current data to be inspected are more matched, thereby improving the detection accuracy of the multiple current data to be inspected.

[0160] In some embodiments, see Figure 4 , Figure 4 This is an optional flowchart of the data processing method provided by an embodiment of the present invention. S105 to S114 can be implemented through S216 to S226, which will be described in conjunction with each step.

[0161] S216, traditional K-means clustering.

[0162] The k-means clustering algorithm is an iterative clustering analysis algorithm. Its steps are to pre-divide the data into K groups, randomly select K objects as the initial cluster centers, and then calculate the distance between each object and each seed cluster center, and assign each object to the cluster center closest to it.

[0163] S217: Obtain the mean and variance, etc., and determine the weight of each feature dimension.

[0164] Calculate the mean, variance, etc. of all feature vectors in the data set on each feature to determine the weight of each feature.

[0165] S218. Determine the optimal K value and the corresponding initial cluster center point set.

[0166] Initialize the K-means clustering algorithm model using the K value determined by the optimization method and the corresponding initial cluster center point as the initialization conditions;

[0167] S219. Initialize the clustering model (select the optimal K value and the corresponding initial cluster center point).

[0168] S220: Calculate the feature weighted distance and assign the object to the class with the closest distance.

[0169] Calculate the data sample points xi other than the cluster center in the data set to each cluster center u j The feature weighted distance dist(x i ,u j ) and classify them into the cluster result set C with the closest distance j middle

[0170] S221. Recalculate each cluster center.

[0171] Recalculate the new mean u of all data points in each cluster j , update the cluster center points of the cluster.

[0172] S222: Whether the cluster center changes.

[0173] If the cluster center point does not change, go to S223; otherwise, go to S220.

[0174] S223. Output multiple intermediate clustering result sets.

[0175] S224. Divide the core area and the boundary area.

[0176] Use the above optimization method to divide the two-branch clustering result set into boundary area and core area

[0177] S225. Delayed decision clustering in boundary domains.

[0178] Delay decision in the boundary domain, that is, perform K-means clustering on the data objects in the boundary domain again

[0179] S226. Output multiple target clustering result sets.

[0180] The clustering result set outputted from the boundary domain is obtained and merged with the result set from the core domain to output multiple target clustering result sets.

[0181] In this embodiment of the present invention, the server incorporates a three-way decision-making approach to optimize uncertain object points. Furthermore, it incorporates feature weighting and K-value validity metrics to more effectively determine the K value and optimize the initial cluster centers. An improved K-means algorithm is used to cluster web log data and establish a signature database of normal and attack behaviors for more accurate web attack detection.

[0182] In some embodiments, S104 to S106 can be implemented through S115 to S125, which will be described in conjunction with each step.

[0183] S115. Calculate the mean of each feature of the multiple feature vectors, and calculate the variance of each feature by combining the eigenvalue and the mean corresponding to each feature.

[0184] In the embodiment of the present invention, the server calculates the mean of each feature of the multiple feature vectors, and calculates the variance of each feature by combining the eigenvalue corresponding to each feature with the mean.

[0185] S116. Compare the variance of each feature with the sum of the variances of each feature to obtain an initial weight for each feature.

[0186] In the embodiment of the present invention, the server compares the variance of each feature with the sum of the variances of each feature to obtain an initial weight of each feature.

[0187] S117 . Calculate Z feature distances corresponding to each feature vector and the Z first center point information by combining the multiple initial weights and the multiple eigenvalues ​​corresponding to the multiple eigenvectors.

[0188] In the embodiment of the present invention, the server calculates Z feature distances corresponding to each feature vector and the Z first center point information in combination with multiple initial weights and multiple feature values ​​corresponding to the multiple feature vectors.

[0189] In this embodiment of the present invention, the server calculates the difference between each eigenvalue of each eigenvector and the eigenvalue corresponding to each piece of first central point information, and then multiplies the difference by the weight of the corresponding feature to obtain the product corresponding to each feature. The server then adds the products of each feature to obtain the feature distance between each eigenvector and the Z pieces of first central point information.

[0190] For example, the server can calculate the feature vector x by formula (1) i With the jth first center point information u j The feature distance between dist(x i ,u j ).

[0191]

[0192] Where D is the eigenvector x i The number of corresponding features, x id is the eigenvector x i The eigenvalue at the dth feature. u jd is the jth first center point information u j The eigenvalue on the dth feature. ω d is the initial weight corresponding to the dth feature. The server will calculate x id with u jd The difference between d Multiplying them, we get xi with u j The server then adds the distances corresponding to the D features to obtain the feature distance dist(x i ,u j ).

[0193] S118. Determine the center point information corresponding to the minimum feature distance among the Z feature distances corresponding to each feature vector, and divide each feature vector into a clustering result set of the corresponding center point information, thereby obtaining Z clustering result sets.

[0194] In an embodiment of the present invention, the server determines the center point information corresponding to the minimum feature distance among the Z feature distances corresponding to each feature vector, divides each feature vector into the clustering result set of the corresponding center point information, and thus obtains Z clustering result sets.

[0195] S119. Calculate Z intra-cluster distances corresponding to the Z clustering result sets.

[0196] In the embodiment of the present invention, the server calculates Z intra-cluster distances corresponding to the Z clustering result sets.

[0197] In the embodiment of the present invention, the clustering result set C i For the i-th clustering result set among the Z clustering result sets, define the intra-class distance Intra(C i ) is the clustering result set C i The sum of the squares of the distances between any two eigenvectors x and y can be calculated using formula (2) to obtain the clustering result set C: i The intra-class distance.

[0198]

[0199] Among them, x, y are the clustering result set C i The eigenvalue of the dth feature of any two eigenvectors in . D is the number of multiple features. ω d is the initial weight of the dth feature. The server calculates the square of the difference between the eigenvalues ​​of x and y on the dth feature and then adds it to ω d Multiply them together to get the product corresponding to the d-th feature. The server adds the products of the calculated D features to get the intra-class distance Intra(C i ). Then the server can use the same method to calculate Z intra-class distances

[0200] S120: Calculate the sum of the inter-class distances between each of the Z clustering result sets.

[0201] In the embodiment of the present invention, the server calculates the sum of the inter-cluster distances between each of the Z clustering result sets.

[0202] In the embodiment of the present invention, the inter-class distance Inter(C i ,C j ) is the clustering result set C i To the clustering result set C j The server can calculate the clustering result set C by formula (3) i To the clustering result set C j Inter-class distance Inter(C i ,C j ).

[0203]

[0204] Among them, q and p are the clustering result set C i and clustering result set C j The number of eigenvectors in . Z is the number of Z clustering result sets. x is the clustering result set C i The mean of the eigenvalues ​​of the dth feature in , y is the clustering result set C j The mean of the eigenvalues ​​of the dth feature in . D is the number of features. ω d is the initial weight of the dth feature. The server calculates the square of the difference between x and y on the dth feature and then adds it to ω d Multiply them together to get the product corresponding to the d-th feature. The server adds the products of the calculated D features to get the clustering result set C. i To the clustering result set C j Inter-class distance Inter(C i ,C j ). Then the server can use the same method to calculate the sum of the distances between classes.

[0205] S121. Calculate the ratio of the sum of Z intra-class distances to the inter-class distances to obtain the validity indicator information of the number Z.

[0206] In the embodiment of the present invention, the server calculates the ratio of the sum of Z intra-class distances to the inter-class distances to obtain the effectiveness indicator information of the number Z.

[0207] In the embodiment of the present invention, the server uses the ratio of the two to balance the distance within a class and the distance between classes, so the effectiveness index information S that defines the number of clusters Z can be calculated using formula (4).

[0208]

[0209] When S is the smallest, the intra-class distance and inter-class distance reach a balance value, which corresponds to the target. Calculate the number of targets k best As shown in formula (5).

[0210]

[0211] in, Characterizes K to be greater than or equal to 2 and less than or equal to n When the number of S corresponding to the minimum is the target number k best .

[0212] S122. Calculate a first mean of each feature in each clustering result set, and calculate a first variance of each feature in each clustering result set by combining the eigenvalue corresponding to each feature with the first mean.

[0213] In the embodiment of the present invention, the server calculates the first mean of each feature in each clustering result set, and calculates the first variance of each feature in each clustering result set by combining the feature value corresponding to each feature with the first mean.

[0214] In the embodiment of the present invention, after the server divides the multiple feature vectors into Z cluster result sets, it obtains the feature vectors with N D-dimensional features in the Z cluster result sets. The server can calculate the first variance of the d-th feature of the Z-th cluster result set by formula (6):

[0215]

[0216] Where N is the number of eigenvectors in the Zth clustering result set. is the first mean of the dth feature of the Zth clustering result set, x id The server calculates the eigenvalue of each eigenvalue and The square value of the difference, add up the various level defense values ​​and then divide by N to get the first variance The server can then use the same method to calculate the first variance of each feature.

[0217] S123. Subtract the first mean of each feature of each pairwise clustering result set in the Z clustering result sets, and then calculate the square of the difference to obtain the cluster center point distance corresponding to each feature of each pairwise clustering result set in the Z clustering result sets.

[0218] In an embodiment of the present invention, the server subtracts the first mean of each feature of the pairwise clustering result sets in the Z clustering result sets, and then calculates the square of the difference to obtain the cluster center point distance corresponding to each feature of the pairwise clustering result sets in the Z clustering result sets.

[0219] In the embodiment of the present invention, the server can calculate the cluster center distance corresponding to the d-th feature by formula (7):

[0220]

[0221] in, is the first mean of the i-th clustering result set on the d-th feature, is the first mean of the jth clustering result set on the dth feature. and The square value of the difference, until i = Z, j = Z. Get the distance of the cluster center point on the dth feature of the Z cluster result set

[0222] S124. Sum the first variance of each feature of each pair of clustering result sets in the Z clustering result sets, and then calculate the square of the sum to obtain the intra-class dispersion degree of each feature.

[0223] In the embodiment of the present invention, the server sums the first variance of each feature of each pair of clustering result sets in the Z clustering result sets, and then calculates the square of the sum to obtain the intra-class dispersion degree of each feature.

[0224] In the embodiment of the present invention, the server can calculate the intra-class discreteness of the dth feature by formula (8):

[0225]

[0226] in, is the first variance of the i-th clustering result set on the d-th feature, The first variance of the j-th clustering result set on the d-th feature. Server calculation and The sum of squares until i = Z, j = Z. Get the degree of intra-class dispersion of the Z clustering result sets on the dth feature

[0227] S125. Calculate the distance between the cluster centers of each feature and the ratio of the distance to the corresponding intra-class dispersion to obtain the clustering quality of each feature. Compare the clustering quality of each feature with the sum of the clustering qualities of multiple features to obtain the Z-level weight of each feature, and then obtain multiple Z-level weights.

[0228] In an embodiment of the present invention, the server calculates the distance between the cluster center points of each feature and the ratio of the distance to the corresponding intra-class discreteness to obtain the clustering quality of each feature, compares the clustering quality of each feature with the sum of the clustering qualities of multiple features to obtain the Z-level weight of each feature, and then obtains multiple Z-level weights.

[0229] In the embodiment of the present invention, the server may use formula (9): Calculate the clustering quality of the dth feature. is the distance between cluster centers on the dth feature, is the intra-class dispersion of the d-th feature.

[0230] In the embodiment of the present invention, the server adopts formula (10): Calculate the Z-level weight ω of the d-th feature d . Among them, R(d) is the clustering quality of the d-th feature, is the sum of the clustering qualities of D features.

[0231] In an embodiment of the present invention, each time the server classifies multiple feature vectors, it calculates multiple weights of the multiple features, so the number of targets determined according to the weights and the information of multiple target center points are more consistent with the multiple current data. Therefore, this solution can enable the server to detect the data to be inspected more accurately.

[0232] In some embodiments, S108 can be implemented through S126 to S128, which will be described in conjunction with each step.

[0233] S126. Determine, from the validity indicator information corresponding to the numbers Z to K, a target number corresponding to the minimum validity indicator information and a plurality of first intermediate clustering result sets corresponding to the target number.

[0234] In the embodiment of the present invention, the server determines the target number corresponding to the minimum effectiveness index information and a plurality of first intermediate clustering result sets corresponding to the target number from the effectiveness index information corresponding to the numbers Z to K.

[0235] S127 . Calculate new means of the plurality of eigenvectors based on the eigenvectors in the plurality of first intermediate clustering result sets to obtain a plurality of new center point information.

[0236] In the embodiment of the present invention, the server calculates new means of the plurality of eigenvectors based on the eigenvectors in the plurality of first intermediate clustering result sets, so as to obtain a plurality of new center point information.

[0237] S128. If the multiple new center point information does not match the corresponding multiple first center point information, the multiple feature vectors are classified according to the target number and the multiple new center point information to obtain new multiple first intermediate clustering result sets, so as to iterate the multiple new center point information until the multiple new center point information matches the previous multiple center point information, and multiple target center point information and multiple intermediate clustering result sets are obtained.

[0238] In an embodiment of the present invention, if the server detects that multiple new center point information do not match the corresponding multiple first center point information, the server classifies the multiple feature vectors according to the number of targets and the multiple new center point information to obtain new multiple first intermediate clustering result sets, so as to iterate the multiple new center point information until the multiple new center point information matches the previous multiple center point information, and obtains multiple target center point information and multiple intermediate clustering result sets.

[0239] In an embodiment of the present invention, the server determines the number of targets when the ratio of the intra-class distance to the inter-class distance is minimized, which makes the division boundaries of each intermediate clustering result set more obvious and the classification of each data to be inspected clearer. Therefore, this solution can enable the server to detect the data to be inspected more accurately.

[0240] In some embodiments, S112 can be implemented through S129 to S131, which will be described in conjunction with each step.

[0241] S129. Traverse the i-th eigenvector in the j-th intermediate clustering result set; i is a positive integer greater than or equal to 1 and less than n; j is a positive integer greater than or equal to 1 and less than k; n is the number of eigenvectors in the corresponding intermediate clustering result set; k is the number of multiple intermediate clustering result sets.

[0242] In an embodiment of the present invention, the server traverses the i-th eigenvector in the j-th intermediate clustering result set, where i is a positive integer greater than or equal to 1 and less than n; j is a positive integer greater than or equal to 1 and less than k; n is the number of eigenvectors in the corresponding intermediate clustering result set; and k is the number of multiple intermediate clustering result sets.

[0243] S130. If the i-th eigenvector belongs to the j-th intermediate clustering result set, and the j-th intermediate clustering result set contains the neighborhood; or, the i-th eigenvector does not belong to the j-th intermediate clustering result set, and the intersection of the j-th intermediate clustering result set and the neighborhood is not an empty set, then the i-th eigenvector is divided into the intermediate boundary domain cluster, otherwise the i-th eigenvector is divided into the core domain cluster.

[0244] In the embodiment of the present invention, the server detects that if the i-th feature vector belongs to the j-th intermediate clustering result set, and the j-th intermediate clustering result set contains the neighborhood; or, if the i-th feature vector does not belong to the j-th intermediate clustering result set, and the intersection of the j-th intermediate clustering result set and the neighborhood is not an empty set, then the i-th feature vector is divided into the intermediate boundary domain cluster, otherwise the i-th feature vector is divided into the core domain cluster

[0245] S131 , stop when the nth eigenvector in the kth intermediate clustering result set is traversed, and obtain the core domain cluster and the intermediate boundary domain cluster.

[0246] In the embodiment of the present invention, the server stops when traversing the nth feature vector in the kth intermediate clustering result set, and obtains the core domain cluster and the intermediate boundary domain cluster.

[0247] In the embodiment of the present invention, the server traverses each feature vector in multiple intermediate clustering result sets to determine the core domain clusters and the boundary domain clusters, thereby making the classification of each feature vector clearer, thereby improving the accuracy of the server's detection of the data to be inspected.

[0248] In some embodiments, see Figure 5 , Figure 5 This is an optional flowchart of a data processing method provided by an embodiment of the present invention. S111 to S114 can be implemented through S227 to S243, which will be described in conjunction with each step.

[0249] S227, traditional K-means clustering algorithm.

[0250] S228. Obtain multiple intermediate clustering result sets, calculate the size of the neighborhood q, and obtain the q neighborhood Neig(xi).

[0251] S229. Traverse the result set Cj, and first set j=1.

[0252] S230 , traverse the data set object xi, first setting i=1.

[0253] S231,xi∈Cj.

[0254] S232, Neig(xi) are completely contained in Cj.

[0255] S233, Neig(xi)∩Cj≠empty set.

[0256] S234. Store in the boundary domain result set.

[0257] S235. Store in the core domain result set.

[0258] S236. Merge the result sets of the two domains.

[0259] S237, i=i+1.

[0260] S238, i≤n.

[0261] S239, j=j+1.

[0262] S240, j≤k.

[0263] S241. Obtain boundary domain and core domain result sets.

[0264] S242. Perform secondary clustering on the boundary domain.

[0265] S243. Output multiple target clustering result sets.

[0266] In the embodiment of the present invention, the server traverses multiple intermediate clustering result sets C, traverses the data objects x in the data set i , if x i ∈C j , and when or if And when Neig q (x i )∩C j ≠Φ, then this x i Divide into boundary domain clusters Otherwise, it is divided into the core domain cluster Merge the result sets in the two domains and continue the cycle. Finally, we get the result sets of the core domain and the boundary domain. Assume the result set of the core domain Only for the result set of the boundary domain Perform secondary K-means clustering, that is, make delayed decisions on the data objects in the boundary domain, and obtain the result set C of the boundary domain B″ Output the final clustering result set C", that is, C" = {C C′ ,C B″ The result set is a result set of a determined core domain obtained by two-branch decision clustering of the data set sample points and a multiple target clustering result set obtained by delayed decision of the data objects in the boundary domain.

[0267] In some embodiments, S101 can be implemented through S132 to S134 , which will be described in conjunction with each step.

[0268] S132: Acquire multiple pieces of current data to be inspected at the current moment.

[0269] In the embodiment of the present invention, the server obtains a plurality of current to-be-inspected data at the current moment.

[0270] S133 , performing impurity and duplication removal processing on the multiple current data to be inspected to obtain multiple intermediate data; each intermediate data includes: multiple characteristic keywords.

[0271] In the embodiment of the present invention, the server performs impurity and duplication removal processing on a plurality of current data to be inspected to obtain a plurality of intermediate data, wherein each intermediate data includes: a plurality of characteristic keywords.

[0272] S134 , performing structural processing on keywords of multiple features of multiple intermediate data to obtain multiple feature vectors corresponding to multiple current data to be inspected.

[0273] In the embodiment of the present invention, the server performs structured processing on keywords of multiple features of multiple intermediate data to obtain multiple feature vectors corresponding to multiple current data to be inspected.

[0274] In some embodiments, see Figure 6 , Figure 6 This is an optional flowchart of a data processing method provided by an embodiment of the present invention. S132 to S134 can be implemented through S244 to S247, which will be explained in conjunction with each step.

[0275] S244. Data preprocessing.

[0276] S245. HTTP data feature extraction.

[0277] S246. Detection model.

[0278] S247. Detect and classify HTTP requests.

[0279] The access log on the server records all Hypertext Transfer Protocol (HTTP) requests. Each record represents an HTTP request and includes the IP address, URL accessed, date and time of access, access results, user website, and configuration information. The following is a partial record from the access_log file on the Apache server.

[0280] 127.0.0.1--[09 / Mar / 2020:11:32:32+0800]"GET / YouweiWechat / servlet / oauthServlet?code=001QMDOr0Xf6Vd1Oi4Mr0nIiOr0QMDOw&state=456HTTP / 1.1"4041017

[0281] 127.0.0.1--[09 / Mar / 2020:11:32:33+0800]"GET / HTTP / 1.1"200 11418

[0282] The purpose of data preprocessing is to remove records in the log that contain abnormal status codes returned by the server, as well as records that are not related to attack detection and analysis. This includes removing information such as graphics, web page style files, abnormal status codes, web crawlers, and reducing the size of log files.

[0283] The data extracted from HTTP request packets is unstructured and requires structural processing. Feature selection is performed on the raw data, and characteristic keywords are matched. For example, common characteristic keywords for Structured Query Language (SQL) injection attacks include "and," "or," "group," and "by," while characteristic keywords for Cross Site Scripting (XSS) attacks include "script," "hash," and "href." Without reducing classification accuracy, the dimensionality of the original feature space is reduced, and the raw data is converted into a fixed-dimensional feature vector, which serves as the input data for Web attack detection methods.

[0284] In some embodiments, S104 to S134 can be implemented through S135 to S138, which will be described in conjunction with each step.

[0285] S135 , matching multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set with multiple result keywords corresponding to each matching result set in the normal behavior matching result set.

[0286] S136: If the matching degree of the multiple keywords with any one of the normal behavior matching result sets reaches a threshold, the current data to be inspected corresponding to the first clustering result set is determined to be normal data.

[0287] S137 . Otherwise, multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set are matched with multiple result keywords corresponding to each matching result set in the attack behavior matching result set.

[0288] S138. If the matching degree of the corresponding multiple result keywords between the multiple keywords and any one of the attack behavior matching result sets reaches a threshold, the current data to be inspected corresponding to the first clustering result set is determined to be attack data, until the multiple target clustering result sets are matched, and the detection results of the multiple current data to be inspected are obtained.

[0289] In some embodiments, S101 also includes S139 to S142, which will be explained in conjunction with each step.

[0290] S139: Process the acquired multiple historical data to obtain multiple historical feature vectors corresponding to the multiple historical data.

[0291] S140. Utilize the relevant data of multiple features corresponding to multiple historical feature vectors to determine the number of historical targets and the center point information of multiple historical targets corresponding to multiple historical intermediate clustering result sets, so as to classify the multiple historical feature vectors to obtain multiple historical intermediate clustering result sets.

[0292] S141. Processing multiple historical intermediate clustering result sets using a clustering model optimized by the number of historical targets and the information of multiple historical target center points, and a three-branch decision clustering algorithm to obtain multiple historical target clustering result sets.

[0293] S142. Detect multiple historical target result sets to obtain multiple detection results corresponding to the multiple historical target result sets, and map the multiple historical target result sets to the multiple detection results to form multiple matching result sets; and divide the multiple matching result sets into two categories: normal behavior matching result sets and attack behavior matching result sets according to the detection results.

[0294] In some embodiments, see Figure 7 , Figure 7 A schematic diagram of the structure of a data processing method provided by an embodiment of the present invention.

[0295] The embodiment of the present invention further provides a data detection device 800 , including: a processing unit 803 , a determination unit 804 and a matching unit 805 .

[0296] The processing unit 803 is configured to process the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected;

[0297] A determining unit 804 is configured to determine the number of targets and information about target centers corresponding to multiple intermediate clustering result sets using relevant data of multiple features corresponding to the multiple feature vectors, so as to classify the multiple feature vectors to obtain the multiple intermediate clustering result sets;

[0298] The processing unit 805 is further configured to process the multiple intermediate clustering result sets using the clustering model optimized by the number of targets and the multiple target center point information and a three-branch decision clustering algorithm to obtain multiple target clustering result sets;

[0299] A matching unit is used to match the multiple target clustering result sets with multiple preset matching result sets to determine the detection results of the multiple current data to be tested; the multiple matching result sets are obtained from multiple historical data through the same processing method as the multiple target clustering result sets.

[0300] In the embodiment of the present invention, the determination unit 804 in the data detection device 800 is used to calculate the mean of each feature corresponding to the multiple feature vectors, and calculate multiple initial weights of the multiple features based on the mean combined with the feature value of the corresponding feature; based on the multiple initial weights, and the number Z of clustering result sets of the obtained initial clustering model and the corresponding Z first center point information, the multiple feature vectors are classified to obtain Z clustering result sets, and the validity index information of the number Z is calculated; Z is a positive integer greater than or equal to 2; multiple Z-level weights corresponding to the multiple features in the Z clustering result sets are calculated; based on the multiple Z-level weights, and the obtained number Z+1 and the corresponding Z+1 first center point information, the multiple feature vectors are classified to obtain Z clustering result sets, and the validity index information of the number Z is calculated; Z is a positive integer greater than or equal to 2; multiple Z-level weights corresponding to the multiple features in the Z clustering result sets are calculated; based on the multiple Z-level weights, and the obtained number Z+1 and the corresponding Z+1 first center point information The method further comprises the following steps: first, classifying the multiple feature vectors according to the center point information, obtaining Z+1 clustering result sets, and calculating the effectiveness index information of the number Z+1 and multiple Z+1 level weights, until the multiple feature vectors are divided into K clustering result sets, obtaining K clustering result sets and the effectiveness index information corresponding to the number K, wherein K is equal to a preset threshold; according to the effectiveness index information corresponding to the numbers Z to K, determining the target number of multiple intermediate clustering result sets and the corresponding multiple first intermediate clustering result sets, and iteratively classifying the multiple feature vectors according to the multiple first intermediate clustering result sets, until the center point information meets the preset conditions, and determining the multiple target center point information and the multiple intermediate clustering result sets.

[0301] In an embodiment of the present invention, the processing unit 803 in the data detection device 800 is used to initialize the initial clustering model using the target number and the multiple target center point information to obtain the optimized clustering model; determine a neighborhood including a predetermined proportion of feature vectors from the multiple feature vectors; traverse each feature vector in the multiple intermediate clustering result sets, and combine the neighborhood to divide the multiple feature vectors into core domain clusters and intermediate boundary domain clusters; the core domain cluster includes: a result set corresponding to the number of the multiple intermediate clustering result sets; the intermediate boundary domain cluster includes: a certain number of feature vectors at the boundary of the multiple intermediate clustering result sets; the intermediate boundary domain cluster is input into the optimized clustering model to obtain a boundary domain cluster; the boundary domain cluster includes: a result set corresponding to the number of the multiple intermediate clustering result sets; the result set in the boundary domain cluster is combined with the corresponding result set in the core domain cluster to obtain the multiple target clustering result sets.

[0302] In an embodiment of the present invention, the determination unit 804 in the data detection device 800 is used to calculate the mean of each feature of the multiple feature vectors, and calculate the variance of each feature by combining the eigenvalue corresponding to each feature with the mean; the variance of each feature is compared with the sum of the variances of each feature to obtain the initial weight of each feature.

[0303] In an embodiment of the present invention, the determination unit 804 in the data detection device 800 is used to combine the multiple initial weights and the multiple eigenvalues ​​corresponding to the multiple eigenvectors to calculate the Z feature distances corresponding to each eigenvector and the Z first center point information; among the Z feature distances corresponding to each eigenvector, determine the center point information corresponding to the minimum feature distance, and divide each eigenvector into the clustering result set of the corresponding center point information, thereby obtaining the Z clustering result sets.

[0304] In an embodiment of the present invention, the determination unit 804 in the data detection device 800 is used to calculate the Z intra-class distances corresponding to the Z clustering result sets; calculate the sum of the inter-class distances between each of the Z clustering result sets; calculate the ratio of the Z intra-class distances to the sum of the inter-class distances, and obtain the effectiveness index information of the number Z.

[0305] In an embodiment of the present invention, the determination unit 804 in the data detection device 800 is used to calculate the first mean of each feature in each clustering result set, and calculate the first variance of each feature in each clustering result set by combining the feature value corresponding to each feature and the first mean; subtract the first mean of each feature in the two clustering result sets in the Z clustering result sets, and then calculate the square of the difference to obtain the cluster center point distance corresponding to each feature in the two clustering result sets in the Z clustering result sets; sum the first variance of each feature in the two clustering result sets in the Z clustering result sets, and then calculate the square of the sum to obtain the intra-class dispersion degree of each feature; calculate the cluster center point distance of each feature and the ratio of the corresponding intra-class dispersion degree to obtain the clustering quality of each feature, and compare the clustering quality of each feature with the sum of the clustering qualities of the multiple features to obtain the Z-level weight of each feature, and then obtain multiple Z-level weights.

[0306] In an embodiment of the present invention, the determination unit 804 in the data detection device 800 is used to determine the target number corresponding to the minimum validity index information and the multiple first intermediate clustering result sets corresponding to the target number in the validity index information corresponding to the numbers Z to K; calculate the new mean of the multiple feature vectors based on the feature vectors in the multiple first intermediate clustering result sets to obtain multiple new center point information; if the multiple new center point information does not match the corresponding multiple first center point information, classify the multiple feature vectors according to the target number and the multiple new center point information to obtain new multiple first intermediate clustering result sets, so as to iterate the multiple new center point information until the multiple new center point information matches the previous multiple center point information, and obtain the multiple target center point information and the multiple intermediate clustering result sets.

[0307] In the embodiment of the present invention, the preset threshold value includes: an integer obtained by square rooting the number of the plurality of current data to be inspected.

[0308] In an embodiment of the present invention, the processing unit 803 in the data detection device 800 is used to traverse the i-th eigenvector in the j-th intermediate clustering result set; i is a positive integer greater than or equal to 1 and less than n; j is a positive integer greater than or equal to 1 and less than k; n is the number of eigenvectors in the corresponding intermediate clustering result set; k is the number of the multiple intermediate clustering result sets; if the i-th eigenvector belongs to the j-th intermediate clustering result set, and the j-th intermediate clustering result set contains the neighborhood; or, the i-th eigenvector does not belong to the j-th intermediate clustering result set, and the intersection of the j-th intermediate clustering result set and the neighborhood is not an empty set, then the i-th eigenvector is divided into the intermediate boundary domain cluster, otherwise the i-th eigenvector is divided into the core domain cluster; until the n-th eigenvector in the k-th intermediate clustering result set is traversed, the process stops and the core domain cluster and the intermediate boundary domain cluster are obtained.

[0309] In an embodiment of the present invention, the processing unit 803 in the data detection device 800 is used to obtain multiple current data to be detected at the current moment; perform impurity removal and deduplication processing on the multiple current data to be detected to obtain multiple intermediate data; each intermediate data includes: keywords of multiple features; perform structured processing on the keywords of multiple features of the multiple intermediate data to obtain multiple feature vectors corresponding to the multiple current data to be detected.

[0310] In an embodiment of the present invention, the multiple matching result sets include two categories: normal behavior matching result sets and attack behavior matching result sets; the matching unit 805 in the data detection device 800 is used to match multiple keywords of the current data to be detected corresponding to the feature vector in the first clustering result set with multiple result keywords corresponding to each matching result set in the normal behavior matching result set; if the matching degree of the multiple keywords with the multiple result keywords corresponding to any matching result set in the normal behavior matching result set reaches a threshold, then the current data to be detected corresponding to the first clustering result set is determined to be normal data; otherwise, the multiple keywords of the current data to be detected corresponding to the feature vector in the first clustering result set are matched with the multiple result keywords corresponding to each matching result set in the attack behavior matching result set; if the matching degree of the multiple keywords with any matching result set in the attack behavior matching result set reaches a threshold, then the current data to be detected corresponding to the first clustering result set is determined to be attack data, until the multiple target clustering result sets are matched, and the detection results of the multiple current data to be detected are obtained.

[0311] In an embodiment of the present invention, the processing unit 803 in the data detection device 800 is used to process the acquired multiple historical data to obtain multiple historical feature vectors corresponding to the multiple historical data; the determination unit 804 is used to use the relevant data of the multiple features corresponding to the multiple historical feature vectors to determine the number of historical targets and the center point information of the multiple historical targets corresponding to the multiple historical intermediate clustering result sets, so as to classify the multiple historical feature vectors to obtain multiple historical intermediate clustering result sets, so that the ratio of the sum of the intra-class distances of each historical intermediate clustering result set to the sum of the inter-class distances between the each historical intermediate clustering result set meets a preset condition; The processing unit 803 is also used to process the multiple historical intermediate clustering result sets through the clustering model optimized by the number of historical targets and the center point information of the multiple historical targets, and the three-branch decision clustering algorithm to obtain multiple historical target clustering result sets; the matching unit 805 is used to detect the multiple historical target result sets, obtain multiple detection results corresponding to the multiple historical target result sets, and map the multiple historical target result sets with the multiple detection results to form the multiple matching result sets; and divide the multiple matching result sets into two categories: normal behavior matching result sets and attack behavior matching result sets according to the detection results.

[0312] In an embodiment of the present invention, the acquired multiple current data to be inspected are processed by the processing unit 803 to obtain multiple feature vectors corresponding to the multiple current data to be inspected; the determination unit 804 uses the relevant data of the multiple features corresponding to the multiple feature vectors to determine the number of targets and the multiple target center point information corresponding to the multiple intermediate clustering result sets, so as to classify the multiple feature vectors to obtain the multiple intermediate clustering result sets; the processing unit 803 processes the multiple intermediate clustering result sets through the clustering model optimized by the number of targets and the multiple target center point information, and the three-branch decision clustering algorithm to obtain multiple target clustering result sets; the matching unit 805 matches the multiple target clustering result sets with the preset multiple matching result sets to determine the detection results of the multiple current data to be inspected; the multiple matching result sets are obtained for multiple historical data by the same processing method as the multiple target clustering result sets. Since this scheme optimizes the uncertain data to be inspected through the three-branch decision clustering algorithm, each data to be inspected can be accurately classified, and the number of targets and the information of multiple target center points are calculated through the relevant data of multiple features, the multiple target clustering result sets and the multiple current data to be inspected are more matched, thereby improving the detection accuracy of the multiple current data to be inspected.

[0313] It should be noted that, in the embodiment of the present invention, if the above-mentioned data detection method is implemented in the form of a software function module and is sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a data detection device (which can be a personal computer, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0314] Correspondingly, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0315] Correspondingly, an embodiment of the present invention provides a data detection device, including a memory 802 and a processor 801, wherein the memory 802 stores a computer program that can be run on the processor 801, and the processor 801 implements the steps in the above method when executing the program.

[0316] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0317] It should be noted that Figure 8 A schematic diagram of a hardware entity of a data detection device provided by an embodiment of the present invention, such as Figure 8 As shown, the hardware entity of the data detection device 800 includes: a processor 801 and a memory 802, wherein;

[0318] The processor 801 generally controls the overall operation of the data detection apparatus 800 .

[0319] The memory 802 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed or processed by the processor 801 and each module in the data detection device 800 (for example, image data, audio data, voice communication data and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0320] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0321] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0322] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0323] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0324] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0325] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0326] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0327] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A data detection method, characterized in that: include: Processing the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected; Determining the number of targets and target center point information corresponding to a plurality of intermediate clustering result sets using relevant data of a plurality of features corresponding to the plurality of feature vectors, so as to classify the plurality of feature vectors to obtain the plurality of intermediate clustering result sets; Initializing the initial clustering model using the number of targets and the information of the multiple target center points to obtain an optimized clustering model; Determining, from the plurality of feature vectors, a neighborhood including a predetermined proportion of feature vectors; Traversing each feature vector in the plurality of intermediate clustering result sets, and combining the neighborhoods to divide the plurality of feature vectors into core domain clusters and intermediate boundary domain clusters; The core domain cluster includes: a result set corresponding to the number of the plurality of intermediate clustering result sets; the intermediate boundary domain cluster includes: a certain number of feature vectors at the boundaries of the plurality of intermediate clustering result sets; Inputting the intermediate boundary domain clusters into the optimized clustering model to obtain boundary domain clusters; the boundary domain clusters include: result sets corresponding to the number of the multiple intermediate clustering result sets; Combining the result set in the boundary domain cluster with the corresponding result set in the core domain cluster to obtain multiple target clustering result sets; The multiple target clustering result sets are matched with multiple preset matching result sets to determine the detection results of the multiple current data to be detected; the multiple matching result sets are obtained from multiple historical data using the same processing method as the multiple target clustering result sets.

2. The data detection method according to claim 1, characterized in that: The method of using the relevant data of the multiple features corresponding to the multiple feature vectors to determine the number of targets and multiple target center point information corresponding to the multiple intermediate clustering result sets, so as to classify the multiple feature vectors to obtain the multiple intermediate clustering result sets, includes: Calculating a mean value of each feature corresponding to the multiple feature vectors, and calculating multiple initial weights of the multiple features based on the mean value combined with the feature value of the corresponding feature; Classifying the plurality of feature vectors according to the plurality of initial weights, the number Z of clustering result sets of the obtained initial clustering model and the corresponding Z first center point information to obtain Z clustering result sets, and calculating validity indicator information of the number Z; Z is a positive integer greater than or equal to 2; Calculating a plurality of Z-level weights corresponding to a plurality of features in the Z clustering result sets; Classifying the plurality of feature vectors according to the plurality of Z-level weights and the obtained number Z+1 and the corresponding Z+1 first center point information to obtain Z+1 clustering result sets, and calculating the validity index information of the number Z+1 and the plurality of Z+1-level weights, until the plurality of feature vectors are divided into K clustering result sets, obtaining K clustering result sets and validity index information corresponding to the number K, where K is equal to a preset threshold; According to the validity index information corresponding to the numbers Z to K, the target number of multiple intermediate clustering result sets and their corresponding multiple first intermediate clustering result sets are determined, and the multiple feature vectors are iteratively classified according to the multiple first intermediate clustering result sets until the center point information meets the preset conditions, and the multiple target center point information and the multiple intermediate clustering result sets are determined.

3. The data detection method according to claim 2, characterized in that: The step of calculating the mean of each feature corresponding to the plurality of feature vectors, and calculating a plurality of initial weights of the plurality of features based on the mean combined with the feature value of the corresponding feature, includes: Calculating the mean of each feature of the multiple feature vectors, and calculating the variance of each feature by combining the eigenvalue corresponding to each feature with the mean; The variance of each feature is compared with the sum of the variances of each feature to obtain an initial weight of each feature.

4. The data detection method according to claim 2, characterized in that: The multiple feature vectors are classified according to the multiple initial weights, the number Z of clustering result sets of the obtained initial clustering model, and the corresponding Z first center point information to obtain Z clustering result sets, including: Calculate, in combination with the multiple initial weights and the multiple eigenvalues ​​corresponding to the multiple eigenvectors, Z feature distances corresponding to each eigenvector and the Z first center point information; Among the Z feature distances corresponding to each feature vector, the center point information corresponding to the minimum feature distance is determined, and each feature vector is divided into a clustering result set of the corresponding center point information, thereby obtaining the Z clustering result sets.

5. The data detection method according to claim 2, characterized in that: The validity indicator information of the calculation number Z includes: Calculating Z intra-class distances corresponding to the Z clustering result sets; Calculate the sum of the inter-class distances between each of the Z clustering result sets; The ratio of the Z intra-class distances to the sum of the inter-class distances is calculated to obtain the effectiveness index information of the number Z.

6. The data detection method according to claim 2, characterized in that: The calculating a plurality of Z-level weights corresponding to the plurality of features in the Z clustering result sets includes: Calculating a first mean of each feature in each clustering result set, and calculating a first variance of each feature in each clustering result set by combining a feature value corresponding to each feature with the first mean; Subtract the first mean of each feature of the pairwise clustering result sets in the Z clustering result sets, and then calculate the square of the difference to obtain the cluster center point distance corresponding to each feature of the pairwise clustering result sets in the Z clustering result sets; Sum the first variance of each feature of each pair of clustering result sets in the Z clustering result sets, and then calculate the square of the sum to obtain the intra-class dispersion degree of each feature; The cluster quality of each feature is obtained by calculating the ratio of the cluster center point distance of each feature to the corresponding intra-class discreteness. The cluster quality of each feature is compared with the sum of the cluster qualities of the multiple features to obtain the Z-level weight of each feature, and then multiple Z-level weights are obtained.

7. The data detection method according to claim 2, characterized in that: The method further comprises: determining a target number of multiple intermediate clustering result sets and corresponding multiple first intermediate clustering result sets based on the validity indicator information corresponding to the numbers Z to K, and iteratively classifying the multiple feature vectors based on the multiple first intermediate clustering result sets until the center point information meets a preset condition, thereby determining the multiple target center point information and the multiple intermediate clustering result sets. Determine the target number corresponding to the minimum effectiveness index information among the effectiveness index information corresponding to the numbers Z to K, and a plurality of first intermediate clustering result sets corresponding to the target number; According to the calculation of the characteristic vectors in the plurality of first intermediate clustering result sets, new means of the plurality of characteristic vectors are calculated to obtain a plurality of new center point information; If the multiple new center point information does not match the corresponding multiple first center point information, the multiple feature vectors are classified according to the target number and the multiple new center point information to obtain new multiple first intermediate clustering result sets, so as to iterate the multiple new center point information until the multiple new center point information matches the previous multiple center point information, and the multiple target center point information and the multiple intermediate clustering result sets are obtained.

8. The data detection method according to claim 2, characterized in that: The preset threshold value includes: an integer obtained by square rooting the number of the multiple current data to be inspected.

9. The data detection method according to claim 1, characterized in that: The traversing each feature vector in the plurality of intermediate clustering result sets and combining the neighborhood to divide the plurality of feature vectors into core domain clusters and intermediate boundary domain clusters includes: Traverse the i-th eigenvector in the j-th intermediate clustering result set; i is a positive integer greater than or equal to 1 and less than n; j is a positive integer greater than or equal to 1 and less than k; n is the number of eigenvectors in the corresponding intermediate clustering result set; k is the number of the multiple intermediate clustering result sets; If the i-th eigenvector belongs to the j-th intermediate clustering result set, and the j-th intermediate clustering result set contains the neighborhood; or, the i-th eigenvector does not belong to the j-th intermediate clustering result set, and the intersection of the j-th intermediate clustering result set and the neighborhood is not an empty set, then the i-th eigenvector is classified into the intermediate boundary domain cluster, otherwise the i-th eigenvector is classified into the core domain cluster; The process stops when the nth eigenvector in the kth intermediate clustering result set is traversed, and the core domain cluster and the intermediate boundary domain cluster are obtained.

10. The data detection method according to claim 1, wherein: The processing of the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected includes: Get multiple current data to be inspected at the current moment; The plurality of current data to be inspected are subjected to impurity and duplication removal processing to obtain a plurality of intermediate data; each intermediate data includes: keywords of a plurality of features; Structural processing is performed on keywords of multiple features of the multiple intermediate data to obtain multiple feature vectors corresponding to the multiple current data to be inspected.

11. The data detection method according to claim 1, wherein: The multiple matching result sets include two categories: normal behavior matching result sets and attack behavior matching result sets; The matching of the plurality of target clustering result sets with the plurality of preset matching result sets to determine the detection results of the plurality of current to-be-detected data includes: Matching multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set with multiple result keywords corresponding to each matching result set in the normal behavior matching result set; If the matching degree of the multiple keywords corresponding to any one of the matching result sets in the normal behavior matching result set reaches a threshold, then the current data to be inspected corresponding to the first clustering result set is determined to be normal data; Otherwise, matching multiple keywords of the current data to be inspected corresponding to the feature vector in the first clustering result set with multiple result keywords corresponding to each matching result set in the attack behavior matching result set; If the matching degree of the multiple keywords corresponding to any one of the attack behavior matching result sets reaches a threshold, the current data to be tested corresponding to the first clustering result set is determined to be attack data, until the multiple target clustering result sets are matched, and the detection results of the multiple current data to be tested are obtained.

12. The data detection method according to claim 1, wherein: Before processing the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected, the method further includes: Process the acquired multiple historical data to obtain multiple historical feature vectors corresponding to the multiple historical data Determining the number of historical targets and central point information of multiple historical targets corresponding to multiple historical intermediate clustering result sets using relevant data of multiple features corresponding to the multiple historical feature vectors, so as to classify the multiple historical feature vectors to obtain the multiple historical intermediate clustering result sets; The plurality of historical intermediate clustering result sets are processed by a clustering model optimized by the number of historical targets and the information of the center points of the plurality of historical targets, and a three-branch decision clustering algorithm to obtain a plurality of historical target clustering result sets; The multiple historical target clustering result sets are detected to obtain multiple detection results corresponding to the multiple historical target clustering result sets, and the multiple historical target clustering result sets are mapped with the multiple detection results to form the multiple matching result sets; the multiple matching result sets are divided into two categories: normal behavior matching result sets and attack behavior matching result sets according to the detection results.

13. A data detection device, characterized in that: include: a processing unit, configured to process the acquired multiple current data to be inspected to obtain multiple feature vectors corresponding to the multiple current data to be inspected; a determining unit, configured to determine the number of targets and information of multiple target center points corresponding to multiple intermediate clustering result sets using relevant data of multiple features corresponding to the multiple feature vectors, so as to classify the multiple feature vectors to obtain the multiple intermediate clustering result sets; The processing unit is further configured to initialize the initial clustering model using the number of targets and the information of the multiple target center points to obtain an optimized clustering model; The processing unit is further configured to determine, from the plurality of feature vectors, a neighborhood including a predetermined proportion of feature vectors; The processing unit is further configured to traverse each feature vector in the plurality of intermediate clustering result sets, and divide the plurality of feature vectors into core domain clusters and intermediate boundary domain clusters in combination with the neighborhood; The core domain cluster includes: a result set corresponding to the number of the plurality of intermediate clustering result sets; the intermediate boundary domain cluster includes: a certain number of feature vectors at the boundaries of the plurality of intermediate clustering result sets; The processing unit is further configured to input the intermediate boundary domain clusters into the optimized clustering model to obtain boundary domain clusters; the boundary domain clusters include: result sets corresponding to the number of the plurality of intermediate clustering result sets; The processing unit is further configured to combine the result set in the boundary domain cluster with the corresponding result set in the core domain cluster to obtain a plurality of target clustering result sets; A matching unit is used to match the multiple target clustering result sets with multiple preset matching result sets to determine the detection results of the multiple current data to be tested; the multiple matching result sets are obtained from multiple historical data through the same processing method as the multiple target clustering result sets.

14. A data detection device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images