Clustering method and device, storage medium, and electronic device
By calculating the density value and target distance of the data points, identifying and excluding outliers, the problem that density-based clustering algorithm cannot accurately detect outliers is solved, and the accuracy of clustering results is improved.
Patent Information
- Application Number
- CN201910970124.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-10-12
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2039-10-12
AI Technical Summary
Density-based clustering algorithms cannot accurately detect outliers in the data, resulting in the clustering results being affected.
By obtaining the location information of the data points, calculating their corresponding density values and target distances, identifying outlier data points, and clustering the data points according to the density values and distances to obtain accurate clustering results.
It can accurately identify outlier data points when the density of data points is dispersed, avoid their impact on clustering results, and improve the accuracy of clustering results.
Smart Images

Figure CN110807474B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to a clustering method, a clustering device, an electronic device, and a computer-readable storage medium. Background Art
[0002] In recent years, with the continuous development of technologies such as computers, the Internet, and the Internet of Things, various types of data have been growing and accumulating at an unprecedented rate. With the substantial increase in the amount of data, data mining and analysis have become increasingly important.
[0003] Clustering algorithm is an important data mining method, which belongs to a descriptive data mining method. There are many common clustering algorithms, including partition clustering, hierarchical clustering, density-based clustering, etc. Among them, density-based clustering algorithm is widely used due to its simplicity and high efficiency.
[0004] However, this density-based clustering algorithm still has some disadvantages. For example, this clustering algorithm cannot accurately detect outliers in the data, which will affect the clustering results.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] The purpose of the present disclosure is to provide a clustering method, a clustering device, an electronic device and a computer-readable storage medium, thereby avoiding to a certain extent the problem of affecting the clustering results due to the inability to accurately detect outliers in the data.
[0007] According to a first aspect of the present disclosure, a clustering method is provided, comprising: obtaining a data set to be detected comprising a plurality of data points, and determining a density value and a target distance corresponding to each of the data points according to position information of each of the data points; calculating an outlier parameter of each of the data points based on each of the density values, determining an outlier data point in the data set to be detected according to the outlier parameters, and determining the data points other than the outlier data points as data points to be clustered; clustering the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result.
[0008] In an exemplary embodiment of the present disclosure, determining the density value and target distance corresponding to each of the data points based on the position information of each of the data points includes: for each of the data points, respectively obtaining the n nearest data points that are closest to the data point; wherein n is a positive integer; for each of the data points, calculating the density value corresponding to the data point based on the data point and its corresponding n nearest data points; for each of the data points, determining the target distance corresponding to the data point based on the density value corresponding to the data point and the density values corresponding to its corresponding n nearest data points.
[0009] In an exemplary embodiment of the present disclosure, for each of the data points, a density value corresponding to the data point is calculated based on the data point and its corresponding n nearest data points, including: determining a first target data point from each of the data points; calculating the ratio of the distance between the first target data point and each of the corresponding n nearest data points to a preset distance, and obtaining n calculation results corresponding to the first target data point according to the ratio and a preset exponential function; and summing the n calculation results corresponding to the first target data point to obtain the density value corresponding to the first target data point.
[0010] In an exemplary embodiment of the present disclosure, for each of the data points, the target distance corresponding to the data point is determined based on the density value corresponding to the data point and the density values corresponding to its corresponding n nearest data points, including: judging the size relationship between the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; when the density value corresponding to the data point is greater than or equal to the density value corresponding to the corresponding n nearest data points, calculating the target distance corresponding to the data point according to a first preset rule; when the density value corresponding to each of the data points is less than the density value corresponding to at least one of the corresponding n nearest data points, calculating the target distance corresponding to the data point according to a second preset rule.
[0011] In an exemplary embodiment of the present disclosure, calculating the target distance corresponding to the data point according to the first preset rule includes: calculating the distance value between the data point and the corresponding n nearest data points, and determining the largest distance value as the target distance corresponding to the data point.
[0012] In an exemplary embodiment of the present disclosure, calculating the target distance corresponding to the data point according to the second preset rule includes: calculating the distance value between the data point and the target's nearest data point, and determining the smallest distance value among the distance values as the target distance corresponding to the data point, wherein the density value corresponding to the target's nearest data point is greater than the density value corresponding to the data point.
[0013] In an exemplary embodiment of the present disclosure, the outlier parameters of each data point are calculated based on each density value, including: obtaining the density value corresponding to each data point and the density values corresponding to the corresponding n nearest data points; for each data point, calculating the outlier parameter of the data point based on the density value corresponding to the data point and the density values corresponding to the corresponding n nearest data points.
[0014] In an exemplary embodiment of the present disclosure, for each of the data points, the outlier parameter of the data point is calculated based on the density value corresponding to the data point and the density values corresponding to its n nearest data points, including: determining a second target data point from each of the data points; calculating the ratio of the density value corresponding to each of the n nearest data points corresponding to the second target data point to the density value corresponding to the second target data point; calculating the average value of the ratios, and configuring the average value as the outlier parameter of the second target data point.
[0015] In an exemplary embodiment of the present disclosure, determining the outlier data points in the data set to be detected based on the outlier parameters, and determining the data points other than the outlier data points as data points to be clustered, includes: judging the relationship between the outlier parameters of each of the data points and a preset threshold; when the outlier parameters of the data point are greater than the preset threshold, determining the data point to be an outlier data point; and determining the data points other than the outlier data points in the data set to be detected as data points to be clustered.
[0016] In an exemplary embodiment of the present disclosure, clustering the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result includes: determining a preset number of central data points among the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered; and clustering the remaining data points to be clustered based on the central data points to obtain a clustering result.
[0017] In an exemplary embodiment of the present disclosure, determining a preset number of central data points among the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered includes: for each of the data points to be clustered, calculating the product of the density value and the target distance corresponding to the data point to be clustered; sorting the data points to be clustered in descending order of the products to obtain a sequence of data points to be clustered; starting from the first data point to be clustered in the sequence of data points to be clustered, extracting a preset number of the data points to be clustered in sequence as central data points.
[0018] In an exemplary embodiment of the present disclosure, clustering the remaining data points to be clustered based on the central data point to obtain a clustering result includes: determining a third target data point from the remaining data points to be clustered; calculating the distance value between the third target data point and each of the central data points, and dividing the third target data point into the cluster cluster where the central data point with the smallest distance value is located to obtain a clustering result.
[0019] According to a second aspect of the present disclosure, a clustering device is provided, comprising: a data calculation module, used to obtain a data set to be detected containing multiple data points, and determine a density value and a target distance corresponding to each data point one by one according to the position information of each data point; an outlier detection module, used to calculate the outlier parameters of each data point based on each density value, determine the outlier data point in the data set to be detected according to the outlier parameters, and determine the data points other than the outlier data points as data points to be clustered; a clustering calculation module, used to cluster the data points to be clustered according to the density value and the target distance corresponding to each data point to be clustered to obtain a clustering result.
[0020] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method described in the first aspect above by executing the executable instructions.
[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect is implemented.
[0022] The exemplary embodiments of the present disclosure may have some or all of the following beneficial effects:
[0023] In a clustering method provided in an exemplary embodiment of the present disclosure, outlier parameters of each data point are calculated based on the density value, outlier data points and data points to be clustered in the data set to be detected are identified based on the outlier parameters, and then clustering is performed based on the density value and distance value corresponding to the data points to be clustered to obtain a clustering result. Since outlier data points can be identified by outlier parameters calculated based on the density value, even when the density dispersion of the data points in the data set to be detected is very different, the data points to be detected can be accurately identified, thereby avoiding the outlier data points in the data set to be detected from affecting the final clustering result.
[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0026] Figure 1 A schematic diagram showing an exemplary system architecture of a clustering method and device to which the embodiments of the present disclosure can be applied;
[0027] Figure 2 A flowchart of a clustering method according to an embodiment of the present disclosure is schematically shown;
[0028] Figure 3 A flowchart of a method for determining a density value and a target distance corresponding to each data point one by one according to the position information of each data point according to an embodiment of the present disclosure is schematically shown;
[0029] Figure 4 A flowchart schematically shows a method for calculating, for each data point, a density value corresponding to the data point based on the data point and its corresponding n nearest data points according to an embodiment of the present disclosure;
[0030] Figure 5 A flowchart schematically shows a method for determining, for each data point, a target distance corresponding to the data point based on a density value corresponding to the data point and density values corresponding to n of the corresponding nearest data points according to an embodiment of the present disclosure;
[0031] Figure 6 A flowchart of a method for respectively calculating the outlier parameter of each data point based on each density value according to an embodiment of the present disclosure is schematically shown;
[0032] Figure 7 A flowchart schematically shows a method for calculating the outlier parameter of each data point based on the density value corresponding to the data point and the density values corresponding to the n nearest data points corresponding to the data point according to an embodiment of the present disclosure;
[0033] Figure 8 A flowchart schematically shows a method for determining outlier data points in a data set to be detected according to the outlier parameters, and determining the data points other than the outlier data points as data points to be clustered according to an embodiment of the present disclosure;
[0034] Fig. 9A flowchart schematically shows a method for clustering the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result according to an embodiment of the present disclosure;
[0035] Fig.10 A flowchart schematically shows a method for determining a preset number of central data points among the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered according to an embodiment of the present disclosure;
[0036] Fig.11 A flowchart of a method for clustering the remaining data points to be clustered based on the central data point according to an embodiment of the present disclosure is schematically shown;
[0037] Fig.12 A flow chart of traffic data clustering according to an embodiment of the present disclosure is schematically shown;
[0038] Fig.13 A schematic diagram showing the dispersion of data points in a data set to be detected when the density dispersion is very different;
[0039] Fig.14 A schematic diagram of product distribution corresponding to each to-be-clustered data point in a to-be-clustered data point sequence according to an embodiment of the present disclosure is schematically shown;
[0040] Fig.15 A block diagram of a clustering device according to an embodiment of the present disclosure is schematically shown;
[0041] Fig.16 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0042] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0043] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0044] Figure 1 A schematic diagram of a system architecture of an exemplary application environment in which a clustering method and device according to an embodiment of the present disclosure can be applied is shown.
[0045] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with display screens, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.
[0046] The clustering method provided in the embodiment of the present disclosure is generally executed by the server 105, and accordingly, the clustering device is generally arranged in the server 105. However, it is easy for those skilled in the art to understand that the clustering method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, 103, and accordingly, the clustering device can also be arranged in the terminal devices 101, 102, 103, and this is not particularly limited in the present exemplary embodiment. For example, in an exemplary embodiment, a user can upload the data set to be detected to the server 105 through the terminal devices 101, 102, 103, and the server clusters the data set to be detected through the clustering method provided in the embodiment of the present disclosure, and finally transmits the clustering result to the terminal devices 101, 102, 103, etc.
[0047] The disclosed embodiment provides a clustering method, which is implemented based on machine learning. Machine learning is a type of artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.
[0048] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0049] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
[0050] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.
[0051] The solution provided by the embodiments of the present disclosure involves artificial intelligence clustering technology, which is specifically described by the following embodiments:
[0052] Clustering algorithm is an important data mining method, which belongs to a descriptive data mining method. There are many common clustering algorithms, including partition clustering, hierarchical clustering, density-based clustering, etc. Among them, density-based clustering algorithm is widely used due to its simplicity and high efficiency. In the relevant density-based clustering algorithm, it is mainly to set a threshold for the density value and target distance of each data point in the data set to be detected to determine whether the data point is an outlier; then find the cluster center according to the density value and target distance, and then cluster the data set to be detected according to the cluster center.
[0053] However, when the density of data points in the test data set is very different, for example, Fig.13 As shown in the figure, when the density dispersion of the data points in the C1 set and the data points in the C2 set is very different, it is difficult to accurately identify the outlier data points o1 and o2 in the data set to be detected by setting thresholds for the density value and the target distance value in the above clustering algorithm. Therefore, the final clustering result will also be affected by the outliers.
[0054] Based on one or more of the above problems, this exemplary embodiment provides a clustering method, which can be applied to the server 105 or to one or more of the terminal devices 101, 102, and 103, which is not particularly limited in this exemplary embodiment.
[0055] Specifically, the clustering method provided in this example implementation can be applied to the process of data clustering processing. For example, in the field of traffic anti-cheating, the traffic data can be clustered by the clustering method provided in this example implementation, and then the cluster cluster can be directly judged as abnormal traffic or normal traffic, avoiding the problem of heavy workload caused by analyzing each traffic data.
[0056] Below, reference Figure 2 As shown, the above steps of this example implementation are described in more detail.
[0057] In step S210, a data set to be detected including a plurality of data points is obtained, and a density value and a target distance corresponding to each data point are determined according to the position information of each data point.
[0058] In this example implementation, the data set to be detected includes multiple data points, and each data point has unique and definite position information in the feature space. The feature space can be established based on the features included in the data points, and can be a multi-dimensional space, for example, a two-dimensional space, a three-dimensional space, a five-dimensional space, etc., and the present disclosure does not impose any special restrictions on this.
[0059] For example, the data points in the data set to be detected are determined based on the data of all irises in a garden. Therefore, the data points can include the length and width of the iris petals at the same time. According to the length and width of the iris petals, a two-dimensional feature space with length as the horizontal coordinate and width as the vertical coordinate is established. At this time, the data points corresponding to each iris and the unique position in the feature space can be determined according to the length and width of all iris petals. For example, if the length and width of a petal of an iris are 2 cm and 0.5 cm respectively, the coordinate (2,0.5) can be determined in the feature space as the position information of the data point corresponding to the iris.
[0060] refer to Figure 3 As shown, in this example implementation, Figure 3 Steps S310 to S330 shown determine the density value and target distance corresponding to each data point.
[0061] In step S310, for each data point, the n nearest data points closest to the data point are obtained respectively; wherein n is a positive integer. In this example implementation, since in actual situations, the density of a data point is less affected by data points that are farther away, when performing density calculation, only the n nearest data points closest to the data point can be considered for calculation. The specific parameter n can be set according to the actual situation of clustering.
[0062] In step S320, for each data point, the density value corresponding to the data point is calculated based on the data point and its corresponding n nearest data points. Figure 4 As shown, the density value corresponding to each data point can be calculated through the following steps S410 to S430:
[0063] In step S410, a first target data point is determined from each of the data points. In this example implementation, although it is necessary to calculate the density value corresponding to each data point in the data set to be detected, the density value corresponding to each data point needs to be calculated separately, so a first target data point can be determined from the data points and its density value can be calculated, and then the other data points can be calculated in sequence, wherein the first target data point can be any one of all the data points.
[0064] In step S420, the ratio of the distance between the first target data point and each of the corresponding n nearest data points to a preset distance is calculated, and n calculation results corresponding to the first target data point are obtained based on the ratio and a preset exponential function.
[0065] In this example implementation, the distance between the first target data point and each of the corresponding n nearest data points is first calculated, and the calculated distances are respectively compared with the preset distances to obtain n ratios. According to the n ratios and the preset exponential function, the n calculation results corresponding to the first target data point are respectively obtained. Specifically, the preset exponential function can be an exponential function with a constant as the base, or can be set to other exponential functions according to the application scenario of the specific clustering method, and the present disclosure does not impose any special restrictions on this.
[0066] For example, when the exponential function is preset to be a negative exponential function of a natural constant, each calculation result can be expressed as:
[0067]
[0068] Among them, d ij is the distance between data point i and data point j; data point j is one of the n nearest data points corresponding to data point i; d c The preset radius.
[0069] In step S430, the n calculation results corresponding to the first target data point are summed to obtain the density value corresponding to the first target data point. In this example implementation, the density value corresponding to the first target data point can be obtained by summing the n calculation results corresponding to the first data point. For example, when the preset exponential function is a negative exponential function of a natural constant, the calculation process can be expressed by formula (1):
[0070]
[0071] Among them, ρ i is the density value corresponding to data point i; data point j is one of the n nearest data points corresponding to data point i, and j ranges from 1 to n; d ij is the distance between data point i and data point j; d c The preset radius.
[0072] In this example implementation, by only considering the n nearest data points to the data point for calculation, the time complexity of the step of calculating the density value can be reduced from O(n) to O(n). 2 ) is reduced to O(n), which shortens the time consumption of the calculation steps. At the same time, due to the reduction in time consumption, this clustering method can also cluster more data points.
[0073] In step S330, for each data point, the target distance corresponding to the data point is determined based on the density value corresponding to the data point and the density values corresponding to the n nearest data points. Figure 5As shown, the target distance corresponding to each data point can be calculated through the following steps S510 to S530:
[0074] In step S510, the magnitude relationship between the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points is determined. In this example implementation, the target distance calculation method corresponding to the data point with the largest density value is different from the target distance calculation method corresponding to other data points. When only considering the n nearest data points, the magnitude relationship between the data point and the corresponding n nearest data points should be determined first, so as to select a suitable calculation method to determine the target distance.
[0075] In step S520, when the density value corresponding to the data point is greater than or equal to the density values corresponding to the corresponding n nearest data points, the target distance corresponding to the data point is calculated according to the first preset rule. In this example implementation, the target distance corresponding to the data point with the largest density value can be determined by the following method: calculate the distance value between the data point and the corresponding n nearest data points, and determine the largest distance value as the target distance corresponding to the data point. For example, when n is 3, the density value of a data point a is greater than the density value of the corresponding three nearest data points, and the calculated distance values between the data point a and the three nearest data points are 10, 12, and 13 respectively. At this time, the largest 13 can be determined as the target distance of the data point a. Specifically, the above method for determining the target distance corresponding to a data point can be expressed by formula (2):
[0076] l i =maxd ij (2)
[0077] Among them, l i is the distance value of data point i; data point j is the n nearest data points corresponding to data point i, j ranges from 1 to n; d ij is the distance between data point i and data point j.
[0078] In step S530, when the density value corresponding to each of the data points is less than the density value corresponding to at least one of the corresponding n nearest data points, the target distance corresponding to the data point is calculated according to the second preset rule. In this example implementation, the target distance corresponding to the data point whose density value is not the largest can be determined by the following method: calculate the distance value between the data point and the target nearest data point, and determine the smallest distance value among the distance values as the target distance corresponding to the data point, wherein the density value corresponding to the target nearest data point is greater than the density value corresponding to the data point. For example, when n is 10, the density value corresponding to a data point b is less than 3 of the 10 nearest data points, which are the nearest data points A, B, and C respectively; the calculated distance values between the data point b and the nearest data points A, B, and C are 11, 15, and 10 respectively. At this time, the smallest 10 can be determined as the target distance of the data point b. Specifically, the above method for determining the target distance corresponding to a data point can be expressed by formula (3):
[0079]
[0080] Among them, l i is the distance value of data point i; data point j is the n nearest data points corresponding to data point i, j ranges from 1 to n; d ij is the distance between data point i and data point j; ρ i and ρ j are the density values corresponding to data point i and data point j respectively.
[0081] Similarly, by considering only the n nearest data points to the data point for calculation, the time complexity of the step of calculating the target distance can be reduced from O(n) to O(n) without almost affecting the calculation result. 2 ) is reduced to O(n), shortening the time consumption of the calculation steps.
[0082] It is easy for those skilled in the art to understand that in other exemplary embodiments of the present disclosure, other preset rules may also be used to calculate the target distance corresponding to the data point. For example, in the above example, the second preset rule may also be to take the average value 12 of the three distance values 11, 15, and 10 as the target distance corresponding to the data point; therefore, the present disclosure includes but is not limited to the above-mentioned target distance calculation method.
[0083] In step S220, outlier parameters of each of the data points are calculated based on each of the density values, outlier data points are determined in the data set to be detected according to the outlier parameters, and the data points other than the outlier data points are determined as data points to be clustered.
[0084] refer to Figure 6 As shown, in this example implementation, Figure 6Steps S610 to S620 shown in the figure calculate the outlier parameters of each data point respectively.
[0085] In step S610, the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points are obtained. In this example implementation, when determining whether a data point is an outlier data point, it can be determined by judging the similarity between the data point and the corresponding n nearest data points, and the similarity between the data point far from the data point and the data point has little effect on judging whether the data point is outlier or not, so the similarity can be judged by obtaining the density value corresponding to the data point and the density values corresponding to the n nearest data points around the data point.
[0086] In step S620, for each data point, the outlier parameter of the data point is calculated based on the density value corresponding to the data point and the density values corresponding to the n nearest data points. Figure 7 As shown, the outlier parameters of the data points can be calculated through the following steps S710 to S730. Where:
[0087] In step S710, a second target data point is determined from each of the data points. In this example implementation, although the outlier parameter of each data point in the data set to be detected needs to be calculated, the outlier parameter of each data point needs to be calculated separately, so a second target data point can be determined from the data points first to calculate its outlier parameter, and then the other data points are calculated in turn, wherein the second target data point can be any one of all the data points.
[0088] In step S720, the ratio of the density value corresponding to each of the n nearest data points corresponding to the second target data point to the density value corresponding to the second target data point is calculated. Specifically, the ratio of the density value corresponding to the n nearest data points around the second target data point to the density value corresponding to the second target data point can be calculated, and then the similarity between the second target data point and the surrounding n nearest data points can be judged according to the n ratios, and then whether the second target data point is outlier can be judged. For example, the ratio of the density value corresponding to a nearest data point D of the second target data point to the second target data point is 4, indicating that the density value corresponding to the nearest data point D is larger, and therefore the density value corresponding to the second target data point and the nearest data point D is quite different.
[0089] In step S730, the average value of the ratios is calculated, and the average value is configured as the outlier parameter of the second target data point. When judging whether the second target data point is an outlier only based on the similarity between a nearest data point and the second target data point, it is possible that the second target data point is misjudged as an outlier because the selected nearest data point happens to be an outlier. Therefore, the average value of the n ratios obtained in step S720 can be calculated to represent the average similarity between the second target data point and the surrounding n nearest data points.
[0090] Specifically, the above method for calculating the outlier parameter of a data point can be expressed by formula (4):
[0091]
[0092] Among them, χ i is the outlier parameter of data point i; data point j is the n nearest data points corresponding to data point i, j ranges from 1 to n; ρ i and ρ j are the density values corresponding to data point i and data point j respectively.
[0093] It is easy for those skilled in the art to understand that in other exemplary embodiments of the present disclosure, other methods may also be used to calculate the outlier parameters of each of the data points. For example, the average value of the difference between the density value corresponding to the second target data point and the density values corresponding to the n nearest data points may be calculated as the outlier parameter of the second target data point, etc. These also fall within the scope of protection of the present disclosure.
[0094] refer to Figure 8 As shown, in this example implementation, Figure 8 Steps S810 to S830 shown in the figure determine outlier data points in the data set to be detected, and determine the data points other than the outlier data points as data points to be clustered.
[0095] In step S810, the relationship between the outlier parameter of each data point and a preset threshold is determined. The preset threshold can be configured according to different ways of calculating outlier parameters and different data sets to be detected. For example, for the outlier parameter calculated in step S730, the preset threshold can be set to 1.
[0096] In step S820, when the outlier parameter of the data point is greater than a preset threshold, the data point is determined to be an outlier data point. For example, assuming that the preset threshold is 1 for the outlier parameter calculated in step S730, when the outlier parameter is much greater than 1, it proves that the density value corresponding to the second target data point is not similar to the density values corresponding to the surrounding n nearest data points, and is much smaller than the density values corresponding to the surrounding n nearest data points, so the second target data point can be determined to be an outlier.
[0097] In step S830, the data points other than the outlier data points in the data set to be detected are determined as data points to be clustered. For example, when it is determined in step S820 that there are 3 outlier data points in a data set to be detected containing 10 data points, 7 data points other than the 3 outlier data points can be determined as data points to be clustered.
[0098] In step S230, the data points to be clustered are clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result.
[0099] refer to Fig. 9 As shown, in this example implementation, Fig. 9 Steps S910 to S920 shown in the figure cluster the data points to be clustered to obtain a clustering result.
[0100] In step S910, a preset number of central data points are determined among the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered.
[0101] In this example implementation, the preset number can be a fixed value defined by the user, or it can be a value selected from multiple custom values based on the distribution of density values and target distance values corresponding to each data point in the data set to be detected. The present disclosure does not impose any special restrictions on this.
[0102] refer to Fig.10 As shown, in this example implementation, Fig.10 Steps S1010 to S1030 shown determine a preset number of central data points among the data points to be clustered.
[0103] in:
[0104] Step S1010, for each data point to be clustered, calculate the product of the density value corresponding to the data point to be clustered and the target distance. Specifically, the density value ρ1 corresponding to the data point to be clustered 1 is multiplied by the target distance l1 to obtain the product.
[0105] Step S1020, sort the data points to be clustered in descending order of the products to obtain a sequence of data points to be clustered. For example, there are 3 data points to be clustered, namely, data point 1 to be clustered, data point 2 to be clustered, and data point 3 to be clustered. The corresponding products calculated in step S1010 are 0.001, 0.05, and 0.15, respectively. At this time, the data points to be clustered can be sorted in order of the size of the products to obtain a sequence of data points to be clustered: data point 3 to be clustered, data point 1 to be clustered, and data point 2 to be clustered.
[0106] Step S1030 , starting from the first data point to be clustered in the sequence of data points to be clustered, a preset number of data points to be clustered are sequentially extracted as central data points.
[0107] In this example implementation, the preset number can be determined according to the distribution of the product. For example, according to the sequence of data points to be clustered obtained in step S1010 and step S1020 and the corresponding product, a graph can be drawn as follows: Fig.14 From the distribution diagram shown, it can be clearly observed that there is a clear difference between the central data points and the non-central data points through the size of the product value of the vertical axis. Therefore, the number of central data points can be determined by the distribution of the product.
[0108] In step S920, the remaining data points to be clustered are clustered based on the central data point to obtain a clustering result.
[0109] refer to Fig.11 As shown, in this example implementation, Fig.11 Steps S1110 to S1120 shown are used to cluster the remaining data points to be clustered.
[0110] In step S1110, a third target data point is determined from the remaining data points to be clustered. In this example implementation, although all remaining data points to be clustered need to be clustered, each remaining data point to be clustered needs to be determined one by one to which it belongs. Therefore, the third target data point can be determined from the remaining data points to be clustered, and then the clusters to which the other remaining data points to be clustered belong can be determined in turn. The third target data point can be any one of the remaining data points to be clustered.
[0111] In step S1120, the distance value between the third target data point and each of the central data points is calculated, and the third target data point is divided into the cluster where the central data point with the smallest distance value is located. For example, assuming that there are two central data points α and β, the distance values between the third target data point and the central data points α and β are calculated to be 3 and 5 respectively, so the third target data point can be divided into the cluster where the central data point α is located.
[0112] In addition, in this example implementation, the distance value calculated in the above method may be Euclidean distance, Manhattan distance, Chebyshev distance, etc.; this also falls within the protection scope of the present disclosure.
[0113] The following takes traffic data as an example. Fig.12 As shown, the implementation details of the technical solution of this example implementation are described in detail:
[0114] Before clustering, a feature space is formed based on the relevant features of the traffic data, such as the number of traffic data requests and the request IP address. Then, by extracting the features of each traffic data, each traffic data is mapped to a unique position coordinate in the feature space, and this coordinate is used as the location information of the data point corresponding to the traffic data.
[0115] Step S1210, determining n nearest data points that are closest to each of the data points according to the position information of each data point;
[0116] Step S1220, based on the position information of each data point, the position information of the corresponding n nearest data points and the defined preset distance, calculate the density value corresponding to each data point according to formula (1);
[0117] Step S1230, determining the target distance corresponding to each data point based on the density value corresponding to each data point calculated by formula (1);
[0118] Specifically, when the density value corresponding to a data point is greater than or equal to the density value corresponding to the corresponding n nearest data points, the target distance corresponding to the data point is calculated according to formula (2); when the density value corresponding to each data point is less than the density value corresponding to at least one data point among the corresponding n nearest data points, the target distance corresponding to the data point is calculated according to formula (3).
[0119] Step S1240, based on the density value corresponding to each data point calculated by formula (1), calculate the outlier parameter of each data point according to formula (4);
[0120] Step S1250, judging whether the data point is an outlier data point according to the magnitude relationship between the outlier parameter and the preset threshold;
[0121] Specifically, data points whose outlier parameters are greater than a preset threshold are determined as outlier data points, and data points whose outlier parameters are less than or equal to the preset threshold are determined as data points to be clustered.
[0122] Step S1260, determining a central data point among the data points to be clustered according to the density value and the target distance corresponding to the data points to be clustered;
[0123] Specifically, the product of the density value and the distance value corresponding to each of the data points to be clustered is calculated, and the corresponding data points to be clustered are sorted according to the size of the product to obtain a sequence of data points to be clustered, and starting from the first data point to be clustered in the sequence of data points to be clustered, a preset number of data points to be clustered are extracted in sequence as central data points.
[0124] Step S1270: cluster the remaining data points to be clustered based on the central data point to obtain a clustering result.
[0125] Specifically, the distance values between the remaining data points to be clustered and each central data point are calculated, and then the data points to be clustered are divided into the clusters where the nearest central data point is located.
[0126] Based on the above clustering method, the clustering results obtained are more accurate, which makes the subsequent classification of traffic data based on the clustering results better. Specifically, after clustering the traffic data using the traditional clustering method and the clustering method in this example implementation, the accuracy and abnormal recall rate of the clustering results are shown in Table 1 below:
[0127] Table 1 Classification accuracy and anomaly recall
[0128] Methods used Accuracy Abnormal recall rate Traditional clustering methods 92.05% 89.97% Clustering Method in This Example Implementation 94.22% 90.96%
[0129] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0130] Furthermore, in this exemplary embodiment, a clustering device is also provided. The clustering device can be applied to a server or a terminal device. Fig.15 As shown, the clustering device 1500 may include a data calculation module 1510, an outlier detection module 1520, and a cluster calculation module 1530. Among them:
[0131] The data calculation module 1510 is used to obtain a data set to be detected that includes multiple data points, and determine the density value and target distance corresponding to each data point according to the position information of each data point; the outlier detection module 1520 is used to calculate the outlier parameters of each data point based on each density value, determine the outlier data point in the data set to be detected according to the outlier parameters, and determine the data points other than the outlier data points as data points to be clustered; the clustering calculation module 1530 is used to cluster the data points to be clustered according to the density value and the target distance corresponding to each data point to be clustered to obtain a clustering result.
[0132] In an exemplary embodiment of the present disclosure, the data calculation module 1510 includes: a data point extraction unit, used to obtain, for each of the data points, the n nearest data points that are closest to the data point; wherein n is a positive integer; for each of the data points, calculating the density value corresponding to the data point based on the data point and its corresponding n nearest data points; for each of the data points, determining the target distance corresponding to the data point based on the density value corresponding to the data point and the density values corresponding to its corresponding n nearest data points.
[0133] In an exemplary embodiment of the present disclosure, the density value calculation unit calculates the density value corresponding to each of the data points based on the data point and its corresponding n nearest data points through the following steps: determining a first target data point from each of the data points; calculating the ratio of the distance between the first target data point and each of the corresponding n nearest data points to a preset distance, and obtaining n calculation results corresponding to the first target data point based on the ratio and a preset exponential function; and summing the n calculation results corresponding to the first target data point to obtain the density value corresponding to the first target data point.
[0134] In an exemplary embodiment of the present disclosure, the target distance calculation unit determines the target distance corresponding to the data point for each of the data points based on the density value corresponding to the data point and the density values corresponding to its corresponding n nearest data points through the following steps: judging the magnitude relationship between the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; when the density value corresponding to the data point is greater than or equal to the density value corresponding to the corresponding n nearest data points, calculating the target distance corresponding to the data point according to a first preset rule; or when the density value corresponding to each of the data points is less than the density value corresponding to at least one of the corresponding n nearest data points, calculating the target distance corresponding to the data point according to a second preset rule.
[0135] In an exemplary embodiment of the present disclosure, calculating the target distance corresponding to the data point according to the first preset rule includes: calculating the distance value between the data point and the corresponding n nearest data points, and determining the largest distance value as the target distance corresponding to the data point.
[0136] In an exemplary embodiment of the present disclosure, calculating the target distance corresponding to the data point according to the second preset rule includes: respectively calculating the distance values between the data point and at least one of the nearest data points whose density value is greater than that of the data point, and determining the smallest distance value among the distance values as the target distance corresponding to the data point.
[0137] In an exemplary embodiment of the present disclosure, the outlier detection module 1520 includes: a density value acquisition unit, used to obtain the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; an outlier parameter calculation unit, used to calculate the outlier parameter of the data point for each of the data points based on the density value corresponding to the data point and the density values corresponding to the corresponding n nearest data points.
[0138] In an exemplary embodiment of the present disclosure, the outlier parameter calculation unit calculates the outlier parameter of the data point for each of the data points based on the density value corresponding to the data point and the density values corresponding to its n nearest data points through the following steps: determining a second target data point from each of the data points; calculating the ratio of the density value corresponding to each of the n nearest data points corresponding to the second target data point to the density value corresponding to the second target data point; calculating the average value of the ratios, and configuring the average value as the outlier parameter of the second target data point.
[0139] In an exemplary embodiment of the present disclosure, the outlier detection module 1520 also includes: a relationship judgment unit, used to judge the relationship between the outlier parameter of each of the data points and a preset threshold; a first determination unit, used to determine that the data point is an outlier data point when the outlier parameter of the data point is greater than the preset threshold; and a second determination unit, used to determine the data points other than the outlier data points in the data set to be detected as data points to be clustered.
[0140] In an exemplary embodiment of the present disclosure, the clustering calculation module 1530 includes: a third determination unit, used to determine a preset number of central data points among the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered; and a clustering unit, used to cluster the remaining data points to be clustered based on the central data points to obtain a clustering result.
[0141] In an exemplary embodiment of the present disclosure, the third determination unit determines a preset number of central data points among the data points to be clustered according to the density values and the target distances corresponding to the data points to be clustered through the following steps: for each of the data points to be clustered, calculating the product of the density value and the target distance corresponding to the data point to be clustered; sorting the data points to be clustered in descending order of the products to obtain a sequence of data points to be clustered; starting from the first data point to be clustered in the sequence of data points to be clustered, extracting a preset number of the data points to be clustered as central data points in sequence.
[0142] In an exemplary embodiment of the present disclosure, the clustering unit clusters the remaining data points to be clustered based on the central data point through the following steps: determining a third target data point from the remaining data points to be clustered; calculating the distance value between the third target data point and each of the central data points, and dividing the third target data point into the cluster where the central data point with the smallest distance value is located.
[0143] The specific details of each module or unit in the above clustering device have been described in detail in the corresponding clustering method, so they will not be repeated here.
[0144] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0145] also, Fig.16 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.
[0146] It should be noted that Fig.16 The computer system 1600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0147] like Fig.16As shown, the computer system 1600 includes a central processing unit (CPU) 1601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1602 or the program loaded from the storage part 1608 to the random access memory (RAM) 1603. In the RAM 1603, various programs and data required for system operation are also stored. The CPU 1601, the ROM 16016 and the RAM 1603 are connected to each other through a bus 1604. An input / output (I / O) interface 1605 is also connected to the bus 1604.
[0148] The following components are connected to the I / O interface 1605: an input section 1606 including a keyboard, a mouse, etc.; an output section 1607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card such as a LAN card, a modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to the I / O interface 1605 as needed. A removable medium 1611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1610 as needed, so that a computer program read therefrom is installed into the storage section 1608 as needed.
[0149] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1609, and / or installed from the removable medium 1611. When the computer program is executed by the central processing unit (CPU) 1601, various functions defined in the method and apparatus of the present application are executed. In some embodiments, the computer system 1600 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0150] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0151] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0152] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0153] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being installed in the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiment. For example, the electronic device may implement the following method. Figure 2 to Figure 12 The various steps shown, etc.
[0154] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0155] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A clustering method, characterized in that: include: Acquire a data set to be detected that includes multiple data points, wherein the data set to be detected includes flow data; For each of the data points, respectively obtain n nearest data points that are closest to the data point; wherein n is a positive integer; For each of the data points, a density value corresponding to the data point is calculated based on the data point and its corresponding n nearest data points; Determine the magnitude relationship between the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; When the density value corresponding to the data point is greater than or equal to the density values corresponding to the corresponding n nearest data points, calculating the target distance corresponding to the data point according to a first preset rule; When the density value corresponding to each of the data points is less than the density value corresponding to at least one of the corresponding n nearest data points, the distance value between the data point and the target nearest data point is calculated, and the average value of each of the distance values is determined as the target distance corresponding to the data point, wherein the density value corresponding to the target nearest data point is greater than the density value corresponding to the data point; Obtaining the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; Determining a second target data point from each of the data points; Calculate the average of the differences between the density value corresponding to the second target data point and the density values corresponding to the corresponding n nearest data points as the outlier parameter of the second target data point; Determining outlier data points in the to-be-detected data set according to the outlier parameters, and determining the data points other than the outlier data points as data points to be clustered; Clustering the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result; It is determined whether each cluster in the clustering result is abnormal traffic or normal traffic.
2. The method according to claim 1, characterized in that The step of calculating, for each of the data points, a density value corresponding to the data point based on the data point and its corresponding n nearest data points, includes: Determining a first target data point from among the data points; Calculating a ratio of a distance between the first target data point and each of the corresponding n nearest data points to a preset distance, and obtaining n calculation results corresponding to the first target data point according to the ratio and a preset exponential function; The n calculation results corresponding to the first target data point are summed to obtain a density value corresponding to the first target data point.
3. The method according to claim 1, characterized in that The calculating the target distance corresponding to the data point according to the first preset rule includes: The distance values between the data point and the corresponding n nearest data points are calculated, and the largest distance value is determined as the target distance corresponding to the data point.
4. The method according to claim 1, characterized in that: The step of determining outlier data points in the to-be-detected data set according to the outlier parameters, and determining the data points other than the outlier data points as to-be-clustered data points, comprises: Determine the magnitude relationship between the outlier parameter of each of the data points and a preset threshold; When the outlier parameter of the data point is greater than a preset threshold, determining the data point to be an outlier data point; The data points other than the outlier data points in the data set to be detected are determined as data points to be clustered.
5. The method according to claim 1, characterized in that The clustering the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result includes: Determining a preset number of central data points among the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered; The remaining data points to be clustered are clustered based on the central data point to obtain a clustering result.
6. The method according to claim 5, characterized in that The step of determining a preset number of central data points from the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered comprises: For each of the data points to be clustered, calculating the product of the density value corresponding to the data point to be clustered and the target distance; Sorting the data points to be clustered in descending order of the products to obtain a sequence of data points to be clustered; Starting from the first data point to be clustered in the sequence of data points to be clustered, a preset number of data points to be clustered are sequentially extracted as central data points.
7. The method according to claim 5, characterized in that The clustering of the remaining data points to be clustered based on the central data point to obtain a clustering result includes: Determining a third target data point from the remaining data points to be clustered; The distance value between the third target data point and each of the central data points is calculated, and the third target data point is divided into a cluster where the central data point with the smallest distance value is located, so as to obtain the clustering result.
8. A clustering device, characterized in that: include: A data calculation module is used to obtain a data set to be detected containing multiple data points, and determine a density value and a target distance corresponding to each data point according to the position information of each data point, wherein the data set to be detected contains flow data; Wherein, the data calculation module includes: A data point extraction unit is used to: for each of the data points, respectively obtain the n nearest data points that are closest to the data point; wherein n is a positive integer; for each of the data points, calculate the density value corresponding to the data point based on the data point and its corresponding n nearest data points; determine the magnitude relationship between the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; when the density value corresponding to the data point is greater than or equal to the density value corresponding to the corresponding n nearest data points, calculate the target distance corresponding to the data point according to a first preset rule; when the density value corresponding to each of the data points is less than the density value corresponding to at least one of the corresponding n nearest data points, calculate the distance value between the data point and the target nearest data point, and determine the average value of each of the distance values as the target distance corresponding to the data point, wherein the density value corresponding to the target nearest data point is greater than the density value corresponding to the data point; An outlier detection module, used to calculate the outlier parameters of each of the data points based on each of the density values, determine the outlier data points in the data set to be detected according to the outlier parameters, and determine the data points outside the outlier data points as the data points to be clustered; wherein the outlier detection module includes: a density value acquisition unit, used to obtain the density value corresponding to each of the data points and the density values corresponding to the corresponding n nearest data points; an outlier parameter calculation unit, used to determine the second target data point from each of the data points; calculate the average value of the difference between the density value corresponding to the second target data point and the density value corresponding to the corresponding n nearest data points as the outlier parameter of the second target data point; A clustering calculation module, used for clustering the data points to be clustered according to the density value and the target distance corresponding to each of the data points to be clustered to obtain a clustering result; The device is also used to determine whether each cluster in the clustering result is abnormal traffic or normal traffic.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.
