High-dimensional vector clustering algorithm based on GPU acceleration

Abstract

The invention discloses a high-dimensional vector clustering algorithm based on GPU acceleration, and the algorithm comprises the steps: obtaining a high-dimensional space vector set, accelerating the construction process of a k-nearest neighbor graph through a GPU, and completing the construction of a global k-nearest neighbor graph; obtaining a high-dimensional space vector set, recursively executing k-means clustering on the high-dimensional space vector set, and completing partition operation of vector objects; independently executing DBSCAN clustering in parallel among different partitions by using a global k-neighbor graph, a partitioning scheme and a GPU thread block to complete partition clustering; vector objects conforming to core point definition are obtained, a core neighbor graph is constructed based on the density direct relation among the vector objects, the core neighbor graph and the partition clustering results are utilized, the partition clustering results are combined, and a global clustering result of the high-dimensional space vector set is obtained. According to the method, the execution time of the DBSCAN algorithm can be remarkably shortened, and the efficiency of large-scale high-dimensional vector clustering is greatly improved on the premise that the clustering precision is guaranteed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of clustering analysis in machine learning, and particularly relates to a high-dimensional vector clustering algorithm based on GPU acceleration. BACKGROUND

[0002] High-dimensional vector clustering is one of the key technologies for large-scale vector data analysis, which can reveal the inherent rules of data distribution by dividing vectors with similar characteristics into the same category, and provide technical support for data analysis tasks such as anomaly detection, group preference identification and personalized recommendation. The density-based spatial clustering algorithm (DBSCAN) in the literature [Ester M, Kriegel H P, Sander J, Xu X. A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proc. of the Second International Conference on Knowledge Discovery and Data Mining. 1996, 96(34): 226-231] has been widely used in natural language processing, image processing, audio analysis, biological data analysis and other fields due to its advantages of not needing to pre-set the number of clusters, being able to identify complex cluster structures and noise points. With the rapid development of artificial intelligence and big data technology, the dimension of vectorized data such as text and image generally reaches hundreds of dimensions, and the data size of vector data set also generally reaches millions; however, as a density-based clustering algorithm, DBSCAN has the problem of low computational efficiency when facing large-scale high-dimensional vectors, and it is difficult to meet the efficient clustering needs in actual application scenarios. Therefore, improving the computational efficiency of DBSCAN algorithm when facing large-scale high-dimensional vectors has become an important issue to be solved.

[0003] The computational overhead of DBSCAN algorithm mainly comes from the neighborhood calculation of each data object. In order to reduce the computational cost, existing methods mostly use tree or grid-based index to accelerate the neighborhood calculation of DBSCAN. However, the tree or grid-based index structure is easily affected by the curse of dimensionality when processing high-dimensional vector data, resulting in a significant decline in the performance of neighbor query. In order to overcome the performance degradation problem caused by the curse of dimensionality, using a neighbor graph structure (such as ε-neighbor graph, K-neighbor graph, HNSW, etc.) as the index structure for DBSCAN calculation has become a mainstream solution. However, existing researches generally construct neighbor graph and DBSCAN clustering based on CPU architecture. Due to the limited parallel computing capability of a small number of CPU cores and the high computational complexity of graph index construction, it is difficult to meet the efficient clustering needs of large-scale high-dimensional vectors.

[0004] To improve the clustering efficiency of large-scale vector data, researchers began to use the powerful parallel computing capability of GPU to accelerate DBSCAN algorithm; the neighborhood calculation of data points is limited within the sub-region, and the near neighbor distance of each data point is calculated in parallel, so as to avoid the distance calculation of each data point relative to all data points in the data set, and reduce the time complexity to O(n 2 ) below. However, in the process of sub-region division, the existing GPU-based DBSCAN algorithm such as document [Bohm C, Noll R, Plant C, Wackersreuther B. Density-based clustering using graphics processors. In: Proc. of the ACM Conference on Information and Knowledge Management. 2009:661-670] will be affected by the high-dimensional space dimension disaster; in addition, the existing GPU-based DBSCAN algorithm does not support the index structure based on graph, and the clustering efficiency of high-dimensional vector data is low. SUMMARY

[0005] In view of the deficiencies of the prior art, in order to cope with the real scene of high-dimensional vector data clustering, the application provides a GPU-accelerated high-dimensional vector clustering algorithm, which fully utilizes the efficient concurrent computing characteristics of GPU, realizes the full-process acceleration of DBSCAN, and improves the efficiency of large-scale high-dimensional vector clustering under the premise of ensuring clustering accuracy.

[0006] A GPU-accelerated high-dimensional vector clustering algorithm, comprising the following steps: (1) obtaining a set of high-dimensional space vectors, using GPU to accelerate the execution of NN-descent algorithm, and constructing a global k-nearest neighbor graph for all vectors in the set; (2) for the set of high-dimensional space vectors, using inter-layer parallel scheme and Tensor core acceleration technology to recursively execute k-means algorithm, and obtaining k-means tree partition scheme; (3) using the k-nearest neighbor graph and the k-means tree partition scheme, assigning a GPU thread block to each partition, and independently performing clustering in parallel between different partitions according to the breadth-first search strategy, to obtain the partition clustering result; (4) constructing a core neighbor graph according to the direct density relationship between the core points of the high-dimensional space vectors, and merging the partition clustering result through the core neighbor graph, to obtain the final global clustering result.

[0007] Further, the specific implementation of step (1) is as follows: S21: Obtain a high-dimensional space vector set and a preset maximum iteration number; S22: Randomly initialize a k-neighbor graph, that is, assign a random k neighbor to each node (corresponding to the vector in the set); S23: Sample the k-neighbor graph in parallel, and retain part of the second-order forward nearest neighbor and the second-order reverse nearest neighbor through sampling to form a sample node set; S24: Sort the sample node set using a parallel double sorting algorithm, form an ordered sequence, and assign a thread to each element in the sequence. The thread is used to determine whether the current element and the previous element are the same. If not, write to the candidate point array in the shared memory to obtain the de-duplicated sample node set; S25: Assign a thread block to each vector in the high-dimensional space vector set. All threads in the thread block are used to load the vector into the register, and then the de-duplicated sample node set is loaded into the shared memory in parallel; S26: Each thread in the thread block calculates the local dimension distance of the vector. After calculating the local dimension distance, use the inter-thread communication function shuf_sync of CUDA (Compute Unified Device Architecture) to exchange data between threads, and then merge the calculation results of each thread to obtain the final distance; S27: Repeat step S26 until the distance of all nodes in the de-duplicated sample node set to the vector to which the thread block belongs is calculated; S28: Sort the candidate points (nodes in the de-duplicated sample node set) according to the distance using a parallel double sorting algorithm, and then merge the neighbor list of the current k-neighbor graph and the candidate point array according to the distance. Keep the first k data points to form an updated neighbor list; S29: Repeat steps S23-S28 until the maximum iteration number is reached, and form the final k-neighbor graph.

[0008] Further, the specific implementation of step (2) is as follows: S31: Obtain a high-dimensional space vector set to be partitioned, a preset tree depth, and a maximum iteration number of k-means; S32: Randomly select k cluster centers, that is, randomly select k data points (corresponding to the vector in the set) in the high-dimensional space vector set as the initial cluster centers of the k-means clustering algorithm; S33: Distribute all data points in the high-dimensional space vector set to each thread block uniformly. The thread block loads the cluster center and the data point belonging to it into the shared memory; S34: The thread block calls different Tensor cores to perform matrix multiplication operations, calculates the distance between the data point and the cluster center, and writes the calculated distance to the shared memory; S35: Each thread in the thread block fetches the distance between the corresponding single data point and each cluster center, and then uses the CUDA inter-thread communication function shuf_sync to select the cluster center with the minimum local distance in different warps (thread groups), and writes the local minimum distance to the shared memory; S36: The thread block reassigns a dedicated thread group to reduce the local minimum distance in the shared memory within the warp, and finally selects the cluster center with the minimum global distance to complete the cluster center assignment of the single data point; S37: Repeat steps S35-S36 until the cluster center assignment of all data points belonging to the thread block is completed; S38: Use GPU parallel to sum the vectors in each cluster and take the average as the new cluster center; S39: Repeat steps S33-S38 until the maximum number of iterations of k-means is reached; S310: Determine whether the preset tree depth has been reached. If yes, the partitioning scheme is formed, and the clustering result of the last round is the partitioning, and the cluster number of the data point is the corresponding partition number. If not, independently execute step S32 and the following steps for each cluster formed in the last round, and start different thread blocks to process different data points in the last round cluster to realize inter-layer parallelism.

[0009] Further, the specific implementation of step (3) is as follows: S41: Obtain the k-nearest neighbor graph, and the neighborhood radius eps and the minimum point number minPts of DBSCAN; S42: Traverse all data points. If in the k-nearest neighbor graph, the distance between the first neighbor of a point and the point is less than or equal to eps, mark the point as a core point, otherwise mark it as a non-core point; S43: According to the information marked in step S42, load the next neighbors of all core points into the global memory of the GPU; S44: Distribute all core points evenly to each thread block, and a single thread in the thread block is responsible for performing a binary search on the single core point neighbor to obtain the number of points in the core point neighbor that satisfy the distance less than or equal to eps; S45: According to the result obtained in step S44, load the data point number in the core point neighbor list that satisfies the distance less than or equal to eps into the global memory of the GPU; S46: Obtain the k-means tree partitioning scheme; S47: Create a cluster table and initialize the cluster number of each data point to the number of the data point; S47: Create a cluster table and initialize the cluster number of each data point to the number of the data point; S48: Assign a thread block to each partition according to the k-means tree partitioning scheme; S49: Create a queue at the shared memory of the thread block and allocate space for indicating whether a data point in the partition has been visited; S410: Find the first unvisited core point in the partition according to the breadth-first search strategy, and then add it to the queue through the host thread; S411: Take out the head element of the queue and perform the dequeue operation on the head element by the host thread; S412: All threads in the thread block process the neighbors of the head element in parallel, that is, for any neighbor node, determine whether the node is a core point: if it is a non-core point, directly merge it into the cluster where the head element is located, and if the non-core point belongs to the partition handled by the current thread block, mark it as visited at the same time; if it is a core point, determine whether the neighbor node belongs to the partition handled by the current thread block, if it does, merge it into the cluster where the head element is located, mark it as visited at the same time, and insert it into the queue through atomic operation; S413: Repeat steps S411-S412 until the queue is empty; S414: Repeat steps S410-S413 until no unvisited core point can be found, thereby completing the clustering in a single partition.

[0010] Further, the specific implementation of step (4) is as follows: S51: Obtain the partition clustering result; S52: Initialize the edge set to be empty, which is located in the global memory of the GPU; S53: Obtain all core points to obtain a core point set; S54: Obtain the core point neighbor list located in the global memory of the GPU; S55: Obtain the k-means tree partitioning scheme to obtain a partition number table; S56: Distribute all points in the core point set evenly to each thread block, and the thread block processes the core point belonging to it in series, that is, for the currently processed core point, use all threads to process its neighbor nodes in the core point neighbor list, and if the neighbor node is a core point and has a different partition number from the currently processed point, add an undirected edge with the currently processed point and the neighbor node as vertices to the edge set; S57: Repeat step S56 until all core points are processed, and obtain the edge set of the core proximity graph; S58: Load the edge set in the global memory of the GPU to the host memory; S59: Traverse all edges in the edge set in the host memory, and merge the clusters where the two vertices of the edge are located, that is, merge the two vertex sets in the way of union-find set; S510: Repeat step S59 until all edges are processed to obtain the final global clustering result.

[0011] A computer device comprises a memory and a processor, the memory has a computer program stored therein, and the processor is configured to execute the computer program to implement the GPU-accelerated high-dimensional vector clustering algorithm.

[0012] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the GPU-accelerated high-dimensional vector clustering algorithm.

[0013] Based on the above technical solutions, the present application has the following beneficial technical effects: 1. The present application designs a parallel k-nearest neighbor graph construction algorithm using GPU acceleration calculation, which can significantly reduce the construction overhead of the k-nearest neighbor graph. By introducing the global k-nearest neighbor graph structure, the distance calculation in the DBSCAN algorithm process can be replaced, the distance calculation is decoupled from the algorithm, and thus the time overhead of clustering is greatly reduced.

[0014] 2. The present application proposes a k-means tree partitioning algorithm based on inter-layer parallelism. The algorithm takes advantage of the k-means algorithm to assign spatially similar points to the same partition, and controls the size and number of partitions by recursively executing the k-means algorithm. The inter-layer parallelization scheme used by the algorithm can greatly improve the utilization of GPU cores and reduce the time overhead of the algorithm.

[0015] 3. The present application uses the global k-nearest neighbor graph index and the k-means tree partitioning scheme to design a parallel clustering algorithm based on breadth-first search and core neighbor graph, which improves the calculation process of the DBSCAN algorithm. On the premise of ensuring clustering accuracy, high-concurrency vector clustering is realized, and the efficiency of large-scale high-dimensional vector clustering is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The figure is a flowchart of the GPU-accelerated high-dimensional vector clustering algorithm of the present application.

[0017] Figure 2 The figure is an algorithm diagram for constructing a k-nearest neighbor graph in the present application.

[0018] Figure 3 The figure is a diagram of the k-means tree partitioning algorithm in the present application.

[0019] Figure 4 The figure is a diagram of the core neighbor graph constructed in the present application. DETAILED DESCRIPTION

[0020] In order to describe the present application more specifically, the technical solutions of the present application are described in detail below in combination with the drawings and specific embodiments.

[0021] The embodiment provides a high-dimensional vector clustering algorithm based on GPU acceleration, and a specific process thereof is as shown in Figure 1 Step S11: Obtain a high-dimensional space vector set, and utilize a calculation process of a GPU accelerated NN-descent algorithm to complete construction of a global k-neighbor graph.

[0022] The high-dimensional space vector set is composed of a plurality of high-dimensional space vector objects, and the vector objects can be generated by a plurality of actual scenes. For example, in a recommendation system or an e-commerce platform, a user and an item can be represented as a high-dimensional vector; in image processing, each image can be converted into a high-dimensional vector; and in bioinformatics, a gene sequence can be converted into a high-dimensional vector. The NN-descent is an efficient nearest neighbor search algorithm, and is particularly suitable for constructing an approximate k-neighbor graph on a large-scale vector dataset. The algorithm is based on the idea that "neighbors of neighbors can also be neighbors", and utilizes second-order neighbor information of data points to update a neighbor set of each data point in an iterative optimization manner. The iterative process reduces redundant calculation when the k-neighbor graph is constructed, and significantly improves the efficiency. In this step, the NN-descent algorithm is executed by using GPU acceleration, and a k-neighbor graph index is constructed for all vector objects, as shown in Figure 2 S21: Obtain a high-dimensional space vector set and a maximum iteration number parameter. The high-dimensional space vector set is composed of a plurality of high-dimensional space vector objects generated by actual scenes, and the maximum iteration number parameter represents a maximum round of optimization of the neighbor set by the NN-descent algorithm.

[0023] S22: Randomly initialize the k-neighbor graph, that is, assign a random k neighbor to each node.

[0024] The k-neighbor graph is stored in the form of an adjacency matrix with n (number of data points) rows and k (number of neighbors) columns, and the number and distance of the neighbor nodes are also saved. In addition, a reverse k-neighbor graph is created synchronously to facilitate the execution of the algorithm, and the reverse k-neighbor graph is optimized synchronously with the forward k-neighbor graph during the execution of the algorithm. At the beginning of the algorithm, a random k other data point is selected as a neighbor for each data point, and the distance is initialized to infinity.

[0025] S23: Sample the k-neighbor graph in parallel, and the sampling retains a part of the second-order forward nearest neighbor and the second-order reverse nearest neighbor to form a sampling node set.

[0026] Specifically, the second-order forward nearest neighbor of the point p in the directed graph refers to a point​​p points reachable from point q by two edges p . p points reachable from point q by two edges.

[0027] S24: Use parallel double sorting algorithm to sort the sample node set and form an ordered sequence. Assign a thread to each element in the sequence. Each thread determines whether the element is the same as the previous element. If not, write it to the candidate point array in shared memory to obtain the de-duplicated sample node set.

[0028] In the k-neighbor graph, the adjacency list of different neighbors of a point may have intersections. For example , , , where represents a data point, represents the adjacency list of data point . In this case, since and have a common neighbor , when sampling the second-order neighbors of , may be sampled multiple times, so we need to de-duplicate the sample node set. Double sorting is a special sorting algorithm that uses the idea of bidirectional scanning to compare and exchange from both ends, gradually sorting the sequence. Thanks to its sorting process, which can be naturally divided into multiple independent tasks, this algorithm is particularly suitable for parallelization. Through parallel processing, double sorting can significantly improve efficiency in large-scale data sorting.

[0029] S25: Assign a thread block to each point in the high-dimensional vector set. All threads in the thread block load the vector corresponding to the point into the register, and then load the de-duplicated sample node set into shared memory in parallel.

[0030] Registers are exclusive to threads, so the vector loading operation is executed in series; shared memory is shared by all threads in the thread block, and the data migration of the sample node set is completed by all threads in parallel.

[0031] S26: Each thread in the thread block calculates the distance of the partial dimension of the vector. After calculating the local dimension distance, use the inter-thread communication function shuf_sync of CUDA to exchange data between threads, and then merge the results of each thread to obtain the final distance.

[0032] Specifically, the distance between the vector to which the thread block belongs and each vector in the set of sampling nodes is calculated, and for a single specific distance calculation, the components in each dimension are evenly distributed to each thread by using the method of adjacent threads accessing continuous memory addresses, and then each thread calculates the local distance of the component it is responsible for. Adjacent threads accessing continuous memory addresses can ensure centralized access of the display memory, realize transaction merging of memory access, thereby improving cache utilization and overall memory access efficiency of the GPU.

[0033] shfl_sync is one of the important functions in CUDA for inter-thread communication, which can efficiently share data within a warp (32 threads) without using shared memory or synchronization overhead. In this embodiment, we add the local distance saved by multiple threads in the warp to the local distance of the specified interval thread each time (the initial interval is 16, for example, the distance of thread 16 is added to thread 0, the distance of thread 17 is added to thread 1, and so on), and then reduce the step size by half each round and continue processing. After 5 rounds of processing, ) the 0th thread obtains the sum of the local distances of 32 threads. By using the above parallel reduction processing, the parallel computing advantage of the GPU can be fully utilized, and the local distance merging operation can be efficiently implemented.

[0034] S27: Repeat step S26 until the distance of all points in the set of sampling nodes to the point to which the thread block belongs is calculated.

[0035] S28: Sort the candidate points according to the distance using a parallel double sorting algorithm, and then merge the existing neighbor list and the candidate point array according to the distance, and keep the first k data points to form an updated neighbor list.

[0036] The existing k-nearest neighbor graph neighbor list is stored in ascending order. After the candidate point distance is sorted, the two ordered lists are merged according to the merge sort strategy, and the first k data points of the merged list are kept to update the original k-nearest neighbor graph neighbor list.

[0037] S29: Repeat steps S23-S28 until the maximum number of iterations is reached to form the final k-nearest neighbor graph.

[0038] Step S12: Obtain a set of high-dimensional space vectors, recursively execute the k-means algorithm using an inter-layer parallel scheme and Tensor core acceleration technology to obtain a k-means tree partitioning scheme.

[0039] The k-means tree is an algorithm for generating a hierarchy by recursive clustering. The process is as follows: first, perform k-means clustering on all data points, and take the k clusters obtained as the k child nodes of the tree, and then perform k-means clustering again on each child node to generate the next layer of nodes. Unlike traditional methods, the k-means tree does not need to partition data by dimension when calculating, and has high efficiency.

[0040] Although the clustering calculation within each node of the k-means tree is suitable for GPU parallelization, as the tree depth increases, the number of data contained in the high-level nodes decreases exponentially, and sequential generation of nodes will result in insufficient utilization of GPU resources, thereby causing a decrease in computing performance. To address this problem, in this step, an inter-layer parallel k-means tree algorithm enhanced by Tensor cores is used. This algorithm generates all nodes in the same layer in parallel, rather than sequentially generating them layer by layer, ensuring that GPU resources are always used efficiently. The core of this algorithm is that the total amount of data at each layer of the k-means tree is constant, so inter-layer parallelism can maximize parallelism.

[0041] Tensor cores are hardware units designed by NVIDIA to accelerate small-scale matrix multiplication. They use the matrix multiplication and addition (MMA) principle to perform matrix operations in parallel by multiple threads in one clock cycle. Compared with traditional CUDA cores, Tensor cores have higher computational parallelism and lower scheduling overhead, and can further improve computational density and efficiency with low-precision data formats. In particular, batch vector Euclidean distance operations can be equivalent to large-scale matrix multiplication. Therefore, Tensor cores can be used to efficiently calculate the distance between multiple data points and each cluster center, thereby significantly improving the operation performance and demonstrating strong acceleration advantages, as shown in Figure 3 The specific implementation process is as follows: S31: Obtain a set of high-dimensional vectors to be partitioned, the degree k of the tree, and the maximum number of iterations of k-means.

[0042] The k-means algorithm is a classic clustering algorithm that divides a set of data points into k non-overlapping clusters, so that the similarity within each cluster is high, and the difference between different clusters is large. The core idea of this algorithm is to assign each data point to the nearest cluster center through iteration, and continuously update the position of the cluster center until the cluster center converges or the cluster assignment result no longer changes significantly. In this embodiment, the degree k of the tree represents the number of clusters to be divided by the k-means algorithm at each node of the tree; the maximum number of iterations of k-means controls the number of times the cluster center is updated, and this parameter is a global parameter applied to all nodes of the k-means tree.

[0043] S32: Randomly select k cluster centers, i.e., randomly select k points in the data set as the initial cluster centers of the k-means clustering algorithm.

[0044] S33: Distribute all data points to each thread block uniformly, and then each thread block loads the cluster centers and the data points belonging to the thread block into shared memory. The shared memory is shared by all threads in the thread block, so the cluster center vector and the data point vector belonging to the thread block are completed by each thread in cooperation.

[0045] S34: The thread block calls different Tensor cores to perform matrix multiplication operations to calculate the distances between the corresponding data points and the cluster centers, and writes the calculated distances to the shared memory.

[0046] Specifically, the left matrix is stored by row first to store the vector of the data points belonging to the thread block, and the right matrix is stored by column first to store the vector of each cluster center, to obtain the vector dot product matrix of the data points belonging to the thread block and all cluster centers; then all threads are mobilized to process the vector dot product matrix in parallel, to perform operations on the dot product in the matrix and the pre-calculated vector self-multiplication product to obtain the Euclidean distance and write it to the shared memory.

[0047] S35: All threads in the thread block respectively take out the distance between a single corresponding data point and each cluster center, and then use the CUDA inter-thread communication function shuf_sync to select the cluster center with the locally minimum distance in different thread groups, and write the locally minimum distance to the shared memory.

[0048] In this embodiment, we compare the distances saved by multiple threads in a warp with the distance saved by a specified interval thread each time, and then replace the smaller value to the smaller thread (the initial interval is 16, for example, the distance of the 16th thread is compared with the distance of the 0th thread, the smaller distance is replaced to the 0th thread; the distance of the 17th thread is compared with the distance of the 1st thread, the smaller distance is replaced to the 1st thread, and so on), and then the step is reduced by half each round and continues to process, after 5 rounds of processing, ) the 0th thread stores the minimum distance of 32 threads.

[0049] S36: The thread block reassigns a dedicated thread group to reduce the locally minimum distance in the shared memory, and finally selects the cluster center with the globally minimum distance to complete the cluster center assignment of a single data point.

[0050] After obtaining the locally minimum distance in different warps, a process similar to step S35 is used to obtain the minimum distance between different warps at a higher level.

[0051] S37: Repeat steps S35-S36 until the cluster center assignment of all data points belonging to the thread block is completed.

[0052] S38: The GPU sums the vectors in each cluster in parallel, then takes the average and updates them as the new cluster centers.

[0053] S39: Repeat steps S33 to S38 until the preset maximum number of iterations is reached.

[0054] S310: Determine whether the preset tree depth has been reached. If it has been reached, the partitioning scheme is formed, and the clustering result of the last round is the partition, with the cluster number to which the data point belongs being the corresponding partition number. If it has not been reached, for each cluster formed in the previous round, step S32 and its subsequent steps are executed independently, and different thread blocks are started to process the data points in different clusters from the previous round to achieve inter-layer parallelism.

[0055] Step S13: Using the k-nearest neighbor graph and partitioning scheme, allocate a GPU thread block for each partition, and perform clustering independently and in parallel in different partitions according to the breadth-first search strategy to obtain the partition clustering results.

[0056] The clustering performed in this implementation is DBSCAN clustering. DBSCAN is a clustering algorithm that does not require pre-specifying the number of clusters, can identify clusters of arbitrary shapes, and is robust to noise. It uses two parameters: the neighborhood radius eps and the minimum number of points minPts to divide core points, boundary points, and noise points, thereby forming clusters in dense regions. DBSCAN is particularly suitable for finding clusters in datasets with obvious spatial or structural characteristics, but it is sensitive to parameters and performs poorly in high-dimensional data.

[0057] DBSCAN's clustering process iterates through all data points, finding the number of neighbors for each point within a given radius eps. If the number is not less than minPts, it is marked as a core point and a new cluster is generated. Simultaneously, the core points in its neighborhood are recursively expanded, gradually forming density-connected clusters. If a point has insufficient neighbors, it is marked as a noise point or a boundary point. Ultimately, the algorithm identifies several dense regions as clusters, and the remaining isolated points as noise points.

[0058] Expanding the core points within the neighborhood typically involves two methods: breadth-first search and depth-first search. To facilitate parallel implementation, this embodiment employs a breadth-first search strategy. The specific implementation process for performing intra-partition clustering is as follows: S41: Obtain the k-nearest neighbor graph and density parameters eps and minPts.

[0059] The k-nearest neighbor graph is stored as an n x k matrix, simultaneously storing the nearest neighbor number matrix and the nearest neighbor distance matrix. The i-th row of the matrix represents the adjacency list of the i-th data point, with neighbors stored in ascending order of distance. The j-th column of the matrix represents the j-th nearest neighbor of the data point in the adjacency list. The density parameter eps represents the neighborhood radius of the DBSCAN algorithm, and minPts represents the minimum number of points.

[0060] S42: Traverse all data points. If, in the k-nearest neighbor graph, the k-th nearest neighbor of a certain point... If the distance between a point and its neighbors is less than or equal to eps, then the point is marked as a core point; otherwise, it is marked as a non-core point.

[0061] In this embodiment, the nearest neighbor list of a data point in the k-nearest neighbor graph does not contain the data point itself (distance is 0). Therefore, if the ordered neighbor list of a point contains the k-th nearest neighbor, the nearest neighbor of that data point is not included in the nearest neighbor list .... If at least `minPts` points within a neighborhood of a given point are less than or equal to `eps`, then the point satisfies the definition of a core point: given an integer `eps`, for a data point... p ,if Then it is called a data point p The core point.

[0062] S43: Based on the information marked in step S42, assign all core points to the following... Each neighbor is loaded into the GPU's global memory.

[0063] Since the GPU's global memory capacity is limited, loading the entire global k-nearest neighbor graph into GPU memory is impractical and would waste a significant amount of storage space. During the expansion process of the DBSCAN algorithm, only points whose distance to the core point is less than or equal to eps are merged into the cluster, and expansion continues in this manner. Based on these properties, in this implementation, a binary search is used to find the position of the last point in the adjacency list of all core points in the k-nearest neighbor graph whose distance to the corresponding core point is less than or equal to eps. The nearest neighbors at the found position and those preceding it are then loaded into GPU memory, thereby reducing the waste of GPU global memory space.

[0064] S44: Distribute all core points evenly to each thread block. Each thread in the thread block is responsible for performing a binary search on the neighbors of the single core point described in step S43 to obtain the number of neighbors of the core point whose distance is less than or equal to eps.

[0065] S45: Based on the results obtained in step S44, load the data point numbers in the neighbor list of the core point that satisfy the distance less than or equal to eps into the GPU global memory.

[0066] The core point neighbor list discards meaningless nearest neighbors during the DBSCAN algorithm expansion process. During execution, there is no need to determine whether the expansion conditions are met based on distance. Therefore, only the node number needs to be saved without loading the distance into the GPU memory, further saving memory space.

[0067] S46: Obtain the k-means tree partitioning scheme.

[0068] Specifically, the clustering results obtained from the last layer of the k-means tree are obtained to get the partitioning scheme; each cluster corresponds to a partition, which is stored in the form of a partition table; the partition table is a data structure similar to a disjoint set, which stores the partition numbers of data points continuously in an array. In this embodiment, the partition table is loaded into the GPU global memory.

[0069] S47: Create a cluster table and initialize the cluster number of each data point to the data point number; the cluster table is also stored in the form of a disjoint set, and each cluster can be regarded as a set in the disjoint set; initially, each point is considered to be a cluster of its own.

[0070] S48: Based on the partitioning scheme obtained in step S46, allocate a thread block for each partition.

[0071] S49: Create a queue in the shared memory of the thread block and allocate space to indicate whether a data point within the partition has been accessed.

[0072] Since each partition is processed independently, and during the breadth-first search expansion process, only points within the current partition are expanded, the number of points entering the queue during single-partition clustering will not exceed the number of partition points. Based on this property, the size of the queue in shared memory and the size of the space used to indicate whether data points within a partition have been accessed can both be fixed at the number of partition points.

[0073] S410: Following a breadth-first search strategy, find the first unvisited core point within the partition, and then add it to the queue via the main thread.

[0074] The DBSCAN algorithm generates new clusters starting from the core point. The enqueue operation is performed by the main thread, which first copies the enqueue element to the memory pointed to by the tail pointer, and then moves the tail pointer one position to the right. When the main thread performs the enqueue operation, other threads need to wait for the main thread to complete its execution through a synchronization mechanism.

[0075] S411: Retrieve the first element of the queue, and then the main thread dequeues the first element.

[0076] Specifically, the first element of the queue is copied to the thread's private register by all threads within the thread block. The dequeue operation is controlled by the main thread to move the head pointer one position forward. When the main thread performs the dequeue operation, the other threads need to wait for the main thread to complete the execution through the synchronization mechanism.

[0077] S412: All threads within the thread block process the neighbors of the head element in parallel (at this point, the neighbor list has been processed by step S45, retaining only points with a distance less than or equal to eps); for each neighbor node, determine whether the point is a core point. If it is a non-core point, it is directly merged into the cluster where the head element of the queue is located. If the non-core point belongs to the partition processed by the current thread block, it is also marked as visited; if it is a core point, determine whether the neighbor node belongs to the partition processed by the current thread block. If it does, it is merged into the cluster where the head element of the queue is located, marked as visited, and inserted into the queue through an atomic operation.

[0078] If the expanded neighbor is a non-core point, assigning it to any adjacent cluster can logically guarantee the correctness of the clustering result. Therefore, its partitioning can be ignored and it can be directly merged into the current cluster. Since this implementation only considers clustering within a single partition, core points of other partitions are ignored when expanding neighbors.

[0079] S413: Repeat steps S411~S412 until the queue is empty; if the queue is empty, it means that the current cluster cannot find a core point neighbor in this partition to continue expanding. At this time, we should continue to find unvisited core points to generate new clusters.

[0080] S414: Repeat steps S410~S413 until no unvisited core points can be found. At this point, clustering within a single partition is complete. If no unvisited core points can be found, it means that the points that have not yet been clustered are noise points relative to the current partition, and the clustering of the current partition ends.

[0081] Step S14: Based on the density direct reach relationship between the core points of the high-dimensional space vector, construct a core nearest neighbor graph, and merge the partition clustering results through the core nearest neighbor graph to obtain the final clustering result.

[0082] like p With the core point, q lie in p Within the ε-neighborhood, it is called q can be p Density directly accessible. If located in partition S... i The core point p With partition S j The core point q Density-based access means that, according to the density-based cluster definition, when performing DBSCAN on the complete dataset, regardless of the cluster origin... p Or byq Departure, final p and q They will all belong to the same cluster. Therefore, during the cluster merging process, p and q The clusters to which they belong should eventually be merged into the same cluster.

[0083] Based on the above observations, firstly, using the nearest neighbor information of the k-nearest neighbor graph, a core nearest neighbor graph is constructed between the core points in the partition, with core points as nodes and density-direction relationships between core points in the partition as edges. The core nearest neighbor graph is a subgraph of the k-nearest neighbor graph. When two core points in a partition are each other's nearest neighbors, an edge is established between the core points. Core points connected by edges and their respective clusters should be merged into the same cluster. Thus, the local cluster merging problem can be transformed into a problem of calculating the connected components of the core nearest neighbor graph. To reduce memory overhead and avoid repeated merging, no edges are established between density-direction core points located in the same partition, because these points have already completed local cluster merging in step S13; edges are established as follows... Figure 4 The core nearest neighbor graph shown is used to merge the clustering results. The specific implementation process is as follows: S51: Obtain the partition clustering results. The partition clustering results are stored in the cluster table, and the cluster number is the number of the core point representing the entire cluster.

[0084] S52: Initialize the edge set to empty; this set is located in the GPU global memory.

[0085] S53: Obtain all core points and get the core point set, which is an array containing the numbers of all core points.

[0086] S54: Get the list of neighbors of the core point in the GPU global memory. The list of neighbors of the core point is a subgraph of the k-nearest neighbor graph. This subgraph only retains the core point as a vertex, and each vertex only retains neighbors whose distance is less than or equal to eps.

[0087] S55: Obtain the k-means tree partitioning scheme and get the partition number table.

[0088] S56: Distribute all points in the core point set evenly to each thread block. Each thread block processes its own core point serially. For each currently processed core point, all threads process its neighbors in the core point nearest neighbor list in parallel. If the neighbor is a core point and its partition number is different from that of the currently processed point, add an undirected edge to the edge set with the current processed point and the neighbor as vertices.

[0089] S57: Continue executing the process described in step S56 until all core points have been processed, and obtain the edge set of the core nearest neighbor graph; the edge set of the core nearest neighbor graph is a subgraph of the core point nearest neighbor list.

[0090] S58: Loads the set of edges in the GPU global memory into main memory.

[0091] S59: Traverse all edges in the main memory edge set, merge the clusters of the two vertices of the edge, that is, merge the sets of the two vertices in the disjoint set method; for the two vertices of a specified edge, first find their root nodes (i.e., cluster numbers) in the disjoint set, and then make one of the root nodes the parent node of the other root node.

[0092] S510: Continue executing the process described in step S59 until all edges have been processed and the final global clustering result is obtained. At this point, it is equivalent to finding all connected components in the core nearest neighbor graph.

[0093] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A GPU-accelerated high-dimensional vector clustering algorithm, characterized in that, The method comprises the following steps: (1) obtaining a high-dimensional space vector set, using GPU to accelerate the execution of the NN-descent algorithm, and constructing a global k-nearest neighbor graph for all vectors in the set; (2) recursively executing the k-means algorithm on the high-dimensional space vector set by using an inter-layer parallel scheme and a Tensor core acceleration technology to obtain a k-means tree partition scheme; (3) using the k-nearest neighbor graph and the k-means tree partition scheme, assigning a GPU thread block to each partition, and independently and parallelly executing clustering among different partitions according to a breadth-first search strategy to obtain a partition clustering result; (4) constructing a core nearest neighbor graph according to the direct density relationship between core points of the high-dimensional space vectors, and merging the partition clustering result through the core nearest neighbor graph to obtain a final global clustering result.

2. The GPU-accelerated high-dimensional vector clustering algorithm according to claim 1, wherein, The specific implementation of step (1) is as follows: S21: obtaining a high-dimensional space vector set and a preset maximum iteration number; S22: randomly initializing the k-nearest neighbor graph, that is, assigning random k neighbors to each node; S23: parallelly sampling the k-nearest neighbor graph, and retaining part of the second-order forward nearest neighbor and the second-order reverse nearest neighbor to form a sampling node set; S24: using a parallel double sorting algorithm to sort the sampling node set to form an ordered sequence, and then assigning a thread to each element in the sequence, which is used to judge whether the current element and the previous element are the same, and if not, writing the candidate point array in the shared memory to obtain a de-duplicated sampling node set; S25: assigning a thread block to each vector in the high-dimensional space vector set, and all threads in the thread block are used to load the vector into the register, and then the de-duplicated sampling node set is loaded into the shared memory in parallel; S26: each thread in the thread block calculates the local dimension distance of the vector, and after the local dimension distance is calculated, the data exchange between threads is performed by using the inter-thread communication function shuf_sync of CUDA, and then the calculation results of the threads are merged to obtain the final distance; S27: repeating step S26 until the distance of all nodes in the de-duplicated sampling node set to the vector to which the thread block belongs is calculated; S28: using a parallel double sorting algorithm to sort the candidate points according to the distance, and then merging the neighbor list of the current k-nearest neighbor graph and the candidate point array according to the distance to retain the first k data points and form an updated neighbor list; S29: repeating steps S23-S28 until the maximum iteration number is reached, and forming a final k-nearest neighbor graph.

3. The GPU-accelerated high-dimensional vector clustering algorithm of claim 1, wherein, The specific implementation of step (2) is as follows: S31: obtaining a high-dimensional space vector set to be partitioned, a preset tree depth, and a maximum iteration number of k-means; S32: randomly selecting k cluster centers, that is, randomly selecting k data points in the high-dimensional space vector set as initial cluster centers of the k-means clustering algorithm; S33: uniformly distributing all data points in the high-dimensional space vector set to thread blocks, and the thread blocks load the cluster centers and the data points into the shared memory; S34: The thread block calls different Tensor cores to perform matrix multiplication operation, calculates the distance between the data point and the cluster center, and writes the calculated distance to the shared memory; S35: Each thread in the thread block takes out the distance between the corresponding single data point and each cluster center, and then selects the cluster center with the minimum local distance in different warps using the CUDA inter-thread communication function shuf_sync, and writes the local minimum distance to the shared memory; S36: The thread block allocates a dedicated thread group to reduce the local minimum distance in the shared memory of the warp, and finally selects the cluster center with the minimum global distance to complete the cluster center allocation of the single data point; S37: Repeat steps S35-S36 until the cluster center allocation of all data points belonging to the thread block is completed; S38: Use GPU parallel to sum the vectors in each cluster and take the average as the new cluster center; S39: Repeat steps S33-S38 until the maximum number of iterations of k-means is reached; S310: Determine whether the preset tree depth is reached, if reached, the partitioning scheme is formed, and the last round of clustering results is the partitioning, and the cluster number of the data point belongs to the corresponding partition number; if not reached, for each cluster formed in the last round, independently execute step S32 and the following steps, and start different thread blocks to process different data points in the last round cluster to realize the parallel between layers.

4. The GPU-accelerated high-dimensional vector clustering algorithm of claim 1, wherein, The specific implementation of step (3) is as follows: S41: Obtain the k-nearest neighbor graph, and the neighborhood radius eps and the minimum point number minPts of DBSCAN; S42: traverse all data points, if the distance between the kth neighbor of a point and the point is less than or equal to eps in the k-neighbor graph, mark the point as a core point, otherwise mark the point as a non-core point; S42: traverse all data points, if the distance between the kth neighbor of a point and the point is less than or equal to eps in the k-neighbor graph, mark the point as a core point, otherwise mark the point as a non-core point; S43: Load all the core points' neighbors into the GPU global memory according to the information marked in step S42. S43: Load all the core points' neighbors into the GPU global memory according to the information marked in step S42. S44: Distribute all core points to each thread block uniformly, and a single thread in the thread block is responsible for performing binary search on the neighbors of a single core point to obtain the number of points in the neighbors of the core point that satisfy the distance less than or equal to eps; S45: According to the result obtained in step S44, load the data point number in the neighbor list of the core point that satisfies the distance less than or equal to eps into the global memory of the GPU; S46: Obtain the k-means tree partitioning scheme; S47: Create a cluster table and initialize the cluster number of each data point to the number of the data point; S48: Assign a thread block to each partition according to the k-means tree partitioning scheme; S49: Create a queue at the shared memory of the thread block, and allocate space to represent whether the data point in the partition has been visited; S410: According to the breadth-first search strategy, find the first unvisited core point in the partition, and then add it to the queue through the master thread; S411: Take out the head element, and use the master thread to perform the dequeue operation on the head element; S412: All threads in the thread block handle the neighbors of the head element in parallel, that is, for any neighbor node, it is determined whether the node is a core point: if it is a non-core point, it is directly merged into the cluster where the head element is located, and if the non-core point belongs to the partition handled by the current thread block, it is marked as visited at the same time; if it is a core point, it is determined whether the neighbor node belongs to the partition handled by the current thread block, if it belongs, it is merged into the cluster where the head element is located, and is marked as visited at the same time, and is inserted into the queue through an atomic operation; S413: Steps S411-S412 are repeatedly executed until the queue is empty; S414: Steps S410-S413 are repeatedly executed until no unvisited core point can be found, thereby completing the clustering in a single partition.

5. The GPU-accelerated high-dimensional vector clustering algorithm of claim 1, wherein, The specific implementation of step (4) is as follows: S51: Obtain the partition clustering result; S52: Initialize the edge set to be empty, which is located in the global memory of the GPU; S53: Obtain all core points to obtain a core point set; S54: Obtain the neighbor list of the core point located in the global memory of the GPU; S55: Obtain the k-means tree partition scheme to obtain a partition number table; S56: Distribute all points in the core point set to each thread block uniformly, and the thread block handles the core point belonging to it in series, that is, for the currently handled core point, all threads handle its neighbor nodes in the core point neighbor list in parallel, if the neighbor node is a core point and the partition number of the current processing point is different, a undirected edge with the current processing point and the neighbor node as vertices is added to the edge set; S57: Step S56 is repeatedly executed until all core points are processed, and the edge set of the core near neighbor graph is obtained; S58: Load the edge set in the global memory of the GPU to the main memory; S59: Traverse all edges in the edge set in the main memory, and merge the clusters where the two vertices of the edge are located, that is, merge the two vertex sets in the way of the union-find set; S510: Step S59 is repeatedly executed until all edges are processed, and the final global clustering result is obtained.

6. A computer device comprising a memory and a processor, said memory having stored therein a computer program, characterized in that: The processor is configured to execute the computer program to implement the GPU-accelerated high-dimensional vector clustering algorithm according to any one of claims 1-5.

7. A computer readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to implement the GPU-accelerated high-dimensional vector clustering algorithm according to any one of claims 1-5.

Images