Picture clustering method and apparatus

Abstract

Embodiments of the present specification provide a picture clustering method and device, the method comprising determining a picture set to be clustered; compressing pictures to be clustered in the picture set to be clustered according to a preset neighbor algorithm, determining picture distances between a target picture to be clustered and other pictures to be clustered according to the compressed pictures to be clustered; connecting pictures to be clustered in the picture set to be clustered according to the picture distances, and clustering the connected pictures to be clustered according to a preset clustering method to obtain a picture clustering result. The picture clustering method utilizes the stable characteristics of connected graph division under high-recall vector retrieval or low-recall vector retrieval, uses the preset neighbor algorithm to independently disassemble subgraphs of the pictures to be clustered, and clusters the pictures according to the disassembled independent subgraphs, thereby greatly reducing the memory and computing resource occupation, and solving the problem that in a large-scale clustering scenario, due to a large number of pictures, the clustering algorithm cannot run in the picture clustering process.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to an image clustering method. Background Technology

[0002] Large-scale vector (e.g., billions of vectors) clustering is widely used in clustering scenarios such as image clustering or text clustering. Existing vector clustering algorithms, such as density clustering or probabilistic clustering, often require calculating the nearest neighbor relationships of all vectors and loading all vectors to achieve clustering. However, large-scale clustering scenarios often exceed the machine's memory capacity and the computational complexity increases superlinearly, which can cause the algorithm to fail to run. Summary of the Invention

[0003] In view of this, embodiments of this specification provide an image clustering method. One or more embodiments of this specification also relate to an image clustering apparatus, an image clustering method and apparatus for traffic scenarios, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, an image clustering method is provided, comprising:

[0005] Determine the set of images to be clustered;

[0006] The images to be clustered in the set of images to be clustered are compressed according to a preset nearest neighbor algorithm. The target image to be clustered and the image distances to other images to be clustered are determined based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered.

[0007] The images to be clustered in the set of images to be clustered are connected according to the image distance, and the connected images are clustered according to a preset clustering method to obtain the image clustering result.

[0008] According to a second aspect of the embodiments of this specification, an image clustering apparatus is provided, comprising:

[0009] The image determination module is configured to determine the set of images to be clustered.

[0010] The distance calculation module is configured to compress the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm, and determine the target image to be clustered and the image distances to other images to be clustered based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the image set to be clustered, and the other images to be clustered are the images to be clustered in the image set to be clustered other than the target image to be clustered.

[0011] The clustering module is configured to connect the images to be clustered in the set of images to be clustered based on the image distance, and to cluster the connected images according to a preset clustering method to obtain the image clustering result.

[0012] According to a third aspect of the embodiments of this specification, an image clustering method for traffic scenarios is provided, including:

[0013] Acquire a video stream in a traffic scene, and split the video stream into video frames. Determine a set of images to be clustered based on the split video frames, wherein the images to be clustered contain pedestrians and / or vehicles.

[0014] The images to be clustered in the set of images to be clustered are compressed according to a preset nearest neighbor algorithm. The target image to be clustered and the image distances to other images to be clustered are determined based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered.

[0015] The images to be clustered in the set of images to be clustered are connected according to the image distance, and the connected images are clustered according to a preset clustering method to obtain the image clustering result.

[0016] According to a fourth aspect of the embodiments of this specification, an image clustering device for traffic scenarios is provided, comprising:

[0017] The image acquisition module is configured to acquire video streams in traffic scenarios, split the video stream into video frames, and determine a set of images to be clustered based on the split video frames, wherein the images to be clustered contain pedestrians and / or vehicles.

[0018] The second calculation module is configured to compress the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm, and determine the target image to be clustered and the image distances to other images to be clustered based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the image set to be clustered, and the other images to be clustered are the images to be clustered in the image set to be clustered other than the target image to be clustered.

[0019] The second clustering module is configured to connect the images to be clustered in the set of images to be clustered based on the image distance, and to cluster the connected images according to a preset clustering method to obtain the image clustering result.

[0020] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0021] Memory and processor;

[0022] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-mentioned image clustering method or the image clustering method in the traffic scenario.

[0023] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described image clustering method or the image clustering method in a traffic scenario.

[0024] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described image clustering method or the image clustering method in a traffic scenario.

[0025] One embodiment of this specification implements an image clustering method and apparatus. The method includes determining a set of images to be clustered, wherein the number of images to be clustered in the set is greater than or equal to a preset threshold; compressing the images to be clustered in the set according to a preset nearest neighbor algorithm; determining a target image to be clustered and image distances to other images to be clustered based on the compressed images, wherein the target image to be clustered is any image to be clustered in the set, and the other images to be clustered are all images to be clustered in the set except the target image; connecting the images to be clustered in the set according to the image distances; and clustering the connected images according to a preset clustering method to obtain an image clustering result.

[0026] This image clustering method leverages the stable connectivity graph partitioning characteristic under either high or low recall vector retrieval. It uses a pre-defined nearest neighbor algorithm to decompose the images to be clustered into independent subgraphs, and then performs image clustering based on these independent subgraphs. This significantly reduces memory and computational resource consumption, and solves the problem that the clustering algorithm cannot run during the image clustering process in large-scale clustering scenarios due to the large number of images. Attached Figure Description

[0027] Figure 1 This is a specific example diagram of an image clustering method applied to traffic scenarios provided in one embodiment of this specification;

[0028] Figure 2 This is a flowchart illustrating an image clustering method provided in one embodiment of this specification;

[0029] Figure 3 This is a schematic diagram of feature vectors, sub-feature vector sets, and cluster centers in an image clustering method provided in one embodiment of this specification;

[0030] Figure 4 This is a schematic diagram of a connected graph in an image clustering method provided in one embodiment of this specification;

[0031] Figure 5 This is a flowchart illustrating the processing procedure of an image clustering method provided in one embodiment of this specification.

[0032] Figure 6 This is a schematic diagram illustrating the image clustering effect in an image clustering method provided in one embodiment of this specification;

[0033] Figure 7 This is a schematic diagram of the structure of an image clustering device provided in one embodiment of this specification;

[0034] Figure 8 This is a flowchart illustrating an image clustering method in a traffic scenario, provided in one embodiment of this specification.

[0035] Figure 9 This is a schematic diagram of the structure of an image clustering device in a traffic scenario provided in one embodiment of this specification;

[0036] Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0037] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0038] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0039] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0040] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0041] Vector representation: Representing objects using a one-dimensional array of floating-point numbers, such as representing images or text using 256 or 512-dimensional features.

[0042] K-Nearest Neighbors Algorithm: A vector retrieval algorithm that finds the K nearest neighbors or the vectors with the lowest distance or highest similarity among feature vectors.

[0043] K-Nearest Neighbor Vector Retrieval: Retrieves the K vectors that are closest to an object in a set of vectors by Euclidean distance or cosine distance.

[0044] Accelerating K-nearest neighbor vector retrieval: Methods such as PQ quantization (product quantization) are used to approximate the symmetric or asymmetric distance between vectors, thereby accelerating the calculation of K-nearest neighbors, but quantization errors exist.

[0045] Recall: The ratio of the number of relevant vectors retrieved to the total number of relevant vectors in the vector library. A lower ratio is considered low recall, and a higher ratio is considered high recall. For example, a ratio less than 30% is considered low recall, and a ratio greater than 80% is considered high recall.

[0046] Low recall vector retrieval: Accelerating K-nearest neighbor vector retrieval results in a low proportion of true K-nearest neighbors among the recalled Topk vectors. Low recall allows for a larger margin of error and faster computation speed.

[0047] High recall vector retrieval: Accelerating K-nearest neighbor vector retrieval. A higher proportion of the Topk values ​​that actually belong to the K-nearest neighbors of the vector indicates high recall. This allows for a smaller margin of error, but the computation speed is slower.

[0048] Connected subgraph partitioning: Set the edges of a graph to 1 (connected) or 0 (disconnected) with a certain threshold, and assign all connected nodes to the same subgraph. There are no edge connections between different subgraphs.

[0049] Density clustering or probabilistic clustering algorithms are clustering methods without prior knowledge of the number of classes. The core idea of ​​density clustering is that each sample is examined for its K nearest neighbor density, and samples with a density greater than that of the nearest neighbor are classified as belonging to the same class.

[0050] Superlinear growth: This means that as the number of clustered objects increases, the computational load or memory usage increases more than the linear growth of the number of objects.

[0051] Similarity recall: The ratio of images belonging to this category recalled in the K-nearest neighbors to the total number of images of this category.

[0052] This specification provides an image clustering method. One or more embodiments of this specification also relate to an image clustering apparatus, an image clustering method and apparatus for traffic scenarios, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0053] See Figure 1 , Figure 1 The illustration shows a specific example diagram of an image clustering method applied to a traffic scenario, provided by an embodiment of this specification.

[0054] Figure 1 The system includes an image acquisition terminal 102, a server 104, and an image display terminal 106. The image acquisition terminal 102 can be understood as a shooting device, such as a camera, and the image display terminal 106 includes, but is not limited to, mobile phones, tablets, and desktop computers. The server 104 can be understood as a physical server or a cloud server. For ease of understanding, this specification describes the embodiments using a cloud server as an example.

[0055] Taking the traffic scenario in the city brain as an example, the image acquisition terminal 102, the server 104, and the image display terminal 106 constitute the visual computing platform in the traffic scenario of the city brain, which is used for image classification, establishing human and vehicle files, etc.

[0056] In practice, the image acquisition terminal 102 acquires (e.g., captures) video streams of pedestrians and / or vehicles in traffic scenes in real time and sends the video streams to the server 104. After receiving the video streams, the server 104 splits the video streams into video frames to obtain a set of images containing pedestrians and / or vehicles to be clustered.

[0057] Then, by using low-recall K-nearest neighbor computation (i.e., low-recall vector retrieval), a preset number of images of each image to be clustered and its nearest neighbors are obtained, as well as the distance between each image and its nearest neighbors. Based on this distance, all images to be clustered are then divided into several independent connected subgraphs. Finally, the images to be clustered in each independent connected subgraph are independently clustered using density clustering or probabilistic clustering algorithms to obtain the clustering results of the images to be clustered in the traffic scene. For example, images containing vehicle 1 are in one class, images containing vehicle 2 are in another class, and images containing vehicle 3 are in yet another class.

[0058] After clustering the images of pedestrians and / or vehicles to be clustered, server 104 can send the clustering results to image display terminal 106. Subsequently, users can use the clustering results to query vehicle and / or pedestrian trajectories, create vehicle and / or pedestrian profiles, etc.

[0059] Furthermore, this image clustering method can be implemented not only on server 104, but also on image acquisition terminal 102, local devices, and IoT (Internet of Things) devices. The specific implementation can be tailored to the actual application, and this specification does not impose any limitations on this embodiment. When the computing resources of the local device, client, and IoT device are sufficient, the image clustering process based on this method can also be implemented locally, on the client, and on the IoT device; this specification embodiment only illustrates the implementation of the image clustering method on server 104.

[0060] The image clustering method for traffic scenarios provided in this specification, when the number of pedestrian and / or vehicle images to be clustered in a traffic scenario is large, uses low-recall K-nearest neighbor computation to first perform independent connected subgraph decomposition on the image clustering task. This significantly reduces computer memory and computing resource consumption, and quickly and accurately completes the clustering of pedestrian and / or vehicle images to be clustered.

[0061] See Figure 2 , Figure 2 A flowchart of an image clustering method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0062] Step 202: Determine the set of images to be clustered.

[0063] Specifically, the image clustering method provided in the embodiments of this specification has different application scenarios, and the images to be clustered in the image set to be clustered are also different. For example, when the image clustering method is applied to a traffic scenario, the images to be clustered in the image set to be clustered can be understood as images containing pedestrians and / or vehicles, that is, the images to be clustered include vehicle images and / or pedestrian images. However, when the image clustering method is applied to a product recommendation scenario, the images to be clustered in the image set to be clustered can be understood as images containing any product. Furthermore, in the embodiments of this specification, the image size, type, etc., of the images to be clustered in the image set to be clustered are not limited.

[0064] For ease of understanding, the following embodiments will use the application of the image clustering method in a traffic scenario as an example, where the images to be clustered in the image set to be clustered are understood to be images containing pedestrians and / or vehicles, to provide a detailed introduction to the image clustering method.

[0065] When this image clustering method is applied to large-scale clustering scenarios, the number of images to be clustered in the image set is greater than or equal to a preset threshold. This preset threshold can be set according to the actual application. When the image clustering method is applied to a traffic scenario, the preset threshold can be set to tens of thousands or more, such as 10 million or 20 million. Therefore, the image clustering method provided in this specification is well-suited for large-scale image clustering scenarios.

[0066] When this image clustering method is applied to a traffic scenario, the images to be clustered in the set can be understood as either images directly obtained from a database storing vehicle and / or pedestrian images in the traffic scenario, or traffic video streams obtained from image acquisition devices in the traffic scenario (e.g., cameras placed at various locations in the traffic scenario). After splitting the traffic video stream into video frames, each split video frame is considered as an image to be clustered. The specific implementation method for determining the set of images to be clustered when the video stream is obtained is as follows:

[0067] The process of determining the set of images to be clustered includes:

[0068] Obtain the video stream corresponding to the target project, and split the video stream into video frames. Determine the set of images to be clustered based on the split video frames.

[0069] As mentioned above, the target project varies depending on the specific application scenario of the image clustering method. For example, if the image clustering method is applied to a transportation scenario, the target project can be understood as an image clustering project in a transportation scenario. However, if the image clustering method is applied to a product recommendation scenario, the target project can be understood as an image clustering project in a product recommendation scenario.

[0070] This explanation will take an image clustering project in the context of traffic as an example.

[0071] Specifically, the video stream corresponding to the image clustering project in the traffic scene is obtained, that is, the traffic video stream acquired by the image acquisition device in the traffic scene. Then, the traffic video stream is split into video frames. Finally, each split video frame is regarded as an image to be clustered, and all video frames are regarded as a set of images to be clustered. There are various ways to implement the video frame splitting of the traffic video stream, such as splitting based on inter-frame difference, splitting based on histogram, splitting based on edge detection, etc. The embodiments in this specification do not limit any of them.

[0072] Furthermore, since a single video frame may contain multiple vehicles and / or pedestrians, for better subsequent image clustering, machine learning models (such as pre-trained image segmentation models) can be used to segment the multiple vehicles and / or pedestrians in a video frame, creating a separate image for each vehicle and pedestrian to be clustered. This increases the number of images to be clustered while reducing the complexity of image clustering. Of course, this method can also be used in other target projects where a single video frame contains multiple objects. The specific implementation is as follows:

[0073] The process of obtaining the video stream corresponding to the target project, splitting the video stream into video frames, and determining the set of images to be clustered based on the split video frames includes:

[0074] Obtain the video stream corresponding to the target project, and perform video frame splitting on the video stream according to the machine learning model. Determine the set of images to be clustered based on the split video frames.

[0075] The image set to be clustered provided in the embodiments of this specification can be not only the image set to be clustered obtained directly from the image storage database, but also the image set to be clustered obtained by splitting the real-time video stream into video frames in a real-time scenario, which greatly increases the applicability of the image clustering method.

[0076] Step 204: Compress the images to be clustered in the set of images to be clustered according to the preset nearest neighbor algorithm, and determine the target image to be clustered and the image distances of other images to be clustered based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered.

[0077] Taking a set of images to be clustered that includes images a, b, and c as an example, the target image to be clustered can be image a, image b, or image c. When the target image is image a, the other images to be clustered can be understood as images b and / or c; when the target image is image b, the other images to be clustered can be understood as images a and / or c; when the target image is image c, the other images to be clustered can be understood as images a and / or b. In practical applications, the other images to be clustered are any one, two, or more images in the set of images to be clustered, excluding the target image.

[0078] As described above, the image clustering method provided in this specification is applied to large-scale image clustering scenarios. To reduce computer performance during image clustering and optimize resource consumption and computation time, this specification implements a low-recall preset nearest neighbor algorithm to compress the images to be clustered in the image set. The compressed images are then used to calculate the target image to be clustered and its distance from other images to be clustered. The specific implementation is as follows:

[0079] The step of compressing the images to be clustered in the image set according to a preset nearest neighbor algorithm, and determining the target image to be clustered and the image distances to other images to be clustered based on the compressed images, includes:

[0080] The images to be clustered in the image set are converted into feature vectors of a preset dimension, and the feature vectors are divided into at least two sub-feature vector sets;

[0081] Cluster the sub-feature vectors in each sub-feature vector set to obtain the cluster center of each sub-feature vector set;

[0082] The sub-feature vectors are compressed based on the cluster centers of each sub-feature vector set. Based on the compressed sub-feature vectors, the target image to be clustered and the image distance between the target image and other images to be clustered are determined.

[0083] The preset dimensions can be set according to the actual application. For example, the preset dimensions can be set to 256 or 512 dimensions.

[0084] Specifically, each image in the set of images to be clustered is converted into a feature vector of a preset dimension, and each feature vector is horizontally divided into at least two sub-feature vectors. Then all feature vectors correspond to at least two sub-feature vector sets.

[0085] Then, clustering is performed on the sub-feature vectors in each sub-feature vector set to obtain at least one cluster center for each sub-feature vector set; finally, vector compression is performed on at least two sub-feature vectors corresponding to the image to be clustered based on at least one cluster center in each sub-feature vector set, and the image distance between each target image to be clustered and other images to be clustered is calculated based on the at least two sub-feature vectors corresponding to the image to be clustered after vector compression.

[0086] See Figure 3 , Figure 3 This diagram illustrates the feature vectors, sub-feature vector sets, and cluster centers in an image clustering method provided by an embodiment of this specification.

[0087] Specifically, the set of images to be clustered includes N images. Each of the N images is then converted into a D-dimensional feature vector, and each D-dimensional feature vector is divided into M sub-feature vectors. All the D-dimensional vectors after division result in M ​​sets of sub-feature vectors, i.e., M sub-feature vector sets.

[0088] Then, clustering is performed within each of the M sub-feature vectors to obtain K cluster centers for each sub-feature vector.

[0089] In this embodiment of the specification, by converting the image to be clustered into a feature vector of a preset dimension, and then segmenting the feature vector and obtaining the cluster center for each segmented sub-feature vector set, vector compression can be achieved based on the cluster center of each sub-feature vector set, which greatly reduces computer memory and improves the processing efficiency of the image clustering method.

[0090] Specifically, based on the cluster centers of each sub-feature vector set, vector compression is performed on at least two sub-feature vectors corresponding to the image to be clustered. There are at least two specific implementation methods for determining the target image to be clustered and its distance from other images to be clustered based on the at least two sub-feature vectors corresponding to the compressed image. The first specific implementation method is as follows:

[0091] The step of compressing the sub-feature vectors based on the cluster centers of each sub-feature vector set, and determining the target image to be clustered and its image distance to other images to be clustered based on the compressed sub-feature vectors, includes:

[0092] Based on the cluster centers of each sub-feature vector set, determine the first cluster centers of at least two sub-feature vectors corresponding to the target image to be clustered, and the second cluster centers of at least two sub-feature vectors corresponding to other images to be clustered;

[0093] Based on the first cluster identifier of the first cluster center and the second cluster identifier of the second cluster center, vector compression is performed on at least two sub-feature vectors corresponding to the target image to be clustered and the other images to be clustered.

[0094] Based on the target image to be clustered after vector compression and at least two sub-feature vectors corresponding to the other images to be clustered, the image distance between the target image to be clustered and the other images to be clustered is determined.

[0095] Specifically, determining the first cluster center of at least two sub-feature vectors corresponding to the target image to be clustered can be understood as determining at least two sub-feature vectors corresponding to the target image to be clustered from the set of at least two sub-feature vectors, that is, the sub-feature vectors corresponding to the target image in the horizontal dimension; then determining the first cluster center corresponding to the at least two sub-feature vectors corresponding to the target image to be clustered based on the cluster centers of the set of at least two sub-feature vectors, that is, taking the nearest cluster center in the set of sub-feature vectors containing the at least two sub-feature vectors corresponding to the target image to be clustered as its first cluster center.

[0096] Similarly, determining the second cluster center of at least two sub-feature vectors corresponding to other images to be clustered can also be understood as first determining at least two sub-feature vectors corresponding to each other image to be clustered from the set of at least two sub-feature vectors, and then taking the nearest cluster center in the set of sub-feature vectors containing the at least two sub-feature vectors of the other image to be clustered as its second cluster center.

[0097] Then, based on the first cluster center of at least two sub-feature vectors corresponding to the target image to be clustered, and the second cluster center of at least two sub-feature vectors corresponding to other images to be clustered, the first cluster identifier of the first cluster center and the second cluster identifier of the second cluster center are determined. Then, based on the first cluster identifier of the first cluster center and the second cluster identifier of the second cluster center, vector compression is performed on the at least two sub-feature vectors corresponding to the target image to be clustered and other images to be clustered. That is, since the number of sub-feature vectors corresponding to the target image to be clustered and other images to be clustered is much greater than the number of cluster centers, each sub-feature vector is compressed. After replacing the vectors with cluster identifiers for the cluster centers, the number of sub-feature vectors corresponding to the target image to be clustered and other images to be clustered can be greatly reduced, thus achieving vector compression. Then, based on at least two sub-feature vectors corresponding to the target image to be clustered and other images to be clustered after vector compression, the image distance between the target image to be clustered and other images to be clustered is determined. Where there are multiple first cluster centers, a unique first cluster identifier needs to be set for each first cluster center. Similarly, when there are multiple second cluster centers, a unique second cluster identifier needs to be set for each second cluster center.

[0098] In practical applications, to achieve vector compression, each cluster center in each sub-feature vector set is assigned a unique identifier. Based on the first cluster identifier of the first cluster center and the second cluster identifier of the second cluster center, vector compression is performed on at least two sub-feature vectors corresponding to the target image to be clustered and other images to be clustered. This can be understood as representing each sub-feature vector corresponding to the target image to be clustered using the first cluster identifier of its nearest first cluster center, and representing each sub-feature vector corresponding to other images to be clustered using the second cluster identifier of its nearest second cluster center. This achieves compression of at least two sub-feature vectors corresponding to the target image to be clustered and at least two sub-feature vectors corresponding to other images to be clustered. When specifically calculating the image distance between the target image to be clustered and other images to be clustered, the image distance is calculated based on the at least two sub-feature vectors corresponding to the target image to be clustered represented by the first cluster identifier, and based on the at least two sub-feature vectors corresponding to other images to be clustered represented by the second cluster identifier.

[0099] Determining the image distance between the target image to be clustered and the other images to be clustered can be understood as calculating the distance between each pair of cluster centers based on the first cluster identifier of the first cluster center of at least two sub-feature vectors corresponding to the target image to be clustered and the second cluster identifier of the second cluster center of at least two sub-feature vectors corresponding to the other images to be clustered, thereby obtaining the image distance between the target image to be clustered and the other images to be clustered.

[0100] In practice, the distance between each pair of cluster centers is known when the vector library is built (that is, the images to be clustered in the set of images to be clustered are converted into feature vectors). Therefore, the distance between each pair of cluster centers can be obtained offline. When calculating the image distance between the target image and other images by using the first cluster center of the sub-feature vector corresponding to the target image and the second cluster center of the sub-feature vector corresponding to other images to be clustered online, the distance can be obtained directly by looking up the table through the distance relationship between the first cluster label of the first cluster center and the second cluster label of the second cluster center, which greatly improves the computational efficiency.

[0101] The second specific implementation method for determining the target image to be clustered and the image distance to other images to be clustered after vector compression based on the cluster centers of each sub-feature vector set is as follows:

[0102] The step of compressing the sub-feature vectors based on the cluster centers of each sub-feature vector set, and determining the target image to be clustered and its image distance to other images to be clustered based on the compressed sub-feature vectors, includes:

[0103] Determine the feature vector of a preset dimension corresponding to the target image to be clustered, and determine the second cluster center of at least two sub-feature vector sets corresponding to other images to be clustered based on the cluster center of each sub-feature vector set;

[0104] Based on the second clustering identifier of the second clustering center, vector compression is performed on at least two sub-feature vectors corresponding to the other images to be clustered.

[0105] Based on the feature vector of the target image to be clustered with a preset dimension, and at least two sub-feature vectors corresponding to the other images to be clustered after vector compression, the image distance between the target image to be clustered and the other images to be clustered is determined.

[0106] For a detailed explanation of the second cluster center of at least two feature vectors corresponding to other images to be clustered, and the second cluster identifier of the second cluster center, please refer to the specific description of the above embodiments, which will not be repeated here.

[0107] In practical applications, after determining the cluster centers of the feature vector of the target image to be clustered (of a preset dimension) and at least two sub-feature vectors of other images to be clustered, vector compression can be performed on the at least two sub-feature vectors of other images to be clustered based on the second cluster identifier of the second cluster center. Then, the image distance between the target image to be clustered and the at least two sub-feature vectors of other images to be clustered after vector compression can be calculated. For a detailed explanation of how to perform vector compression on the at least two sub-feature vectors of other images to be clustered based on the second cluster identifier of the second cluster center, please refer to the detailed description in the above embodiments.

[0108] In specific implementation, the above embodiment calculates the distance between each pair of cluster centers by using the first cluster center of at least two sub-feature vectors corresponding to the target image to be clustered after vector compression, and the second cluster center of at least two sub-feature vectors corresponding to each other image to be clustered. However, in this embodiment, only each other image to be clustered (e.g., represented by y) uses cluster centers (e.g., represented by q(y)), while the target image to be clustered (e.g., represented by x) does not. Therefore, when calculating the distance, the distance between x and q(y) is calculated. Specifically, the cluster center of each sub-feature vector corresponding to y after quantization, i.e., q(y), can be obtained in each sub-feature vector set. Then, the distance between q(y) and x in each sub-feature vector set can be calculated, and the square root of the sum of the squares of all distances in each sub-feature vector set is the final distance between x and y.

[0109] In practice, because it uses an asymmetric distance calculation method that calculates the cluster center of the target image and each sub-feature vector of the other images to be clustered, it only has the quantization error of the cluster centers of the other images and each sub-feature vector of the other images to be clustered. This allows for a more accurate calculation of the distance between the target image and other images to be clustered, thus improving the calculation accuracy.

[0110] Furthermore, in another possible implementation, to further improve the calculation efficiency of the image distance between the target image to be clustered and other images to be clustered, this specification also provides another implementable low-recall nearest neighbor algorithm in the embodiments. The specific implementation is as follows:

[0111] The step of compressing the images to be clustered in the image set according to a preset nearest neighbor algorithm, and determining the target image to be clustered and the image distances to other images to be clustered based on the compressed images, includes:

[0112] Cluster the images to be clustered in the set of images to be clustered to obtain the clustered set of images to be clustered and the cluster centers corresponding to the images to be clustered in the set of images to be clustered;

[0113] Based on the images to be clustered in the image set to be clustered and the cluster centers corresponding to the images to be clustered in the image set to be clustered, determine the residual vector of the images to be clustered in the image set to be clustered;

[0114] Determine the residual vector of the target image to be clustered and the residual vectors of other images to be clustered from the set of images to be clustered;

[0115] Vector compression is performed on the residual vector of the target image to be clustered and the residual vectors of the other images to be clustered. Based on the vector-compressed residual vectors of the target image to be clustered and the residual vectors of the other images to be clustered, the image distance between the target image to be clustered and the other images to be clustered is determined.

[0116] Specifically, the process involves clustering the images in the image set to be clustered, resulting in multiple sets of images to be clustered, and cluster centers for each image in each set. For example, a coarse-grained clustering algorithm (k-means clustering) can be used to create K buckets (the image set), each with its own cluster center. This allows us to determine each image and its cluster center. In practical applications, clustering is also based on the feature vectors of each image in the image set. The resulting image set can be understood as the set of feature vectors corresponding to the images, and the cluster center for each image can be understood as the cluster center of the feature vectors corresponding to each image.

[0117] After obtaining the cluster center of the feature vector corresponding to each image to be clustered, we can calculate the feature vector corresponding to each image to be clustered and its corresponding cluster center using vector subtraction to obtain the residual vector of each image to be clustered, which is the residual vector of the feature vector corresponding to each image to be clustered.

[0118] Then, the residual vector of the target image to be clustered and the residual vector of each other image to be clustered are determined from the set of images to be clustered. Vector compression is performed on the residual vector of the target image to be clustered and the residual vector of each other image to be clustered. Finally, the image distance between the target image to be clustered and each other image to be clustered is calculated based on the vector-compressed residual vector of the target image to be clustered and the residual vector of each other image to be clustered.

[0119] Specifically, the method for calculating the image distance between the target image and each other image to be clustered, based on the residual vector of the target image and the residual vector of each other image to be clustered, can be found in the first and second implementations described above, which determine the image distance between the target image and other images to be clustered after vector compression based on the cluster centers of each sub-feature vector set. Specifically, this involves performing product quantization on the residual vector (e.g., dividing the residual vector into at least two feature vector sets as described in the above embodiments, clustering the sub-feature vectors in each sub-feature vector set to obtain the cluster center of each sub-feature vector set, and then compressing the sub-feature vectors based on the cluster centers of each sub-feature vector set), achieving vector compression, and then calculating the image distance between the target image and each other image to be clustered using the compressed vector. This improves computational efficiency while reducing memory usage and increasing computational resources.

[0120] Step 206: Connect the images to be clustered in the set of images to be clustered according to the image distance, and cluster the connected images according to the preset clustering method to obtain the image clustering result.

[0121] Specifically, the step of connecting the images to be clustered in the set of images to be clustered based on the image distance, and then clustering the connected images according to a preset clustering method to obtain the image clustering result includes:

[0122] Based on the image distance, the images in the set of images to be clustered are connected to obtain a connected graph;

[0123] Cluster the images to be clustered in the connected graph using density or probability clustering methods to obtain image clustering results.

[0124] Specifically, the images in the set to be clustered are connected based on their distance to obtain a connected graph. For example, if the set of images to be clustered includes 100 images, then the K-nearest neighbors of each of these 100 images are calculated. The images with distances within a threshold (such as 0.5) are considered connected, and the others are not connected. This results in a graph of 100 image nodes, where each edge represents a high degree of similarity between two images.

[0125] Then, the images to be clustered in the connected graph are clustered according to density or probability clustering methods to obtain the image clustering results.

[0126] In the embodiments of this specification, after the images to be clustered in the set of images to be clustered are divided into independent connected graphs, the images to be clustered in the connected graphs can be clustered according to ordinary density or probability clustering methods to quickly obtain the image clustering results.

[0127] In practice, the images in the set to be clustered are connected based on their distances to obtain a connected graph. The specific implementation method is as follows:

[0128] The step of connecting the images in the set of images to be clustered according to the image distance to obtain a connected graph includes:

[0129] Starting with the target image to be clustered, connect the target image to be clustered and other images to be clustered whose image distance is greater than or equal to a preset distance threshold to obtain a connected graph.

[0130] The preset distance threshold can be set according to the actual application. For example, the preset distance threshold can be set to 0.5 or 0.6.

[0131] Specifically, the images in the set of images to be clustered are connected based on image distance to obtain a connected graph. This can be understood as determining other images whose image distance is greater than a preset distance threshold based on the image distance between each target image to be clustered and other images to be clustered, and connecting them to obtain several independent connected graphs.

[0132] Then, the images to be clustered in each connected subgraph are clustered according to density or probability clustering methods to obtain the image clustering results, that is, the classes after multiple images are aggregated.

[0133] See Figure 4 , Figure 4 This diagram illustrates a connected graph in an image clustering method provided in one embodiment of this specification.

[0134] Figure 4 The dataset includes images to be clustered: a, b, c, d, e, f, g, and h. After calculating the distance between each pair of images to be clustered, images with a distance greater than 0.5 are connected to form a cluster. Figure 4 Several connected graphs in the graph.

[0135] In the embodiments of this specification, a fast low-recall nearest neighbor algorithm is used to calculate image distance. By connecting the images to be clustered in the set of images to be clustered according to the image distance, several independent connected subgraphs are obtained. Subsequently, the images to be clustered in each connected subgraph can be clustered according to density or probability clustering methods, so as to achieve large-scale image clustering while significantly reducing computer memory and computing resource consumption.

[0136] In large-scale image clustering scenarios, images in the set to be clustered are connected based on their distances, resulting in multiple connected graphs. After obtaining the image clustering results for each connected graph using density or probability clustering methods, these results need to be merged to obtain the final, complete image clustering result for all images in the set to be clustered. The specific implementation is as follows:

[0137] The connected graph consists of at least two elements;

[0138] Accordingly, the step of clustering the images to be clustered in the connected graph according to a density or probability clustering method to obtain the image clustering results includes:

[0139] Cluster the images to be clustered in each of the at least two connected graphs according to density or probability clustering methods to obtain at least two initial image clustering results;

[0140] The image clustering result is determined based on the at least two initial image clustering results, wherein the image clustering result includes the clustered image set and the category of the clustered image set.

[0141] In practice, the clustering of images to be clustered in each connected graph is performed independently, and is not limited to the density clustering method or probability clustering method disclosed in the embodiments of this specification. Any other clustering method can be used.

[0142] Using the above example, for instance, 100 images to be clustered can be divided into three connected graphs: one with 60 images, one with 30 images, and one with 10 images. Then, density clustering or probabilistic clustering methods can be used to cluster these three connected graphs into 3, 2, and 2 classes respectively. The final image clustering result is 3 + 2 + 2 = 7 classes, and the image clusters between different connected graphs do not need to be merged.

[0143] The image clustering method provided in this specification utilizes the characteristic that connected graph partitioning is basically stable under high-recall vector retrieval or low-recall vector retrieval. It uses a preset nearest neighbor algorithm to decompose the images to be clustered into independent subgraphs, and then performs image clustering based on these independent subgraphs. This significantly reduces memory and computational resource consumption, and solves the problem that in large-scale clustering scenarios, the clustering algorithm cannot run due to the large number of images. The following, in conjunction with the appendix... Figure 5 Taking the application of an image clustering method provided in this specification in a traffic scenario as an example, the image clustering method will be further explained. Figure 5 The flowchart of an image clustering method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0144] Step 502: Obtain a set of images to be clustered, containing pedestrians and / or vehicles in a traffic scene.

[0145] Step 504: Calculate the image distance between each image to be clustered and other images in the set of images to be clustered, based on the low recall K-nearest neighbor calculation.

[0146] The specific implementation of the low-recall K-nearest neighbor calculation can be found in the above embodiments, and will not be repeated here.

[0147] Step 506: Starting with each image to be clustered, connect each image to be clustered and the images to be clustered that have a distance greater than 0.5 from other images to be clustered to obtain several independent connected subgraphs.

[0148] Step 508: Perform independent clustering on each independent connected subgraph using density clustering or probabilistic clustering methods, and then use the multiple classes after independent clustering as the image clustering results for the traffic scene containing pedestrians and / or vehicles.

[0149] See Figure 6 , Figure 6 This diagram illustrates the image clustering effect of an image clustering method provided in one embodiment of this specification.

[0150] Figure 6 The 'a' part represents the average proportion of elements (images) in each class belonging to the same connected subgraph; Figure 6 The b part represents the number of classes in which all images of a class belong to the same connected subgraph, divided by the total number of classes.

[0151] like Figure 6 As shown, the accelerated computation of single-graph KNN (K-Nearest Neighbor) can be understood as the accelerated computation of the first and second low-recall methods in the above embodiments, and... Figure 6 The horizontal axis represents the recall rate of images of the same class. When m (number of images) = 8 and the KNN recall rate = 0.3, it means that each image randomly recalls 8 * 0.3 = 2.4 random images of the same class and connects them.

[0152] pass Figure 6 It can be seen that under low KNN same-image recall, images of the same type generally belong to the same connected subgraph. For classes with 16 or more images, with a KNN same-image recall of only 0.3, approximately 98% of images of the same type belong to the same connected subgraph. For classes with 8 or more images, with a KNN same-image recall of only 0.6, approximately 95% of images of the same type belong to the same connected subgraph. For classes with 4 or more images, with a KNN same-image recall of only 0.8, approximately 95% of images of the same type belong to the same connected subgraph. This means that the clustering effect of this subgraph segmentation is basically consistent with that of high recall in low-recall scenarios.

[0153] The image clustering method provided in this specification utilizes the characteristics of stable connected subgraph partitioning under both high-recall vector retrieval and low-recall vector retrieval. It uses fast low-recall vector retrieval to independently decompose the connected subgraph for the image clustering task. While significantly reducing computer memory and computing resource consumption and accelerating high concurrency, it achieves results that are basically consistent with single large-scale image clustering under high-recall vector retrieval, thus improving the user experience.

[0154] Corresponding to the above method embodiments, this specification also provides embodiments of image clustering devices. Figure 7 A schematic diagram of an image clustering device according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0155] Image determination module 702 is configured to determine the set of images to be clustered;

[0156] The first calculation module 704 is configured to compress the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm, and determine the target image to be clustered and the image distances to other images to be clustered based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the image set to be clustered, and the other images to be clustered are the images to be clustered in the image set to be clustered other than the target image to be clustered.

[0157] The first clustering module 706 is configured to connect the images to be clustered in the set of images to be clustered according to the image distance, and to cluster the connected images according to a preset clustering method to obtain the image clustering result.

[0158] Optionally, the first computing module 704 is further configured to:

[0159] The images to be clustered in the image set are converted into feature vectors of a preset dimension, and the feature vectors are divided into at least two sub-feature vector sets;

[0160] Cluster the sub-feature vectors in each sub-feature vector set to obtain the cluster center of each sub-feature vector set;

[0161] The sub-feature vectors are compressed based on the cluster centers of each sub-feature vector set. Based on the compressed sub-feature vectors, the target image to be clustered and the image distance between the target image and other images to be clustered are determined.

[0162] Optionally, the first computing module 704 is further configured to:

[0163] Based on the cluster centers of each sub-feature vector set, determine the first cluster centers of at least two sub-feature vectors corresponding to the target image to be clustered, and the second cluster centers of at least two sub-feature vectors corresponding to other images to be clustered;

[0164] Based on the first cluster identifier of the first cluster center and the second cluster identifier of the second cluster center, vector compression is performed on at least two sub-feature vectors corresponding to the target image to be clustered and the other images to be clustered.

[0165] Based on the target image to be clustered after vector compression and at least two sub-feature vectors corresponding to the other images to be clustered, the image distance between the target image to be clustered and the other images to be clustered is determined.

[0166] Optionally, the first computing module 704 is further configured to:

[0167] Determine the feature vector of a preset dimension corresponding to the target image to be clustered, and determine the second cluster center of at least two sub-feature vector sets corresponding to other images to be clustered based on the cluster center of each sub-feature vector set;

[0168] Based on the second clustering identifier of the second clustering center, vector compression is performed on at least two sub-feature vectors corresponding to the other images to be clustered.

[0169] Based on the feature vector of the target image to be clustered with a preset dimension, and at least two sub-feature vectors corresponding to the other images to be clustered after vector compression, the image distance between the target image to be clustered and the other images to be clustered is determined.

[0170] Optionally, the first computing module 704 is further configured to:

[0171] Cluster the images to be clustered in the set of images to be clustered to obtain the clustered set of images to be clustered and the cluster centers corresponding to the images to be clustered in the set of images to be clustered;

[0172] Based on the images to be clustered in the image set to be clustered and the cluster centers corresponding to the images to be clustered in the image set to be clustered, determine the residual vector of the images to be clustered in the image set to be clustered;

[0173] Determine the residual vector of the target image to be clustered and the residual vectors of other images to be clustered from the set of images to be clustered;

[0174] Vector compression is performed on the residual vector of the target image to be clustered and the residual vectors of the other images to be clustered. Based on the vector-compressed residual vectors of the target image to be clustered and the residual vectors of the other images to be clustered, the image distance between the target image to be clustered and the other images to be clustered is determined.

[0175] Optionally, the first clustering module 706 is further configured to:

[0176] Based on the image distance, the images in the set of images to be clustered are connected to obtain a connected graph;

[0177] Cluster the images to be clustered in the connected graph using density or probability clustering methods to obtain image clustering results.

[0178] Optionally, the first clustering module 706 is further configured to:

[0179] Starting with the target image to be clustered, connect the target image to be clustered and other images to be clustered whose image distance is greater than or equal to a preset distance threshold to obtain a connected graph.

[0180] Optionally, the connected graph consists of at least two graphs;

[0181] Accordingly, the first clustering module 706 is further configured as follows:

[0182] Cluster the images to be clustered in each of the at least two connected graphs according to density or probability clustering methods to obtain at least two initial image clustering results;

[0183] The image clustering result is determined based on the at least two initial image clustering results, wherein the image clustering result includes the clustered image set and the category of the clustered image set.

[0184] Optionally, the image determination module 702 is further configured to:

[0185] Obtain the video stream corresponding to the target project, and split the video stream into video frames. Determine the set of images to be clustered based on the split video frames.

[0186] Optionally, the image determination module 702 is further configured to:

[0187] Obtain the video stream corresponding to the target project, and perform video frame splitting on the video stream according to the machine learning model. Determine the set of images to be clustered based on the split video frames.

[0188] Optionally, the images to be clustered include vehicle images and / or pedestrian images.

[0189] The image clustering device provided in this specification utilizes the characteristic that the partitioning of connected graphs is basically stable under high recall vector retrieval or low recall vector retrieval. It uses a preset nearest neighbor algorithm to decompose the images to be clustered in the image clustering task into independent subgraphs, and performs image clustering based on the decomposed independent subgraphs. This significantly reduces memory and computing resource consumption and solves the problem that the clustering algorithm cannot run during the image clustering process due to the large number of images in large-scale clustering scenarios.

[0190] The above is a schematic scheme of an image clustering device according to this embodiment. It should be noted that the technical solution of this image clustering device and the technical solution of the image clustering method described above belong to the same concept. For details not described in detail in the technical solution of the image clustering device, please refer to the description of the technical solution of the image clustering method described above.

[0191] See Figure 8 , Figure 8 A flowchart of an image clustering method in a traffic scenario according to an embodiment of this specification is shown; the specific steps are as follows.

[0192] Step 802: Obtain the video stream in the traffic scene, and split the video stream into video frames. Determine the set of images to be clustered based on the split video frames, wherein the images to be clustered contain pedestrians and / or vehicles.

[0193] Step 804: Compress the images to be clustered in the set of images to be clustered according to the preset nearest neighbor algorithm, and determine the target image to be clustered and the image distances of other images to be clustered based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered.

[0194] Step 806: Connect the images to be clustered in the set of images to be clustered according to the image distance, and cluster the connected images according to a preset clustering method to obtain the image clustering result.

[0195] The specific details of the implementation process of the image clustering method in this traffic scenario can be found in the detailed description of the above embodiments, and will not be repeated here.

[0196] The image clustering method for traffic scenarios provided in this specification utilizes the characteristic that connected graph partitioning is basically stable under high-recall vector retrieval or low-recall vector retrieval. It uses a preset nearest neighbor algorithm to decompose the images to be clustered in the image clustering task into independent subgraphs, and performs image clustering based on the decomposed independent subgraphs. This significantly reduces memory and computing resource consumption and solves the problem that the clustering algorithm cannot run during the image clustering process in large-scale traffic image clustering scenarios due to the large number of images.

[0197] Corresponding to the above method embodiments, this specification also provides an embodiment of an image clustering device for traffic scenarios. Figure 9 This specification illustrates a schematic diagram of an image clustering device for a traffic scenario according to one embodiment. Figure 9 As shown, the device includes:

[0198] Image acquisition module 902 is configured to acquire video streams in traffic scenarios, split the video stream into video frames, and determine a set of images to be clustered based on the split video frames, wherein the images to be clustered contain pedestrians and / or vehicles.

[0199] The second calculation module 904 is configured to compress the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm, and determine the target image to be clustered and the image distances to other images to be clustered based on the compressed images to be clustered. The target image to be clustered is any image to be clustered in the image set to be clustered, and the other images to be clustered are the images to be clustered in the image set to be clustered other than the target image to be clustered.

[0200] The second clustering module 906 is configured to connect the images to be clustered in the set of images to be clustered according to the image distance, and to cluster the connected images according to a preset clustering method to obtain the image clustering result.

[0201] The image clustering device for traffic scenarios provided in this specification utilizes the characteristic that connected graph partitioning is basically stable under high or low recall vector retrieval. It uses a preset nearest neighbor algorithm to decompose the images to be clustered in the image clustering task into independent subgraphs, and performs image clustering based on the decomposed independent subgraphs. This significantly reduces memory and computing resource consumption and solves the problem that the clustering algorithm cannot run during the image clustering process in large-scale traffic image clustering scenarios due to the large number of images.

[0202] The above is an illustrative scheme of an image clustering device in a traffic scenario according to this embodiment. It should be noted that the technical solution of this image clustering device in a traffic scenario and the technical solution of the image clustering method in a traffic scenario described above belong to the same concept. Details not described in detail in the technical solution of the image clustering device in a traffic scenario can be found in the description of the technical solution of the image clustering method in a traffic scenario described above.

[0203] See Figure 10 , Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0204] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0205] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0206] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 1000 can also be a mobile or stationary server.

[0207] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described image clustering method.

[0208] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the image clustering method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the image clustering method described above.

[0209] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described image clustering method.

[0210] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the image clustering method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the image clustering method described above.

[0211] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described image clustering method.

[0212] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the image clustering method described above. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the image clustering method described above.

[0213] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0214] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0215] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0216] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0217] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image clustering method, comprising: Determine the set of images to be clustered; Compression of the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm includes: converting the images to be clustered in the image set to be clustered into feature vectors of a preset dimension, and dividing the feature vectors into at least two sub-feature vector sets; clustering the sub-feature vectors in each sub-feature vector set; and representing each feature sub-vector corresponding to the image to be clustered with its nearest cluster center and cluster identifier; the preset nearest neighbor algorithm includes: low recall K-nearest neighbor algorithm; The target image to be clustered and the image distance to other images to be clustered are determined based on the compressed images to be clustered and the low recall K-nearest neighbor algorithm. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered. Based on the image distance, connect the images in the image set to be clustered that have an image distance greater than a preset distance threshold to obtain several independent connected graphs. Then, cluster the connected images according to a preset clustering method to obtain image clustering results. Finally, merge the image clustering results corresponding to all connected graphs to obtain the image clustering results corresponding to all images in the image set to be clustered.

2. The image clustering method according to claim 1, wherein compressing the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm, and determining the target image to be clustered and the image distances to other images to be clustered based on the compressed images to be clustered, includes: The images to be clustered in the image set are converted into feature vectors of a preset dimension, and the feature vectors are divided into at least two sub-feature vector sets; Cluster the sub-feature vectors in each sub-feature vector set to obtain the cluster center of each sub-feature vector set; Based on the cluster center of each sub-feature vector set, the sub-feature vectors are compressed. Based on the compressed sub-feature vectors, the target image to be clustered and the image distance between the target image and other images to be clustered are determined.

3. The image clustering method according to claim 2, wherein the step of compressing the sub-feature vectors based on the cluster centers of each sub-feature vector set, and determining the target image to be clustered and the image distances to other images to be clustered based on the compressed sub-feature vectors, includes: Based on the cluster centers of each sub-feature vector set, determine the first cluster centers of at least two sub-feature vectors corresponding to the target image to be clustered, and the second cluster centers of at least two sub-feature vectors corresponding to other images to be clustered; Based on the first cluster identifier of the first cluster center and the second cluster identifier of the second cluster center, vector compression is performed on at least two sub-feature vectors corresponding to the target image to be clustered and the other images to be clustered. Based on the target image to be clustered after vector compression and at least two sub-feature vectors corresponding to the other images to be clustered, the image distance between the target image to be clustered and the other images to be clustered is determined.

4. The image clustering method according to claim 2, wherein the step of compressing the sub-feature vectors based on the cluster centers of each sub-feature vector set, and determining the target image to be clustered and the image distances to other images to be clustered based on the compressed sub-feature vectors, includes: Determine the feature vector of a preset dimension corresponding to the target image to be clustered, and determine the second cluster center of at least two sub-feature vector sets corresponding to other images to be clustered based on the cluster center of each sub-feature vector set; Based on the second clustering identifier of the second clustering center, vector compression is performed on at least two sub-feature vectors corresponding to the other images to be clustered. Based on the feature vector of the target image to be clustered with a preset dimension, and at least two sub-feature vectors corresponding to the other images to be clustered after vector compression, the image distance between the target image to be clustered and the other images to be clustered is determined.

5. The image clustering method according to claim 1, wherein compressing the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm, and determining the target image to be clustered and the image distances to other images to be clustered based on the compressed images to be clustered, includes: Cluster the images to be clustered in the set of images to be clustered to obtain the clustered set of images to be clustered and the cluster centers corresponding to the images to be clustered in the set of images to be clustered; Based on the images to be clustered in the image set to be clustered and the cluster centers corresponding to the images to be clustered in the image set to be clustered, determine the residual vector of the images to be clustered in the image set to be clustered; Determine the residual vector of the target image to be clustered and the residual vectors of other images to be clustered from the set of images to be clustered; Vector compression is performed on the residual vector of the target image to be clustered and the residual vectors of the other images to be clustered. Based on the vector-compressed residual vectors of the target image to be clustered and the residual vectors of the other images to be clustered, the image distance between the target image to be clustered and the other images to be clustered is determined.

6. The image clustering method according to claim 1, wherein connecting the images to be clustered in the set of images to be clustered based on the image distance, and clustering the connected images to be clustered according to a preset clustering method to obtain the image clustering result includes: Based on the image distance, the images in the set of images to be clustered are connected to obtain a connected graph; Cluster the images to be clustered in the connected graph using density or probability clustering methods to obtain image clustering results.

7. The image clustering method according to claim 6, wherein connecting the images to be clustered in the set of images to be clustered based on the image distance to obtain a connected graph includes: Starting with the target image to be clustered, connect the target image to be clustered and other images to be clustered whose image distance is greater than or equal to a preset distance threshold to obtain a connected graph.

8. The image clustering method according to claim 6, wherein the connected graph comprises at least two nodes; Accordingly, the step of clustering the images to be clustered in the connected graph according to a density or probability clustering method to obtain the image clustering results includes: Cluster the images to be clustered in each of the at least two connected graphs according to density or probability clustering methods to obtain at least two initial image clustering results; The image clustering result is determined based on the at least two initial image clustering results, wherein the image clustering result includes the clustered image set and the category of the clustered image set.

9. The image clustering method according to claim 1, wherein determining the set of images to be clustered includes: Obtain the video stream corresponding to the target project, and split the video stream into video frames. Determine the set of images to be clustered based on the split video frames.

10. The image clustering method according to claim 9, wherein obtaining the video stream corresponding to the target item, splitting the video stream into video frames, and determining the set of images to be clustered based on the split video frames includes: Obtain the video stream corresponding to the target project, and perform video frame splitting on the video stream according to the machine learning model. Determine the set of images to be clustered based on the split video frames.

11. An image clustering device, comprising: The image determination module is configured to determine the set of images to be clustered. The first calculation module is configured to compress the images to be clustered in the image set according to a preset nearest neighbor algorithm, including: converting the images to be clustered in the image set into feature vectors of a preset dimension, and dividing the feature vectors into at least two sub-feature vector sets; clustering the sub-feature vectors in each sub-feature vector set; and representing each feature sub-vector corresponding to the image to be clustered with its nearest cluster center and cluster identifier; the preset nearest neighbor algorithm includes: a low-recall K-nearest neighbor algorithm; The target image to be clustered and the image distance to other images to be clustered are determined based on the compressed images to be clustered and the low recall K-nearest neighbor algorithm. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered. The first clustering module is configured to connect images in the image set to be clustered that have a distance greater than a preset distance threshold based on the image distance, thereby obtaining several independent connected graphs. Then, it clusters the connected images according to a preset clustering method to obtain image clustering results. Finally, it merges the image clustering results corresponding to all connected graphs to obtain the image clustering results corresponding to all images in the image set to be clustered.

12. An image clustering method for traffic scenarios, comprising: Acquire a video stream in a traffic scene, and split the video stream into video frames. Determine a set of images to be clustered based on the split video frames, wherein the images to be clustered contain pedestrians and / or vehicles. Compression of the images to be clustered in the image set to be clustered according to a preset nearest neighbor algorithm includes: converting the images to be clustered in the image set to be clustered into feature vectors of a preset dimension, and dividing the feature vectors into at least two sub-feature vector sets; clustering the sub-feature vectors in each sub-feature vector set; and representing each feature sub-vector corresponding to the image to be clustered with its nearest cluster center and cluster identifier; the preset nearest neighbor algorithm includes: low recall K-nearest neighbor algorithm; The target image to be clustered and the image distance to other images to be clustered are determined based on the compressed images to be clustered and the low recall K-nearest neighbor algorithm. The target image to be clustered is any image to be clustered in the set of images to be clustered, and the other images to be clustered are the images to be clustered in the set of images to be clustered except for the target image to be clustered. Based on the image distance, connect the images in the image set to be clustered that have an image distance greater than a preset distance threshold to obtain several independent connected graphs. Then, cluster the connected images according to a preset clustering method to obtain image clustering results. Finally, merge the image clustering results corresponding to all connected graphs to obtain the image clustering results corresponding to all images in the image set to be clustered.

13. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the image clustering method according to any one of claims 1 to 10.

14. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the image clustering method according to any one of claims 1 to 10.

Images