A method and apparatus for image clustering

Abstract

The application provides a method and device for image clustering to solve the problem of one person with multiple files in the image clustering method. The method comprises: obtaining at least two image sets, each image in the image set containing a target type object, and the similarity of the target type object features of any two images in each image set exceeding a first threshold; determining at least one feature in each image set, the feature including a time feature and / or a space feature; wherein the time feature is used to describe the time information corresponding to the image, and the space information is used to describe the position information of the target type object contained in the image; determining a first image set and a second image set from the image set, wherein the features of the first image set and the features of the second image set satisfy a preset condition; and merging the first image set and the second image set.

Description

A method and apparatus for image clustering Technical Field

[0001] This application relates to the field of computer technology, specifically to a method and apparatus for image clustering. Background Technology

[0002] Current image clustering schemes primarily rely on the similarity of target features contained within an image to complete the clustering task. To ensure clustering accuracy, a relatively high similarity threshold is often set.

[0003] However, this situation can lead to difficulties in grouping images of the same target in different forms into the same set, resulting in a low overall clustering recall rate. For example, in the process of portrait clustering, images of the same person often appear in different sets due to differences such as wearing a mask or not wearing a mask, bright or dim lighting, or wearing a hat or not wearing a hat. This results in the so-called "one person, multiple files" problem.

[0004] Currently, there is an urgent need for a method to improve the accuracy of image clustering. Summary of the Invention

[0005] This application provides a method and apparatus for image clustering, which is used to improve the accuracy of image clustering.

[0006] In a first aspect, this application provides an image clustering method, which specifically includes: acquiring at least two image sets, each image in the image set containing a target type object, the similarity of the target type object features between any two images in each image set exceeding a first threshold, the target type object features being used to describe the target type object; determining at least one feature in each image set, the feature including a temporal feature and / or a spatial feature; wherein the temporal feature is used to describe the temporal information corresponding to the image, and the spatial information is used to describe the location information of the target type object contained in the image; determining a first image set and a second image set from the image sets, wherein the features of the first image set and the features of the second image set satisfy a preset condition; and merging the first image set and the second image set.

[0007] In this scheme, image sets with similar temporal and / or spatial features are identified and merged, thereby achieving clustering based on the spatiotemporal information of the target type objects contained in the image sets. This results in image sets being clustered not only based on the similarity of target type object features between images, but also on the spatiotemporal information of the target type objects, effectively utilizing more information carried by the images and improving the accuracy of image clustering. Furthermore, by using the similarity of temporal and spatial features, image sets without target type object features exceeding a first threshold can also be merged, avoiding the problem of images containing different target type objects being classified into different image sets due to an excessively high first threshold. This also increases the number of images contained in sets of the same target type object, improving image recall.

[0008] Optionally, the preset conditions include one or more of the following: the similarity between the temporal features of the first image set and the temporal features of the second image set exceeds a second threshold; the similarity between the spatial features of the first image set and the spatial features of the second image set exceeds a third threshold.

[0009] In this approach, the preset conditions satisfied by the first image set and the second image set include three possible scenarios, allowing the solution to be adjusted according to actual usage, thus providing flexibility.

[0010] Optionally, the feature further includes the target type object feature; the preset condition further includes: the similarity between the target type object feature of the first image set and the target type object feature of the second image set exceeds a fourth threshold; wherein the fourth threshold is less than the first threshold.

[0011] In this approach, a minimum similarity is set for the target type object features of an image set whose temporal and spatial similarity exceeds a second threshold and a third threshold, thereby further improving the accuracy of image clustering results.

[0012] Optionally, the method further includes: acquiring a plurality of sample images, each sample image containing the target type object, each sample image carrying an identity marker, the identity marker being used to indicate the identity information of the target type object contained in the corresponding sample image; clustering the plurality of sample images according to the target type object features to obtain at least two sample image sets, wherein the similarity of the target type object features of any two images in each sample image set exceeds a first threshold; merging sample image sets with the same identity marker in the sample image sets to obtain a processed image set; determining a second threshold according to the temporal features of the processed image set; and determining a third threshold according to the spatial features of the processed image set.

[0013] In this approach, the distribution of similarity between temporal and spatial features after one-stage clustering of sample images carrying identity tags is statistically analyzed to determine the specific values ​​of the second and third thresholds. This approach yields more accurate threshold results and improves the universality of the proposed solution.

[0014] Optionally, determining the second threshold based on the temporal features of the processed image set includes: determining a first statistical value corresponding to each processed image set, wherein the first statistical value is a statistical value of the similarity of the temporal features between each image in each processed image set; and determining the second threshold based on the normal distribution result of the first statistical value corresponding to each processed image set.

[0015] This method allows the server to determine the second, third, and fourth thresholds more accurately based on the sample images, further improving the accuracy and universality of the solution.

[0016] Optionally, determining the first image set and the second image set from the image set includes: determining the first image set from the image set, wherein the number of images in the first image set exceeds a fifth threshold; and determining at least one target image set from each of the other image sets in the image set that satisfies the preset condition with the first image set, wherein the target image set includes the second image set.

[0017] In this approach, the server can select an image set whose number of images exceeds a fifth threshold as a benchmark, and then select other image sets that meet preset conditions for merging. This reduces the computational load required in the image clustering process by using only a subset of sets as the standard, improving the efficiency of the method, while maintaining the accuracy of the results.

[0018] Optionally, each image in the image set carries a tracking identification number (ID); the method further includes: determining a third image set and a fourth image set from the image set, wherein all images in the third image set and the fourth image set carry the same tracking ID; and merging the third image set and the fourth image set.

[0019] In this approach, the server can merge the image set after the first-stage clustering based on the tracking ID, further improving the accuracy of image clustering.

[0020] Optionally, the similarity of the target type object features of any two images in the third image set and the fourth image set exceeds a sixth threshold, wherein the sixth threshold is less than the first threshold.

[0021] In this approach, when clustering images with the same tracking ID, the server can appropriately lower the similarity requirement for the features of their target object type. This allows for the clustering of images with different shapes even when the target object type exhibits different forms, further improving the accuracy of the clustering results.

[0022] Optionally, each of the sample images carries the tracking ID, and the method further includes: determining a first sample image set from the sample images, wherein any image in the first sample image set carries the same tracking ID and a different identity marker as at least one image in the first sample image set other than the arbitrary image; determining a first similarity of the target type object features between sample images with the same tracking ID in the first sample image set; and determining a sixth threshold based on the result of a normal distribution of the first similarity in the first sample image set.

[0023] In this approach, a sixth threshold is obtained by statistically analyzing sample images, which improves the accuracy and universality of the solution.

[0024] Secondly, an image clustering apparatus is provided, comprising: an acquisition module for acquiring at least two image sets, each image in the image set containing a target type object, wherein the similarity of the target type object features between any two images in each image set exceeds a first threshold, the target type object features being used to describe the target type object; a processing module for determining at least one feature in each image set, the feature including a temporal feature and / or a spatial feature; wherein the temporal feature is used to describe the temporal information corresponding to the image, and the spatial information is used to describe the location information of the target type object contained in the image; determining a first image set and a second image set from the image sets, wherein the features of the first image set and the features of the second image set satisfy a preset condition; and a merging module for merging the first image set and the second image set.

[0025] Optionally, the preset conditions include one or more of the following: the similarity between the temporal features of the first image set and the temporal features of the second image set exceeds a second threshold; the similarity between the spatial features of the first image set and the spatial features of the second image set exceeds a third threshold.

[0026] Optionally, the feature further includes the target type object feature; the preset condition further includes: the similarity between the target type object feature of the first image set and the target type object feature of the second image set exceeds a fourth threshold; wherein the fourth threshold is less than the first threshold.

[0027] Optionally, the processing module is further configured to acquire several sample images, each sample image containing the target type object, and each sample image carrying an identity marker, the identity marker indicating the identity information of the target type object contained in the corresponding sample image; the processing module is further configured to cluster the several sample images according to the target type object features to obtain at least two sample image sets, wherein the similarity of the target type object features of any two images in each sample image set exceeds the first threshold; merge sample image sets with the same identity marker in the sample image sets to obtain one or more processed image sets; determine the second threshold according to the temporal features of the processed image sets; and determine the third threshold according to the spatial features of the processed image sets.

[0028] Optionally, when the processing module determines the second threshold based on the temporal features of the processed image set, it is specifically configured to: determine a first statistical value corresponding to each processed image set, wherein the first statistical value is a statistical value of the similarity of the temporal features between each image in each processed image set; and determine the second threshold based on the normal distribution result of the first statistical value corresponding to each processed image set.

[0029] Optionally, when the processing module determines the first image set and the second image set from the image set, it is specifically configured to: determine the first image set from the image set, wherein the number of images in the first image set exceeds a fifth threshold; and determine at least one target image set from each of the other image sets in the image set that satisfies the preset condition with the first image set, wherein the target image set includes the second image set.

[0030] Optionally, each image in the image set carries a tracking identification number (ID); the processing module is further configured to determine a third image set and a fourth image set from the image set, wherein all images in the third image set and the fourth image set carry the same tracking ID; the merging module is further configured to merge the third image set and the fourth image set.

[0031] Optionally, the similarity of the target type object features of any two images in the third image set and the fourth image set exceeds a sixth threshold, wherein the sixth threshold is less than the first threshold.

[0032] Optionally, each of the sample images carries the tracking ID, and the processing module is further configured to determine a first sample image set from the sample images, wherein any image in the first sample image set carries the same tracking ID and a different identity marker as at least one image in the first sample image set other than the arbitrary image; determine a first similarity of the target type object features between sample images with the same tracking ID in the first sample image set; and determine a sixth threshold based on the result of the normal distribution of the first similarity in the first sample image set.

[0033] Thirdly, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, causes the device to perform the method described in the first aspect or any optional embodiment of the first aspect.

[0034] Fourthly, a computer-readable storage medium is provided for storing instructions that, when executed, cause a method as described in the first aspect or any alternative embodiment of the first aspect to be implemented.

[0035] The technical effects or advantages of one or more technical solutions provided in the second, third and fourth aspects of this application can all be explained by the corresponding technical effects or advantages of one or more technical solutions provided in the first aspect. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 is a schematic diagram of a possible application scenario provided by an embodiment of this application;

[0038] Figure 2 is a flowchart of an image clustering method provided in an embodiment of this application;

[0039] Figure 3 is a flowchart of the method for determining a threshold provided in an embodiment of this application;

[0040] Figure 4 is a schematic diagram of one or more processed image sets provided in this application;

[0041] Figure 5 is a schematic diagram of an image clustering device provided in an embodiment of this application;

[0042] Figure 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0044] It should be understood that in the description of the embodiments of this application, terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order. In the description of the embodiments of this application, "multiple" refers to two or more.

[0045] The term "and / or" in the embodiments of this application is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0046] Current image clustering methods rely solely on the similarity of features of the objects contained within an image to complete the clustering task. For example, methods for clustering face images are based on the similarity of features such as faces and bodies. To ensure the accuracy of the clustering results, these methods typically set a high similarity threshold. This leads to the difficulty of clustering images with different appearances of the same object (e.g., the difference between the same person wearing a hat and not wearing a hat, or wearing a mask and not wearing a mask) into the same image set, resulting in a low overall clustering recall rate.

[0047] In view of this, this application provides an image clustering method and apparatus to solve the problem. This method can be applied to semi-enclosed scenarios such as supermarkets, shopping malls, sales offices, parking lots, and zoos, utilizing information beyond the physical characteristics of the target object, such as spatiotemporal information, to assist in clustering, thereby improving the accuracy of clustering.

[0048] Referring to Figure 1, which is a schematic diagram of a possible application scenario provided by an embodiment of this application, the scenario shows an aisle between shelves in a small supermarket. In this aisle, two people, A and B, are walking towards each other, and a camera a is recording the scene above the shelf. It should be understood that the scenario shown in Figure 1 is only one of the possible scenarios in which this method is applicable, and this application does not limit the specific application scenarios of this method in real life.

[0049] Referring to Figure 2, which is a flowchart of an image clustering method provided in an embodiment of this application, this method can be executed by any device with computing capabilities. For example, the executing entity of this method can be a server device capable of acquiring images, or a terminal device that directly acquires images and has computing functions, such as a mobile phone, tablet, laptop, or camera. For ease of description, the following description uses a server as the executing entity of this method. The specific steps of this method are as follows:

[0050] Step S201: The server obtains at least two image sets, each of which contains a target type object, and the similarity of the target type object features between any two images in these sets exceeds a first threshold. The target type object features are used to describe the target type object.

[0051] The target type objects include different types of objects, such as people, vehicles, or animals. When the target type objects contained in the above images are different, the target type object features during clustering will also be different. For example, when the target type object is a person, its features include facial or body features; when the target type object is a vehicle, its features include license plate features. For ease of explanation, this paper uses people as an example; in real life, this application does not limit the specific types of target type objects.

[0052] It should be noted that the specific value of the first threshold can be set according to actual needs, and this application does not impose any restrictions. For example, when it is necessary to ensure that each of the at least two image sets contains objects of the same target type, a higher first threshold can be set.

[0053] Optionally, the at least two image sets obtained by the server can be image sets that have been initially clustered based on the similarity of the target type object features, or they can be obtained by the server after obtaining several images containing target type objects and then performing clustering operations based on the target type object features. This application does not impose any restrictions.

[0054] For example, when the server needs to obtain the aforementioned at least two image sets, it can directly obtain the at least two image sets after one-stage clustering from the terminal device that captured the images or other servers that communicate with the terminal device. The criterion for this one-stage clustering is that the similarity of facial features exceeds a threshold: as long as the similarity of facial features contained in any two images exceeds a first threshold, the two images can be grouped into the same image set. Alternatively, when the server needs to obtain the aforementioned at least two image sets, it can obtain several images containing people from the terminal that captured the images or other servers that communicate with the terminal, and then the server itself performs one-stage clustering on these images to further obtain the aforementioned at least two image sets after one-stage clustering.

[0055] Step S202: The server determines at least one feature for each of the above at least two image sets, the feature including temporal features and / or spatial features.

[0056] Among them, the time feature is used to describe the time information corresponding to the image. For example, the time when the image was taken can be used as the time feature corresponding to the image.

[0057] Spatial features are used to describe the location information of target-type objects contained in an image. Optionally, for the location information of a target-type object, the server can use an item with a fixed position in the image as a reference to indicate the location information of the target-type object. Taking the scenario shown in Figure 1 as an example, after the server acquires the image, it can represent the location information of person A based on the relative distance between shelf 1 and person A in the image, thereby further determining the spatial features of the image. Alternatively, the terminal device for acquiring the image can be set to a binocular or higher camera or a time-of-flight (ToF) camera. These types of cameras can directly establish a coordinate system in the target area, thereby directly acquiring the location coordinate information of the target object when capturing the image. When determining the spatial features of the image, the server can determine the corresponding spatial features of the image based on the coordinate information carried in the images captured by these cameras.

[0058] The temporal and spatial features described above are used to describe the specific temporal and spatial information of an image. For a set of images, the temporal and spatial features can be the sum of the temporal or spatial information of all images in the set, or they can be statistical values ​​of the temporal or spatial information of all images in the set. The data format of at least one specific feature can be set according to actual needs, and there are no restrictions on the application.

[0059] Step S203: The server determines a first image set and a second image set from at least two image sets, wherein the features of the first image set and the features of the second image set satisfy a preset condition.

[0060] Optionally, the above preset conditions include one or more of the following: 1. The similarity between the temporal features of the first image set and the temporal features of the second image set exceeds a second threshold; 2. The similarity between the spatial features of the first image set and the spatial features of the second image set exceeds a third threshold.

[0061] For example, depending on actual needs, preset conditions can include at least the following three types:

[0062] Type 1: The first image set and the second image set only need to have a temporal feature similarity exceeding the second threshold; Type 2: The first image set and the second image set only need to have a spatial feature similarity exceeding the third threshold; Type 3: The first image set and the second image set need to have both a temporal feature similarity exceeding the second threshold and a spatial feature similarity exceeding the third threshold.

[0063] In this approach, the preset conditions satisfied by the first image set and the second image set include three possible scenarios, allowing the solution to be adjusted according to actual usage, thus providing flexibility.

[0064] Step S204: The server merges the first image set and the second image set determined above.

[0065] In this scheme, image sets with similar temporal and / or spatial features are identified and merged, thereby achieving clustering based on the spatiotemporal information of the target type objects contained in the image sets. This results in image sets being clustered not only based on the similarity of target type object features between images, but also on the spatiotemporal information of the target type objects, effectively utilizing more information carried by the images and improving the accuracy of image clustering. Furthermore, by using the similarity of temporal and spatial features, image sets without target type object features exceeding a first threshold can also be merged, avoiding the problem of images containing different target type objects being classified into different image sets due to an excessively high first threshold. This also increases the number of images contained in sets of the same target type object, improving image recall.

[0066] It should be noted that the above description of this solution only uses the first image set and the second image set as examples. However, in practical applications, there may be more than two sets that meet the preset conditions. Therefore, when merging, all sets that meet the preset conditions can be merged at once to obtain a new merged image set.

[0067] Optionally, in addition to the aforementioned temporal and spatial features, at least one feature may also include target type object features, and the preset conditions may also include: the similarity of the target type object features between the first image set and the second image set exceeds a fourth threshold, and the fourth threshold is less than the first threshold.

[0068] For example, in addition to the aforementioned limitations on temporal and spatial features, the preset conditions may also include limitations on target type object features. For instance, after determining the image sets whose temporal feature similarity exceeds the second threshold and whose spatial feature similarity exceeds the third threshold based on the temporal and spatial features of the image sets, it is then determined whether the similarity of their target type object features exceeds the fourth threshold. If it is determined that they exceed the fourth threshold, these sets can be merged.

[0069] In this approach, a minimum similarity is set for the target type object features of an image set whose temporal and spatial similarity exceeds a second threshold and a third threshold, thereby further improving the accuracy of image clustering results.

[0070] The above describes the specific clustering steps in the image clustering method provided in this application. The determination of the second, third, and fourth thresholds can be made by the user according to different situations, or by the server after statistical analysis of several sample images.

[0071] Referring to Figure 3, which is a flowchart of a method for determining a threshold provided in an embodiment of this application, the specific steps of the method for determining the threshold are as follows:

[0072] Step S301: The server obtains several sample images, each of which contains a target type object, and each sample image carries an identity tag, which is used to indicate the identity information of the target type object contained in each sample image.

[0073] Optionally, the identity markers included in the sample images can be specifically annotated by the sample image provider. In this way, after the clustering operation is completed, the accuracy of the clustering results can be directly determined based on the identity markers carried by each sample image.

[0074] Step S302: The server clusters the above sample images according to the target type object features to obtain at least two sample image sets, wherein the similarity of the target type object features of any two sample images in each of these sample image sets exceeds a first threshold.

[0075] In this step, the server performs one-stage clustering on these sample images based solely on the target type object features between the images, obtaining at least two clustered sample image sets. Furthermore, since the first threshold is the same as the first threshold in step S201, there may be a problem in the sample image sets after clustering in step S302 where images of the same target type object are assigned to different sample image sets because the similarity of the target type objects required by the first threshold is too high. To address this problem, the following steps are performed.

[0076] Step S303: The server merges the sample image sets corresponding to the same identity tag in the above at least two sample image sets to obtain one or more processed image sets.

[0077] Based on the identity markers carried by the sample images, the server can merge all image sets with the same identity markers to obtain a complete and accurate clustered image set. Furthermore, the problem of objects containing the same target type being classified into different image sets (i.e., "one person, multiple files") will not occur within the aforementioned processed image sets.

[0078] Step S304: The server determines a second threshold based on the temporal characteristics of the above-mentioned image set and a third threshold based on its spatial characteristics.

[0079] In this approach, the distribution of similarity between temporal and spatial features after one-stage clustering of sample images carrying identity tags is statistically analyzed to determine the specific values ​​of the second and third thresholds. This approach yields more accurate threshold results and improves the universality of the proposed solution.

[0080] Optionally, the determination of the second, third, and fourth thresholds can be accomplished using the following methods.

[0081] 1. Determination of the second threshold

[0082] First, the server can determine a first statistical value for each image set in the above-mentioned image processing set. This first statistical value is a statistical value of the similarity of temporal features between images in each sample image set.

[0083] It should be noted that after merging at least two sample image sets in step 303, each sample image in the resulting merged set of one or more processed images still carries the set information of the original at least two sample image sets. Referring to Figure 4, which is a schematic diagram of one or more processed image sets provided in this application, set 1 and set 2 are two sets in the merged set of one or more processed images. These two sets also contain six subsets: set a, set b, set c, set d, set e, and set f. These six subsets are sets from the aforementioned at least two sample image sets. In other words, even after obtaining the aforementioned merged set of one or more processed images, the server can still determine the set information to which any image in the set belonged before merging.

[0084] Furthermore, the determination of the aforementioned first statistical value can be achieved in different ways. For ease of understanding, these methods will be explained below using sets 1 and 2 shown in Figure 4 as examples.

[0085] Method 1: The server can calculate the temporal feature similarity of each subset in set 1 and set 2 respectively, and finally calculate the first statistical value corresponding to set 1 and set 2.

[0086] For example, the server calculates the similarity between images in each of the three sets a, b, and c, and then calculates the first statistical value for set 1 and the first statistical value for set 2 based on these three sets.

[0087] Method 2: The server can calculate the temporal feature similarity between any two images in set 1 and set 2 respectively, and then determine the first statistical value corresponding to set 1 and set 2 based on these similarities.

[0088] Secondly, the server determines the second threshold based on the normal distribution of the first statistical values ​​corresponding to each sample image set in one or more processed image sets obtained above.

[0089] For example, the server can use mathematical tools to fit the first statistical value corresponding to each set of sample images to obtain its normal distribution result, and determine the second threshold according to the 3σ principle of normal distribution.

[0090] The 3σ principle, also known as the Raida criterion, means that a set of test data is first assumed to contain only random errors. The standard deviation is calculated and processed to obtain the standard deviation. An interval is determined according to a certain probability. It is believed that any error exceeding this interval is not a random error but a gross error, and data containing such errors should be discarded.

[0091] In this application, the normal distribution result of the first statistical value is used to determine the possible range of similarity, and then the second threshold is determined.

[0092] 2. Determination of the third and fourth thresholds

[0093] The determination of the third and fourth thresholds can be done by referring to the method described above for determining the second threshold. Simply change the statistical content to the second statistical value corresponding to the spatial features of each sample image set and the third statistical value corresponding to the target type object features of each sample image set. The rest can be done using the same method, and will not be elaborated further here.

[0094] Using the above method, the server can determine the second, third, and fourth thresholds more accurately based on the sample images, further improving the accuracy and universality of this solution.

[0095] Optionally, when performing step S203 above to determine the first image set and the second image set from at least two image sets, the specific steps include:

[0096] First, a first image set is determined from at least two image sets, wherein the number of images in the first image set exceeds a fifth threshold;

[0097] Next, the other image sets in at least two image sets, excluding the first image set, are traversed to determine at least one target image set that satisfies the preset conditions of the first image set, and the at least one image set includes the second image set.

[0098] In this way, the server can sequentially merge the first image set with the at least one target image set to obtain the image clustering result.

[0099] For example, suppose at least two image sets contain a large number of subsets. In this case, directly determining the first and second image sets from these at least two image sets would require traversing all subsets within the at least two image sets, resulting in excessive computation and overloading the server. Therefore, a fifth threshold can be set. The server first determines the first image set whose number of images exceeds the fifth threshold, and then compares the temporal characteristics of the first image set with the temporal characteristics of other image sets to obtain the second threshold for merging.

[0100] In this approach, the server can select an image set whose number of images exceeds a fifth threshold as a benchmark, and then select other image sets that meet preset conditions for merging. This reduces the computational load required in the image clustering process by using only a subset of sets as the standard, improving the efficiency of the method, while maintaining the accuracy of the results.

[0101] It should be noted that in actual use, the fifth threshold can be set by the user based on the number of images in the entire image set, or it can be calculated by the server after training with sample images. This application does not impose any restrictions.

[0102] Optionally, the calculation of the specific value of the fifth threshold can be accomplished using the following method.

[0103] The first step is to acquire several sample images, each of which carries an identity marker that can identify the identity information of the target type object contained in each sample image.

[0104] It should be noted that the sample image in this method can be the same as the sample image in step S301 above, or it can be a sample image obtained from other different sources; this application does not impose any restrictions. For ease of explanation, the following will use the example of the sample image being the same as that in step S301 above.

[0105] The second step involves clustering the sample images based on the similarity of their target type object features, resulting in at least two sets of sample images. The specific clustering process is described in step S302 and will not be repeated here.

[0106] The third step is to count the number of images in the set of sample images corresponding to the same identity tag. For example, suppose that sample image sets 1, 2, 3, and 4 are clustered image sets where all images correspond to the same identity tag. In this case, record the number of images in the set with the largest number of images in sample image sets 1, 2, 3, and 4. Then, record the number of images in the set with the largest number of images in the image sets corresponding to other identity tags.

[0107] In this way, based on the number of images, it can be determined whether a certain image set can be identified as the main file corresponding to the target type object. Then, based on each main file, other image sets are traversed, and the next stage of image merging is carried out.

[0108] The fourth step is to use mathematical tools to fit the number of images in the main archives corresponding to each identity mark to obtain its normal distribution result, and then determine the fifth threshold according to the 3σ principle of normal distribution.

[0109] The specific method for determining the fifth threshold is similar to that for determining the second threshold, and will not be repeated here.

[0110] It should be noted that the fifth threshold can be a simple numerical value. For example, a fifth threshold of 500 means that an image set containing more than 500 images can be considered a primary archival image set. Alternatively, the fifth threshold can be a ratio. For example, a fifth threshold of 60% means that an image set containing more than 60% of all images to be clustered can be considered a primary archival image set. In practical applications, the specific form of the fifth threshold can be determined according to requirements, and this application does not impose any restrictions.

[0111] In this method, the fifth threshold is determined by the clustering results of sample images, which has high accuracy and high universality.

[0112] Optionally, in addition to the identity marker, the images containing the target type object may also carry a tracking identity document (ID). The tracking ID is a marker that the terminal device marks for the target type object in each frame of the image in real time when the target type object is captured, so that it can better track and identify the target object.

[0113] After the images carrying the tracking ID have completed the first stage of clustering, the server can determine the third and fourth image sets carrying the same tracking ID from at least two clustered image sets, and then merge the third and fourth image sets.

[0114] In this approach, the server can merge the image set after the first-stage clustering based on the tracking ID, further improving the accuracy of image clustering.

[0115] Optionally, for the determined third and fourth image sets, the following restriction also applies: the similarity of the target type object features of any two images in the third and fourth image sets exceeds a sixth threshold, wherein the sixth threshold is less than the first threshold.

[0116] In other words, when clustering images with the same tracking ID, the server can appropriately lower the similarity requirement for the features of their target object type. This way, even if the target object type changes form (e.g., a person in a portrait changes from wearing a mask to not wearing one), clustering can still be achieved for both forms, further improving the accuracy of the clustering results.

[0117] Optionally, the following methods can be used to determine the sixth threshold:

[0118] First, several sample images carrying tracking IDs and identity tags are acquired. These sample images can be sample images determined by the other thresholds mentioned above, or they can be other different sample images. This application does not impose any restrictions.

[0119] Secondly, a first set of sample images is obtained from the above sample images, wherein any image in the set corresponds to the same tracking ID and a different identity tag as at least one other image in the set.

[0120] The images in the first sample image set are sample images with incorrect tracking ID labeling, that is, target objects with different identities are labeled with the same tracking ID for various possible reasons. At this time, the server can count the similarity between these images with the same tracking ID, so as to provide a basis for how to handle images with the same tracking ID when performing subsequent image clustering.

[0121] Next, the server determines the first similarity of target type object features between sample images with the same tracking ID in the first sample image set, and determines the sixth threshold based on the normal distribution result of the first similarity.

[0122] The sixth threshold determined by the above method represents the maximum similarity when images with different identity tags are assigned to the same tracking ID. That is, when the similarity of target type object features between two images with the same tracking ID is lower than the maximum similarity, the two images are highly likely not to correspond to the same target object. Conversely, when the similarity of target type object features between two images with the same tracking ID is greater than the maximum similarity, the two are very likely to correspond to the same target object.

[0123] In this approach, a sixth threshold is obtained by statistically analyzing sample images, which improves the accuracy and universality of the solution.

[0124] The above describes the clustering scheme performed on high-quality images containing valid information during image clustering. However, in practical applications, the images acquired by terminal devices always include some low-quality images (such as a side view of a target object in a portrait, or a vehicle whose photographic position is mostly obscured). These images do not significantly affect the clustering results, but they result in a considerable number of different image sets. This type of image data can generally be called "waste images," and the image sets containing only these "waste images" are the erroneous data that need to be removed during the image clustering process. The following methods can be used to remove such image sets or "waste images."

[0125] Optionally, for the at least two image sets obtained in step S201, some "dead" images will inevitably exist in these sets during the first-stage clustering process. Therefore, based on some judgment criteria, image sets containing the aforementioned "dead" images can be filtered out from these at least two image sets and removed in a timely manner to facilitate further image clustering; alternatively, the "dead" images can be removed after the merging of the at least two image sets is completed, so that users can obtain clearer and more understandable image clustering results.

[0126] The above judgment criteria include at least one or more of the following: 1. The number of images in the set is less than the seventh threshold; 2. The average portrait quality score of the images in the set is less than the eighth threshold.

[0127] For example, the seventh and eighth thresholds mentioned above can be determined in the following manner.

[0128] Method 1: Users set the specific values ​​or percentages of the seventh and eighth thresholds based on actual usage. For example, when the acquired image set is small, users can set the seventh threshold to 2 and the eighth threshold to 20. That is, when the number of images in the image set is less than or equal to 2 and the average quality score of the images in the set is less than 20, it can be determined that it is "waste" data that needs to be deleted.

[0129] Method 2: Determine the seventh and eighth thresholds based on the data in the sample images. The specific method is similar to the method used to determine the fifth and sixth thresholds described above. The difference is that in determining the seventh and eighth thresholds, the sample images carry artificially labeled discarded images. After performing a one-stage clustering on these sample images, the number of images in the image set containing these discarded images and their average quality scores are counted. The seventh and eighth thresholds are then determined based on their normal distribution.

[0130] In this method, the images that need to be clustered can be cleaned of unusable images, making the clustering results more concise and clear, reducing the amount of computation required for clustering, and improving the clustering efficiency.

[0131] Based on the same inventive concept, embodiments of this application also provide an image clustering device.

[0132] Referring to Figure 5, an image clustering device is provided in an embodiment of this application. The device may be the server mentioned above or a chip or integrated circuit in the device. The device includes modules / units / technical means for executing the method executed by the server in the above method embodiment.

[0133] For example, the device 500 includes:

[0134] The acquisition module 501 is used to acquire at least two image sets, each image in the at least two image sets contains a target type object, and the similarity of the target type object features of any two images in each of the at least two image sets exceeds a first threshold, wherein the target type object features are used to describe the target type object;

[0135] Processing module 502 is configured to determine at least one feature of each of the at least two image sets, the at least one feature including a temporal feature and / or a spatial feature; wherein the temporal feature is used to describe the temporal information corresponding to the image, and the spatial information is used to describe the location information of the target type object contained in the image; and to determine a first image set and a second image set from the at least two image sets, wherein the at least one feature of the first image set and the at least one feature of the second image set satisfy a preset condition;

[0136] The merging module 503 is used to merge the first image set and the second image set.

[0137] As an example, the device described in FIG5 can be used to perform the method described in the embodiment shown in FIG2. Therefore, the functions that each functional module of the device can achieve can be referred to the description of the embodiment shown in FIG2, and will not be repeated here.

[0138] It should be noted that although several modules or sub-modules of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units described above can be embodied in a single module. Conversely, the features and functions of a module described above can be further divided and embodied by multiple modules.

[0139] As one possible product form of the above-mentioned device, referring to Figure 6, this application embodiment also provides an electronic device 600, including:

[0140] At least one processor 601; and a communication interface 603 communicatively connected to the at least one processor 601; the at least one processor 601 causes the electronic device 600 to perform the method steps performed by any device in the above method embodiments through the communication interface 603 by executing instructions stored in the memory 602.

[0141] Optionally, the memory 602 is located outside the electronic device 600.

[0142] Optionally, the electronic device 600 includes the memory 602, which is connected to the at least one processor 601. The memory 602 stores instructions that can be executed by the at least one processor 601. Figure 6 shows, with dashed lines, that the memory 602 is optional for the electronic device 600.

[0143] The processor 601 and the memory 602 can be coupled through an interface circuit or integrated together; no restriction is imposed here.

[0144] This application embodiment does not limit the specific connection medium between the processor 601, memory 602, and communication interface 603. In Figure 6, the processor 601, memory 602, and communication interface 603 are connected via a bus 604, indicated by a thick line. The connection methods between other components are merely illustrative and not intended to be limiting. The bus can be an address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 6, but this does not imply that there is only one bus or one type of bus. It should be understood that the processor mentioned in this application embodiment can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0145] For example, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0146] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAM (DR RAM).

[0147] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0148] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0149] As another possible product form, this application embodiment also provides a computer-readable storage medium for storing instructions that, when executed, cause a computer to perform the method steps performed by any of the devices in the above method examples.

[0150] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0154] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image clustering method, characterized in that, include: Acquire at least two image sets, each image in the image set containing a target type object, wherein the similarity of the target type object features between any two images in each image set exceeds a first threshold, and the target type object features are used to describe the target type object; determine at least one feature in each image set, the feature including temporal features and / or spatial features; wherein the temporal features are used to describe the time information corresponding to the image, and the spatial features are used to describe the location information of the target type object contained in the image; determine a first image set and a second image set from the image sets, wherein the features of the first image set and the features of the second image set satisfy a preset condition; merge the first image set and the second image set; wherein, in acquiring Before obtaining at least two image sets, the method further includes: acquiring a plurality of sample images, each sample image containing the target type object, each sample image carrying an identity marker, the identity marker indicating the identity information of the target type object contained in the corresponding sample image; clustering the plurality of sample images according to the target type object features to obtain at least two sample image sets, wherein the similarity of the target type object features of any two images in each sample image set exceeds a first threshold; merging sample image sets with the same identity marker in the sample image sets to obtain one or more processed image sets; determining a second threshold according to the temporal features of the processed image sets; and determining a third threshold according to the spatial features of the processed image sets.

2. The method as described in claim 1, characterized in that, The preset conditions include one or more of the following: the similarity between the temporal features of the first image set and the temporal features of the second image set exceeds a second threshold; the similarity between the spatial features of the first image set and the spatial features of the second image set exceeds a third threshold.

3. The method as described in claim 1 or 2, characterized in that, The feature also includes the target type object feature; the preset condition further includes: the similarity between the target type object feature of the first image set and the target type object feature of the second image set exceeds a fourth threshold, wherein the fourth threshold is less than the first threshold.

4. The method as described in claim 1, characterized in that, Determining the second threshold based on the temporal features of the processed image set includes: determining a first statistical value corresponding to each processed image set, wherein the first statistical value is a statistical value of the similarity of the temporal features between each image in each processed image set; and determining the second threshold based on the normal distribution result of the first statistical value corresponding to each processed image set.

5. The method as described in claim 1, characterized in that, Determining the first image set and the second image set from the image set includes: determining the first image set from the image set, wherein the number of images in the first image set exceeds a fifth threshold; and determining at least one target image set from each of the other image sets in the image set that satisfies the preset condition with the first image set, wherein the target image set includes the second image set.

6. The method as described in claim 1, characterized in that, Each image in the image set carries a tracking identification number (ID); the method further includes: determining a third image set and a fourth image set from the image set, wherein all images in the third image set and the fourth image set carry the same tracking ID; and merging the third image set and the fourth image set.

7. The method as described in claim 6, characterized in that, The similarity of the target type object features of any two images in the third image set and the fourth image set exceeds a sixth threshold, where the sixth threshold is less than the first threshold.

8. The method as described in claim 7, characterized in that, Each of the sample images carries the tracking ID, and the method further includes: determining a first sample image set from the sample images, wherein any image in the first sample image set carries the same tracking ID and a different identity marker as at least one image in the first sample image set other than the arbitrary image; determining a first similarity of the target type object features between sample images with the same tracking ID in the first sample image set; and determining a sixth threshold based on the result of a normal distribution of the first similarity in the first sample image set.

9. An image clustering apparatus, characterized in that, include: An acquisition module is configured to acquire at least two image sets, each image in the image set containing a target type object, wherein the similarity of the target type object features between any two images in each image set exceeds a first threshold, and the target type object features are used to describe the target type object; a processing module is configured to determine at least one feature in each image set, the feature including a temporal feature and / or a spatial feature; wherein the temporal feature is used to describe the time information corresponding to the image, and the spatial feature is used to describe the location information of the target type object contained in the image; from the image sets, a first image set and a second image set are determined, wherein the features of the first image set and the features of the second image set satisfy a preset condition; a merging module is configured to merge the first image set and the second image set. The system includes a set of images and a second set of images; a training module for acquiring several sample images, each sample image containing the target type object, and each sample image carrying an identity marker indicating the identity information of the target type object contained in the corresponding sample image; clustering the several sample images according to the target type object features to obtain at least two sample image sets, wherein the similarity of the target type object features of any two images in each sample image set exceeds a first threshold; merging sample image sets with the same identity marker to obtain one or more processed image sets; determining a second threshold according to the temporal features of the processed image sets; and determining a third threshold according to the spatial features of the processed image sets.

10. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor performs the method as described in any one of claims 1-8 by executing the instructions stored in the memory.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause the method as described in any one of claims 1-8 to be implemented.

Images