An image clustering method, device, terminal and computer readable storage medium

Abstract

The application provides an image clustering method, device, terminal and computer readable storage medium. In the image clustering method, a to-be-clustered image and a historical image set are obtained; the body image set is divided into at least two sub-image sets based on the shooting angle of the target in the body image; in response to the first similarity between the to-be-clustered body image and the sub-image set exceeding a first similarity threshold, it is determined that the to-be-clustered body image is attributed to the body image set to obtain an updated body image set; in response to the second similarity between the to-be-clustered face image and the face image set exceeding a second similarity threshold, it is determined that the to-be-clustered face image is attributed to the face image set to obtain an updated face image set; and based on the association relationship between the to-be-clustered face image and the to-be-clustered body image, the updated image sets to which the to-be-clustered face image and the to-be-clustered body image belong are merged. The application improves the recall rate of image clustering while ensuring the accuracy of image clustering.

Description

Technical Field

[0001] This invention relates to the field of image clustering technology, and in particular to an image clustering method, apparatus, terminal, and computer-readable storage medium. Background Technology

[0002] With the rapid development of technology, surveillance cameras are ubiquitous in cities, and facial recognition data has become an important tool for solving crimes. Based on the facial trajectories in these data files, it's possible to reconstruct the trajectories of all individuals within a given area, enabling surveillance of key regions. Most existing facial clustering technologies are based on deep learning, extracting body feature vectors from images and calculating similarity between images based on these vectors. However, due to variations in capture angles and image clarity across different scenarios, there are instances where the similarity between a person's image and historical images in the data file is low. Current strategies often rely on calculating the average centroid of the data file based on the features of all images, then calculating similarity between this average centroid and the images to be clustered, resulting in low recall rates. Summary of the Invention

[0003] The main technical problem solved by this invention is to provide an image clustering method, device, terminal and computer-readable storage medium to solve the problem of low recall rate in the prior art of human image clustering.

[0004] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is: to provide an image clustering method, the image clustering method comprising:

[0005] Obtain the images to be clustered and the historical image set. The images to be clustered include face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target.

[0006] Based on the shooting angle of the target in the body image, the body image set is divided into at least two sub-image sets;

[0007] In response to the first similarity between the body image to be clustered and the sub-image set exceeding the first similarity threshold, it is determined that the body image to be clustered belongs to the body image set to obtain an updated body image set;

[0008] In response to the second similarity between the face image to be clustered and the face image set exceeding the second similarity threshold, it is determined that the face image to be clustered belongs to the face image set to obtain the updated face image set;

[0009] Based on the association between the face images to be clustered and the body images to be clustered, the updated face image set to which the face images to be clustered belong and the updated body image set to which the body images to be clustered belong are merged.

[0010] Before the step of merging the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs, based on the association between the face image to be clustered and the body image to be clustered, the method further includes:

[0011] Based on the face images in the updated face image set and the face images to be clustered, determine the updated face centroids in the updated face image set;

[0012] If the third similarity between a face image or a face image to be clustered and the updated face centroid is less than the third similarity threshold, then the face image or face image to be clustered corresponding to the third similarity is removed.

[0013] Based on the body images in the updated body image set and the body images to be clustered, determine the updated body centroid of the updated body image set;

[0014] If the fourth similarity between the body image or the body image to be clustered and the updated body centroid is less than the fourth similarity threshold, then the body image or the body image to be clustered corresponding to the fourth similarity is removed.

[0015] This involves acquiring images to be clustered and historical image sets. Images to be clustered include face images and body images of the same target object. Historical image sets include multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target.

[0016] Obtain multiple human images to be clustered, where each human image contains the target object;

[0017] The human images to be clustered are subjected to body and face detection and extraction to obtain the face images and body images to be clustered that have a relationship.

[0018] Based on the shooting angle of the target in the body images, the body image set is divided into at least two sub-image sets, including:

[0019] Based on the shooting angle of the target in the body image, the body images in the body image set are divided into three sub-image sets, and the shooting angle of the target in each sub-image set is within the corresponding preset angle range; among them, the three sub-image sets include the front body sub-image set, the side body sub-image set, and the back body sub-image set.

[0020] Specifically, in response to a first similarity threshold between the body image to be clustered and the sub-image set exceeding a first similarity threshold, determining to assign the body image to the body image set to obtain an updated body image set includes:

[0021] Feature extraction is performed on the body image to be clustered and the body image respectively to obtain the body feature information corresponding to the body image to be clustered and the body image respectively;

[0022] Based on the body feature information of the body images contained in each sub-image set, determine the body centroid corresponding to each sub-image set;

[0023] The body feature information of the body image to be clustered is compared with the first similarity and the first similarity threshold between the body centroids of each sub-image set and the body feature information of the body image to be clustered.

[0024] If the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to at least one sub-image set is not less than the first similarity threshold, then it is determined that the body image to be clustered is assigned to the body image set to obtain an updated body image set.

[0025] Specifically, based on the body feature information of the body images contained in each sub-image set, the body centroid corresponding to each sub-image set is determined, including:

[0026] Based on the body feature information of each body image in the sub-image set, the similarity between each body image is calculated.

[0027] The body centroid of the sub-image set is determined based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image.

[0028] Specifically, based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image, the body centroid of the sub-image set is determined, including:

[0029] The similarity between body images is compared with a similarity threshold;

[0030] If the similarity is less than the similarity threshold, the similarity corresponding to the body image is deleted.

[0031] If the similarity is not less than the similarity threshold, the similarity of the body image is retained.

[0032] The body centroid of the sub-image set is determined by summing the products between the body feature information of each body image and the similarity retained by the corresponding body image.

[0033] Specifically, in response to the second similarity between the face image to be clustered and the face image set exceeding a second similarity threshold, determining to assign the face image to be clustered to the face image set to obtain an updated face image set includes:

[0034] Feature extraction is performed on the face image and the face image to be clustered separately to obtain the face feature information corresponding to the face image and the face image to be clustered respectively;

[0035] Based on the facial feature information of the facial images in the face image set, determine the centroid of the face corresponding to the face image set;

[0036] The second similarity between the facial feature information of the face image to be clustered and the face centroid of the face image set is compared with the second similarity threshold.

[0037] If the second similarity between the facial feature information of the face image to be clustered and the centroid of the face image set is not less than the second similarity threshold, then the face image to be clustered is determined to belong to the face image set to obtain the updated face image set.

[0038] Among them, determining the face centroid corresponding to the face image set based on the facial feature information of the face images in the face image set includes:

[0039] In response to the presence of a face image in the face image set, the facial feature information of the face image is determined as the face centroid of the face image set.

[0040] Among them, the face centroid includes the first centroid, the second centroid, and the third centroid;

[0041] Based on the facial feature information of the face images in the face image set, the centroid of the face corresponding to the face image set is determined, including:

[0042] In response to the fact that the face image set contains two face images, the face feature information corresponding to the two face images are determined as the first centroid and the second centroid, respectively.

[0043] The third centroid is obtained by performing differential processing on the facial feature information corresponding to the two facial images.

[0044] Specifically, the facial feature information corresponding to the two facial images is differentially processed to obtain the third centroid, including:

[0045] The facial feature information corresponding to the two facial images is averaged to obtain the third centroid.

[0046] Among them, the face centroid includes the first centroid, the second centroid, the third centroid, and the fourth centroid;

[0047] Based on the facial feature information of the face images in the face image set, the centroid of the face corresponding to the face image set is determined, including:

[0048] If the face image set contains at least three face images, the face similarity between each face image is calculated.

[0049] The facial feature information corresponding to the two facial images with the smallest facial similarity is selected as the first centroid and the second centroid, respectively.

[0050] The facial feature information of the face image whose sum of the face similarity to the first centroid and the face similarity to the second centroid is the minimum value is selected as the third centroid;

[0051] The facial feature information of all face images in the face image set is averaged to obtain the fourth centroid.

[0052] Specifically, in response to the second similarity between the facial feature information of the face image to be clustered and the facial centroid of the face image set being not less than a second similarity threshold, it is determined that the face image to be clustered is assigned to the face image set to obtain an updated face image set, including:

[0053] If the second similarity between the facial feature information of the face image to be clustered and any centroid of the face centroid is not less than the second similarity threshold, then the face image to be clustered is determined to belong to the face image set to obtain the updated face image set.

[0054] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide an image clustering device, the image clustering device comprising:

[0055] The acquisition module is used to acquire images to be clustered and historical image sets. Images to be clustered include face images and body images of the same target object. Historical image sets include multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target.

[0056] A classification module is used to divide a body image set into at least two sub-image sets based on the shooting angle of the target in the body image;

[0057] The body clustering module is used to determine that the body image to be clustered belongs to the body image set and update the body image set in response to the first similarity between the body image to be clustered and the sub-image set exceeding the first similarity threshold.

[0058] The face clustering module is used to determine the updated face image set by classifying the face image to be clustered into the face image set in response to the second similarity between the face image to be clustered and the face image set exceeding the second similarity threshold.

[0059] The merging module is used to merge the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs, based on the association relationship between the face image to be clustered and the body image to be clustered.

[0060] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. The processor is used to execute program data to implement the steps in the above-mentioned image clustering method.

[0061] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the above-mentioned image clustering method.

[0062] The beneficial effects of this invention are as follows: Unlike existing technologies, it provides an image clustering method, apparatus, terminal, and computer-readable storage medium. The image clustering method acquires an image to be clustered and a historical image set. The image to be clustered includes face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target. Based on the shooting angle of the target in the body image, the body image set is divided into at least two sub-images. The system is as follows: In response to a first similarity threshold between the body image to be clustered and the sub-image set, it determines that the body image to be clustered belongs to the body image set to obtain an updated body image set; In response to a second similarity threshold between the face image to be clustered and the face image set, it determines that the face image to be clustered belongs to the face image set to obtain an updated face image set; Based on the association between the face image to be clustered and the body image to be clustered, it merges the updated face image set to which the face image to be clustered belongs with the updated body image set to which the body image to be clustered belongs. In this application, face images and body images to be clustered are clustered separately. By calculating the similarity between the body images to be clustered and body images taken from different angles, the same target object's body images are prevented from belonging to different image sets due to differences in shooting angle and clarity, thereby improving the clustering accuracy of the body images to be clustered. Based on the association between the body images to be clustered and the face images to be clustered, the updated face image set and the updated body image set corresponding to the same target object are merged. This not only ensures the image clustering accuracy in the merged image set but also improves the image clustering recall rate. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating the image clustering method provided by the present invention;

[0065] Figure 2 This is a flowchart illustrating an embodiment of the image clustering method provided by the present invention;

[0066] Figure 3 This is a flowchart illustrating a specific embodiment of the image clustering method provided by the present invention;

[0067] Figure 4 yes Figure 2 A flowchart illustrating a specific embodiment of step S205 in the provided image clustering method;

[0068] Figure 5 It is an undirected line graph between body images in a frontal body image set in one embodiment;

[0069] Figure 6 yes Figure 2 A flowchart illustrating a specific embodiment of step S208 in the provided image clustering method;

[0070] Figure 7 It is an undirected line graph between face images in a face image set in one embodiment;

[0071] Figure 8 This is a schematic diagram of the framework of an embodiment of the image clustering device provided by the present invention;

[0072] Figure 9 This is a schematic diagram of the framework of another embodiment of the image clustering device provided by the present invention;

[0073] Figure 10 This is a schematic diagram of the framework of an embodiment of the terminal provided by the present invention;

[0074] Figure 11 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided by the present invention. Detailed Implementation

[0075] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0076] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0077] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.

[0078] To enable those skilled in the art to better understand the technical solution of the present invention, the image clustering method provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0079] Please see Figure 1 , Figure 1 This is a flowchart illustrating the image clustering method provided by the present invention. This embodiment provides an image clustering method, which includes the following steps.

[0080] S11: Obtain the images to be clustered and the historical image set.

[0081] Specifically, multiple human images to be clustered are acquired, each containing the target object. Body and face detection are performed on these images to obtain related face and body images to be clustered. The images to be clustered include face and body images of the same target object. The historical image set includes multiple face and body image sets, each face image set including at least one target face image, and each body image set including multiple target body images.

[0082] S12: Based on the shooting angle of the target in the body image, divide the body image set into at least two sub-image sets.

[0083] Specifically, based on the shooting angle of the target in the body image, the body images in the body image set are divided into three sub-image sets, and the shooting angle of the target in each sub-image set is within the corresponding preset angle range; wherein, the three sub-image sets include a front body sub-image set, a side body sub-image set, and a back body sub-image set.

[0084] S13: In response to the first similarity between the body image to be clustered and the sub-image set exceeding the first similarity threshold, determine that the body image to be clustered belongs to the body image set to obtain an updated body image set.

[0085] Specifically, feature extraction is performed on the body image to be clustered and the body image respectively to obtain the body feature information corresponding to the body image to be clustered and the body image respectively; based on the body feature information of the body images contained in each sub-image set, the body centroid corresponding to each sub-image set is determined; the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to each sub-image set is compared; in response to the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to at least one sub-image set being not less than the first similarity threshold, it is determined that the body image to be clustered belongs to the body image set to obtain the updated body image set.

[0086] In one embodiment, the similarity between body images is calculated based on the body feature information of each body image in the sub-image set; the body centroid of the sub-image set is determined based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image.

[0087] In one specific embodiment, the similarity between body images is compared with a similarity threshold; if the similarity is less than the similarity threshold, the similarity corresponding to the body image is deleted; if the similarity is not less than the similarity threshold, the similarity corresponding to the body image is retained; the body centroid of the sub-image set is determined by summing the products between the body feature information of each body image and the retained similarity corresponding to the body image.

[0088] S14: In response to the second similarity between the face image to be clustered and the face image set exceeding the second similarity threshold, determine that the face image to be clustered belongs to the face image set to obtain the updated face image set.

[0089] Specifically, feature extraction is performed on the face image and the face image to be clustered, respectively, to obtain the face feature information corresponding to the face image and the face image to be clustered. Based on the face feature information of the face images in the face image set, the face centroid corresponding to the face image set is determined. The second similarity between the face feature information of the face image to be clustered and the face centroid of the face image set is compared. In response to the second similarity between the face feature information of the face image to be clustered and the face centroid of the face image set being not less than the second similarity threshold, it is determined that the face image to be clustered belongs to the face image set to obtain the updated face image set.

[0090] In one specific embodiment, in response to the fact that the face image set contains a face image, the facial feature information of the face image is determined as the face centroid of the face image set.

[0091] In one specific embodiment, the face centroid includes a first centroid, a second centroid, and a third centroid. Responding to a face image set containing two face images, the facial feature information corresponding to the two face images is determined as the first centroid and the second centroid, respectively; the facial feature information corresponding to the two face images is then subjected to differential processing to obtain the third centroid. Specifically, the facial feature information corresponding to the two face images is then averaged to obtain the third centroid.

[0092] In one specific embodiment, the face centroid includes a first centroid, a second centroid, a third centroid, and a fourth centroid. In response to the face image set containing at least three face images, the face similarity between each face image is calculated; the face feature information corresponding to the two face images with the smallest face similarity is selected as the first centroid and the second centroid, respectively; the face feature information of the face image whose face similarity to the first centroid and the sum of the face similarity to the second centroid is the minimum value is selected as the third centroid; the face feature information of all face images in the face image set is averaged to obtain the fourth centroid.

[0093] If the second similarity between the facial feature information of the face image to be clustered and any centroid of the face centroid is not less than the second similarity threshold, then the face image to be clustered is determined to belong to the face image set to obtain the updated face image set.

[0094] In one optional embodiment, based on the face images in the updated face image set and the face images to be clustered, the updated face centroid of the updated face image set is determined; in response to a third similarity between the face image or the face image to be clustered and the updated face centroid being less than a third similarity threshold, the face image or the face image to be clustered corresponding to the third similarity is removed; based on the body images in the updated body image set and the body images to be clustered, the updated body centroid of the updated body image set is determined; in response to a fourth similarity between the body image or the body image to be clustered and the updated body centroid being less than a fourth similarity threshold, the body image or the body image to be clustered corresponding to the fourth similarity is removed.

[0095] S15: Based on the association between the face images to be clustered and the body images to be clustered, merge the updated face image set to which the face images to be clustered belong and the updated body image set to which the body images to be clustered belong.

[0096] Specifically, since the face images and body images to be clustered belong to the same target object, the updated face image set to which the face images to be clustered belong and the updated body image set to which the body images to be clustered belong can be merged based on the association between them to obtain the human body image set of the target object. The human body image set includes the face images and body images of the target object.

[0097] In the image clustering method provided in this embodiment, the following steps are taken: First, an image to be clustered and a historical image set are obtained. The image to be clustered includes face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target. Based on the shooting angle of the target in the body image, the body image set is divided into at least two sub-image sets. In response to a first similarity threshold between the body image to be clustered and a sub-image set, it is determined that the body image to be clustered belongs to the body image set to obtain an updated body image set. In response to a second similarity threshold between the face image to be clustered and a face image set, it is determined that the face image to be clustered belongs to the face image set to obtain an updated face image set. Finally, based on the association between the face image to be clustered and the body image to be clustered, the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs are merged. In this application, face images and body images to be clustered are clustered separately. By calculating the similarity between the body images to be clustered and body images taken from different angles, the same target object's body images are prevented from belonging to different image sets due to differences in shooting angle and clarity, thereby improving the clustering accuracy of the body images to be clustered. Based on the association between the body images to be clustered and the face images to be clustered, the updated face image set and the updated body image set corresponding to the same target object are merged. This not only ensures the image clustering accuracy in the merged image set but also improves the image clustering recall rate.

[0098] Please see Figure 2 and Figure 3 , Figure 2 This is a flowchart illustrating an embodiment of the image clustering method provided by the present invention; Figure 3 This is a flowchart illustrating a specific embodiment of the image clustering method provided by the present invention. This embodiment provides an image clustering method comprising the following steps.

[0099] S201: Obtain multiple images of human bodies to be clustered.

[0100] Specifically, images are acquired by image acquisition devices installed at preset locations to capture images near those locations, resulting in human body images to be clustered. These images contain target objects, such as pedestrians. Each human body image may contain one or more target objects.

[0101] S202: Perform body and face detection and extraction on the human images to be clustered to obtain the face images and body images to be clustered that have a correlation.

[0102] Specifically, an object detection network is used to perform face and body part detection on the human images to be clustered, resulting in human images to be clustered that include face images corresponding to each face and body images corresponding to each body. The face images to be clustered are images of human faces, and the body images are images of the human torso and limbs. In other words, the body images to be clustered include images of all parts of the human body except the head. The human images to be clustered can include face and body images of multiple target objects. That is, face and body images corresponding to the same target object share the same identification information, such as an ID. Face and body images corresponding to the same target object are associated with each other.

[0103] In one embodiment, all face images and body images corresponding to multiple human images to be clustered are classified so that all face images to be clustered belong to a set of face images to be clustered, and all body images to be clustered belong to a set of body images to be clustered.

[0104] S203: Obtain historical image set.

[0105] Specifically, a historical image set is obtained, which includes multiple sets of face images and multiple sets of body images. In one embodiment, if the historical image set includes multiple face images and multiple body images, the multiple face images are clustered to obtain a face image set, and the body images are clustered to obtain a body image set.

[0106] Each face image set includes at least one face image containing the target. That is, multiple face images in each face image set represent the same target. Each body image set includes multiple body images of the target. That is, multiple body images in each body image set represent the same target.

[0107] Because the changes in a target object's face over time are relatively smaller than changes in the body, while the similarity of the target object's body can decrease depending on its clothing, the image to be clustered can be compared with body images from different angles to improve the accuracy of body clustering.

[0108] S204: Based on the shooting angle of the target in the body image, divide the body image set into at least two sub-image sets.

[0109] Specifically, based on the shooting angle of the target in the body image, the body image can be divided into two sub-image sets. The sub-image sets can be a frontal body image set and a non-frontal body image set. Specifically, based on whether the angle of the target's frontal view in the body image is within a preset angle range, all body images in the body image set are further divided into a frontal body image set and a non-frontal body image set. In other embodiments, the number of sub-image sets can also be set according to actual conditions.

[0110] In one embodiment, to improve the clustering accuracy of the body images to be clustered, the body images in the body image set are divided into three sub-image sets based on the shooting angle of the target in the body image. The shooting angle of the target in each sub-image set is within a corresponding preset angle range. The three sub-image sets include a frontal body sub-image set, a side body sub-image set, and a rear body sub-image set. For example, the angle of the frontal body in the body images in the frontal body sub-image set is within the range of [0°, 60°]. That is, the body images in the frontal body sub-image set are images of the target body captured from the front. The angle of the frontal body in the body images in the side body sub-image set is within the range of (60°, 120°). That is, the body images in the side body sub-image set are images of the target body captured from the side. The angle of the frontal body in the body images in the rear body sub-image set is within the range of (120°, 180°). That is, the body images in the rear body sub-image set are images of the target body captured from the rear.

[0111] S205: In response to the first similarity between the body image to be clustered and the sub-image set exceeding the first similarity threshold, determine that the body image to be clustered belongs to the body image set to obtain an updated body image set.

[0112] Specifically, the method for clustering the body images to be clustered with the body image set in the historical image set to obtain an updated body image set includes the following steps.

[0113] Please see Figure 4 , Figure 4 yes Figure 2 A flowchart illustrating a specific embodiment of step S205 in the provided image clustering method.

[0114] S2051: Extract features from the body image to be clustered and the body image respectively to obtain the body feature information corresponding to the body image to be clustered and the body image respectively.

[0115] Specifically, the trained deep learning network extracts features from the body images to be clustered and the body images in the body image set, obtaining the body feature information of the body images to be clustered and the body feature information of the body images. Specifically, the body feature information is a body feature map, that is, the body feature information is a body feature vector.

[0116] S2052: Determine the body centroid corresponding to each sub-image set based on the body feature information of the body images contained in each sub-image set.

[0117] Specifically, based on the body feature information of each body image in the sub-image set, the similarity between each body image is calculated; based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image, the body centroid of the sub-image set is determined.

[0118] In one embodiment, the similarity between body images is compared with a similarity threshold; if the similarity is less than the similarity threshold, the similarity corresponding to the body image is deleted; if the similarity is not less than the similarity threshold, the similarity corresponding to the body image is retained; the body centroid of the sub-image set is determined by summing the products between the body feature information of each body image and the retained similarity corresponding to the body image.

[0119] In one specific embodiment, taking a frontal body sub-image set as an example, the body centroid of the frontal body sub-image set is calculated. The frontal body sub-image set includes five body images, and the similarity between the body feature vectors of each body image is calculated based on the body feature vectors corresponding to the five body images.

[0120] Please see Figure 5 , Figure 5 It is an undirected line graph connecting the various body images in a frontal body image set in one embodiment.

[0121] To improve the accuracy of body centroid determination in the sub-image set, the similarity between body feature vectors of each body image is compared to a similarity threshold. If the similarity is greater than the threshold, an undirected straight line is established between the body feature vectors of the two body images corresponding to the similarity. This undirected straight line represents the similarity between the body feature vectors of the two body images. For example, given five body images S1, S2, S3, S4, and S5, similarity comparisons determine that the similarities between S1 and S3, S4, S4 and S3, and S2 and S4 all exceed the similarity threshold. Specifically, the fact that the similarity between S1 and S3 both exceed the threshold indicates a higher probability that the target bodies in the corresponding body images of S1 and S3 belong to the same target.

[0122] The more body images associated with each body image in a sub-image set, that is, the more undirected lines connecting the body feature vectors of each body image on the undirected connection graph, the more similar body images there are in the sub-image set, the stronger the representativeness of the body image, and the more likely it is to be a correctly clustered body image. On the other hand, body images with only a few undirected lines are more likely to be outliers and have weaker representativeness.

[0123] In one specific embodiment, the body centroid of the frontal body sub-image set is calculated based on the following formula.

[0124] C=(d 13 +d 14 )*t1+(d 24 +d 25 )*t2+(d 13 +d 34 )*t3+(d 14 +d 34 +d 24 )*t4+d 25 *t5 (Formula 1)

[0125] In the formula: C is the body's center of mass; t a For body image S a The body feature vector, a = 1, 2, 3, 4, 5; d xy Let x be the similarity between the body feature vector x and the body feature vector y of the body image. Both x and y can take values ​​of 1, 2, 3, 4, and 5.

[0126] This step allows us to calculate the body centroid corresponding to each sub-image set.

[0127] S2053: Compare the body feature information of the body image to be clustered with the first similarity between the body centroids corresponding to each sub-image set.

[0128] Specifically, based on the body feature vector of the body image to be clustered and the body centroids corresponding to all sub-image sets of the body image set, a first similarity between the body feature vector of the body image to be clustered and each body centroid is calculated. For example, Manhattan distance or Euclidean distance can be calculated based on the body feature vector of the body image to be clustered and each body centroid to obtain the first similarity between the body feature vector of the body image to be clustered and each body centroid. In other embodiments, the first similarity between the body feature vector of the body image to be clustered and each body centroid can also be calculated in other ways.

[0129] The first similarity between the body feature vector of the body image to be clustered and each body centroid is compared with the first similarity threshold.

[0130] S2054: In response to the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to at least one sub-image set being not less than the first similarity threshold, it is determined that the body image to be clustered is assigned to the body image set to obtain an updated body image set.

[0131] Specifically, if the first similarity between the body feature vector of the body image to be clustered and at least one body centroid in the body image set is not less than a first similarity threshold, then the body image to be clustered is determined to belong to the body image set.

[0132] The process iterates through all body images corresponding to the body image to be clustered, assigning each body image to its corresponding body image set to obtain an updated body image set. This updated body image set consists of body images from the already clustered body images and body images from the historical image set. Each updated body image set contains body images of the same target.

[0133] S206: Based on the body images in the updated body image set and the body images to be clustered, determine the updated body centroid of the updated body image set.

[0134] Specifically, based on the body feature vectors of the body images contained in each updated body image set and the body feature vectors of the body images to be clustered belonging to that updated body image set, the average centroid of each updated body image set is calculated, and this average centroid is used as the updated body centroid. Then, the updated body centroid of the updated body image set replaces the average centroid of the body image set in the historical image set.

[0135] S207: In response to the fourth similarity between the body image or the body image to be clustered and the updated body centroid being less than the fourth similarity threshold, the body image or the body image to be clustered corresponding to the fourth similarity is removed.

[0136] Specifically, to improve the recall rate of the updated body image set, a fourth similarity is calculated between the body images in the updated body image set, the body images to be clustered, and the updated body centroids. The fourth similarity is compared with a fourth similarity threshold to purify the images contained in the updated body image set, removing body images that may not belong to the target corresponding to the updated body image set.

[0137] If the fourth similarity between the body image or the body image to be clustered and the updated body centroid is less than the fourth similarity threshold, then the body image or the body image to be clustered corresponding to the fourth similarity is determined not to belong to the target body image corresponding to the updated body image set, and the body image or the body image to be clustered corresponding to the fourth similarity is removed.

[0138] Iterate through and update all body images and all body images to be clustered in the body image set.

[0139] S208: In response to the second similarity between the face image to be clustered and the face image set exceeding the second similarity threshold, determine that the face image to be clustered belongs to the face image set to obtain an updated face image set.

[0140] Specifically, the method for clustering the face images to be clustered with the face image set in the historical image set to obtain an updated face image set includes the following steps.

[0141] Please see Figure 6 , Figure 6 yes Figure 2 A flowchart illustrating a specific embodiment of step S208 in the provided image clustering method.

[0142] S2081: Perform feature extraction on the face image and the face image to be clustered, respectively, to obtain the face feature information corresponding to the face image and the face image to be clustered.

[0143] Specifically, the trained deep learning network extracts features from the face images to be clustered and the face images in the face image set, obtaining the facial feature information of the face images to be clustered and the facial feature information of the face images. Specifically, the facial feature information is a face feature map, that is, a face feature vector.

[0144] S2082: Determine the centroid of the face corresponding to the face image set based on the facial feature information of the face images in the face image set.

[0145] In one embodiment, in response to the presence of a face image in the face image set, the facial feature information of the face image is determined as the face centroid of the face image set.

[0146] In another embodiment, the face centroid includes a first centroid, a second centroid, and a third centroid; in response to the face image set containing two face images, the face feature information corresponding to the two face images is determined as the first centroid and the second centroid, respectively; the face feature information corresponding to the two face images is differentially processed to obtain the third centroid; the face feature information corresponding to the two face images is averaged to obtain the third centroid.

[0147] In another embodiment, the face centroid includes a first centroid, a second centroid, a third centroid, and a fourth centroid. In response to the presence of at least three face images in the face image set, the face similarity between each face image is calculated. To ensure that the face centroid of the face image set more comprehensively represents the face images in the set, a greedy algorithm is used to determine the face centroid. Specifically, the face feature information corresponding to the two face images with the lowest face similarity is selected as the first centroid and the second centroid, respectively; the face feature information of the face image whose face similarity to the first centroid and the sum of the face similarity to the second centroid are minimized is selected as the third centroid; and the face feature information of all face images in the face image set is averaged to obtain the fourth centroid.

[0148] Please see Figure 7 , Figure 7 It is an undirected graph connecting the face images in a face image set in one embodiment.

[0149] In one specific embodiment, the face image set includes four face images A1, A2, A3, and A4. The face similarity between the face images is calculated based on their facial feature vectors. Since the face similarity between A1 and A2 is the smallest, the facial feature vectors corresponding to face images A1 and A2 are determined as the first centroid and the second centroid, respectively. The sum of the face similarity between face images A3 and A1 and between A3 and A2 yields a first value of 1.40; the sum of the face similarity between face images A4 and A1 and between A4 and A2 yields a second value of 1.55. Since the first value is less than the second value, to more comprehensively reflect the face features of the face image set, the facial feature vector of face image A3 corresponding to the second value is selected as the third centroid. The fourth centroid is obtained by summing and averaging the facial feature vectors corresponding to face images A1, A2, A3, and A4 respectively.

[0150] S2083: Compare the facial feature information of the face image to be clustered with the second similarity between the face centroids of the face image set.

[0151] Specifically, based on the facial feature vectors of the face images to be clustered and the corresponding centroids of the face images in the set of face images, a second similarity is calculated between the facial feature vectors of the face images to be clustered and each face centroid. For example, Manhattan distance or Euclidean distance can be calculated based on the facial feature vectors of the face images to be clustered and each face centroid to obtain the second similarity between the facial feature vectors of the face images to be clustered and each face centroid. In other embodiments, the second similarity between the facial feature vectors of the face images to be clustered and each face centroid can also be calculated in other ways.

[0152] The second similarity between the facial feature vectors of the face images to be clustered and the centroids of each face is compared with the second similarity threshold.

[0153] S2084: In response to the fact that the second similarity between the facial feature information of the face image to be clustered and the face centroid of the face image set is not less than the second similarity threshold, it is determined that the face image to be clustered belongs to the face image set to obtain the updated face image set.

[0154] Specifically, in response to the second similarity between the facial feature information of the face image to be clustered and any centroid in the face centroid not being less than the second similarity threshold, it is determined that the face image to be clustered belongs to the face image set to obtain the updated face image set.

[0155] In one embodiment, if the second similarity between the face feature vector of the face image to be clustered and at least one face centroid in the face image set is not less than a second similarity threshold, then the face image to be clustered is determined to belong to the face image set.

[0156] The process iterates through all face images corresponding to the face image to be clustered, assigning each face image to its corresponding face image set to obtain an updated face image set. This updated face image set consists of the face images from the already clustered face images and face images from the historical image set. Each updated face image set contains face images of the same target.

[0157] S209: Based on the face images in the updated face image set and the face images to be clustered, determine the updated face centroids of the updated face image set.

[0158] Specifically, based on the facial feature vectors of the facial images contained in each updated face image set and the facial feature vectors of the face images to be clustered belonging to that updated face image set, the average centroid of each updated face image set is calculated, and this average centroid is used as the updated face centroid. Then, the updated face centroid of the updated face image set replaces the average centroid of the face image set in the historical image set.

[0159] S210: In response to the fact that the third similarity between the face image or the face image to be clustered and the updated face centroid is less than the third similarity threshold, the face image or the face image to be clustered corresponding to the third similarity is removed.

[0160] Specifically, to improve the recall rate of the updated face image set, a third similarity is calculated between the face images in the updated face image set, the face images to be clustered, and the centroids of the updated faces. The third similarity is compared with a third similarity threshold to purify the images contained in the updated face image set, removing face images that may not belong to the target corresponding to the updated face image set.

[0161] If the third similarity between the face image or the face image to be clustered and the updated face centroid is less than the third similarity threshold, then the face image or the face image to be clustered corresponding to the third similarity is determined not to belong to the target face image corresponding to the updated face image set, and the face image or the face image to be clustered corresponding to the third similarity is removed.

[0162] Iterate through and update all face images and all face images to be clustered in the face image set.

[0163] S211: Based on the association between the face images to be clustered and the body images to be clustered, merge the updated face image set to which the face images to be clustered belong and the updated body image set to which the body images to be clustered belong.

[0164] Specifically, since the face images and body images of the same target object have the same identification information, the updated face image set to which the face images of the same target object belong and the updated body image set to which the body images belong can be merged based on the identification information to obtain the human body image set of the target object. The human body image set includes the face images and body images of the target object.

[0165] In the image clustering method provided in this embodiment, the following steps are taken: First, an image to be clustered and a historical image set are obtained. The image to be clustered includes face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target. Based on the shooting angle of the target in the body image, the body image set is divided into at least two sub-image sets. In response to a first similarity threshold between the body image to be clustered and a sub-image set, it is determined that the body image to be clustered belongs to the body image set to obtain an updated body image set. In response to a second similarity threshold between the face image to be clustered and a face image set, it is determined that the face image to be clustered belongs to the face image set to obtain an updated face image set. Finally, based on the association between the face image to be clustered and the body image to be clustered, the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs are merged. In this application, face images and body images to be clustered are clustered separately. By calculating the similarity between the body images to be clustered and body images taken from different angles, the same target object's body images are prevented from belonging to different image sets due to differences in shooting angle and clarity, thereby improving the clustering accuracy of the body images to be clustered. Based on the association between the body images to be clustered and the face images to be clustered, the updated face image set and the updated body image set corresponding to the same target object are merged. This not only ensures the image clustering accuracy in the merged image set but also improves the image clustering recall rate.

[0166] Please see Figure 8 , Figure 8 This is a schematic diagram of the framework of an embodiment of the image clustering device provided by the present invention. This embodiment provides an image clustering device 60, which includes an acquisition module 61, a classification module 62, a body clustering module 63, a face clustering module 64, and a merging module 65.

[0167] The acquisition module 61 is used to acquire images to be clustered and historical image sets. Images to be clustered include face images and body images of the same target object. Historical image sets include multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target.

[0168] The acquisition module 61 is used to acquire multiple human images to be clustered, which are images containing target objects; to perform body and face detection and extraction on the human images to be clustered, and to obtain the face images and body images to be clustered that have a relationship.

[0169] The classification module 62 is used to divide the body image set into at least two sub-image sets based on the shooting angle of the target in the body image.

[0170] The classification module 62 is used to divide the body images in the body image set into three sub-image sets based on the shooting angle of the target in the body image. The shooting angle of the target in each sub-image set is within the corresponding preset angle range. The three sub-image sets include a front body sub-image set, a side body sub-image set, and a back body sub-image set.

[0171] The body clustering module 63 is used to determine, in response to a first similarity between the body image to be clustered and the sub-image set exceeding a first similarity threshold, to assign the body image to be clustered to the body image set to obtain an updated body image set.

[0172] The body clustering module 63 is used to extract features from the body image to be clustered and the body image respectively, to obtain the body feature information corresponding to the body image to be clustered and the body image respectively; based on the body feature information of the body images contained in each sub-image set, to determine the body centroid corresponding to each sub-image set; to compare the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to each sub-image set; in response to the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to at least one sub-image set being not less than the first similarity threshold, it is determined that the body image to be clustered belongs to the body image set to obtain an updated body image set.

[0173] The body clustering module 63 is also used to calculate the similarity between body images based on the body feature information of each body image in the sub-image set; and to determine the body centroid of the sub-image set based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image.

[0174] The body clustering module 63 is also used to compare the similarity between body images with a similarity threshold; in response to the similarity being less than the similarity threshold, the similarity corresponding to the body image is deleted; in response to the similarity being not less than the similarity threshold, the similarity corresponding to the body image is retained; and the body centroid of the sub-image set is determined by summing the products between the body feature information of each body image and the retained similarity corresponding to the body image.

[0175] The face clustering module 64 is used to determine that the face image to be clustered belongs to the face image set to obtain an updated face image set in response to the second similarity between the face image to be clustered and the face image set exceeding the second similarity threshold.

[0176] The face clustering module 64 is also used to extract features from the face image and the face image to be clustered, respectively, to obtain the face feature information corresponding to the face image and the face image to be clustered, respectively; based on the face feature information of the face images in the face image set, to determine the face centroid corresponding to the face image set; to compare the second similarity between the face feature information of the face image to be clustered and the face centroid of the face image set; in response to the second similarity between the face feature information of the face image to be clustered and the face centroid of the face image set not being less than the second similarity threshold, to determine that the face image to be clustered belongs to the face image set to obtain the updated face image set.

[0177] In one embodiment, the face clustering module 64 is further configured to determine the face feature information of the face image as the face centroid of the face image set in response to the face image set containing a face image.

[0178] In one specific embodiment, the facial feature information corresponding to the two facial images is averaged to obtain a third centroid.

[0179] In one embodiment, the face centroid includes a first centroid, a second centroid, and a third centroid. The face clustering module 64 is further configured to, in response to the presence of two face images in the face image set, determine the face feature information corresponding to the two face images as the first centroid and the second centroid, respectively; and perform differential processing on the face feature information corresponding to the two face images to obtain the third centroid.

[0180] In one specific embodiment, the face centroid includes a first centroid, a second centroid, a third centroid, and a fourth centroid. The face clustering module 64 is further configured to, in response to a face image set containing at least three face images, calculate the face similarity between each face image; select the face feature information corresponding to the two face images with the smallest face similarity as the first centroid and the second centroid, respectively; select the face feature information of the face image whose sum of the face similarity corresponding to the first centroid and the face similarity corresponding to the second centroid is the minimum, as the third centroid; and average the face feature information of all face images in the face image set to obtain the fourth centroid.

[0181] In one specific embodiment, the face clustering module 64 is further configured to determine that the face image to be clustered belongs to the face image set to obtain an updated face image set in response to the second similarity between the face feature information of the face image to be clustered and any centroid of the face centroid not being less than the second similarity threshold.

[0182] The merging module 65 is used to merge the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs, based on the association relationship between the face image to be clustered and the body image to be clustered.

[0183] Please see Figure 9 , Figure 9 This is a schematic diagram of another embodiment of the image clustering device provided by the present invention.

[0184] In an optional embodiment, the image clustering device 60 further includes a verification module 66, which is used to determine the updated face centroid of the updated face image set based on the face images in the updated face image set and the face images to be clustered; in response to a third similarity between the face image or the face image to be clustered and the updated face centroid being less than a third similarity threshold, the face image or the face image to be clustered corresponding to the third similarity is removed; based on the body images in the updated body image set and the body images to be clustered, the updated body centroid of the updated body image set is determined; in response to a fourth similarity between the body image or the body image to be clustered and the updated body centroid being less than a fourth similarity threshold, the body image or the body image to be clustered corresponding to the fourth similarity is removed.

[0185] The image clustering device provided in this embodiment includes an acquisition module for acquiring images to be clustered and historical image sets. The images to be clustered include face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target. A classification module is used to divide the body image set into at least two sub-image sets based on the shooting angle of the target in the body image. A body clustering module is used to determine that the body image to be clustered belongs to the body image set to obtain an updated body image set in response to a first similarity exceeding a first similarity threshold. A face clustering module is used to determine that the face image to be clustered belongs to the face image set to obtain an updated face image set in response to a second similarity exceeding a second similarity threshold. A merging module is used to merge the updated face image set to which the face image to be clustered belongs with the updated body image set to which the body image to be clustered belongs based on the association relationship between the face image to be clustered and the body image to which the body image to be clustered belongs. By clustering the face images and body images to be clustered separately, and by calculating the similarity between the body images to be clustered and body images taken from different angles, the clustering accuracy of the body images to be clustered is improved, thus avoiding the body images of the same target object belonging to different image sets due to differences in shooting angle and clarity. Based on the association between the body images to be clustered and the face images to be clustered, the updated face image set and the updated body image set corresponding to the same target object are merged, which improves the image clustering recall rate while ensuring the image clustering accuracy in the merged image set.

[0186] Please see Figure 10 , Figure 10This is a schematic diagram of a terminal embodiment provided by the present invention. The terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described image clustering method embodiments. In a specific implementation scenario, the terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0187] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described image clustering method embodiments. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.

[0188] The above scheme, the image clustering method includes: acquiring images to be clustered and historical image sets, wherein the images to be clustered include face images and body images of the same target object, and the historical image set includes multiple face image sets and body image sets, each face image set including at least one face image of the target, and each body image set including multiple body images of the target; dividing the body image set into at least two sub-image sets based on the shooting angle of the target in the body image; in response to a first similarity between the body image to be clustered and the sub-image set exceeding a first similarity threshold, determining that the body image to be clustered belongs to the body image set to obtain an updated body image set; in response to a second similarity between the face image to be clustered and the face image set exceeding a second similarity threshold, determining that the face image to be clustered belongs to the face image set to obtain an updated face image set; and merging the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs based on the association relationship between the face image to be clustered and the body image to which the body image to be clustered belongs.

[0189] Please see Figure 11 , Figure 11This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present invention. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above-described image clustering method embodiments.

[0190] The above scheme, the image clustering method includes: acquiring images to be clustered and historical image sets, wherein the images to be clustered include face images and body images of the same target object, and the historical image set includes multiple face image sets and body image sets, each face image set including at least one face image of the target, and each body image set including multiple body images of the target; dividing the body image set into at least two sub-image sets based on the shooting angle of the target in the body image; in response to a first similarity between the body image to be clustered and the sub-image set exceeding a first similarity threshold, determining that the body image to be clustered belongs to the body image set to obtain an updated body image set; in response to a second similarity between the face image to be clustered and the face image set exceeding a second similarity threshold, determining that the face image to be clustered belongs to the face image set to obtain an updated face image set; and merging the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs based on the association relationship between the face image to be clustered and the body image to which the body image to be clustered belongs.

[0191] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0192] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0194] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0195] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0196] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0197] The above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image clustering method, characterized in that, The image clustering method includes: Obtain images to be clustered and historical image sets. The images to be clustered include face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target. Based on the shooting angle of the target in the body image, the body image set is divided into at least two sub-image sets; In response to a first similarity between the body image to be clustered and the sub-image set exceeding a first similarity threshold, it is determined that the body image to be clustered belongs to the body image set to obtain an updated body image set; In response to the second similarity between the face image to be clustered and the face image set exceeding a second similarity threshold, it is determined that the face image to be clustered should be assigned to the face image set to obtain an updated face image set; Based on the association between the face image to be clustered and the body image to be clustered, the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs are merged. The step of determining to assign the body image to the body image set to obtain an updated body image set in response to a first similarity threshold between the body image to be clustered and the sub-image set includes: Feature extraction is performed on the body image to be clustered and the body image respectively to obtain the body feature information corresponding to the body image to be clustered and the body image respectively; Based on the body feature information of each body image in the sub-image set, the similarity between each body image is calculated. Based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image, the body centroid of the sub-image set is determined; The first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to each of the sub-image sets is compared with the first similarity threshold. In response to the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to at least one of the sub-image sets being not less than the first similarity threshold, it is determined that the body image to be clustered is assigned to the body image set to obtain the updated body image set.

2. The image clustering method according to claim 1, characterized in that, Before the step of merging the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs based on the association relationship between the face image to be clustered and the body image to be clustered, the method further includes: Based on the face images in the updated face image set and the face images to be clustered, determine the updated face centroid of the updated face image set; If the third similarity between the face image or the face image to be clustered and the updated face centroid is less than the third similarity threshold, then the face image or the face image to be clustered corresponding to the third similarity is removed. Based on the body images in the updated body image set and the body images to be clustered, determine the updated body centroid of the updated body image set; If the fourth similarity between the body image or the body image to be clustered and the updated body centroid is less than the fourth similarity threshold, then the body image or the body image to be clustered corresponding to the fourth similarity is removed.

3. The image clustering method according to claim 1, characterized in that, The process of acquiring images to be clustered and historical image sets, wherein the images to be clustered include face images and body images of the same target object, and the historical image sets include multiple face image sets and body image sets, each face image set including at least one face image of the target, and each body image set including multiple body images of the target, includes: Acquire multiple human images to be clustered, wherein the human images to be clustered are images containing the target object; The human images to be clustered are subjected to body and face detection and extraction to obtain the face images and body images to be clustered that have a correlation.

4. The image clustering method according to claim 1, characterized in that, Based on the shooting angle of the target in the body image, the body image set is divided into at least two sub-image sets, including: Based on the shooting angle of the target in the body image, the body images in the body image set are divided into three sub-image sets, and the shooting angle of the target in each sub-image set is within a corresponding preset angle range; wherein, the three sub-image sets include a front body sub-image set, a side body sub-image set, and a back body sub-image set.

5. The image clustering method according to claim 1, characterized in that, The step of determining the body centroid of the sub-image set based on the body feature information of each body image in the sub-image set and the similarity to the corresponding body image includes: The similarity between the body images is compared with a similarity threshold; If the similarity is less than the similarity threshold, then the similarity corresponding to the body image is deleted; If the similarity is not less than the similarity threshold, then the similarity corresponding to the body image is retained; The body centroid of the sub-image set is determined by summing the products between the body feature information of each body image and the similarity retained for the corresponding body image.

6. The image clustering method according to claim 1, characterized in that, The step of determining to assign the face image to the face image set to obtain an updated face image set in response to a second similarity threshold between the face image to be clustered and the face image set includes: Feature extraction is performed on the face image and the face image to be clustered, respectively, to obtain the face feature information corresponding to the face image and the face image to be clustered, respectively; Based on the facial feature information of the facial images in the facial image set, the centroid of the face corresponding to the facial image set is determined; The second similarity between the facial feature information of the face image to be clustered and the face centroid of the face image set is compared with the second similarity threshold. If the second similarity between the facial feature information of the face image to be clustered and the face centroid of the face image set is not less than the second similarity threshold, then it is determined that the face image to be clustered is assigned to the face image set to obtain the updated face image set.

7. The image clustering method according to claim 6, characterized in that, The step of determining the face centroid corresponding to the face image set based on the face feature information of the face images in the face image set includes: In response to the fact that the face image set contains a face image, the facial feature information of the face image is determined as the face centroid of the face image set.

8. The image clustering method according to claim 6, characterized in that, The face centroid includes a first centroid, a second centroid, and a third centroid; The step of determining the centroid of the face corresponding to the face image set based on the facial feature information of the face images in the face image set includes: In response to the fact that the face image set contains two face images, the face feature information corresponding to the two face images are determined as the first centroid and the second centroid, respectively. The third centroid is obtained by performing differential processing on the facial feature information corresponding to the two facial images respectively.

9. The image clustering method according to claim 8, characterized in that, The step of performing differential processing on the facial feature information corresponding to the two facial images respectively to obtain the third centroid includes: The facial feature information corresponding to the two facial images is averaged to obtain the third centroid.

10. The image clustering method according to claim 6, characterized in that, The face centroid includes a first centroid, a second centroid, a third centroid, and a fourth centroid; The step of determining the centroid of the face corresponding to the face image set based on the facial feature information of the face images in the face image set includes: In response to the fact that the face image set contains at least three face images, the face similarity between each face image is calculated; The facial feature information corresponding to the two facial images with the smallest facial similarity is selected as the first centroid and the second centroid, respectively. The facial feature information of the face image whose sum of the face similarity to the first centroid and the face similarity to the second centroid is the minimum value is selected as the third centroid; The fourth centroid is obtained by averaging the facial feature information of all the facial images in the set of facial images.

11. The image clustering method according to any one of claims 6 to 10, characterized in that, The step of determining that the updated face image set is obtained by assigning the face image to the face image set in response to the second similarity between the face feature information of the face image to be clustered and the face centroid of the face image set is not less than the second similarity threshold includes: If the second similarity between the facial feature information of the face image to be clustered and any centroid of the face centroid is not less than the second similarity threshold, then it is determined that the face image to be clustered belongs to the face image set to obtain the updated face image set.

12. An image clustering device, characterized in that, The image clustering device includes: The acquisition module is used to acquire images to be clustered and historical image sets. The images to be clustered include face images and body images of the same target object. The historical image set includes multiple face image sets and body image sets. Each face image set includes at least one face image of the target, and each body image set includes multiple body images of the target. A classification module is used to divide the body image set into at least two sub-image sets based on the shooting angle of the target in the body image; A body clustering module is configured to: determine that the body image to be clustered belongs to the body image set to obtain an updated body image set in response to a first similarity between the body image to be clustered and the sub-image set exceeding a first similarity threshold; further configured to: extract features from the body image to be clustered and the body image respectively to obtain body feature information corresponding to the body image to be clustered and the body image respectively; calculate the similarity between each body image based on the body feature information of each body image in the sub-image set; determine the body centroid of the sub-image set based on the body feature information of each body image in the sub-image set and the similarity corresponding to the body image; compare the first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to each sub-image set with the first similarity threshold; and determine that the body image to be clustered belongs to the body image set to obtain the updated body image set in response to a first similarity between the body feature information of the body image to be clustered and the body centroid corresponding to at least one sub-image set not less than the first similarity threshold. A face clustering module is used to determine, in response to the second similarity between the face image to be clustered and the face image set exceeding a second similarity threshold, to assign the face image to be clustered to the face image set to obtain an updated face image set; The merging module is used to merge the updated face image set to which the face image to be clustered belongs and the updated body image set to which the body image to be clustered belongs, based on the association relationship between the face image to be clustered and the body image to be clustered.

13. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, the processor being configured to execute program data to implement the steps in the image clustering method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image clustering method as described in any one of claims 1 to 11.

Images