Image clustering method, image clustering apparatus, and computer storage medium
By calculating the similarity between faces and bodies in large captured images and employing a multi-threshold strategy and conflict judgment, the portrait clustering process is optimized, solving the problem of low recall rate in portrait clustering and achieving higher recall and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2023-03-13
- Publication Date
- 2026-05-08
AI Technical Summary
In existing technologies, the recall rate of portrait clustering is low, mainly because the similarity of portrait images of the same person is lower than the threshold due to different angles and times of capture devices.
By acquiring small images of faces and bodies from large captured images, the similarity between faces and bodies is calculated, and image clustering is performed based on similarity thresholds. The similarity between small captured images of faces and bodies is adjusted to improve the similarity of human images. A multi-threshold strategy and judgment of intra-image and inter-image contradictions are adopted to optimize the clustering process.
It improves the recall and accuracy of image clustering. By adjusting the similarity of captured large images, it enhances the accuracy and recall of portrait clustering.
Smart Images
Figure CN116386107B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image clustering technology, and in particular to an image clustering method, an image clustering device, and a computer storage medium. Background Technology
[0002] With the continuous development of technology, facial recognition cameras are ubiquitous. These cameras capture images of people to create profiles and reconstruct each person's trajectory. Currently, most profile creation is based on extracting small facial and body images from pictures, using deep learning technology to extract features and form feature vectors. Profile creation is then based on the similarity between these vectors. However, due to differences in capture devices, capture angles, and capture times, the similarity of images of the same person may be slightly lower than the profile creation threshold. Therefore, how to effectively improve the recall rate of profile creation is a problem that urgently needs to be solved. Summary of the Invention
[0003] This application provides an image clustering method, an image clustering apparatus, and a computer storage medium.
[0004] One technical solution adopted in this application is to provide an image clustering method, the image clustering method comprising:
[0005] Obtain the first and second large-scale images captured;
[0006] Based on the first captured large image, obtain several first face captured small images and several first body captured small images;
[0007] Based on the second large image, obtain several small images of second faces and several small images of second bodies;
[0008] Obtain the face similarity of each pair of face capture images and the human body similarity of each pair of human body capture images. Each pair of face capture images includes any first face capture image and any second face capture image, and each pair of human body capture images includes any first human body capture image and any second human body capture image.
[0009] Based on the face similarity of all pairs of face capture images and the human body similarity of all pairs of human body capture images, the portrait similarity of the first large capture image and the second large capture image is obtained.
[0010] When the portrait similarity is less than the large image similarity threshold, image clustering is performed on the face capture small images of the face capture small image pair whose face similarity is greater than or equal to the first face clustering threshold, and on the human body capture small image pair whose human body similarity is greater than or equal to the first human body clustering threshold.
[0011] When the face similarity is greater than or equal to the large image similarity threshold, image clustering is performed on the face capture small images of the face capture small image pair whose face similarity is greater than or equal to the second face clustering threshold, and on the human body capture small image pair whose human body similarity is greater than or equal to the second human body clustering threshold.
[0012] Wherein, the first face clustering threshold is greater than the second face clustering threshold, and the first human body clustering threshold is greater than the second human body clustering threshold.
[0013] The step of obtaining the portrait similarity between the first large image and the second large image based on the face similarity of all pairs of small face images and the human similarity of all pairs of small human images includes:
[0014] When the face similarity of the face capture small image pair is greater than or equal to the first face clustering threshold, the ratio of the face similarity to the first face clustering threshold is used as the portrait similarity of the face capture small image pair and its associated human body capture small image pair.
[0015] When the face similarity of the face capture small image pair is less than the first face clustering threshold and greater than or equal to the second face clustering threshold, the weighted sum of the ratio of the face similarity to the first face clustering threshold and the ratio of the human body similarity of the human body capture small image pair associated with the face capture small image pair to the first human body clustering threshold is taken as the human image similarity of the face capture small image pair.
[0016] When the facial similarity of the face capture image pair is less than the second face clustering threshold, the calculation of the facial similarity of the face capture image and its associated human body capture image is cancelled.
[0017] The sum of the portrait similarity scores of all the captured small image pairs is used as the portrait similarity score between the first captured large image and the second captured large image.
[0018] The image clustering method further includes:
[0019] When the face capture image pair has no associated human body capture image pair, the human body capture image pair with zero human body similarity is associated with the face capture image pair.
[0020] The step of using the sum of the facial similarity scores of all the captured small image pairs as the facial similarity score between the first captured large image and the second captured large image includes:
[0021] When there are no associated face capture pairs in the human body capture image pairs, and the human body similarity of the human body capture image pairs is greater than or equal to the second human body clustering threshold, the weight of the ratio of the human body similarity to the first human body clustering threshold is used as the human image similarity of the human body capture image pairs.
[0022] The sum of the portrait similarity scores of all the face capture pairs and the portrait similarity scores of all the human body capture pairs is used as the portrait similarity score of the first large capture image and the second large capture image.
[0023] The step of obtaining a plurality of first face capture small images based on the first large capture image includes:
[0024] Based on the first large captured image, obtain several small captured images of candidate faces;
[0025] Obtain the first candidate similarity between any two candidate face capture images from the plurality of candidate face capture images;
[0026] The similarity calculation of the faces of two candidate face capture images with a similarity greater than or equal to the first face clustering threshold is cancelled, and the plurality of first face capture images are obtained.
[0027] The image clustering method, after obtaining the facial similarity of each pair of captured face images, further includes:
[0028] Based on the face similarity of all the face capture small image pairs, determine whether there exists a face capture small image whose face similarity with at least two face capture small images in another large capture image is greater than or equal to the first face clustering threshold.
[0029] If it exists, cancel the facial similarity calculation for that small captured face image.
[0030] The image clustering method further includes, after obtaining the face similarity of each pair of face capture images and the human body similarity of each pair of human body capture images:
[0031] Based on the face similarity of all pairs of face capture images and the human similarity of all pairs of human body capture images, determine whether there are any face capture images that meet the inter-image contradiction condition.
[0032] If it exists, cancel the facial similarity calculation for that small captured face image;
[0033] The inter-image contradiction condition is as follows: the face similarity between the first target face capture image and the second target face capture image in another large capture image is greater than or equal to the second face clustering threshold; the human body similarity between the first target human body capture image associated with the first target face capture image and the second target human body capture image associated with the second target face capture image is less than the second human body clustering threshold; and the human body similarity between the first target human body capture image associated with the first target face capture image and other human body capture images is greater than or equal to the second human body clustering threshold.
[0034] The image clustering method further includes, after obtaining the face similarity of each pair of face capture images and the human body similarity of each pair of human body capture images:
[0035] Based on the face similarity of all pairs of face capture images and the human body similarity of all pairs of human body capture images, determine whether there are any human body capture images that meet the contradictory conditions of human body and face.
[0036] If it exists, cancel the facial similarity calculation for that human body snapshot.
[0037] The human face contradiction condition is as follows: the human body similarity between the first target human body capture small image and the second target human body capture small image in another large image is greater than or equal to the second human body clustering threshold, and the human body similarity between the first target face capture small image associated with the first target human body capture small image and the second target face capture small image associated with the second target human body capture small image is less than the second face clustering threshold.
[0038] Another technical solution adopted in this application is to provide an image clustering device, which includes a memory and a processor coupled to the memory;
[0039] The memory is used to store program data, and the processor is used to execute the program data to implement the image clustering method described above.
[0040] Another technical solution adopted in this application is to provide a computer storage medium for storing program data, which, when executed by a computer, is used to implement the image clustering method described above.
[0041] The beneficial effects of this application are: the image clustering device acquires a first large-scale captured image and a second large-scale captured image; based on the first large-scale captured image, it acquires several first small-scale captured images of faces and several small-scale captured images of bodies; based on the second large-scale captured image, it acquires several small-scale captured images of faces and several small-scale captured images of bodies; it acquires the face similarity of each pair of face captured images and the body similarity of each pair of body captured images, wherein each pair of face captured images includes any one first small-scale captured image of a face and any one second small-scale captured image of a face, and each pair of body captured images includes any one first small-scale captured image of a body and any one second small-scale captured image of a body; based on the face similarity of all pairs of face captured images and the body similarity of all pairs of body captured images, it acquires... The image clustering method of this application calculates the facial similarity between the first and second large-scale captured images. When the facial similarity is less than the large-scale image similarity threshold, image clustering is performed on small-scale captured images of faces whose facial similarity is greater than or equal to the first face clustering threshold, and on small-scale captured images of bodies whose human body similarity is greater than or equal to the first body clustering threshold. When the facial similarity is greater than or equal to the large-scale image similarity threshold, image clustering is performed on small-scale captured images of faces whose facial similarity is greater than or equal to the second face clustering threshold, and on small-scale captured images of bodies whose human body similarity is greater than or equal to the second body clustering threshold. The first face clustering threshold is greater than the second face clustering threshold, and the first body clustering threshold is greater than the second body clustering threshold. This image clustering method can adjust the similarity of small-scale captured facial and body images based on the similarity of the large-scale captured images, thereby improving the recall rate of image clustering. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 This is a flowchart illustrating an embodiment of the image clustering method provided in this application;
[0044] Figure 2 yes Figure 1 The flowchart of the specific sub-steps of step S12 in the image clustering method shown is as follows:
[0045] Figure 3 This is a schematic diagram of an embodiment of the image clustering device provided in this application;
[0046] Figure 4 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation
[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0048] This application provides a method to obtain the portrait similarity between large images based on the similarity between small images of faces and bodies in a large image. If the portrait similarity is higher than the similarity threshold, it is considered that there is a portrait association relationship between the large images. Based on this association relationship, the portrait similarity can be appropriately increased, thereby improving the recall rate of image clustering without reducing the accuracy of image clustering.
[0049] The following is a definition of the terms used in this application:
[0050] Large snapshot: refers to a snapshot taken by a camera when capturing a person's image.
[0051] Captured small image: refers to a single face or body image extracted from a captured large image.
[0052] Portrait capture thumbnail: refers to the face image extracted from the large capture image and the associated body image. If there is no associated face or body image, the portrait capture thumbnail refers to a single face or body capture thumbnail.
[0053] Face-body correlation: refers to the correlation between the face image and the body image of the same person in a large-scale snapshot.
[0054] First face clustering threshold: If the similarity between small face capture images is higher than the first face clustering threshold, they are considered to be face images of the same person and can be directly clustered. At the same time, the human body images associated with the corresponding face can also be clustered.
[0055] Second face clustering threshold: If the similarity between small face capture images is higher than the second face clustering threshold but lower than the first face clustering threshold, they are considered to be face images of the same person, and other methods are needed to determine whether they are the same person.
[0056] First human body clustering threshold: If the similarity between small human body images is higher than the first human body clustering threshold, and there are no associated face images or the similarity between associated face images is higher than the first face clustering threshold, then human body clustering can be performed directly. If the similarity between associated face images is lower than the first face clustering threshold but higher than the second face clustering threshold, then other methods need to be combined to determine whether they are the same person.
[0057] Second human body clustering threshold: If the similarity between small human body images is higher than the second human body clustering threshold but lower than the first human body clustering threshold, they are considered to be images of the same person and need to be combined with other methods to determine whether they are the same person.
[0058] Intra-image contradiction: This refers to a situation where a large snapshot image contains multiple smaller snapshot images with similarity scores higher than the first face clustering threshold or the first human body clustering threshold. In such cases, similarity calculations are not performed on the smaller snapshot images, and other intervention methods are required. This type of situation generally occurs when twins or multiple people are wearing the same uniform.
[0059] First-image contradiction: This refers to a situation where the similarity between a small image captured in one large image and multiple small images captured in another large image is higher than the first face clustering threshold or the first human body clustering threshold. In this case, the similarity of the captured small image is not calculated, and other methods are needed to intervene. This type of situation generally occurs when twins or multiple people are wearing the same uniform.
[0060] The second type of contradiction occurs when a small face image in one large captured image has a similarity higher than the second face clustering threshold, but the similarity between related human bodies is lower than the second human body clustering threshold. Furthermore, if one of the related human bodies has a similarity higher than the second human body clustering threshold with other human bodies in the large image, then the similarity calculation for that small captured image is not performed, and other intervention methods are required. This type of situation generally occurs when multiple people are wearing the same uniform.
[0061] Human-face contradiction: This refers to a situation where the similarity between a small human image captured in one large image and a small human image captured in another large image is higher than the second human clustering threshold, but the similarity between the associated faces is lower than the second face clustering threshold. In this case, the similarity of the captured small image is not calculated, and other methods are required to intervene. This type of situation generally occurs when multiple people are wearing the same uniform.
[0062] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the image clustering method provided in this application.
[0063] The image clustering method of this application is applied to an image clustering device, which can be a server or a system consisting of a server and a local terminal working together. Accordingly, all components of the image clustering device, such as units, sub-units, modules, and sub-modules, can be located entirely in the server, or separately in the server and the local terminal.
[0064] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed servers, or as a single software program or software module; no specific limitation is made here. In some possible implementations, the image clustering method of this application embodiment can be implemented by a processor calling computer-readable instructions stored in memory.
[0065] Specifically, such as Figure 1 As shown, the image clustering method in this application embodiment specifically includes the following steps:
[0066] Step S11: Obtain the first large-scale image and the second large-scale image.
[0067] Step S12: Based on the first large-capture image, obtain several small-capture images of the first face and several small-capture images of the first human body.
[0068] In this embodiment of the application, the image clustering device collects large-scale images of all portrait capture points within a designated area using a capture camera or similar device, resulting in a set P of large-scale portrait capture images, assuming it to be {P1, P2, ..., P...}. n} Where n represents the total number of large-scale human portrait capture images. Deep learning techniques are used to extract features from each large-scale capture image for each face and body, forming a feature vector, and thus obtaining the relationship between faces and bodies. For example, assuming large-scale capture images p m There are 2 small images of captured faces and 3 small images of captured bodies, respectively. and Among them, small images of facial capture Human body snapshots Interrelation of face and body images; capture of small face images Unrelated human body snapshot thumbnails, human body snapshot thumbnails Unrelated small images of faces.
[0069] Furthermore, the image clustering device first calculates the face similarity between small face captures in the same large capture image based on face feature vectors, or calculates the human body similarity between small human body captures in the same large capture image based on human body feature vectors; if there are inconsistencies within the same large capture image, then the image similarity, i.e., face similarity and human body similarity calculation, is not performed on the small capture images that cause inconsistencies.
[0070] For details on the process of identifying contradictions within the diagram, please refer to [link / reference needed]. Figure 2 , Figure 2 yes Figure 1The flowchart of the specific sub-steps of step S12 in the image clustering method shown is illustrated.
[0071] Specifically, such as Figure 2 As shown, the image clustering method in this application embodiment specifically includes the following steps:
[0072] Step S121: Obtain several candidate face capture images based on the first large capture image.
[0073] Step S122: Obtain the first candidate similarity between pairs of candidate face capture images from several candidate face capture images.
[0074] Step S123: Cancel the portrait similarity calculation of two candidate face capture images with a first candidate similarity greater than or equal to the first face clustering threshold, and obtain several first face capture images.
[0075] In this embodiment of the application, if there are multiple small images of faces with a face similarity higher than the first face clustering threshold in the large image, or multiple small images of human bodies with a body similarity higher than the first human body clustering threshold, then it is determined that there is an intra-image contradiction in these small images, and similarity calculation is not performed on these small images.
[0076] Step S13: Based on the second large-capture image, obtain several second small-capture images of faces and several second small-capture images of bodies.
[0077] Step S14: Obtain the face similarity of each pair of face capture images and the human body similarity of each pair of human body capture images. Each pair of face capture images includes any first face capture image and any second face capture image, and each pair of human body capture images includes any first human body capture image and any second human body capture image.
[0078] In this embodiment, the image clustering device then calculates the face similarity of each small face capture image between different large capture images based on the face feature vector. If there is a contradiction between two large capture images that satisfies the first contradiction or the second contradiction, the face similarity calculation is not performed on the small capture images of that part.
[0079] The process for determining the contradiction between the first images is as follows: Based on the face similarity of all groups of face capture image pairs, the image clustering device determines whether there exists a face capture image whose face similarity with at least two face capture images in another capture image is greater than or equal to the first face clustering threshold; if so, face similarity calculation is not performed on the face capture images of that part.
[0080] The process for determining the contradiction between the two images is as follows: The image clustering device determines whether there are any face capture images that meet the conditions for contradiction between the two images based on the face similarity of all pairs of face capture images and the human similarity of all pairs of human capture images; if so, the face similarity of the face capture images in that part is not calculated.
[0081] The inter-image contradiction condition is as follows: the face similarity between the first target face capture image and the second target face capture image in another large capture image is greater than or equal to the second face clustering threshold, and the human body similarity between the first target human body capture image associated with the first target face capture image and the second target human body capture image associated with the second target face capture image is less than the second human body clustering threshold, and the human body similarity between the first target human body capture image associated with the first target face capture image and other human body capture images is greater than or equal to the second human body clustering threshold.
[0082] Among them, the threshold for the first face clustering is greater than the threshold for the second face clustering, and the threshold for the first human body clustering is greater than the threshold for the second human body clustering.
[0083] It should be noted that the inconsistencies within the image, between the first images, and between the second images mentioned above refer to either small images captured of faces or small images captured of the human body. The determination of inconsistencies within the image, between the first images, and between the second images for small images captured of the human body will not be elaborated upon here.
[0084] The process for determining human-face contradictions is as follows: The image clustering device determines whether there are any human-face images that meet the human-face contradiction condition based on the face similarity of all pairs of face capture images and the human similarity of all pairs of human-face capture images; if so, the face similarity of that part of the face capture images is not calculated.
[0085] The human-face contradiction condition is as follows: the human body similarity between the small image of the first target human body and the small image of the second target human body in another large image is greater than or equal to the second human body clustering threshold, and the human body similarity between the small image of the first target face associated with the small image of the first target human body and the small image of the second target face associated with the small image of the second target human body is less than the second face clustering threshold.
[0086] Through the above process, the image clustering device will no longer perform similarity calculations on small face and human body images that meet the criteria of intra-image contradiction, inter-image contradiction, inter-image contradiction, and human-face contradiction. In other words, they will no longer be used to calculate the similarity of human images between large images.
[0087] Step S15: Based on the face similarity of all groups of face capture small image pairs and the human body similarity of all groups of human body capture small image pairs, obtain the portrait similarity of the first large capture image and the second large capture image.
[0088] In this embodiment of the application, the image clustering device calculates the face similarity map and the human body similarity map of the small face capture images among different large capture images.
[0089] For example, there exists p i and p j Two large snapshots, one of which is a large snapshot (p). i There are 5 small images of captured faces and 5 small images of captured bodies, respectively. and Among them, small images of facial capture Human body snapshots Face capture thumbnail Human body snapshots Face capture thumbnail Human body snapshots Face capture thumbnail Human body snapshots These are related to each other's faces and bodies, and are small images of captured faces. Unrelated human body snapshot thumbnails, human body snapshot thumbnails Unrelated small facial capture images. Larger capture image (edited) j There are 5 small images of captured faces and 5 small images of captured bodies, respectively. and Among them, small images of facial capture Human body snapshots Face capture thumbnail Human body snapshots Face capture thumbnail Human body snapshots Face capture thumbnail Human body snapshots These are related to each other's faces and bodies, and are small images of captured faces. Unrelated human body snapshot thumbnails, human body snapshot thumbnails Unrelated small images of faces.
[0090] Then, the image clustering device calculates the face capture thumbnails respectively. and The facial similarity between the two faces, assuming the calculated facial similarity scores are as follows:
[0091] The image clustering device selects the two small images of faces with the highest similarity, assuming First face clustering threshold, removing captured small images. Then, take the two remaining small images of faces with the highest similarity, and assume the first face clustering threshold. The second face clustering threshold is used to remove small captured images. Then, take the two remaining small images of faces with the highest similarity, and assume the first face clustering threshold. The second face clustering threshold is used to remove small captured images. Then, take the two remaining small images of faces with the highest similarity, assuming... The second face clustering threshold is then used to terminate the calculation of face capture image similarity. Therefore, p i and p j The facial similarity between the two large snapshots is:
[0092] Similarly, calculate the small image captured of the human body. and The similarity is calculated, and the two most similar human body snapshots are selected. Let's assume... Human body clustering threshold is 1, small captured images are removed. Then, take the two remaining small images of the human body with the highest similarity, assuming... First human body clustering threshold, removing captured small images. Then, take the two remaining small images of the human body with the highest similarity, assuming... First human body clustering threshold, removing captured small images. Then, take the two remaining small human images with the highest similarity and assume the first human clustering threshold. The second human body clustering threshold is used to remove captured small images. Then, take the two remaining small images of the human body with the highest similarity, assuming... The second human body clustering threshold is then used to terminate the similarity calculation of the captured small images of the human body. Therefore, p i and p j The similarity of the human figures in the two large snapshots is:
[0093] As shown above, p i and p j The facial similarity between the two large snapshots is Human body similarity Since each person's face is unique, but the human body can change with clothing, and clustering errors can occur due to multiple people wearing similar clothes, it is necessary to determine whether there are contradictions between the faces and bodies in two large snapshot images.
[0094] Specifically, if the similarity s between two small images of a face is... f If the similarity is greater than or equal to the first face clustering threshold, then by default, the two captured face images are associated with the same person (if associated captured face images exist), and the similarity between the two captured face images is...
[0095] If the first face clustering threshold > s f If the similarity s between the two captured face images is greater than or equal to the second face clustering threshold, then the similarity s between the captured human body images associated with these two captured face images is determined. b (If a related human body snapshot exists, otherwise s) b =0), the similarity between these two portrait snapshots is 0. Where α is a constant greater than 0.5 and less than 1.
[0096] If s f If the second face clustering threshold is exceeded, the similarity between these two face capture images and their associated human body capture images will be discarded.
[0097] In addition, when there are no related face capture pairs in the human body capture pairs, and the human body similarity of the human body capture pairs is greater than or equal to the second human body clustering threshold, the weight of the ratio of human body similarity to the first human body clustering threshold is used as the human image similarity of the human body capture pairs.
[0098] Therefore, p i and p j The similarity between the two candid photos is:
[0099]
[0100] Step S16: When the face similarity is less than the large image similarity threshold, perform image clustering on the face capture small image pairs with face similarity greater than or equal to the first face clustering threshold, and on the human body capture small image pairs with human body similarity greater than or equal to the first human body clustering threshold.
[0101] Step S17: When the face similarity is greater than or equal to the large image similarity threshold, perform image clustering on the face capture small image pairs with face similarity greater than or equal to the second face clustering threshold, and on the human body capture small image pairs with human body similarity greater than or equal to the second human body clustering threshold.
[0102] In this embodiment of the application, if the similarity of the human figures between two large captured images is greater than or equal to the large image similarity threshold, then the two large captured images are considered to be highly similar, that is, there are small captured images of the same human figure in the large captured images, and there is a trajectory accompaniment relationship between multiple human figures.
[0103] Therefore, appropriately increasing the similarity between the small face images in these two large captured images to a threshold higher than the face clustering threshold two, or the similarity between the small human body images to a threshold higher than the human body clustering threshold two, is sufficient. That is, if the original face similarity in these two large images is less than the face clustering threshold one but greater than the face clustering threshold two, then clustering can be performed; similarly, if the human body similarity is less than the human body clustering threshold one but greater than the human body clustering threshold two, then clustering can also be performed. Therefore, p i and p j Image clustering can be performed on the small images of the person captured in the large image (i1 and j1, i2 and j2, i3 and j3, i6 and j6), but if this method is not used, only the small images of the person captured (i1 and j1) can be clustered.
[0104] In this embodiment, the image clustering device acquires a first large-capture image and a second large-capture image; based on the first large-capture image, it acquires several first small-capture images of faces and several small-capture images of bodies; based on the second large-capture image, it acquires several second small-capture images of faces and several small-capture images of bodies; it acquires the face similarity of each pair of face small-capture images and the body similarity of each pair of body small-capture images, wherein each pair of face small-capture images includes any one first small-capture image of a face and any one second small-capture image of a face, and each pair of body small-capture images includes any one first small-capture image of a body and any one second small-capture image of a body; based on the face similarity of all pairs of face small-capture images and the body similarity of all pairs of body small-capture images, it acquires... The image clustering method of this application calculates the facial similarity between the first and second large-scale captured images. When the facial similarity is less than the large-scale image similarity threshold, image clustering is performed on small-scale captured images of faces whose facial similarity is greater than or equal to the first face clustering threshold, and on small-scale captured images of bodies whose human body similarity is greater than or equal to the first body clustering threshold. When the facial similarity is greater than or equal to the large-scale image similarity threshold, image clustering is performed on small-scale captured images of faces whose facial similarity is greater than or equal to the second face clustering threshold, and on small-scale captured images of bodies whose human body similarity is greater than or equal to the second body clustering threshold. The first face clustering threshold is greater than the second face clustering threshold, and the first body clustering threshold is greater than the second body clustering threshold. This image clustering method can adjust the similarity of small-scale captured facial and body images based on the similarity of the large-scale captured images, thereby improving the recall rate of image clustering. The image clustering method in this application also proposes the concepts of intra-image contradiction, inter-image contradiction, and face / body contradiction to improve the accuracy of image clustering; based on the trajectory association relationship of multiple portraits in a large captured image, the accuracy and recall of image clustering are improved.
[0105] The above embodiments are merely one common example of this application and do not constitute any limitation on the technical scope of this application. Therefore, any minor modifications, equivalent changes, or alterations made to the above content based on the substance of the solution of this application shall still fall within the scope of the technical solution of this application.
[0106] Please continue reading Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the image clustering device provided in this application. The image clustering device 300 of this application embodiment includes a processor 31, a memory 32, an input / output device 33, and a bus 34.
[0107] The processor 31, memory 32, and input / output device 33 are respectively connected to the bus 34. The memory 32 stores program data, and the processor 31 is used to execute the program data to implement the image clustering method described in the above embodiments.
[0108] In this embodiment, processor 31 can also be referred to as a CPU (Central Processing Unit). Processor 31 may be an integrated circuit chip with signal processing capabilities. Processor 31 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 31 can be any conventional processor.
[0109] This application also provides a computer storage medium; please refer to the following: Figure 4 , Figure 4 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 400 stores program data 41, which is used to implement the image clustering method of the above embodiment when executed by the processor.
[0110] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored on a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored on a storage medium and includes several instructions to cause a computer device (which may be a human-computer interface, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0111] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. An image clustering method, characterized in that, The image clustering method includes: Obtain the first and second large-scale images captured; Based on the first captured large image, obtain several first face captured small images and several first body captured small images; Based on the second large image, obtain several small images of second faces and several small images of second bodies; Obtain the face similarity of each pair of face capture images and the human body similarity of each pair of human body capture images. Each pair of face capture images includes any first face capture image and any second face capture image, and each pair of human body capture images includes any first human body capture image and any second human body capture image. Based on the face similarity of all pairs of face capture images and the human body similarity of all pairs of human body capture images, the portrait similarity of the first large capture image and the second large capture image is obtained. When the portrait similarity is less than the large image similarity threshold, image clustering is performed on the face capture small images of the face capture small image pair whose face similarity is greater than or equal to the first face clustering threshold, and on the human body capture small image pair whose human body similarity is greater than or equal to the first human body clustering threshold. When the face similarity is greater than or equal to the large image similarity threshold, image clustering is performed on the face capture small images of the face capture small image pair whose face similarity is greater than or equal to the second face clustering threshold, and on the human body capture small image pair whose human body similarity is greater than or equal to the second human body clustering threshold. Wherein, the first face clustering threshold is greater than the second face clustering threshold, and the first human body clustering threshold is greater than the second human body clustering threshold.
2. The image clustering method according to claim 1, characterized in that, The step of obtaining the portrait similarity between the first large image and the second large image based on the face similarity of all pairs of small face images and the human similarity of all pairs of small human images includes: When the face similarity of the face capture small image pair is greater than or equal to the first face clustering threshold, the ratio of the face similarity to the first face clustering threshold is used as the portrait similarity of the face capture small image pair and its associated human body capture small image pair. When the face similarity of the face capture small image pair is less than the first face clustering threshold and greater than or equal to the second face clustering threshold, the weighted sum of the ratio of the face similarity to the first face clustering threshold and the ratio of the human body similarity of the human body capture small image pair associated with the face capture small image pair to the first human body clustering threshold is taken as the human image similarity of the face capture small image pair. When the facial similarity of the face capture image pair is less than the second face clustering threshold, the calculation of the facial similarity of the face capture image and its associated human body capture image is cancelled. The sum of the portrait similarity scores of all the captured small image pairs is used as the portrait similarity score between the first captured large image and the second captured large image.
3. The image clustering method according to claim 2, characterized in that, The image clustering method further includes: When the face capture image pair has no associated human body capture image pair, the human body capture image pair with zero human body similarity is associated with the face capture image pair.
4. The image clustering method according to claim 2, characterized in that, The step of summing the portrait similarity scores of all the captured small image pairs as the portrait similarity score between the first captured large image and the second captured large image includes: When there are no associated face capture pairs in the human body capture image pairs, and the human body similarity of the human body capture image pairs is greater than or equal to the second human body clustering threshold, the weight of the ratio of the human body similarity to the first human body clustering threshold is used as the human image similarity of the human body capture image pairs. The sum of the portrait similarity scores of all the face capture pairs and the portrait similarity scores of all the human body capture pairs is used as the portrait similarity score of the first large capture image and the second large capture image.
5. The image clustering method according to claim 1, characterized in that, The process of obtaining several small first face capture images based on the first large capture image includes: Based on the first large captured image, obtain several small captured images of candidate faces; Obtain the first candidate similarity between any two candidate face capture images from the plurality of candidate face capture images; The similarity calculation of the faces of two candidate face capture images with a similarity greater than or equal to the first face clustering threshold is cancelled, and the plurality of first face capture images are obtained.
6. The image clustering method according to claim 1, characterized in that, After obtaining the face similarity of each pair of captured face images, the image clustering method further includes: Based on the face similarity of all the face capture small image pairs, determine whether there exists a face capture small image whose face similarity with at least two face capture small images in another large capture image is greater than or equal to the first face clustering threshold. If it exists, cancel the facial similarity calculation for that small captured face image.
7. The image clustering method according to claim 1, characterized in that, After obtaining the face similarity of each pair of captured face images and the body similarity of each pair of captured body images, the image clustering method further includes: Based on the face similarity of all pairs of face capture images and the human similarity of all pairs of human body capture images, determine whether there are any face capture images that meet the inter-image contradiction condition. If it exists, cancel the facial similarity calculation for that small captured face image; The inter-image contradiction condition is as follows: the face similarity between the first target face capture image and the second target face capture image in another large capture image is greater than or equal to the second face clustering threshold; the human body similarity between the first target human body capture image associated with the first target face capture image and the second target human body capture image associated with the second target face capture image is less than the second human body clustering threshold; and the human body similarity between the first target human body capture image associated with the first target face capture image and other human body capture images is greater than or equal to the second human body clustering threshold.
8. The image clustering method according to claim 1, characterized in that, After obtaining the face similarity of each pair of captured face images and the body similarity of each pair of captured body images, the image clustering method further includes: Based on the face similarity of all pairs of face capture images and the human body similarity of all pairs of human body capture images, determine whether there are any human body capture images that meet the contradictory conditions of human body and face. If it exists, cancel the facial similarity calculation for that human body snapshot. The human face contradiction condition is as follows: the human body similarity between the first target human body capture small image and the second target human body capture small image in another large image is greater than or equal to the second human body clustering threshold, and the human body similarity between the first target face capture small image associated with the first target human body capture small image and the second target face capture small image associated with the second target human body capture small image is less than the second face clustering threshold.
9. An image clustering device, characterized in that, The image clustering device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the image clustering method as described in any one of claims 1 to 8.
10. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the image clustering method as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Portrait clustering method and device and storage medium
CN112818867A