Face clustering method, electronic device, and storage medium

Abstract

The application discloses a face clustering method, an electronic device and a computer readable storage medium. The method comprises the following steps: obtaining a to-be-merged class cluster; determining a mean centroid and at least two recommended centroids of the to-be-merged class cluster, the mean centroid is used for representing all face images in the to-be-merged class cluster, each recommended centroid is used for representing face images in different sub-class clusters of the to-be-merged class cluster, and a first similarity between different face images in a same sub-class cluster is less than a first similarity threshold; and determining whether to merge the to-be-merged class cluster with a historical class cluster based on a second similarity about the mean centroid and a third similarity about the recommended centroid between the to-be-merged class cluster and the historical class cluster. In this way, the face clustering effect can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a face clustering method, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Face clustering plays an important role in the field of intelligent security. The process of face clustering is to cluster face images in an incremental time period to obtain a to-be-merged class cluster, and then merge the to-be-merged class cluster with a historical class cluster. The goal of face clustering is to cluster face images belonging to the same person into the same class cluster, so as to realize the case of one person one file. However, the existing face clustering method has poor effect on face clustering, and the accuracy and recall rate of the face clustering result are not high enough, and the cases of wrong file and multiple files are prone to occur. SUMMARY

[0003] The present application provides a face clustering method, an electronic device and a computer readable storage medium, which can solve the problem of poor effect of the existing face clustering method on face clustering.

[0004] To solve the above technical problems, one technical solution adopted by the present application is to provide a face clustering method. The method comprises: obtaining a to-be-merged class cluster; determining a mean centroid and at least two recommended centroids of the to-be-merged class cluster, the mean centroid being used to represent all face images in the to-be-merged class cluster, and each recommended centroid being used to represent face images in a different sub-class cluster of the to-be-merged class cluster, the first similarity between different face images in the same sub-class cluster being less than a first similarity threshold; and determining whether to merge the to-be-merged class cluster with a historical class cluster based on a second similarity about the mean centroid and a third similarity about the recommended centroid between the to-be-merged class cluster and the historical class cluster.

[0005] To solve the above technical problems, another technical solution adopted by the present application is to provide an electronic device, which comprises a processor and a memory connected with the processor, wherein the memory stores program instructions; and the processor is configured to execute the program instructions stored in the memory to implement the above method.

[0006] To solve the above technical problems, still another technical solution adopted by the present application is to provide a computer readable storage medium, which stores program instructions, and the program instructions can implement the above method when executed.

[0007] In the above manner, the second similarity between the historical class cluster and the class cluster to be merged in terms of the mean centroid is not only used as the basis for determining whether to merge, but the second similarity between the historical class cluster and the class cluster to be merged in terms of the mean centroid and the third similarity between the recommended centroid and the class cluster to be merged are used as the basis for determining whether to merge together, and the recommended centroid can represent the face image in the sub-class cluster of the class cluster to be merged. Therefore, the method provided by the present application can improve the face clustering effect. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a flowchart of an embodiment of the face clustering method of the present application;

[0009] Figure 2 is Figure 1 is a specific flowchart of S12 in

[0010] Figure 3 is a schematic diagram of the face image in the class cluster to be merged in the feature space;

[0011] Figure 4 is Figure 1 is a specific flowchart of S13 in

[0012] Figure 5 is a flowchart of another embodiment of the face clustering method of the present application;

[0013] Figure 6 is a schematic diagram of a specific example of the face clustering method of the present application;

[0014] Figure 7 is a flowchart of still another embodiment of the face clustering method of the present application;

[0015] Figure 8 is a schematic diagram of the node in the feature space;

[0016] Figure 9 is a schematic diagram of still another specific example of the face clustering method of the present application;

[0017] Figure 10 is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0018] Figure 11 is a structural schematic diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0019] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the scope of the present application.

[0020] The terms "first", "second", "third" in the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", "third" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0021] In this document, the term "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean that the same embodiment is referred to, nor does it mean that independent or alternative embodiments are mutually exclusive or alternative to each other. A person of ordinary skill in the art explicitly and implicitly understands that the embodiments described herein can be combined with other embodiments without conflict.

[0022] Figure 1 is a flowchart of an embodiment of the face clustering method of the present application. It should be noted that the present embodiment is not limited to the order of the flowchart shown in Figure 1 . As shown in Figure 1 , the present embodiment can include:

[0023] S11: Obtain a to-be-merged cluster.

[0024] The to-be-merged cluster is a cluster of face images in a to-be-clustered time period (also referred to as an incremental time period). The to-be-merged cluster is obtained by clustering face images in the to-be-clustered time period.

[0025] As an implementation manner, a set of face images in the to-be-clustered time period can be obtained, and the set of face images is clustered to obtain a plurality of to-be-merged clusters.

[0026] As another implementation manner, a set of face images in the to-be-clustered time period can be obtained, and the set of face images is divided into a plurality of subsets of face images, and each subset of face images is clustered into a plurality of to-be-merged clusters. Among them, the set of face images can be divided into a plurality of subsets of face images according to a similar spatiotemporal domain, that is, face images with similar shooting time and similar shooting space (position) are divided into the same subset of face images. Of course, the set of face images can also be divided into a plurality of subsets of face images according to a similar spatial domain, etc.

[0027] In addition, the set of face images can be cleaned and de-waste before clustering the set of face images into multiple subsets of face images.

[0028] S12: determining a mean centroid of the to-be-merged class cluster and at least two recommended centroids.

[0029] The mean centroid is used to represent all face images in the to-be-merged class cluster, and each recommended centroid is used to represent face images in a different sub-class cluster of the to-be-merged class cluster. The first similarity between different face images in the same sub-class cluster is less than the first similarity threshold.

[0030] The similarity between the face images / centroids mentioned in the present application is the similarity of the face images / centroids in the feature space, that is, the similarity of the features of the face images / centroids.

[0031] The mean centroid can be obtained by weighted average calculation of the face images in the to-be-merged class cluster. Specifically, the weights of the face images in the to-be-merged class cluster can be determined; and the face images in the to-be-merged class cluster are weighted and averaged according to the corresponding weights to obtain the mean centroid of the to-be-merged class cluster.

[0032] The face images that actually participate in the calculation of the mean centroid can be all face images in the to-be-merged class cluster. However, in order to improve the accuracy of the mean centroid, the face images that actually participate in the calculation of the mean centroid can be part of the face images in the to-be-merged class cluster. For example, the quality score of each face image is obtained, and the face images with a quality score higher than a score threshold are used to calculate the mean centroid, wherein the quality score is determined by factors such as the occlusion of the face in the face image, the corresponding light intensity of the face image, the resolution, etc. For another example, the face images other than outliers / isolated points are used to calculate the mean centroid, wherein the outliers / isolated points are face images with a first similarity to other face images that is too large.

[0033] In the weighted average calculation of the mean centroid, the weight of the face image that does not actually participate in the calculation is 0, and the weight of the different face images that actually participate in the calculation is greater than 0. Moreover, the weights of the different face images that actually participate in the calculation can be the same or different. For example, the weights of the different face images that actually participate in the calculation are all 1. For another example, the higher the quality score of the different face images that actually participate in the calculation, the higher the weight.

[0034] The calculation formula for calculating the mean centroid by weighted average can be as follows:

[0035] ;

[0036] wherein, represents the mean centroid, represents the weight of the kth face image, represents the kth face image.

[0037] All the recommended centroids can collectively represent all the face images in the to-be-merged class cluster. In other words, the distribution of the face images in the to-be-merged class cluster in the feature space (feature distribution) can be comprehensively reflected by all the recommended centroids. The to-be-merged class cluster can be divided into at least two sub-class clusters based on the first similarity of different face images in the to-be-merged class cluster in the feature space, and one recommended centroid can be selected from each sub-class cluster.

[0038] For better understanding, refer to Figure 2 The obtaining of the recommended centroids in S12 can include the following sub-steps:

[0039] S121: Obtain the first similarity between each pair of different face images in the to-be-merged class cluster.

[0040] S122: Divide the to-be-merged class cluster into at least two sub-class clusters based on the first similarity.

[0041] The number of sub-class clusters is the same as the number of recommended centroids. The number of recommended centroids can be a fixed value. For example, the number of recommended centroids is 3, 5, etc.

[0042] Two face images corresponding to a first similarity less than a first similarity threshold value are divided into the same sub-class cluster. The first similarity threshold value is negatively correlated with the number of recommended centroids.

[0043] S123: Select one face image from each sub-class cluster as a recommended centroid of the to-be-merged class cluster.

[0044] The way of selecting the recommended centroid from the sub-class cluster can be the same as or different from the way of determining the mean centroid of the to-be-merged class cluster. For example, a weighted average of all face images included in the sub-class cluster can be performed to obtain the recommended centroid corresponding to the sub-class cluster. For another example, a face image with a high quality score can be selected from the sub-class cluster as the recommended centroid corresponding to the sub-class cluster. For another example, a face image with the minimum average similarity to other face images in the sub-class cluster can be selected as the recommended centroid.

[0045] The above S121-S123 can be regarded as a process of selecting recommended centroids for the to-be-merged class cluster by using graph cutting. In the following, taking the selection of three recommended centroids by using graph cutting as an example for description: Figure 3

[0046] Figure 3 is a schematic diagram of the face images in the to-be-merged class cluster in the feature space (distribution of the features of the face images in the to-be-merged class cluster). As shown in Figure 3 ​As shown, the to-be-merged class cluster includes face images a1-a3, b1-b2, and c1-c4. The to-be-merged class cluster is divided into three sub-class clusters 1-3. The A region represents sub-class cluster 1, which includes three face images (a1-a3). The B region represents sub-class cluster 2, which includes two face images (b1-b2). The C region represents sub-class cluster 3, which includes four face images (c1-c4). In the A region, a2 is selected as the recommended centroid. In the B region, b1 is selected as the recommended centroid. In the C region, c2 is selected as the recommended centroid.

[0047] S13: Based on the second similarity between the to-be-merged class cluster and the historical class cluster with respect to the mean centroid and the third similarity with respect to the recommended centroid, it is determined whether to merge the to-be-merged class cluster with the historical class cluster.

[0048] In the process of the first initialization of the historical class cluster, the mean centroid is calculated in the same way as the mean centroid of the to-be-merged class cluster. For the calculation of the mean centroid in the subsequent process, please refer to the calculation of the mean centroid of the target class cluster. In the process of the first initialization of the historical class cluster, a recommended centroid is preset. For example, the image with the highest quality score in the historical class cluster is taken as the recommended centroid of the historical class cluster. For the calculation of the recommended centroid in the subsequent process, please refer to the calculation of the recommended centroid of the target class cluster.

[0049] There are multiple third similarities between the to-be-merged class cluster and the historical class cluster. The third similarity mentioned later in this application is greater than the third similarity threshold, which can be that all the multiple third similarities are greater than the third similarity threshold, or that the mean of the multiple third similarities is greater than the third similarity threshold, and so on.

[0050] As an implementation manner, the second similarity and the third similarity can be directly calculated in S13. If the second similarity is greater than the second similarity threshold and the third similarity is greater than the third similarity threshold, it is determined to merge the to-be-merged class cluster with the historical class cluster. Otherwise, it is determined not to merge the to-be-merged class cluster with the historical class cluster. It can be understood that, in this implementation manner, the second similarity being greater than the second similarity threshold means that the to-be-merged class cluster meets the condition of merging with the historical class cluster under the restriction of the second similarity; the third similarity being greater than the third similarity threshold means that the to-be-merged class cluster meets the condition of merging with the historical class cluster under the restriction of the third similarity. Therefore, compared with the manner of determining to merge the to-be-merged class cluster with the historical class cluster only when the second similarity is greater than the second similarity threshold, this implementation manner can improve the effect of face clustering.

[0051] As another implementation manner, the second similarity and the third similarity can be directly calculated; the second similarity is adjusted by using the third similarity; it is judged whether the adjusted second similarity is greater than the second similarity threshold; if the adjusted second similarity is greater than the second similarity threshold, it is determined that the to-be-merged class cluster and the historical class cluster are merged, otherwise it is determined that the to-be-merged class cluster is not merged.

[0052] The adjustment manner of the second similarity by using the third similarity can be that if the third similarity is greater than the third similarity threshold, the second similarity is increased by a first preset ratio; if the third similarity is equal to the third similarity threshold, the second similarity is kept unchanged; if the third similarity is less than the third similarity threshold, the second similarity is decreased by a second preset ratio. The first preset ratio / second preset ratio can be fixed. Alternatively, the first preset ratio / second preset ratio can be set according to the difference between the third similarity and the second similarity. For example, the first preset ratio / second preset ratio is positively correlated with the difference. The third similarity threshold can be equal to the second similarity, and the third similarity threshold can also be other values.

[0053] It can be understood that in this implementation manner, in the case that the second similarity is greater than the similarity threshold, the to-be-merged class cluster and the historical class cluster are not directly merged; instead, the third similarity is calculated again, and the second similarity is adjusted by using the third similarity; in the case that the adjusted second similarity is greater than the similarity threshold, the to-be-merged class cluster and the historical class cluster are merged. Therefore, this implementation manner can determine whether the to-be-merged class cluster and the historical class cluster are merged by taking the second similarity as the main basis and the third similarity as the auxiliary basis, and can improve the effect of face clustering.

[0054] For reference Figure 4 As another implementation manner, S13 can include the following sub-steps:

[0055] S131: Calculate the second similarity between the to-be-merged class cluster and the historical class cluster about the mean centroid.

[0056] S132: Judge whether the second similarity is greater than the second similarity threshold.

[0057] It can be understood that the second similarity being greater than the second similarity threshold means that the to-be-merged class cluster meets the condition of being merged with the historical class cluster under the limitation of the second similarity. On the contrary, the second similarity being not greater than the second similarity threshold means that the to-be-merged class cluster does not meet the condition of being merged with the historical class cluster under the limitation of the second similarity.

[0058] If yes, S133 is executed; otherwise, it is determined that the to-be-merged class cluster and the historical class cluster are not merged.

[0059] S133: Calculate a third similarity between the to-be-merged class cluster and the historical class cluster with respect to the recommended centroid.

[0060] S134: Adjust the second similarity by using the third similarity.

[0061] If the third similarity is greater than the third similarity threshold, the second similarity is adjusted to be larger; if the third similarity is not greater than the third similarity threshold, the second similarity is adjusted to be smaller. For details of the adjustment manner, refer to the description of the foregoing implementation manner, which is not described herein.

[0062] S135: Determine whether to merge the to-be-merged class cluster and the historical class cluster based on the adjusted second similarity.

[0063] If the adjusted second similarity is still greater than the second similarity threshold, it is determined to merge the to-be-merged class cluster and the historical class cluster; otherwise, it is determined not to merge the to-be-merged class cluster and the historical class cluster.

[0064] It can be understood that if only the second similarity is used as the basis to determine whether to merge the to-be-merged class cluster and the historical class cluster, the situation of false merging and missing merging is likely to occur, resulting in poor clustering effect. The situation of missing merging refers to that the to-be-merged class cluster and the historical class cluster actually correspond to the same person, but the determination result is that the to-be-merged class cluster and the historical class cluster are not merged. The situation of false merging refers to that the to-be-merged class cluster and the historical class cluster actually correspond to different persons, but the determination result is that the to-be-merged class cluster and the historical class cluster are merged (false merging).

[0065] Through the implementation of the embodiment, the present application is not only based on the second similarity between the historical class cluster and the to-be-merged class cluster with respect to the mean centroid as the basis to determine whether to merge, but also based on the second similarity between the historical class cluster and the to-be-merged class cluster with respect to the mean centroid and the third similarity between the historical class cluster and the to-be-merged class cluster with respect to the recommended centroid as the basis to determine whether to merge, and the recommended centroid can represent the face image in the sub-class cluster of the to-be-merged class cluster. Therefore, the method provided by the present application can improve the effect of face clustering.

[0066] In addition, if it is determined to merge the to-be-merged class cluster and the historical class cluster, the mean centroid and the recommended centroid of the target class cluster obtained after merging need to be further obtained. The target class cluster serves as a new historical class cluster, and subsequent new face clustering tasks are performed on the basis of the mean centroid and the recommended centroid of the new historical class cluster.

[0067] The following lists several ways to obtain the mean centroid of the target class cluster:

[0068] In the first mode, the mean centroid of the target cluster can be directly obtained by weighted average of the mean centroids of the to-be-merged cluster and the historical cluster. The weights of the mean centroids of the to-be-merged cluster and the historical cluster can be the same. Alternatively, the weight of the mean centroid of the historical cluster can be set to be greater than the weight of the mean centroid of the to-be-merged cluster, so as to reduce the probability of mean centroid drift.

[0069] In the second mode, the mean centroid of the target cluster can be determined based on the sum of the number of face images corresponding to the to-be-merged cluster and the historical cluster. For details, refer to Figure 5 In this mode, the mean centroid of the target cluster can be determined by the following steps:

[0070] S21: Determine whether the sum of the number of face images corresponding to the to-be-merged cluster and the historical cluster is greater than a number threshold.

[0071] If the sum is greater than the number threshold, perform S22; otherwise, perform S23.

[0072] S22: Take the mean centroid of the cluster corresponding to the face image with a greater number of face images as the mean centroid of the target cluster.

[0073] S23: Perform weighted average of the mean centroids of the to-be-merged cluster and the historical cluster by taking the number of corresponding face images as the weight, to obtain the mean centroid of the target cluster.

[0074] It can be understood that in the case of greater than the number threshold, if the weighted average result of the mean centroids corresponding to the to-be-merged cluster and the historical cluster is directly taken as the mean centroid of the target cluster, the mean centroid may drift in the iteration process, affecting the accuracy of the subsequent new clustering task. Therefore, in this step, the weighted average result is only taken as the mean centroid of the target cluster in the case of not greater than the number threshold. Thus, the probability of mean centroid drift can be reduced, and the reliability of the mean centroid can be improved.

[0075] In the third mode, the weight of the face image when calculating the mean centroid of the cluster is taken as the weight of the face image in this step. The first weight sum of the face image corresponding to the to-be-merged cluster can be obtained, and the second weight sum of the face image corresponding to the historical cluster can be obtained; the first weight sum and the second weight sum are added to obtain a third weight sum.

[0076] It can be understood that the greater the third weight sum, the higher the probability of drift of the mean centroid of the target cluster obtained by weighted average. Therefore, the mean centroid of the target cluster can be obtained by the following method:

[0077] determining whether the third weight sum is greater than the weight threshold value; if yes, taking the mean value centroid of the class cluster corresponding to the greater one of the first weight sum and the second weight sum as the mean value centroid of the target class cluster, and taking the greater one of the first weight sum and the second weight sum as the weight sum of the target class cluster; otherwise, performing weighted averaging on the mean value centroids of the class cluster to be merged and the historical class cluster to obtain the mean value centroid of the target class cluster according to the corresponding weight sum, and adding the first weight sum and the second weight sum to obtain the weight sum of the target class cluster.

[0078] The calculation formula of the first weight sum can be:

[0079] ,

[0080] wherein wgt1 represents the first weight, and wgtk represents the weight of the kth face image in the class cluster to be merged.

[0081] The second weight sum in the initialization process can be the weight sum of the face images in the historical class cluster. The second weight sum in the non-initialization process can be obtained in the manner as described below for obtaining the weight sum of the target class cluster.

[0082] In combination with Figure 6 For example, the mean value centroid of the class cluster to be merged is set as ave1, and the first weight sum is set as wgt1. The mean value centroid of the historical class cluster is set as ave2, and the second weight sum is set as wgt2. The mean value centroid of the target class cluster is set as ave, and the weight sum is set as wgt. The weight threshold value is set as HW.

[0083] If wgt1+wgt2>HW, then the greater one of wgt1 and wgt2 is determined as wgt, and the mean value centroid corresponding to the greater one of wgt1 and wgt2 is determined as ave; otherwise, ave and wgt are calculated by the following formula:

[0084] ,

[0085] .

[0086] It can be understood that, for the third manner, if the weight of the face image that actually participates in the mean value centroid calculation is determined according to the quality score, then the greater the third weight sum is, the higher the overall quality level of the face images in the class cluster to be merged and the historical class cluster is. Therefore, it can be understood that, based on the overall quality level of the face images in the class cluster to be merged and the historical class cluster, the mean value centroid of the target class cluster is determined. For example, if the overall quality level is greater than a quality level threshold value, then the mean value centroids of the class cluster to be merged and the historical class cluster are weighted and averaged to obtain the mean value centroid of the target class cluster; otherwise, the mean value centroid of the class cluster corresponding to the greater one of the first weight sum and the second weight sum is taken as the mean value centroid of the target class cluster.

[0087] If the weight of the face image that truly participates in the mean centroid calculation is 1, the third weight is greater, which means that the sum of the number of face images that truly participate in the mean centroid calculation is greater, and then this implementation mode can be understood as that the mean centroid of the target class cluster is determined based on the sum of the number of face images that truly participate in the mean centroid calculation corresponding to the to-be-merged class cluster and the historical class cluster. In this case, mode three is equivalent to mode two described above.

[0088] The following lists several ways to obtain the recommended centroid of the target class cluster:

[0089] The number of recommended centroids of the target class cluster can be the same as or different from the number of recommended centroids of the to-be-merged class cluster.

[0090] Mode four is that the recommended centroid of the target class cluster is determined in the same way as the mean centroid of the to-be-merged class cluster. When the number of face images in the target class cluster is large, the calculation amount is large in this mode.

[0091] Mode five is that the recommended centroid of the target class cluster is determined based on the recommended centroid of the to-be-merged class cluster and the historical class cluster. For details, refer to Figure 7 This mode can determine the recommended centroid of the target class cluster through the following steps:

[0092] S31: Take all the recommended centroids of the to-be-merged class cluster and the historical class cluster as nodes.

[0093] For example, the number of recommended centroids of the to-be-merged class cluster is 3, and the number of recommended centroids of the historical class cluster is also 3, so the number of nodes is 6.

[0094] S32: Calculate the fourth similarity between different nodes.

[0095] For example, the fourth similarity between different nodes in the 6 nodes is calculated.

[0096] S33: According to the fourth similarity from small to large, connect the corresponding two nodes until the number of node connected graphs obtained is equal to the number of recommended centroids.

[0097] For details, refer to Figure 8 For details, refer to Figure 8 is a schematic diagram of the feature space of the node. Figure 8 In the figure, the fourth similarity between nodes 1 and 2 is the smallest, nodes 1 and 2 are connected, and 4 node connected graphs are obtained, which are 1-2, 3, 4, and 5; after the first connection, the fourth similarity between nodes 2 and 3 is the smallest, nodes 2 and 3 are connected, and 3 node connected graphs are obtained, which are 1-2-3, 4, and 5.

[0098] S34: selecting a node from each node-connected graph as a recommended centroid of the target class cluster.

[0099] It can be understood that the nodes with the highest similarity can be connected to form a node-connected subgraph through S31-S34. Thus, the node similarity between different node-connected subgraphs is the lowest, and the node similarity within the same node-connected subgraph is the highest. Furthermore, the recommended centroid selected from different node-connected subgraphs can be used to fully represent the face images of different sub-class clusters of the target class cluster. Therefore, when the target class cluster (new historical class cluster) is applied to a new face clustering task, the face clustering effect can be improved.

[0100] Hereinafter, the process of S31-S34 is referred to as selecting a recommended centroid for the target class cluster by using a connected iteration method.

[0101] The method provided by the present application will be described in detail in the form of a specific example as follows: Figure 9

[0102] S1: local clustering to form a class cluster: dividing face data (a face image set) into multiple face image subsets, clustering each face image subset, and forming multiple to-be-merged class clusters.

[0103] S2: calculating a mean centroid of the to-be-merged class cluster, and selecting a recommended centroid of the to-be-merged class cluster by using a graph-cut method.

[0104] S3: increment and historical class cluster merging: based on a first similarity between the to-be-merged class cluster and the historical class cluster with respect to the mean centroid, and a second similarity with respect to the recommended centroid.

[0105] S4: jointly determining whether to merge the to-be-merged class cluster and the historical class cluster based on the first similarity and the second similarity.

[0106] S5-S6: if a target class cluster is obtained by merging, updating a mean centroid of the target class cluster, and selecting a recommended centroid for the target class cluster by using a connected iteration method.

[0107] Figure 10 Fig. 1 is a structural schematic diagram of an embodiment of an electronic device according to the present application. As shown in the figure, the electronic device includes a processor 21 and a memory 22 coupled to the processor 21. Figure 10

[0108] ​​The memory 22 stores program instructions for implementing the method of any of the above embodiments. The processor 21 is configured to execute the program instructions stored in the memory 22 to implement the steps of the above method embodiments. The processor 21 can also be referred to as a CPU (Central Processing Unit). The processor 21 can be an integrated circuit chip having a processing capability. The processor 21 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0109] Figure 11 is a structural schematic diagram of an embodiment of the computer readable storage medium of the present application. As shown in Figure 11 the computer readable storage medium 30 of the embodiment of the present application stores program instructions 31, which, when executed, implement the method provided by the above embodiments of the present application. The program instructions 31 can form a program file and be stored in the above computer readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor executes all or part of the steps of the method of each embodiment of the present application. The aforementioned computer readable storage medium 30 includes a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, etc. various media that can store program codes, or a computer, a server, a mobile phone, a tablet, etc. terminal device.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0111] In addition, the various functional units in the embodiments of the present application can be integrated in one processing unit, or each can exist physically as a separate unit, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware, or in the form of a software functional unit. The above is only an implementation of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A face clustering method, characterized in that, The method comprises the following steps: obtaining a to-be-merged class cluster, the to-be-merged class cluster being obtained by clustering face images in a to-be-clustered time period; determining a mean centroid and at least two recommended centroids of the to-be-merged class cluster, the mean centroid being used to represent all face images in the to-be-merged class cluster, each recommended centroid being used to represent face images in a different sub-class cluster of the to-be-merged class cluster, a first similarity between different face images in the same sub-class cluster being less than a first similarity threshold; determining whether to merge the to-be-merged class cluster with a historical class cluster based on a second similarity between the to-be-merged class cluster and the historical class cluster with respect to the mean centroid and a third similarity between the to-be-merged class cluster and the historical class cluster with respect to the recommended centroid, comprising: calculating the second similarity between the to-be-merged class cluster and the historical class cluster with respect to the mean centroid; if the second similarity is greater than a second similarity threshold, calculating the third similarity between the to-be-merged class cluster and the historical class cluster with respect to the recommended centroid; adjusting the second similarity by using the third similarity; determining whether to merge the to-be-merged class cluster with the historical class cluster based on the adjusted second similarity; or, in response to the second similarity being greater than the second similarity threshold and the third similarity being greater than a third similarity threshold, determining to merge the to-be-merged class cluster with the historical class cluster; wherein the determination of the at least two recommended centroids of the to-be-merged class cluster comprises: obtaining the first similarity between each pair of different face images in the to-be-merged class cluster; dividing the to-be-merged class cluster into at least two sub-class clusters based on the first similarity; and selecting one face image from each sub-class cluster as a recommended centroid of the to-be-merged class cluster.

2. The method of claim 1, wherein, The adjustment of the second similarity by using the third similarity comprises: if the third similarity is greater than the third similarity threshold, increasing the second similarity by a first preset proportion; if the third similarity is not greater than the third similarity threshold, decreasing the second similarity by a second preset proportion.

3. The method of claim 1, wherein, The method further comprises: if it is determined to merge the to-be-merged class cluster with the historical class cluster, obtaining a mean centroid and a recommended centroid of a target class cluster obtained after the merging.

4. The method of claim 3, wherein the obtaining of the mean centroid of the target class cluster comprises: if a sum of the number of face images corresponding to the to-be-merged class cluster and the historical class cluster is greater than a number threshold, taking a mean centroid of the one with a greater number of face images corresponding to the to-be-merged class cluster and the historical class cluster as the mean centroid of the target class cluster; if the sum of the number of face images corresponding to the to-be-merged class cluster and the historical class cluster is not greater than the number threshold, performing weighted averaging on the mean centroids of the to-be-merged class cluster and the historical class cluster with the number of face images corresponding to the to-be-merged class cluster and the historical class cluster as weights to obtain the mean centroid of the target class cluster.

5. The method of claim 3, wherein the obtaining of the recommended centroid of the target class cluster comprises: taking all recommended centroids of the to-be-merged class cluster and the historical class cluster as nodes; ​ ​ calculate a fourth similarity between each pair of the nodes; connect two nodes corresponding to the fourth similarity in ascending order until the number of connected graphs of the nodes is equal to the number of recommended centroids; select a node from each connected graph of the nodes as a recommended centroid of the target class cluster.

6. The method of claim 1, wherein the determining the mean centroid of the class cluster to be merged comprises: determining a weight of each face image in the class cluster to be merged; performing a weighted average of the face images in the class cluster to be merged according to the corresponding weights to obtain the mean centroid of the class cluster to be merged.

7. The method of claim 1, wherein the obtaining the class cluster to be merged comprises: obtaining a set of face images in a time period to be clustered; dividing the set of face images into a plurality of subsets of face images; clustering each subset of face images into a plurality of class clusters to be merged. A device comprising a processor and a memory connected to the processor, wherein the memory stores program instructions; and the processor is configured to execute the program instructions stored in the memory to implement the method of any one of claims 1-7. A storage medium storing program instructions, which when executed, implement the method of any one of claims 1-7.

8. An electronic device, comprising: A storage medium storing program instructions, which when executed, implement the method of any one of claims 1-7. ​ ​ 9. A computer-readable storage medium, characterized in that, ​

Images