Facial image clustering method, device, electronic device, and storage medium

Abstract

This application provides a method, an apparatus, an electronic device, and a storage medium for face image clustering, which includes: using a face recognition model to perform feature extraction on samples in a face dataset to obtain corresponding features for each sample; calculating cosine distances between features and constructing a connection graph including all samples; searching the connection graph based on connected components to obtain low-level subgraphs, aggregating the low-level subgraphs to obtain a first candidate cluster; calculating quality scores and overlap scores for the first candidate cluster, filtering the first candidate cluster according to the quality scores, and obtaining a second candidate cluster; using a graph convolutional neural network to output a probability value for each vertex in the second candidate cluster, and removing noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; and searching for shared vertices between each of the other third candidate clusters and a benchmark cluster, and removing the shared vertices.

Description

[Technical Field]

[0001] The present application relates to the field of face recognition technology, and in particular to a method, apparatus, electronic device and storage medium for face image clustering. [Background technology]

[0002] With the development of facial recognition technology, a lot of data is required to train and obtain a highly accurate model. Although it is relatively easy to collect this data, labeling it requires a lot of effort. Therefore, it is considered to use unlabeled data to improve the effectiveness of facial recognition. An intuitive way is to cluster the unlabeled data to generate pseudo-labels, and then directly input these data into a supervised model.

[0003] However, existing clustering methods, such as K-means clustering, spectral clustering, and hierarchical clustering, all rely on many assumptions. For example, the initial assumption of K-means clustering is that cluster centers exist, but in reality, this assumption may be incorrect (e.g., in the case of a slant distribution). Spectral clustering requires that class distributions be balanced. These assumptions make it difficult to apply these methods to complex clustering scenes and tend to generate noisy clusters. Furthermore, complex distributions are a major challenge in large-scale face image clustering tasks, and traditional clustering methods often generate noisy clusters when dealing with these complex distributions.

[0004] Therefore, conventional face image clustering algorithms have the problem that they are difficult to apply to complex clustering scenes, and tend to generate clusters containing noise, which reduces the effectiveness of face image clustering. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a method, apparatus, electronic device and storage medium for face image clustering to solve the problems existing in the prior art, such as difficulty in applying to complex clustering scenes, tendency to generate noisy clusters and reduced effectiveness of face image clustering.

[0006] In a first aspect of the embodiment of the present application, the steps include: obtaining a face dataset for clustering; using a trained face recognition model to perform feature extraction on samples in the face dataset to obtain corresponding features of each sample; calculating corresponding cosine distances between the features of the samples; constructing a connection graph including all samples, with each sample as a vertex and the cosine distance as a link; searching the connection graph based on connected components to obtain low-level subgraphs that satisfy a predetermined condition; performing an aggregation operation on the low-level subgraphs to obtain first candidate clusters; and using a graph convolutional neural network to calculate corresponding quality scores and overlap scores for each first candidate cluster, and then performing aggregation on the low-level subgraphs according to the quality scores. A method for face image clustering is provided, including the steps of: filtering a first candidate cluster to obtain a second candidate cluster; taking the second candidate cluster as an input of a graph convolutional neural network, outputting a corresponding probability value of each vertex in the second candidate cluster, and removing noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; taking the third candidate cluster with the highest overlap score as a benchmark cluster, searching for shared vertices between each other third candidate cluster and the benchmark cluster, removing shared vertices in the other third candidate clusters, and taking the benchmark cluster and the other third candidate clusters from which the shared vertices have been removed as corresponding face image clustering results of the face dataset.

[0007] In a second aspect of the embodiment of the present application, the present invention includes a feature extraction module, which is configured to obtain a face dataset for clustering, and use a trained face recognition model to perform feature extraction on samples in the face dataset, to obtain corresponding features of each sample; a connectivity graph construction module, which is configured to calculate corresponding cosine distances between the features of the samples, and construct a connectivity graph including all the samples, with each sample as a vertex and the cosine distance as a link; a search and aggregation module, which is configured to search the connectivity graph based on connected components, to obtain low-level subgraphs that satisfy a predetermined condition, and perform aggregation operations on the low-level subgraphs to obtain first candidate clusters; and a graph convolutional neural network, which is configured to calculate corresponding quality scores and overlap scores for each first candidate cluster, and to select the first candidate clusters according to the quality scores. A face image clustering apparatus is provided, comprising: a candidate cluster selection module configured to select a raster and obtain a second candidate cluster; a noise point removal module configured to take the second candidate cluster as an input of a graph convolutional neural network, output a corresponding probability value of each vertex in the second candidate cluster, and remove noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; and a shared vertex removal module configured to take the third candidate cluster with the highest overlap score as a benchmark cluster, search for shared vertices between each other third candidate cluster and the benchmark cluster, remove shared vertices in the other third candidate clusters, and take the benchmark cluster and the other third candidate clusters from which the shared vertices have been removed as corresponding face image clustering results of the face dataset.

[0008] In a third aspect of an embodiment of the present application, there is provided an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the electronic device implementing the steps of the above method when the processor executes the computer program.

[0009] In a fourth aspect of an embodiment of the present application, there is provided a computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above method when executed by a processor.

[0010] The at least one technical solution used in the embodiments of the present application has the following beneficial effects:

[0011] The method includes the steps of obtaining a face dataset for clustering, using a trained face recognition model to perform feature extraction on samples in the face dataset to obtain corresponding features of each sample, calculating corresponding cosine distances between the features of the samples, constructing a connection graph including all samples, with each sample as a vertex and the cosine distance as a link, searching the connection graph based on connected components to obtain low-level subgraphs that satisfy a predetermined condition, performing an aggregation operation on the low-level subgraphs to obtain first candidate clusters, using a graph convolutional neural network to calculate corresponding quality scores and overlap scores for each first candidate cluster, selecting the first candidate clusters according to the quality scores, and obtaining second candidate clusters, and By following these steps, the face image clustering method of the present application can be applied to complex clustering scenes, avoid generating noisy clusters, and improve the effect of face image clustering. [Brief explanation of the drawings]

[0012] In order to more clearly explain the technical solutions in the embodiments of the present application, the following briefly introduces drawings necessary for describing the embodiments or prior art. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative work.

[0013] [Figure 1] 1 is a method flowchart of face image clustering provided in an embodiment of the present application; [Figure 2] 1 is a schematic diagram of a GCN network provided in an embodiment of the present application. [Figure 3] 1 is a schematic configuration diagram of a face image clustering device provided in an embodiment of the present application; [Figure 4] 1 is a schematic configuration diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0014] In the following description, for purposes of explanation, not limitation, specific details, such as particular system structures and techniques, are provided to thoroughly understand the embodiments of the present application. However, those skilled in the art should understand that the present application can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary details.

[0015] As explained in the background art, with the development of facial recognition technology, a lot of data is required to train and obtain a highly accurate model. Although it is relatively easy to collect such data, labeling it requires a great deal of effort. Therefore, it is considered to use unlabeled data to improve the effectiveness of facial recognition. An intuitive way is to cluster the unlabeled data to generate pseudo-labels, and then directly input these data into a supervised model.

[0016] However, existing clustering methods, such as K-means clustering, spectral clustering, and hierarchical clustering, all rely on many assumptions. For example, K-means assumes that there is an initial process, i.e., the existence of cluster centers. However, in reality, this assumption may be incorrect (e.g., in the case of a slant distribution). Spectral clustering requires that class distributions be balanced. These assumptions make it difficult to apply these methods to complex clustering scenes and tend to generate noisy clusters. In particular, complex distributions are a major challenge in the task of clustering large-scale face images, and traditional clustering methods often generate noisy clusters when dealing with these complex distributions.

[0017] Considering that the graph convolutional neural network (GCN) is an extension of the traditional convolutional neural network (CNN) to process graph-structured data, and the GCN has a powerful ability to construct complex graph models, this application combines the graph convolutional neural network GCN to separate the face image clustering problem into a candidate cluster detection problem and a segmentation problem, and a large number of primitive clusters can be obtained through candidate cluster detection, and then the noise points in the candidate clusters are removed through the segmentation model, and finally the clustered clusters are obtained.

[0018] 1 is a flowchart of a face image clustering method provided in an embodiment of the present application. The face image clustering method of FIG. 1 may be performed by a server. As shown in FIG. 1, the face image clustering method specifically includes: S101: obtaining a face dataset for clustering, and performing feature extraction on samples in the face dataset using a trained face recognition model to obtain corresponding features for each sample; Calculating the corresponding cosine distances between the features of the samples, and constructing a connectivity graph including all the samples, with each sample as a vertex and the cosine distance as a link (S102); S103: searching the connection graph based on the connected components, obtaining a low-level subgraph that satisfies a predetermined condition, and performing an aggregation operation on the low-level subgraph to obtain a first candidate cluster; S104: calculating corresponding quality scores and overlap scores of each first candidate cluster using a graph convolutional neural network, and selecting the first candidate clusters according to the quality scores to obtain second candidate clusters; S105: taking the second candidate cluster as input of a graph convolutional neural network, outputting a corresponding probability value of each vertex in the second candidate cluster, and removing noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; S106: determining the third candidate cluster with the highest overlap score as a benchmark cluster, searching for shared vertices between each of the other third candidate clusters and the benchmark cluster, removing shared vertices in the other third candidate clusters, and determining the benchmark cluster and the other third candidate clusters from which the shared vertices have been removed as corresponding face image clustering results of the face dataset.

[0019] Specifically, in the embodiment of the present application, the face image dataset includes a large number of samples, each corresponding to one face image, for example, the face dataset includes hundreds of thousands or even millions of face images. The face image clustering method in the embodiment of the present application includes six parts, including extracting face image features, constructing an original image, obtaining candidate subgraphs by a detection module, selecting high-quality candidate subgraphs by a GCN, culling noise points in the high-quality candidate subgraphs by a segmentation module, and removing overlapping parts between candidate subgraphs. The contents of these six parts will be described in detail below with reference to a specific embodiment.

[0020] In some embodiments, the step of performing feature extraction on samples in the face dataset using the trained face recognition model to obtain corresponding features for each sample includes the steps of pre-training the face recognition model using the collected disclosure data to obtain a pre-trained face recognition model, extracting and labeling a predetermined number of samples from the face dataset and performing secondary training on the pre-trained face recognition model using the labeled samples to obtain a secondary trained face recognition model, and performing feature extraction on the remaining samples in the face dataset using the secondary trained face recognition model to obtain corresponding features for each sample.

[0021] Specifically, the extraction of facial features in the embodiments of the present application is very important for facial image clustering, and effective facial features can obtain better clustering results. Therefore, when extracting corresponding facial features for each sample (facial image) in a facial dataset, first, a facial recognition model is pre-trained using collected public data, and then a predetermined number of samples (e.g., 50,000 facial images) are extracted and labeled from the data to be clustered (i.e., the facial dataset), and the pre-trained facial recognition model is trained using the labeled data (i.e., secondary training of the pre-trained facial recognition model), and feature extraction is performed on the remaining samples in the facial dataset using the secondary trained facial recognition model to obtain corresponding facial features for each sample.

[0022] Furthermore, in the embodiment of the present application, feature extraction is performed for all samples in the original dataset (face dataset), the cosine distance between corresponding features of each sample is calculated, and the original image (i.e., the connection graph) is constructed. The connection graph construction method uses each sample as a vertex of the connection graph, and the calculated cosine distance between sample features is used as a link of the connection graph. When constructing the links, only the k nearest vertices are selected for each vertex, and finally a connection graph including all samples can be obtained.

[0023] In some embodiments, the step of searching the connection graph based on the connected components and obtaining a low-level subgraph that satisfies the predetermined condition includes the steps of: removing links in the connection graph whose cosine distance is smaller than a set threshold; obtaining a connected subgraph from the connection graph based on the connection graph from which the links have been removed and the connected components; removing the connected subgraph from the connection graph; and storing the connection graph from which the connected subgraph has been removed in a list; and determining, among the connected subgraphs, graphs whose number of vertices is lower than a fixed threshold as low-level subgraphs; gradually increasing the set threshold according to a preset threshold ladder; and obtaining all low-level subgraphs until the connection graph from which the connected subgraphs have been removed and stored in the list becomes empty.

[0024] Specifically, in the embodiment of the present application, when the detection module obtains candidate subgraphs, the detection module borrows the concept of a detection algorithm, so that a large number of candidate boxes (proposals) can be generated, and the candidate boxes are also candidate clusters. The generation of candidate clusters mainly includes two steps: step 1 for generating low-level subgraphs and step 2 for generating high-level subgraphs. The contents of the two steps for generating candidate clusters will be described in detail below with reference to specific embodiments.

[0025] Furthermore, a low-level subgraph refers to a graph containing a small number of vertices, where the vertices in the graph are similar and can therefore be replaced by connected components. However, if the connected components are directly derived from the original image (i.e., the connected graph), the number of low-level subgraphs may become very large. Therefore, in order to maintain high connectivity between other subgraphs, in the embodiment of the present application, connection links lower than a certain threshold are removed, and the number of vertices in the connected subgraph is forced to be lower than a certain value to generate the low-level subgraph. Below, the process and principle of generating a low-level subgraph will be described in detail with reference to a specific embodiment, specifically including the following:

[0026] Suppose the constructed and obtained connected graph is A, A is the original image, and R is for storing a graph obtained by removing connected subgraphs from the connected graph A. In practical applications, R may be regarded as a list or a data storage structure, and the connected graph obtained by removing connected subgraphs is stored in list R.

[0027] First, a threshold is used to remove links in the connection graph whose cosine distance is lower than the threshold, and then multiple connected subgraphs are derived based on the connected components in the connection graph from which the links have been removed. That is, each time, links lower than the threshold are first removed from connection graph A, and then the connected subgraphs generated by removing the links are obtained.

[0028] Next, the connected subgraphs are searched for graphs whose number of nodes (i.e., the number of vertices) is less than a fixed threshold T, and these searched graphs are designated as low-level subgraphs. After that, the threshold is gradually increased, and the above operation is repeated until the list R is empty and the search is completed.

[0029] Note that a maximal connected subgraph of an undirected graph G is called a connected component of G. Any connected graph has only one connected component, which is the graph itself, while an unconnected undirected graph has multiple connected components. In the embodiment of this application, connected subgraphs are searched from the connected graph based on the connected components, so that one connected component can correspond to one connected subgraph, and all the searched connected subgraphs are called low-level subgraphs.

[0030] In some embodiments, the step of performing an aggregation operation on the low-level subgraphs to obtain a first candidate cluster includes the steps of: identifying corresponding central vertices of each low-level subgraph; taking the central vertices of the low-level subgraphs as vertices and the connections between the central vertices as links; aggregating the low-level subgraphs to obtain a high-level subgraph; searching for connected components based on the aggregated high-level subgraph to obtain a new connected subgraph; performing a next aggregation iteration based on the new connected subgraph to obtain a new high-level subgraph; and storing the high-level subgraph and the new high-level subgraph all as the first candidate cluster.

[0031] Specifically, after searching all low-level subgraphs, compared with the required candidate clusters, the low-level subgraphs are still too conservative. While one low-level subgraph is likely to belong to the same person, samples of the same person may be divided into different low-level subgraphs. This is inspired by multi-scale candidate boxes in target detection. In this application, a higher-level graph is constructed on the low-level subgraphs. In the process of constructing a high-level subgraph, in the embodiment of this application, the centers of the low-level subgraphs are used as vertices, and the connections between vertices are used as links, and a large number of multi-scale candidate clusters can be obtained by using them multiple times.

[0032] Furthermore, assuming there are three low-level subgraphs A, B, and C, when aggregating the low-level subgraphs to obtain a high-level subgraph, first identify the corresponding central vertices of each subgraph, and the central vertices are obtained by averaging all the vertices in the low-level subgraphs. The central vertices of the low-level subgraphs A, B, and C are taken as the vertices of the high-level subgraph, and the connections between the central vertices are taken as links. According to this operation, after one iteration, several low-level subgraphs can be combined into a high-level subgraph.

[0033] After the first iteration to obtain high-level subgraphs, continue to search for connected components on these high-level subgraphs to obtain new connected subgraphs, calculate central vertices for these new connected subgraphs, continue to aggregate based on the central vertices, and repeat this operation to obtain new high-level subgraphs after two iterations, and finally store all high-level subgraphs generated during the above iteration process as first candidate clusters.

[0034] It should be noted that the embodiments of the present application obtain a large number of multi-scale candidate clusters through repeated operations of searching for aggregated-connected components. In actual applications, the number of iterations of searching for aggregated-connected components may be two or more, and the specific number of iterations can be determined according to actual needs.

[0035] In some embodiments, the step of using a graph convolutional neural network to calculate corresponding quality scores and overlap scores for each first candidate cluster, selecting the first candidate clusters according to the quality scores, and obtaining second candidate clusters includes the steps of: inputting the first candidate clusters into a graph convolutional neural network, and using the graph convolutional neural network to calculate corresponding quality scores and overlap scores for each first candidate cluster; and selecting the first candidate clusters according to the corresponding quality scores of each first candidate cluster and a predetermined quality score threshold, and selecting corresponding first candidate clusters with quality scores higher than the quality score threshold as second candidate clusters, wherein a GCN network is used as the graph convolutional neural network.

[0036] Specifically, after obtaining all candidate sub-clusters (i.e., first candidate clusters) through the detection module, the embodiment of the present application selects a high-quality graph based on a GCN network. A GCN (Graph Convolutional Neural Network) can actually function as a feature extractor, similar to a CNN, but its target is graph data. A GCN cleverly designs a method for extracting features from graph data, and can then use these features to perform node classification, graph classification, and link prediction on the graph data, thereby obtaining a graph embedding. The following describes the calculation steps and principles of the graph convolutional neural network (GCN), and GCN calculation mainly includes the following three steps:

[0037] Step 1: Each node sends and converts its own characteristic information, and then sends it to its neighboring nodes. In this step, the characteristic information of the node is extracted and converted.

[0038] Step 2: Each node receives and collects the feature information of its neighboring nodes. In this step, the local structure information of the node is fused, that is, the information of the node and all its neighboring nodes is fused.

[0039] Step 3: Transform: After collecting previous information, perform nonlinear transformation to increase the expressive power of the model.

[0040] The above-mentioned GCN network processing can obtain a large number of candidate clusters, and since GCN has a strong information aggregation ability, in this application, GCN is used to select high-quality candidate graphs from high-level subgraphs (i.e., first candidate clusters). Hereinafter, the process of selecting high-quality candidate graphs using a GCN network will be described in detail with reference to the configuration diagram of the GCN network provided in the embodiments of this application. Figure 2 is a schematic configuration diagram of the GCN network provided in the embodiments of this application. As shown in Figure 2, the process of selecting high-quality candidate graphs using the GCN network specifically includes the following:

[0041] The training process of the GCN network provided in the embodiments of the present application includes a step of inputting each first candidate cluster into a GCN network and calculating its score (including a quality score and an overlap score). Given a training set with class labels, the training process of the GCN network can obtain the ground-truth IoU (overlap score) and IoP (quality score), and includes a step of training the GCN network using the mean squared error (MSE). The estimation process involves using the trained GCN network to predict the IoU and IoP scores of each proposal (first candidate cluster). Here, the IoP score is used to determine whether a proposal needs to be subdivided, i.e., whether noise points need to be removed. Meanwhile, the IoU score is used to remove overlaps between proposals. A higher IoU score means that more overlaps need to be removed.

[0042] Furthermore, the GCN network in the embodiment of the present application uses the following formula when calculating the score, which is: JPEG2025526866000002.jpg35170

[0043] Here, the IoP score is also referred to as the candidate cluster score, and the higher the IoP score, the higher the quality of the candidate cluster. A candidate cluster whose IoP score is greater than the quality score threshold is selected as a high-quality candidate graph, and noise points are removed from the high-quality candidate graph.

[0044] In some embodiments, the steps of inputting the second candidate clusters to a graph convolutional neural network, outputting a corresponding probability value of each vertex in the second candidate clusters, and removing noise points in the second candidate clusters according to the probability values to obtain a third candidate cluster include: inputting the second candidate clusters to a graph convolutional neural network and calculating a corresponding probability value of each vertex in each second candidate cluster using the graph convolutional neural network; determining vertices with probability values lower than a threshold as noise points, removing the noise points from the second candidate clusters, and determining the second candidate clusters from which the noise points have been removed as the third candidate cluster, wherein the probability value represents the probability that the vertex is not a noise point, and the larger the probability value, the less likely the vertex is a noise point.

[0045] Specifically, after selecting a high-quality candidate graph, in the embodiment of the present application, a segmentation module is used to cull noise points in the high-quality candidate graph. Because the candidate clusters generated through the previous process are still impure, in the embodiment of the present application, a segmentation module using a GCN network is constructed to remove outliers.

[0046] In the embodiment of the present application, when defining outliers, the normal method is to set all vertices with different labels as outliers, but this method is not very effective when all vertices in a cluster have almost half the labels. Therefore, to avoid defining outliers by manually adjusting parameters and to promote the model's learning of different partitioning patterns, the embodiment of the present application randomly selects one vertex as a seed multiple times, so that each candidate cluster can obtain multiple training samples.

[0047] Furthermore, the GCN network used to cull noise points in high-quality candidate graphs and the GCN network used to select high-quality candidate graphs have the same structure, so the structure of the GCN network will not be described redundantly. The main difference is the predicted target value. The GCN network here is not used to predict the quality score of the entire candidate cluster, but to output a probability value for each vertex v in the candidate cluster, which indicates the likelihood that the vertex is a true member rather than an outlier (i.e., a noise point).

[0048] In the training process of the GCN network in the embodiment of this application, one vertex is randomly selected from the candidate cluster as a seed, and the vertex with the same label as the seed is considered a positive vertex, and the others are considered outliers. By applying the method multiple times using randomly selected seeds, multiple training samples can be obtained from each candidate cluster, and cross-entropy is used as the loss function for training.

[0049] Furthermore, the trained GCN network outputs a corresponding probability value for each vertex in each second candidate cluster, which represents the probability that the vertex is not a noise point (i.e., not an outlier), and the inference process of the GCN network is to reserve the prediction result with the most positive vertices (threshold 0.5).

[0050] In some embodiments, the step of selecting the third candidate cluster with the highest overlap score as the benchmark cluster, searching for shared vertices between each of the other third candidate clusters and the benchmark cluster, and removing shared vertices in the other third candidate clusters includes the steps of: sorting the third candidate clusters according to their overlap scores, selecting the third candidate cluster with the highest overlap score as the benchmark cluster, and retaining the benchmark cluster; and comparing each of the other third candidate clusters with the benchmark cluster in turn according to the sorting result, identifying shared vertices between each of the other third candidate clusters and the benchmark cluster, selecting the shared vertices as overlapping portions between the other third candidate clusters and the benchmark cluster, and removing the overlapping portions from the other third candidate clusters.

[0051] Specifically, after removing outliers in the second candidate cluster, the embodiment of the present application uses a modified NMS algorithm to remove overlaps between the candidate sub-clusters (i.e., separate the overlapping clusters). Hereinafter, the operation of removing overlaps using the NMS algorithm will be described with reference to a specific embodiment, which specifically includes the following:

[0052] Since overlaps may occur between the third candidate clusters, i.e., the existence of shared vertices, conflicting results may occur in face recognition. Therefore, the embodiment of the present application proposes a modified NMS algorithm, which first sorts the third candidate clusters according to IoU score (overlap score), reserves the third candidate cluster with the highest IoU score as the most reliable candidate cluster (i.e., benchmark cluster), compares each of the other third candidate clusters with the benchmark cluster according to IoU score, and detects whether any vertices between each of the other third candidate clusters and the benchmark cluster belong to shared vertices, and finally removes the shared vertices in the other third candidate clusters to obtain candidate clusters with the overlaps removed.

[0053] That is, the basic principle of the NMS algorithm in the embodiment of this application is to obtain the ranks in reverse order according to the IoU score (overlap score), then collect candidate cluster proposals from the ranks, and finally gradually remove the vertices in the previous cluster from top to bottom.

[0054] According to the technical solutions provided in the embodiments of the present application, the face image clustering method provided in the embodiments of the present application has the following advantages:

[0055] (1) By analyzing traditional clustering algorithms, this application proposes a face image clustering algorithm with a detection module and a segmentation module, which can process more complex clustering data and generate a large amount of data for face recognition.

[0056] (2) Based on the idea of traditional detection algorithms, this application proposes a Detection module, which can generate a large number of candidate graphs from the original connection graph, and at the same time, further fuse these candidate graphs to obtain a high-level candidate graph.

[0057] (3) In this application, a segmentation module is constructed based on a graph convolutional neural network (GCN), which outputs a score for each vertex in the candidate graph, indicating the probability that it is a true member rather than an outlier, and the module processing can remove outliers in the candidate graph.

[0058] Below are apparatus embodiments of the present application for carrying out the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0059] 3 is a schematic diagram of a face image clustering device provided in an embodiment of the present application. As shown in FIG. 3, the face image clustering device includes: a feature extraction module 301 configured to obtain a face dataset for clustering, and to perform feature extraction on samples in the face dataset using a trained face recognition model to obtain corresponding features for each sample; a connectivity graph construction module 302 configured to calculate corresponding cosine distances between the features of the samples and to construct a connectivity graph including all the samples, with each sample as a vertex and the cosine distances as links; a search and aggregation module 303 configured to search the connectivity graph based on its connected components, obtain low-level subgraphs that satisfy a predetermined condition, and perform an aggregation operation on the low-level subgraphs to obtain first candidate clusters; a candidate cluster selection module 304 configured to use a graph convolutional neural network to calculate corresponding quality scores and overlap scores for each first candidate cluster, and to select the first candidate clusters according to the quality scores to obtain second candidate clusters; a noise point removal module 305, configured to take the second candidate cluster as an input of a graph convolutional neural network, output a corresponding probability value of each vertex in the second candidate cluster, and remove noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; a duplicate removal module 306 configured to: take the third candidate cluster with the highest overlap score as a benchmark cluster, search for shared vertices between each other third candidate cluster and the benchmark cluster, remove the shared vertices in the other third candidate clusters, and take the benchmark cluster and the other third candidate clusters from which the shared vertices have been removed as corresponding face image clustering results of the face dataset.

[0060] In some embodiments, the feature extraction module 301 of FIG. 3 pre-trains a face recognition model using the collected disclosure data to obtain a pre-trained face recognition model, extracts and labels a predetermined number of samples from the face dataset, performs secondary training on the pre-trained face recognition model using the labeled samples to obtain a secondary trained face recognition model, and uses the secondary trained face recognition model to perform feature extraction on the remaining samples in the face dataset to obtain corresponding features for each sample.

[0061] In some embodiments, the search aggregation module 303 in FIG. 3 removes links in the connection graph whose cosine distance is smaller than a set threshold, obtains connected subgraphs from the connection graph based on the connection graph from which the links have been removed and the connected components, removes the connected subgraphs from the connection graph, stores the connection graph from which the connected subgraphs have been removed in a list, and determines that the number of vertices in the connected subgraphs is lower than a fixed threshold as a low-level subgraph. The set threshold is gradually increased according to a preset threshold ladder to obtain all low-level subgraphs until the connection graph from which the connected subgraphs have been removed stored in the list becomes empty.

[0062] In some embodiments, the search aggregation module 303 in FIG. 3 identifies corresponding central vertices of each low-level subgraph, takes the central vertices of the low-level subgraphs as vertices and the connections between the central vertices as links, aggregates the low-level subgraphs to obtain a high-level subgraph, searches for connected components based on the aggregated high-level subgraph to obtain a new connected subgraph, performs the next aggregation iteration based on the new connected subgraph to obtain a new high-level subgraph, and stores the high-level subgraph and the new high-level subgraph together as first candidate clusters.

[0063] In some embodiments, the candidate cluster selection module 304 of FIG. 3 inputs the first candidate clusters into a graph convolutional neural network, calculates corresponding quality scores and overlap scores for each first candidate cluster using the graph convolutional neural network, and selects the first candidate clusters according to the corresponding quality scores of each first candidate cluster and a predetermined quality score threshold, and selects the corresponding first candidate clusters with quality scores higher than the quality score threshold as second candidate clusters, and a GCN network is used as the graph convolutional neural network.

[0064] In some embodiments, the noise point removal module 305 of FIG. 3 inputs the second candidate clusters into a graph convolutional neural network, calculates a corresponding probability value for each vertex in each second candidate cluster using the graph convolutional neural network, determines the vertices with a probability value lower than a threshold as noise points, removes the noise points from the second candidate cluster, and determines the second candidate cluster from which the noise points have been removed as a third candidate cluster, where the probability value represents the probability that the vertex is not a noise point, and the larger the probability value, the less likely the vertex is a noise point.

[0065] In some embodiments, the overlap removal module 306 of FIG. 3 sorts the third candidate clusters according to their overlap scores, selects the third candidate cluster with the highest overlap score as the benchmark cluster, and retains the benchmark cluster; and, according to the sorting result, compares each of the other third candidate clusters with the benchmark cluster in turn, identifies shared vertices between each of the other third candidate clusters and the benchmark cluster, selects the shared vertices as overlapping portions between the other third candidate clusters and the benchmark cluster, and removes the overlapping portions from the other third candidate clusters.

[0066] 4 is a schematic diagram of an electronic device 4 provided in an embodiment of the present application. As shown in FIG. 4, the electronic device 4 of the embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable by the processor 401. When the processor 401 executes the computer program 403, it performs the steps in each of the above method embodiments. Alternatively, when the processor 401 executes the computer program 403, it performs the functions of each module / unit in each of the above device embodiments.

[0067] For example, the computer program 403 may be divided into one or more modules / units, which may be stored in the memory 402 and executed by the processor 401 to accomplish the present application. The one or more modules / units may be a series of computer program command sections capable of performing specific functions, which are intended to explain the process of the computer program 403 being executed in the electronic device 4.

[0068] The electronic device 4 may be an electronic device such as a desktop computer, a laptop computer, a palmtop computer, or a cloud server. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. As will be understood by those skilled in the art, FIG. 4 is merely an example of the electronic device 4 and is not intended to limit the electronic device 4, which may include more or fewer components than those shown, or may combine certain components or different components; for example, the electronic device may include input / output devices, network access devices, buses, etc.

[0069] Processor 401 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, any common processor, etc.

[0070] The memory 402 may be an internal storage unit of the electronic device 4, such as a hard disk or RAM of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash card, etc., that is provided in the electronic device 4. Furthermore, the memory 402 may include not only the internal storage unit of the electronic device 4 but also an external storage device. The memory 402 is intended to store computer programs and other programs and data required by the electronic device. The memory 402 may also be used to temporarily store data that has been output or that is to be output.

[0071] Those skilled in the art will understand that, for convenience and brevity, only the division of the above functional units and modules has been used as an example. However, in actual applications, the above functions can be assigned to different functional units or modules as needed, i.e., all or part of the above-described functions can be achieved by dividing the internal structure of the device into different functional units or modules. The functional units and modules in the embodiments may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of hardware or software functional units. The specific names of the functional units and modules are provided solely for the purpose of distinguishing them from one another and do not limit the scope of protection of the present application. For the specific operating processes of the units and modules in the above system, reference may be made to the corresponding processes in the above-described method embodiments, and further description will be omitted here.

[0072] In the above embodiments, the description of each embodiment has its own emphasis, and for the details or parts not described in an embodiment, reference can be made to the relevant descriptions of other embodiments.

[0073] Those skilled in the art can recognize that the units and algorithm steps of each example described in the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented by hardware or software is determined by the specific application and design constraints of the technical solution. Those skilled in the art may implement the described functions using different methods for each specific application, but such implementation should not be considered to go beyond the scope of this application.

[0074] It should be understood that the disclosed devices / computer devices and methods in the embodiments provided in this application can be implemented in other ways. For example, the device / computer device embodiments described above are merely illustrative, and the division into modules or units is merely a logical division of functions. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into other systems, or some features may be omitted or not implemented. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through several interfaces, devices, or units, and may be electrical, mechanical, or other types.

[0075] Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, i.e., located in one location or distributed across multiple network units, some or all of which may be selected according to actual needs to achieve the objectives of the solutions of this embodiment.

[0076] In addition, each functional unit in the present application may be integrated into one processing unit, each unit may exist physically independently, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional unit.

[0077] The integrated module / unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the present application recognizes that the realization of all or part of the processes in the above-described method embodiments can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-described method embodiments can be realized. The computer program may include computer program code, which may be in source code format, object code format, an executable file, or some intermediate format. The computer-readable storage medium may include any entity or device capable of carrying computer program code, such as a recording medium, a U-disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier wave signal, an electrical communication signal, and a software distribution medium. Furthermore, the content contained on a computer-readable storage medium may be increased or decreased as required by the legislation and patent practice of a jurisdiction. For example, in some jurisdictions, the legislation and patent practice may require that a computer-readable storage medium not include electrical carrier signals and telecommunications signals.

[0078] The above-mentioned embodiments are only for illustrating the technical solutions of the present invention, but are not intended to limit the same. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art may still amend the technical solutions described in the above-mentioned embodiments or equivalently replace some technical features therein, but it should be understood that such amendments or replacements shall not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and all of them shall fall within the protection scope of the present invention.

Claims

1. obtaining a face dataset for clustering, and performing feature extraction on samples in the face dataset using a trained face recognition model to obtain corresponding features for each sample; calculating corresponding cosine distances between features of said samples and constructing a connectivity graph including all samples, with each said sample as a vertex and said cosine distances as links; searching the connection graph based on connected components to obtain a low-level subgraph that satisfies a predetermined condition, and performing an aggregation operation on the low-level subgraph to obtain a first candidate cluster; Calculating corresponding quality scores and overlap scores for each of the first candidate clusters using a graph convolutional neural network, and filtering the first candidate clusters according to the quality scores to obtain second candidate clusters; taking the second candidate cluster as an input of the graph convolutional neural network, outputting a corresponding probability value of each vertex in the second candidate cluster, and removing noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; determining the third candidate cluster with the highest overlap score as a benchmark cluster, searching for shared vertices between each of the other third candidate clusters and the benchmark cluster, removing shared vertices in the other third candidate clusters, and determining the benchmark cluster and the other third candidate clusters from which the shared vertices have been removed as corresponding face image clustering results of the face dataset. A method for face image clustering, comprising:

2. performing feature extraction on samples in the face dataset using a trained face recognition model to obtain corresponding features for each sample; Pre-training a facial recognition model using the collected disclosure data to obtain a pre-trained facial recognition model; extracting and labeling a predetermined number of samples from the face dataset, and performing secondary training on the pre-trained face recognition model using the labeled samples to obtain a secondary trained face recognition model; and performing feature extraction on the remaining samples in the face dataset using the secondary trained face recognition model to obtain corresponding features for each sample.

2. The method of claim 1 .

3. The step of searching the connection graph based on connected components to obtain a low-level subgraph that satisfies a predetermined condition includes: removing links in the connection graph whose cosine distance is smaller than a set threshold, obtaining a connected subgraph from the connection graph based on the connection graph from which the links have been removed and connected components, removing the connected subgraph from the connection graph, and storing the connection graph from which the connected subgraph has been removed in a list; a step of determining a graph among the connected subgraphs whose number of vertices is less than a fixed threshold as a low-level subgraph, gradually increasing the set threshold according to a preset threshold ladder, and acquiring all low-level subgraphs until the connected graph stored in the list after the connected subgraphs have been removed becomes empty.

2. The method of claim 1 .

4. performing an aggregation operation on the low-level subgraph to obtain a first candidate cluster; Identifying corresponding central vertices of each of the low-level subgraphs, defining the central vertices of the low-level subgraphs as vertices and the connections between the central vertices as links, and aggregating the low-level subgraphs to obtain a high-level subgraph; searching for connected components based on the aggregated high-level subgraph to obtain a new connected subgraph; performing a next aggregation iteration based on the new connected subgraph to obtain a new high-level subgraph; and storing the high-level subgraph and the new high-level subgraph all as the first candidate cluster.

2. The method of claim 1 .

5. The step of calculating corresponding quality scores and overlap scores for each of the first candidate clusters using a graph convolutional neural network, filtering the first candidate clusters according to the quality scores, and obtaining second candidate clusters includes: inputting the first candidate clusters into the graph convolutional neural network and calculating corresponding quality scores and overlap scores for each of the first candidate clusters using the graph convolutional neural network; and a step of sorting the first candidate clusters according to a corresponding quality score of each of the first candidate clusters and a predetermined quality score threshold, and selecting a corresponding first candidate cluster having a quality score higher than the quality score threshold as the second candidate cluster, wherein a GCN network is used as the graph convolutional neural network.

2. The method of claim 1 .

6. The step of taking the second candidate cluster as an input of the graph convolutional neural network, outputting a corresponding probability value of each vertex in the second candidate cluster, and removing noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster includes: inputting the second candidate clusters into the graph convolutional neural network and calculating a corresponding probability value for each vertex in each of the second candidate clusters using the graph convolutional neural network; determining the vertices whose probability values are lower than a threshold as noise points, removing the noise points from the second candidate cluster, and determining the second candidate cluster from which the noise points have been removed as the third candidate cluster, Here, the probability value represents the probability that the vertex is not a noise point, and the larger the probability value, the smaller the possibility that the vertex is a noise point.

2. The method of claim 1 .

7. the step of selecting the third candidate cluster with the highest overlap score as a benchmark cluster, searching for shared vertices between each of the other third candidate clusters and the benchmark cluster, and removing the shared vertices in the other third candidate clusters, sorting the third candidate clusters according to the overlap score, selecting the third candidate cluster with the highest overlap score as a benchmark cluster, and retaining the benchmark cluster; and comparing each of the other third candidate clusters with the benchmark cluster in turn according to the sorting result, identifying a shared vertex between each of the other third candidate clusters and the benchmark cluster, determining the shared vertex as an overlap between the other third candidate cluster and the benchmark cluster, and removing the overlap from the other third candidate clusters.

2. The method of claim 1 .

8. a feature extraction module configured to obtain a face dataset for clustering, and to perform feature extraction on samples in the face dataset using a trained face recognition model to obtain corresponding features for each sample; a connectivity graph construction module configured to calculate corresponding cosine distances between features of said samples and to construct a connectivity graph including all samples, with each said sample as a vertex and said cosine distances as links; a searching and aggregating module configured to search the connectivity graph based on connected components, obtain a lower-level subgraph that satisfies a predetermined condition, and perform an aggregation operation on the lower-level subgraph to obtain a first candidate cluster; a candidate cluster selection module configured to calculate corresponding quality scores and overlap scores for each of the first candidate clusters using a graph convolutional neural network, and to select the first candidate clusters according to the quality scores to obtain second candidate clusters; a noise point removal module configured to take the second candidate cluster as an input of the graph convolutional neural network, output a corresponding probability value of each vertex in the second candidate cluster, and remove noise points in the second candidate cluster according to the probability value to obtain a third candidate cluster; a shared vertex removal module configured to: determine the third candidate cluster with the highest overlap score as a benchmark cluster; search for shared vertices between each of the other third candidate clusters and the benchmark cluster; remove shared vertices in the other third candidate clusters; and determine the benchmark cluster and the other third candidate clusters from which the shared vertices have been removed as corresponding face image clustering results of the face dataset. A face image clustering device characterized by:

9. 10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the electronic device implementing the method of claim 1 when the processor executes the computer program.

10. A computer-readable storage medium on which a computer program is stored, The computer program, when executed by a processor, implements the method of claim 1. A computer-readable storage medium comprising:

Images