A face clustering method, device and equipment

Abstract

The application discloses a face clustering method, device and equipment, and is used for an offline face clustering method of a supervised model, and a wrong node identification strategy is adopted to further improve the accuracy of a clustering result. The method comprises the following steps: obtaining a first KNN graph, wherein the first KNN graph is obtained by removing the undirected edges in an original KNN graph for the first time, and the original KNN graph is generated according to a face feature set corresponding to a face image set; according to the group degree of a non-isolated node in the first KNN graph, removing the undirected edges containing the non-isolated node again to obtain a second KNN graph, wherein the group degree represents the similarity between the non-isolated node and a first-order neighbor node of the non-isolated node, and the first-order neighbor node is a node directly connected to the non-isolated node through an undirected edge; and determining a face clustering result corresponding to the face image set according to the connected subgraph contained in the second KNN graph.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device and equipment for face clustering. Background Art

[0002] Face clustering is used to solve the problem of which faces belong to the same person. Offline face clustering technology is responsible for archiving by person, trying to ensure that all faces of the same person are classified into the same file. At the same time, the same person is tried to be in the same file as much as possible, that is, "one person, one file". The accuracy of the clustering technology results can be reflected by two statistical indicators. One is the accuracy rate, that is, whether all faces in a file belong to the same person. The more faces are misclassified, the lower the accuracy rate; the other is the recall rate, that is, whether all faces of the same person are in the same file. The more faces are missed, the lower the recall rate.

[0003] Current offline face clustering methods fall into two categories: unsupervised and supervised. Unsupervised approaches lack robustness, resulting in inferior clustering results compared to supervised learning approaches. Supervised approaches, while superior in both accuracy and recall, often contain a large number of erroneous and isolated nodes. These erroneous nodes can lead to multiple instances of a single file, reducing clustering accuracy. Isolated nodes can also lead to multiple instances of a single file, reducing recall. Summary of the Invention

[0004] The present invention provides a method, apparatus and device for face clustering, which are used for an offline face clustering method of a supervised model, and adopts a strategy of error node recognition to further improve the accuracy of clustering results.

[0005] In a first aspect, an embodiment of the present invention provides a method for face clustering, comprising:

[0006] Obtaining a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0007] removing undirected edges containing the non-isolated nodes again according to the sociability of the non-isolated nodes in the first KNN graph to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated nodes and their first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated nodes via undirected edges;

[0008] A face clustering result corresponding to the face image set is determined based on the connected subgraphs included in the second KNN graph.

[0009] The face clustering method provided in this embodiment is based on the current clustering process. For the first KNN graph obtained by fine segmentation, the negative connections in the first KNN graph that cannot be successfully identified by the current supervised recognition model are removed again according to the sociability of the non-isolated nodes in the first KNN graph. This solves the problem of multiple people in one file and improves the accuracy of the clustering results.

[0010] As an optional implementation, the sociability of non-isolated nodes in the first KNN graph is determined in the following manner:

[0011] Determining the similarity between the non-isolated node and the first-order neighbor nodes of the non-isolated node;

[0012] The maximum similarity among the similarities is determined as the sociability of the non-isolated node.

[0013] As an optional implementation manner, the step of removing undirected edges containing non-isolated nodes again according to the sociability of the non-isolated nodes in the first KNN graph to obtain a second KNN graph includes:

[0014] If the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

[0015] As an optional implementation manner, before determining the face clustering result corresponding to the face image set based on the connected subgraphs included in the second KNN graph, the method further includes:

[0016] For any isolated node in the second KNN graph, obtaining a second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents a connection relationship between the isolated node, the first-order neighbor nodes of the isolated node, and the second-order neighbor nodes in the original KNN graph, wherein the second-order neighbor nodes are nodes other than the isolated node that are directly connected to the first-order neighbor nodes through undirected edges;

[0017] Inputting the second-order neighborhood subgraph into a first graph convolutional network model, and outputting predicted values ​​of undirected edges contained in the second-order neighborhood subgraph, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent connections between nodes corresponding to the same face;

[0018] The isolated nodes are reconnected in the second KNN graph according to the predicted values ​​and the undirected edges corresponding to the predicted values ​​to obtain a reconnected second KNN graph.

[0019] As an optional implementation manner, reconnecting the isolated nodes in the second KNN graph according to the target predicted value and the undirected edge corresponding to the target predicted value to obtain the reconnected second KNN graph includes:

[0020] If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value;

[0021] The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0022] As an optional implementation, the first graph convolutional network model is determined in the following manner:

[0023] Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph;

[0024] The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

[0025] As an optional implementation, the sample second-order neighborhood subgraph is determined in the following manner:

[0026] Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set;

[0027] Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features;

[0028] According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

[0029] As an optional implementation manner, obtaining the first KNN graph includes:

[0030] Inputting the original KNN graph into a second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph;

[0031] Removing the undirected edges whose scores are lower than a third threshold to obtain an original KNN graph after the removal;

[0032] According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than a fourth threshold are removed to obtain a first KNN graph.

[0033] As an optional implementation, the intimacy between the nodes is determined in the following manner:

[0034] The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

[0035] As an optional implementation, the second graph convolutional network model is determined in the following manner:

[0036] Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample;

[0037] The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

[0038] As an optional implementation manner, determining the face clustering result corresponding to the face image set based on the connected subgraph contained in the second KNN graph includes:

[0039] Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the reconnected second KNN graph.

[0040] As an optional implementation manner, determining the face clustering result corresponding to the face image set based on the connected subgraph contained in the second KNN graph includes:

[0041] Any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

[0042] In a second aspect, an embodiment of the present invention provides a face clustering device, comprising:

[0043] an acquiring unit, configured to acquire a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0044] a removing unit, configured to remove undirected edges containing the non-isolated node again according to the sociability of the non-isolated node in the first KNN graph, to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated node and its first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated node via undirected edges;

[0045] A determining unit is configured to determine a face clustering result corresponding to the face image set according to a connected subgraph included in the second KNN graph.

[0046] As an optional implementation manner, the removal unit is specifically configured to determine the sociability of the non-isolated nodes in the first KNN graph in the following manner:

[0047] Determining the similarity between the non-isolated node and the first-order neighbor nodes of the non-isolated node;

[0048] The maximum similarity among the similarities is determined as the sociability of the non-isolated node.

[0049] As an optional implementation manner, the removing unit is specifically configured to:

[0050] If the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

[0051] As an optional implementation, before determining the face clustering result corresponding to the face image set based on the connected subgraphs included in the second KNN graph, the method further includes a reconnection unit configured to:

[0052] For any isolated node in the second KNN graph, obtaining a second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents a connection relationship between the isolated node, the first-order neighbor nodes of the isolated node, and the second-order neighbor nodes in the original KNN graph, wherein the second-order neighbor nodes are nodes other than the isolated node that are directly connected to the first-order neighbor nodes through undirected edges;

[0053] Inputting the second-order neighborhood subgraph into a first graph convolutional network model, and outputting predicted values ​​of undirected edges contained in the second-order neighborhood subgraph, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent connections between nodes corresponding to the same face;

[0054] The isolated nodes are reconnected in the second KNN graph according to the predicted values ​​and the undirected edges corresponding to the predicted values ​​to obtain a reconnected second KNN graph.

[0055] As an optional implementation manner, the reconnection unit is specifically configured to:

[0056] If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value;

[0057] The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0058] As an optional implementation manner, the reconnection unit is specifically configured to determine the first graph convolutional network model in the following manner:

[0059] Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph;

[0060] The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

[0061] As an optional implementation manner, the reconnection unit is specifically configured to determine the sample second-order neighborhood subgraph in the following manner:

[0062] Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set;

[0063] Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features;

[0064] According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

[0065] As an optional implementation manner, the acquiring unit is specifically configured to:

[0066] Inputting the original KNN graph into a second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph;

[0067] Removing the undirected edges whose scores are lower than a third threshold to obtain an original KNN graph after the removal;

[0068] According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than a fourth threshold are removed to obtain a first KNN graph.

[0069] As an optional implementation manner, the acquisition unit is specifically configured to determine the intimacy between the nodes in the following manner:

[0070] The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

[0071] As an optional implementation manner, the acquisition unit is specifically configured to determine the second graph convolutional network model in the following manner:

[0072] Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample;

[0073] The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

[0074] As an optional implementation manner, the determining unit is specifically configured to:

[0075] Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the reconnected second KNN graph.

[0076] As an optional implementation manner, the determining unit is specifically configured to:

[0077] Any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

[0078] In a third aspect, an embodiment of the present invention further provides a face clustering device, comprising a processor and a memory, wherein the memory is configured to store a program executable by the processor, and the processor is configured to read the program in the memory and perform the following steps:

[0079] Obtaining a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0080] removing undirected edges containing the non-isolated nodes again according to the sociability of the non-isolated nodes in the first KNN graph to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated nodes and their first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated nodes via undirected edges;

[0081] A face clustering result corresponding to the face image set is determined based on the connected subgraphs included in the second KNN graph.

[0082] As an optional implementation manner, the processor is specifically configured to determine the sociability of non-isolated nodes in the first KNN graph in the following manner:

[0083] Determining the similarity between the non-isolated node and the first-order neighbor nodes of the non-isolated node;

[0084] The maximum similarity among the similarities is determined as the sociability of the non-isolated node.

[0085] As an optional implementation, the processor is specifically configured to execute:

[0086] If the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

[0087] As an optional implementation manner, before determining the face clustering result corresponding to the face image set based on the connected subgraph included in the second KNN graph, the processor is further configured to execute:

[0088] For any isolated node in the second KNN graph, obtaining a second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents a connection relationship between the isolated node, the first-order neighbor nodes of the isolated node, and the second-order neighbor nodes in the original KNN graph, wherein the second-order neighbor nodes are nodes other than the isolated node that are directly connected to the first-order neighbor nodes through undirected edges;

[0089] Inputting the second-order neighborhood subgraph into a first graph convolutional network model, and outputting predicted values ​​of undirected edges contained in the second-order neighborhood subgraph, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent connections between nodes corresponding to the same face;

[0090] The isolated nodes are reconnected in the second KNN graph according to the predicted values ​​and the undirected edges corresponding to the predicted values ​​to obtain a reconnected second KNN graph.

[0091] As an optional implementation, the processor is specifically configured to execute:

[0092] If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value;

[0093] The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0094] As an optional implementation, the processor is specifically configured to determine the first graph convolutional network model in the following manner:

[0095] Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph;

[0096] The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

[0097] As an optional implementation manner, the processor is specifically configured to determine the sample second-order neighborhood subgraph in the following manner:

[0098] Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set;

[0099] Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features;

[0100] According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

[0101] As an optional implementation, the processor is specifically configured to execute:

[0102] Inputting the original KNN graph into a second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph;

[0103] Removing the undirected edges whose scores are lower than a third threshold to obtain an original KNN graph after the removal;

[0104] According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than a fourth threshold are removed to obtain a first KNN graph.

[0105] As an optional implementation manner, the processor is specifically configured to determine the intimacy between the nodes in the following manner:

[0106] The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

[0107] As an optional implementation, the processor is specifically configured to determine the second graph convolutional network model in the following manner:

[0108] Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample;

[0109] The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

[0110] As an optional implementation, the processor is specifically configured to execute:

[0111] Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the reconnected second KNN graph.

[0112] As an optional implementation, the processor is specifically configured to execute:

[0113] Any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

[0114] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, it is used to implement the steps of the method described in the first aspect above.

[0115] These and other aspects of the present application will be more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0117] Figure 1 A flowchart of an implementation method of a face clustering method provided by an embodiment of the present invention;

[0118] Figure 2 A flowchart for implementing reconnection of isolated nodes provided by an embodiment of the present invention;

[0119] Figure 3 A schematic diagram of the structure of a second graph convolutional network model provided by an embodiment of the present invention;

[0120] Figure 4 A schematic diagram of the structure of a first graph convolutional network model provided by an embodiment of the present invention;

[0121] Figure 5 A specific implementation flow chart of face clustering provided by an embodiment of the present invention;

[0122] Figure 6 A schematic diagram of a face clustering device provided by an embodiment of the present invention;

[0123] Figure 7 A schematic diagram of a face clustering device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0124] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0125] In embodiments of the present invention, the term "and / or" describes the association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0126] The application scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Persons skilled in the art will appreciate that as new application scenarios emerge, the technical solutions provided by the embodiments of the present invention will also be applicable to similar technical problems. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0127] Face clustering is used to solve the problem of which faces belong to the same person. Offline face clustering technology is responsible for archiving by person, trying to ensure that all faces of the same person are classified into the same file. At the same time, the same person is tried to be in the same file as much as possible, that is, "one person, one file". The accuracy of the clustering technology results can be reflected by two statistical indicators. One is the accuracy rate, that is, whether all faces in a file belong to the same person. The more faces are misclassified, the lower the accuracy rate; the other is the recall rate, that is, whether all faces of the same person are in the same file. The more faces are missed, the lower the recall rate.

[0128] Current offline face clustering methods fall into two categories: unsupervised and supervised. Unsupervised methods cluster facial images based on specific similarity metrics and manually designed sample merging strategies. These methods lack robustness, resulting in inferior clustering results compared to supervised learning-based clustering algorithms. Supervised methods cluster facial images by using a learnable model to partition the K-Nearest Neighbor (KNN) graph of facial features into multiple subgraphs. While these methods offer significant advantages in clustering accuracy and recall, the final clustering results often contain a large number of erroneous and isolated nodes. These erroneous nodes can lead to multiple instances in a single cluster, reducing clustering accuracy, while isolated nodes can lead to multiple instances in a single cluster, reducing recall.

[0129] Example 1: If the connections between nodes belonging to the same person in the KNN graph are called positive connections, and the connections between nodes that do not belong to the same person are called negative connections, then the main problem with the current face clustering method based on KNN subgraph segmentation is that the supervised learning model does not learn enough about feature nodes in the neighbor node set where the proportion of similar nodes is small, resulting in a large number of incorrectly clustered erroneous nodes and isolated nodes in the fine segmentation stage of the KNN graph.

[0130] When a supervised learning model predicts the probability of each undirected edge being a positive link, it may make some incorrect predictions. For example, it may predict a probability value for a positive link that is lower than a probability threshold, or a probability value for a negative link that is higher than a probability threshold. In this case, some negative links may still exist within the multiple subgraphs obtained by initially partitioning the KNN graph. These negative links can lead to three main types of nodes in the KNN subgraph: the first type of nodes are those that are connected only to a large number of non-similar nodes, referred to as error nodes; the second type of nodes are those that are connected to both a small number of similar nodes and a small number of non-similar nodes, referred to as bridge nodes; and the third type of nodes are those that are connected to a large number of similar nodes and very few non-similar nodes, referred to as internal anomaly nodes. If these special nodes are not addressed, some files in the clustering results will contain a large number of faces of different people, resulting in multiple faces in a single file, reducing the accuracy of the clustering results.

[0131] To further remove negative links that are not successfully identified by supervised learning models, current face clustering methods primarily employ heuristic negative link removal schemes. For example, these schemes employ multiple thresholds to perform a phased filtering of the remaining undirected edges in the KNN graph based on model-predicted probabilities, or utilize prior knowledge to design specific metrics for further filtering of the remaining undirected edges in the KNN graph. However, for feature nodes whose neighboring nodes contain a relatively small proportion of similar nodes, the model has insufficiently learned these nodes. Therefore, after the initial segmentation of the KNN graph, the residual negative links on these nodes cannot be identified by heuristic negative link removal schemes. Consequently, these heuristic negative link removal schemes not only fail to completely remove these negative links, but also erroneously remove some positive links. This erroneous removal of positive links can result in some bridge nodes becoming disconnected from their corresponding clusters of similar nodes, or even from all other feature nodes, leaving them as isolated feature nodes. This can lead to multiple clusters being identified, reducing the recall of the clustering results. The residual negative connections will cause some nodes to be unable to be completely separated from their corresponding non-similar nodes and become wrong nodes, resulting in the occurrence of multiple nodes in one file and reducing the accuracy of the clustering results.

[0132] The face clustering method based on the supervised learning model in this embodiment first uses a specific feature extraction model to extract the features of the face image, and then clusters the face images based on the facial features. It should be noted that this embodiment is applied to offline face clustering scenarios, such as clustering and archiving face images stored locally, and classifying face images belonging to the same face into a document corresponding to the same face, so as to facilitate various processes such as face verification in the subsequent process. The implementation steps of the face clustering method based on the supervised learning model involved in this embodiment are as follows:

[0133] Step (1) uses a specific feature extraction model to extract the corresponding facial features from each face image, and then obtains a set of facial features corresponding to all face images.

[0134] Step (2) calculates the K most similar facial features of each facial feature in the feature set based on a predefined feature similarity metric (such as cosine similarity).

[0135] In step (3), each facial feature is regarded as a node, and each node is connected to its corresponding K most similar facial feature nodes with an undirected edge, thereby forming a KNN (K-Nearest Neighbor) graph.

[0136] Step (4) uses a learnable model to score the connections in the KNN graph, and divides the KNN graph into multiple subgraphs according to the scores to obtain a preliminary segmentation result of the KNN graph.

[0137] Step (5) uses a manually designed segmentation strategy to further refine the initial segmentation result of the KNN graph to obtain a refined segmentation result of the KNN graph.

[0138] Step (6) considers each subgraph of the fine segmentation result as a cluster (archive) and archives it, ending the clustering process.

[0139] The face clustering method provided in this embodiment is based on the current clustering process. For the first KNN graph obtained by fine segmentation, the negative connections in the first KNN graph that cannot be successfully identified by the current supervised recognition model are removed again according to the sociability of the non-isolated nodes in the first KNN graph. This solves the problem of multiple people in one file and improves the accuracy of the clustering results.

[0140] like Figure 1 As shown, the implementation process of a face clustering method provided in this embodiment is as follows:

[0141] Step 100: Obtain a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0142] Among them, the connection between nodes belonging to the same face in the KNN graph is called a positive connection, and the connection between nodes that do not belong to the same face is called a negative connection.

[0143] In some embodiments, this embodiment can obtain the original KNN graph through existing methods, and obtain the first KNN graph by removing the negative connections in the original KNN graph. The process of removing the negative connections in the original KNN graph in this embodiment can be obtained by existing methods, and this embodiment does not impose too many restrictions on this.

[0144] In some embodiments, the original KNN graph is determined as follows:

[0145] Obtain an offline face image set to be classified, wherein the face image set to be classified includes facial images of multiple faces, perform facial feature extraction on each face image in the face image set to obtain corresponding facial features, and combine the facial features corresponding to each face image into a facial feature set as the facial feature set corresponding to the face image set;

[0146] Each facial feature in the facial feature set is taken as a node, and the K facial features that are most similar to the facial feature are screened from the facial feature set. Each node is connected to the nodes corresponding to its K corresponding facial features through an undirected edge, thereby forming the original KNN graph.

[0147] In some embodiments, this embodiment provides an example of obtaining a first KNN graph, as shown below:

[0148] (1) Inputting the original KNN graph into the second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph;

[0149] During implementation, for each undirected edge in the original KNN graph, the second graph convolutional network model is used to score the classification result of the undirected edge to obtain the score of each undirected edge.

[0150] In some embodiments, the second graph convolutional network model is determined as follows:

[0151] Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample;

[0152] The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

[0153] (2) removing the undirected edges whose scores are lower than the third threshold, and obtaining the original KNN graph after the removal;

[0154] After removing undirected edges with scores below the third threshold, a preliminary segmentation of the original KNN graph is achieved. However, many negative connections may still exist in the original KNN graph after the removal. Therefore, the original KNN graph needs to be segmented again to obtain a refined segmentation result (the first KNN graph). The details are shown below.

[0155] (3) According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than the fourth threshold are removed to obtain a first KNN graph.

[0156] In some embodiments, the intimacy between the nodes is determined by:

[0157] The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

[0158] In the implementation, it is assumed that the nodes at both ends of any undirected edge in the original KNN graph after removal are v1 and v2 respectively; the intimacy NI of v1 and v2 is calculated using the following formula (1):

[0159]

[0160] Among them, NI represents the intimacy between nodes, k represents the number of common neighbor nodes of nodes v1 and v2 in the original KNN graph after removal, that is, the total number of nodes that are directly connected to both nodes v1 and v2; n1 represents the number of undirected edges containing node v1, and n2 represents the number of undirected edges containing node v2.

[0161] In implementation, a preset fourth threshold is used to filter the undirected edges in the original KNN graph after removal, removing undirected edges with an intimacy NI lower than the fourth threshold, thereby obtaining a refined segmentation result of the original KNN graph and obtaining the first KNN graph. The fourth threshold can be set to a fixed value, such as 0.7.

[0162] Step 101: Based on the sociability of the non-isolated node in the first KNN graph, undirected edges containing the non-isolated node are removed again to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated node and its first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated node via undirected edges.

[0163] In some embodiments, the sociability of non-isolated nodes in the first KNN graph is determined as follows:

[0164] Determine the similarity between the non-isolated node and its first-order neighbor nodes; and determine the maximum similarity among the similarities as the sociability of the non-isolated node. In implementation, the first-order neighbor nodes of the non-isolated node represent nodes directly connected to the non-isolated node via undirected edges.

[0165] It should be noted that after obtaining the first KNN graph, in order to further remove the negative connections in the first KNN graph, it is necessary to search for suspected error nodes in the first KNN graph that may contain negative connections, and then remove all connections corresponding to these suspected error nodes. The specific implementation method is as follows:

[0166] For the i-th non-isolated node v in the first KNN graph i (where a non-isolated node represents a node connected by an undirected edge), the node v is calculated using the following formula (2): i NC i :

[0167]

[0168] Among them, nbr i Represents node v i The set of first-order neighbor nodes in the first KNN graph, i.e., the node v i The set of directly connected nodes; S j Represents node v i The cosine similarity of the feature with the j-th first-order neighbor node.

[0169] In some embodiments, if the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

[0170] After obtaining the sociability NC of all non-isolated nodes in the first KNN graph, the undirected edges in the first KNN graph are filtered using a preset first threshold value, such as a fixed value of 0.7, to remove all undirected edges connected to non-isolated nodes with a sociability NC below the first threshold value, thereby obtaining a second KNN graph.

[0171] Step 102: Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the second KNN graph.

[0172] In some embodiments, any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

[0173] The face clustering method provided in this embodiment, after initially and finely segmenting the original KNN graph to obtain a first KNN graph, further utilizes the sociability of non-isolated nodes to filter out negative connections, thereby reducing the probability of clustering multiple people in a group and improving clustering accuracy. In the first KNN graph, the sociability of each non-isolated node is calculated. Any undirected edge in the first KNN graph containing at least one non-isolated node with a sociability less than a first threshold is filtered out.

[0174] In some embodiments, after removing undirected edges multiple times through the above steps, this embodiment may remove a large number of undirected edges belonging to positive connections. In order to avoid the situation where one person has multiple files, this embodiment can also reconnect the isolated nodes in the second KNN graph, thereby improving the recall rate of face clustering and reducing the probability of one person having multiple files.

[0175] In some embodiments, as Figure 2 As shown, after obtaining the second KNN graph and before determining the face clustering result corresponding to the face image set based on the connected subgraph contained in the second KNN graph, the method for re-predicting and reconnecting isolated nodes is specifically as follows:

[0176] Step 200: For any isolated node in the second KNN graph, obtain a second-order neighborhood subgraph of the isolated node from the original KNN graph;

[0177] The second-order neighborhood subgraph represents the connection relationship between the isolated node, the first-order neighbor nodes of the isolated node, and the second-order neighbor nodes in the original KNN graph, wherein the second-order neighbor nodes are nodes other than the isolated node that are directly connected to the first-order neighbor nodes through undirected edges;

[0178] In some embodiments, the sample second-order neighborhood subgraph is determined by:

[0179] Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set;

[0180] Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features;

[0181] According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

[0182] Step 201: Input the second-order neighborhood subgraph into a first graph convolutional network model, and output the predicted values ​​of the undirected edges contained in the second-order neighborhood subgraph;

[0183] Among them, the predicted value represents the probability that the corresponding undirected edge belongs to a positive connection, and the positive connection represents the connection between nodes corresponding to the same face; similarly, the negative connection represents the connection between nodes that do not correspond to the same face.

[0184] In some embodiments, the first graph convolutional network model is determined as follows:

[0185] Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph;

[0186] The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

[0187] In some embodiments, the first graph convolutional network model in this embodiment includes a first graph convolutional network layer and a first fully connected network layer.

[0188] In some embodiments, in the process of training the first initial model, the true value of the predicted value of the positive undirected edge is set to 1, and the true value of the predicted value of the negative undirected edge is set to 0. Then, the training loss of the first initial model is calculated using the cross entropy loss function, and the weight parameters are updated using the SGD algorithm. For the i-th sample node v in the training set, i , suppose that all first-order neighbor sample nodes of the sample node in the original KNN graph constitute a set V nbr , V nbr There are a total of sample nodes corresponding to the sample face features of m people, and the number of sample nodes belonging to each person’s sample face features is [x1, x2, x3, …, x m ], y is V nbr and sample node v i The total number of first-order neighbor sample nodes belonging to the same face, S avg is the sample node v i The average cosine similarity with all similar first-order neighbor sample nodes, where the similar first-order neighbor sample nodes are used to represent sample nodes belonging to the same face and are represented as the same sample cluster in the original KNN graph.

[0189] When the sample node v i When the following formula is satisfied:

[0190] y>2*max j∈[1,m] x j and S avg >0.7 Formula (5);

[0191] The first initial model will skip the sample node v during training. i Otherwise, extract the sample node v in the original KNN graph i The second-order neighborhood subgraph of is used for training the first initial model. Other hyperparameters during training can use common default settings. The purpose of skipping certain sample nodes during training is to skip sample nodes that are less likely to become isolated points during the clustering process, allowing the model to focus on learning sample nodes that are likely to become isolated points, thereby improving the accuracy of the output results and the recall rate of face clustering.

[0192] Step 202: Reconnect the isolated nodes in the second KNN graph according to the target prediction value and the undirected edges corresponding to the target prediction value to obtain a reconnected second KNN graph.

[0193] It should be noted that in this embodiment, when reconnecting the isolated nodes in the second KNN graph, the original KNN graph is needed. First, any isolated node v is obtained from the original KNN graph. i The corresponding second-order neighborhood subgraph g i , wherein the second-order neighborhood subgraph includes the isolated node, the first-order neighbor nodes directly connected to the isolated node, the second-order neighbor nodes directly connected to each first-order neighbor node, and the connected undirected edges. The first graph convolutional network layer in the first graph convolutional network model is used to perform feature aggregation on each node in the second-order neighborhood subgraph, wherein the feature aggregation process is: summing the feature vector corresponding to each node in the second-order neighborhood subgraph, the first-order feature vector corresponding to the first-order neighbor nodes of the node, and the second-order feature vector corresponding to the second-order neighbor nodes, and then calculating the average value to obtain the aggregated feature vector obtained after the node feature aggregation. Through this feature aggregation process, the second-order neighborhood subgraph corresponding to each isolated node in the second KNN graph is feature aggregated to obtain the aggregated feature vector corresponding to each node in the second-order neighborhood subgraph.

[0194] The mathematical expression of the feature aggregation process is shown in the following formula (3):

[0195]

[0196] in, Represents the output of the first graph convolutional network layer, N sub Represents the second-order neighborhood subgraph g iThe total number of nodes, d1 represents the feature dimension of each node in the output of the first graph convolutional network layer, Represents the second-order neighborhood subgraph g i The corresponding results of all nodes in the second graph convolutional network layer output; is a diagonal matrix and A i is the second-order neighborhood subgraph g i The corresponding adjacency matrix; Represents the element in the j-th row and k-th column of the adjacency matrix; It is a learnable parameter matrix used to transform features into a new feature space; d2 represents the feature dimension of each node in the output of the second graph convolutional network layer; σ is a nonlinear function, and the ReLU function is generally used.

[0197] It should be noted that in the above-mentioned process of calculating feature aggregation, the results output by the second graph convolutional network layer are used. Since this embodiment needs to calculate the feature aggregation results of the second-order neighborhood subgraph, a fast and portable calculation method is to obtain the feature aggregation results corresponding to the second-order neighborhood subgraph of the isolated node from the results output by the second graph convolutional network layer in the second graph convolutional network model, that is, to obtain the feature aggregation vector corresponding to each node in the second-order neighborhood subgraph from the results output by the second graph convolutional network layer. Since the second graph convolutional network model is used to perform preliminary segmentation on the original KNN graph, and the second graph convolutional network model includes a second graph convolutional network layer for performing first-order feature aggregation on each node in the original KNN graph to obtain the feature aggregation vector of each node, and the present application uses the feature aggregation vector obtained by first-order feature aggregation of each node in the second-order neighborhood subgraph of each isolated node, and performs first-order feature aggregation again, thereby obtaining the corresponding feature aggregation vector obtained by performing second-order feature aggregation on each node in the second-order neighborhood subgraph. The feature aggregation process of this embodiment can include more neighbor nodes, thereby improving the receptive field of the model.

[0198] In some embodiments, after obtaining F2, for the isolated node v i , for each undirected edge connected in the original KNN graph, let the feature vectors of the two nodes connected by the undirected edge be and F′ a and F′ b After concatenation along the feature dimension, the vectors are input into the first fully connected network layer. For example, a 5*1 vector and a 5*1 vector are concatenated into a 10*1 vector according to the vector dimension. Finally, the target prediction value of the undirected edge belonging to the positive connection is output. The mathematical expression of this process is shown in formula (4):

[0199] P′=σ([F′a ,F′ b ]W′2)W′1 formula (4);

[0200] Among them, P′ represents the target prediction value of the undirected edge belonging to the positive connection, is the weight parameter of the second fully connected network layer, d′ hid is the output feature dimension of the second fully connected network layer; σ is a nonlinear function, and the ReLU function is generally used; is the weight parameter of the first fully connected network layer.

[0201] It should be noted that if Figure 3 As shown, the second graph convolutional network model includes a second graph convolutional network layer 300 and a second fully connected network layer 301. The second graph convolutional network layer of the second graph convolutional network model is used to perform feature aggregation on each node in the input original KNN graph to obtain a corresponding feature aggregation vector, and the feature aggregation process is to perform feature aggregation on the node and the first-order neighbor nodes of the node to obtain a vector. According to the feature aggregation vector of each node obtained after feature aggregation, the second fully connected network layer is used to classify the undirected edges containing two nodes to obtain the score of each undirected edge, thereby filtering out the undirected edges according to the score.

[0202] like Figure 4 As shown, the first graph convolutional network model includes a first graph convolutional network layer 400 and a first fully connected network layer 401. The first graph convolutional network layer is used to perform feature aggregation on each node in the input second-order neighborhood subgraph to obtain a corresponding feature aggregation vector. The feature aggregation process is to perform feature aggregation on the node, the first-order neighbor nodes of the node, and the second-order neighbor nodes of the node to obtain a vector. According to the feature aggregation vector of each node obtained after feature aggregation, the first fully connected network layer is used to classify the undirected edges containing two nodes to obtain the predicted value of each undirected edge, so as to reconnect the isolated nodes according to the predicted value.

[0203] In some embodiments, reconnection of isolated nodes is performed as follows:

[0204] If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value;

[0205] The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0206] During implementation, if the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, the target nodes other than the isolated nodes connected to the undirected edges corresponding to the maximum predicted value are determined; in the second KNN graph, the isolated nodes and the target nodes included in the undirected edges are reconnected to obtain a reconnected second KNN graph.

[0207] In the implementation, for the isolated node v in the second KNN graph i , find the isolated node v i The maximum predicted value among all the corresponding undirected edges in the original KNN graph is calculated. If the maximum predicted value is greater than a preset second threshold, the undirected edge is re-added to the second KNN graph. The second threshold is a preset fixed value, such as 0.9. After traversing all isolated nodes in the second KNN graph and adding undirected edges that meet the above conditions, the resulting second KNN graph is called the reconnected second KNN graph (for distinction, it can also be called the third KNN graph).

[0208] In some embodiments, after reconnecting the isolated nodes in the second KNN graph using the above method, a face clustering result corresponding to the face image set is determined based on the connected subgraphs contained in the reconnected second KNN graph (third KNN graph). In implementation, any connected subgraph in the reconnected second KNN graph can be used as a cluster to determine the face clustering result corresponding to the face image set, where the cluster represents a set of face images of the same person.

[0209] This embodiment, targeting the offline face clustering model, adds the recognition and processing of erroneous and isolated nodes in the refined segmentation results (first KNN graph) of the original KNN graph. The erroneous nodes in the first KNN graph are identified and processed based on the sociability of non-isolated nodes, reducing the occurrence of multiple people in one file in the final clustering results and improving the accuracy of the clustering results. The first graph convolutional network model is used to re-predict the undirected edges of each isolated node in the original KNN graph based on the second-order neighborhood subgraph, reconnecting some isolated nodes to their corresponding similar nodes, effectively reducing the occurrence of multiple people in one file and further improving the recall rate of the clustering results.

[0210] like Figure 5 As shown, this embodiment also provides a specific face clustering method, and the implementation process of the method is as follows:

[0211] Step 500: Input the original KNN graph into the second graph convolutional network model, and output the score of the undirected edges of the original KNN graph;

[0212] Step 501: Remove undirected edges with scores lower than a third threshold, and remove undirected edges with intimacy between nodes corresponding to the undirected edges in the original KNN graph lower than a fourth threshold, to obtain a first KNN graph.

[0213] Step 502: If the sociability of the non-isolated nodes in the first KNN graph is less than the first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph;

[0214] Step 503: For any isolated node in the second KNN graph, obtain the second-order neighborhood subgraph of the isolated node from the original KNN graph;

[0215] Step 504: Input the second-order neighborhood subgraph into the first graph convolutional network model, and output the predicted values ​​of the undirected edges contained in the second-order neighborhood subgraph;

[0216] The predicted value represents the probability that the corresponding undirected edge belongs to a positive connection;

[0217] Step 505: If the maximum predicted value among the predicted values ​​of the undirected edges is higher than the second threshold, the undirected edge corresponding to the maximum predicted value is determined; and the two nodes included in the undirected edge are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0218] Step 506: Take any connected subgraph in the reconnected second KNN graph as a cluster, and determine the face clustering result corresponding to the face image set.

[0219] This embodiment addresses current face clustering methods based on supervised learning models and employs strategies for error node identification and isolated node re-prediction to further improve the accuracy and recall of clustering results. After obtaining a refined segmentation result from the KNN graph, the statistical characteristics of the error nodes in the refined segmentation result are used to delineate a set of suspected error nodes. All connections corresponding to these suspected error nodes are then severed, turning them into isolated nodes, thereby further removing any remaining negative connections in the refined segmentation result. The first graph convolutional network model is then used to re-predict the probability that all undirected edges of all isolated nodes in the original KNN graph are positive connections, restoring some of the positive connections. This, in turn, improves both the accuracy and recall of the final clustering result.

[0220] Example 2: Based on the same inventive concept, the embodiment of the present invention also provides a device for face clustering. Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0221] like Figure 6 As shown, the device includes:

[0222] An acquiring unit 600 is configured to acquire a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges from an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0223] a removing unit 601 configured to remove undirected edges containing the non-isolated node again based on the sociability of the non-isolated node in the first KNN graph, to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated node and its first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated node via undirected edges;

[0224] The determining unit 602 is configured to determine a face clustering result corresponding to the face image set according to the connected subgraphs included in the second KNN graph.

[0225] As an optional implementation manner, the removing unit 601 is specifically configured to determine the sociability of the non-isolated nodes in the first KNN graph in the following manner:

[0226] Determining the similarity between the non-isolated node and the first-order neighbor nodes of the non-isolated node;

[0227] The maximum similarity among the similarities is determined as the sociability of the non-isolated node.

[0228] As an optional implementation manner, the removing unit 601 is specifically configured to:

[0229] If the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

[0230] As an optional implementation, before determining the face clustering result corresponding to the face image set based on the connected subgraphs included in the second KNN graph, the method further includes a reconnection unit configured to:

[0231] For any isolated node in the second KNN graph, obtaining a second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents a connection relationship between the isolated node, the first-order neighbor nodes of the isolated node, and the second-order neighbor nodes in the original KNN graph, wherein the second-order neighbor nodes are nodes other than the isolated node that are directly connected to the first-order neighbor nodes through undirected edges;

[0232] Inputting the second-order neighborhood subgraph into a first graph convolutional network model, and outputting predicted values ​​of undirected edges contained in the second-order neighborhood subgraph, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent connections between nodes corresponding to the same face;

[0233] The isolated nodes are reconnected in the second KNN graph according to the predicted values ​​and the undirected edges corresponding to the predicted values ​​to obtain a reconnected second KNN graph.

[0234] As an optional implementation manner, the reconnection unit is specifically configured to:

[0235] If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value;

[0236] The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0237] As an optional implementation manner, the reconnection unit is specifically configured to determine the first graph convolutional network model in the following manner:

[0238] Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph;

[0239] The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

[0240] As an optional implementation manner, the reconnection unit is specifically configured to determine the sample second-order neighborhood subgraph in the following manner:

[0241] Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set;

[0242] Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features;

[0243] According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

[0244] As an optional implementation manner, the acquiring unit 600 is specifically configured to:

[0245] Inputting the original KNN graph into a second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph;

[0246] Removing the undirected edges whose scores are lower than a third threshold to obtain an original KNN graph after the removal;

[0247] According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than a fourth threshold are removed to obtain a first KNN graph.

[0248] As an optional implementation, the acquisition unit 600 is specifically configured to determine the intimacy between the nodes in the following manner:

[0249] The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

[0250] As an optional implementation, the acquisition unit 600 is specifically configured to determine the second graph convolutional network model in the following manner:

[0251] Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample;

[0252] The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

[0253] As an optional implementation manner, the determining unit 602 is specifically configured to:

[0254] Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the reconnected second KNN graph.

[0255] As an optional implementation manner, the determining unit 602 is specifically configured to:

[0256] Any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

[0257] Example 3. Based on the same inventive concept, the embodiment of the present invention also provides a device for face clustering. Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0258] like Figure 7 As shown, the device includes a processor 700 and a memory 701, wherein the memory 701 is used to store a program executable by the processor 700, and the processor 700 is used to read the program in the memory 701 and perform the following steps:

[0259] Obtaining a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0260] removing undirected edges containing the non-isolated nodes again according to the sociability of the non-isolated nodes in the first KNN graph to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated nodes and their first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated nodes via undirected edges;

[0261] A face clustering result corresponding to the face image set is determined based on the connected subgraphs included in the second KNN graph.

[0262] As an optional implementation manner, the processor 700 is specifically configured to determine the sociability of non-isolated nodes in the first KNN graph in the following manner:

[0263] Determining the similarity between the non-isolated node and the first-order neighbor nodes of the non-isolated node;

[0264] The maximum similarity among the similarities is determined as the sociability of the non-isolated node.

[0265] As an optional implementation manner, the processor 700 is specifically configured to execute:

[0266] If the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

[0267] As an optional implementation manner, before determining the face clustering result corresponding to the face image set based on the connected subgraph included in the second KNN graph, the processor 700 is further configured to execute:

[0268] For any isolated node in the second KNN graph, obtaining a second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents a connection relationship between the isolated node, the first-order neighbor nodes of the isolated node, and the second-order neighbor nodes in the original KNN graph, wherein the second-order neighbor nodes are nodes other than the isolated node that are directly connected to the first-order neighbor nodes through undirected edges;

[0269] Inputting the second-order neighborhood subgraph into a first graph convolutional network model, and outputting predicted values ​​of undirected edges contained in the second-order neighborhood subgraph, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent connections between nodes corresponding to the same face;

[0270] The isolated nodes are reconnected in the second KNN graph according to the predicted values ​​and the undirected edges corresponding to the predicted values ​​to obtain a reconnected second KNN graph.

[0271] As an optional implementation manner, the processor 700 is specifically configured to execute:

[0272] If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value;

[0273] The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

[0274] As an optional implementation, the processor 700 is specifically configured to determine the first graph convolutional network model in the following manner:

[0275] Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph;

[0276] The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

[0277] As an optional implementation manner, the processor 700 is specifically configured to determine the sample second-order neighborhood subgraph in the following manner:

[0278] Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set;

[0279] Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features;

[0280] According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

[0281] As an optional implementation manner, the processor 700 is specifically configured to execute:

[0282] Inputting the original KNN graph into a second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph;

[0283] Removing the undirected edges whose scores are lower than a third threshold to obtain an original KNN graph after the removal;

[0284] According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than a fourth threshold are removed to obtain a first KNN graph.

[0285] As an optional implementation manner, the processor 700 is specifically configured to determine the intimacy between the nodes in the following manner:

[0286] The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

[0287] As an optional implementation, the processor 700 is specifically configured to determine the second graph convolutional network model in the following manner:

[0288] Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample;

[0289] The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

[0290] As an optional implementation manner, the processor 700 is specifically configured to execute:

[0291] Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the reconnected second KNN graph.

[0292] As an optional implementation manner, the processor 700 is specifically configured to execute:

[0293] Any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

[0294] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0295] Obtaining a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images;

[0296] removing undirected edges containing the non-isolated nodes again according to the sociability of the non-isolated nodes in the first KNN graph to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated nodes and their first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated nodes via undirected edges;

[0297] A face clustering result corresponding to the face image set is determined based on the connected subgraphs included in the second KNN graph.

[0298] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0299] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that performs the functions specified in one or more boxes.

[0300] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0301] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0302] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for face clustering, characterized in that: The method includes: Obtaining a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images; removing undirected edges containing the non-isolated nodes again according to the sociability of the non-isolated nodes in the first KNN graph to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated nodes and their first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated nodes via undirected edges; Determine the face clustering result corresponding to the face image set according to the connected subgraph contained in the second KNN graph; before determining the face clustering result corresponding to the face image set according to the connected subgraph contained in the second KNN graph, the method further comprises: for any isolated node in the second KNN graph, obtain the second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents the connection relationship between the isolated node, the first-order neighboring nodes and the second-order neighboring nodes in the original KNN graph, wherein the The second-order neighbor nodes are nodes that are directly connected to the first-order neighbor nodes through undirected edges except for the isolated nodes; the second-order neighborhood subgraph is input into the first graph convolutional network model, and the predicted values ​​of the undirected edges contained in the second-order neighborhood subgraph are output, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent the connections between nodes corresponding to the same face; according to the predicted values ​​and the undirected edges corresponding to the predicted values, the isolated nodes are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

2. The method according to claim 1, characterized in that The sociability of the non-isolated nodes in the first KNN graph is determined as follows: Determining the similarity between the non-isolated node and the first-order neighbor nodes of the non-isolated node; The maximum similarity among the similarities is determined as the sociability of the non-isolated node.

3. The method according to claim 1, characterized in that The step of removing undirected edges containing the non-isolated nodes in the first KNN graph again according to the sociability of the non-isolated nodes to obtain a second KNN graph includes: If the sociability of the non-isolated nodes in the first KNN graph is less than a first threshold, the undirected edges containing the non-isolated nodes are removed again to obtain a second KNN graph.

4. The method according to claim 1, wherein The step of reconnecting the isolated nodes in the second KNN graph according to the predicted values ​​and the undirected edges corresponding to the predicted values ​​to obtain a reconnected second KNN graph includes: If the maximum predicted value among the predicted values ​​of the undirected edges is higher than a second threshold, determining the undirected edge corresponding to the maximum predicted value; The nodes included in the undirected edges are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

5. The method according to claim 1, wherein The first graph convolutional network model is determined as follows: Taking the sample second-order neighborhood subgraph as input, determining the loss function value according to the output result and the true predicted value of the undirected edge corresponding to the sample second-order neighborhood subgraph; The loss function value is used as an optimization condition, and the stochastic gradient descent (SGD) algorithm is used to train the weight parameters in the first initial model to obtain a trained first graph convolutional network model; wherein the first initial model includes a first graph convolutional network layer and a first fully connected network layer.

6. The method according to claim 5, characterized in that The sample second-order neighborhood subgraph is determined by: Performing feature extraction on the sample face images in the sample face image set to obtain a corresponding sample face feature set; Generating an original sample KNN graph according to the sample face feature set, wherein the sample nodes in the original sample KNN graph represent the sample face features; According to the sample nodes for training in the original sample KNN graph, a sample second-order neighborhood subgraph of the sample nodes is determined, wherein the sample nodes for training represent sample nodes that are more likely to become isolated points during the first removal of undirected edges in the original sample KNN graph.

7. The method according to claim 1, characterized in that The obtaining of the first KNN graph includes: Inputting the original KNN graph into a second graph convolutional network model, and outputting the score of the undirected edges of the original KNN graph; Removing the undirected edges whose scores are lower than a third threshold to obtain an original KNN graph after the removal; According to the intimacy between the nodes corresponding to the undirected edges in the original KNN graph after the removal, the undirected edges with intimacy lower than a fourth threshold are removed to obtain a first KNN graph.

8. The method according to claim 7, characterized in that The intimacy between the nodes is determined as follows: The intimacy between the nodes is determined according to the number of edges containing each of the nodes and the total number of nodes directly connected to each of the nodes.

9. The method according to claim 7, characterized in that The second graph convolutional network model is determined as follows: Taking the original KNN graph of the sample as input, determining the loss function value according to the true score of the undirected edges corresponding to the output result and the original KNN graph of the sample; The loss function value is used as the optimization condition, and the stochastic gradient descent SGD algorithm is used to train the weight parameters in the second initial model to obtain a trained second graph convolutional network model; wherein the second initial model includes a second graph convolutional network layer and a second fully connected network layer.

10. The method according to claim 1, characterized in that The determining, based on the connected subgraphs included in the second KNN graph, a face clustering result corresponding to the face image set includes: Determine a face clustering result corresponding to the face image set based on the connected subgraphs included in the reconnected second KNN graph.

11. The method according to any one of claims 1 to 10, characterized in that: The determining, based on the connected subgraphs included in the second KNN graph, a face clustering result corresponding to the face image set includes: Any connected subgraph in the second KNN graph is taken as a cluster to determine a face clustering result corresponding to the face image set, wherein the cluster represents a face image set of the same face.

12. A face clustering device, characterized in that: include: an acquiring unit, configured to acquire a first KNN graph, wherein the first KNN graph is obtained by first removing undirected edges in an original KNN graph, wherein the original KNN graph is generated based on a set of facial features corresponding to a set of facial images; a removing unit, configured to remove undirected edges containing the non-isolated node again according to the sociability of the non-isolated node in the first KNN graph, to obtain a second KNN graph, wherein the sociability represents the similarity between the non-isolated node and its first-order neighbor nodes, where the first-order neighbor nodes are nodes directly connected to the non-isolated node via undirected edges; A determination unit is configured to determine a face clustering result corresponding to the face image set according to a connected subgraph included in the second KNN graph; before determining the face clustering result corresponding to the face image set according to the connected subgraph included in the second KNN graph, the method further includes a reconnection unit configured to: for any isolated node in the second KNN graph, obtain a second-order neighborhood subgraph of the isolated node from the original KNN graph, wherein the second-order neighborhood subgraph represents a connection relationship between the isolated node, the first-order neighboring nodes of the isolated node, and the second-order neighboring nodes in the original KNN graph. system, wherein the second-order neighbor nodes are nodes that are directly connected to the first-order neighbor nodes through undirected edges except the isolated nodes; the second-order neighborhood subgraph is input into the first graph convolutional network model, and the predicted values ​​of the undirected edges contained in the second-order neighborhood subgraph are output, wherein the predicted values ​​represent the probability that the corresponding undirected edges belong to positive connections, and the positive connections represent the connections between nodes corresponding to the same face; according to the predicted values ​​and the undirected edges corresponding to the predicted values, the isolated nodes are reconnected in the second KNN graph to obtain a reconnected second KNN graph.

13. A face clustering device, characterized in that: The device includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and execute the steps of any one of the methods of claims 1 to 11.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images