A face clustering method based on graph convolutional neural network

Through the face clustering method based on graph convolution neural network, the nodes are divided using local density values, and only adaptive subgraphs are constructed for low-density nodes for inference, which solves the problem of low-scale face data clustering efficiency and improves the The accuracy and efficiency of clustering are achieved.

CN114511905BActive Publication Date: 2024-11-05HARBIN ENG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210066025.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-20
Publication Date
2024-11-05
Estimated Expiration
2042-01-20

AI Technical Summary

Technical Problem

The existing face clustering method has high time complexity when processing large-scale face data sets, traditional methods require strict data structures, and the highly overlapping of constructed subgraphs leads to waste of computing resources and low efficiency.

Method used

The face clustering method based on graph convolution neural networks, by extracting face features, calculating local density values ​​and dividing high-density and low-density nodes, only adaptive subgraphs are constructed for low-density nodes to perform graph convolutional neural network inference, combined with The pseudo-label propagation strategy iteratively removes edges that do not meet the requirements to form a final cluster.

Benefits of technology

It reduces the input data scale of graph convolutional neural networks, improves clustering efficiency and accuracy, reduces computing resource consumption, and obtains richer context information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114511905B_ABST
    Figure CN114511905B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of face clustering technology, and specifically relates to a face clustering method based on a graph convolutional neural network. The present invention first extracts features from face data, regards face features as nodes, and calculates local density values ​​of all nodes; then, based on the local density value, the data is divided into two parts: high-density nodes and low-density nodes. The high-density nodes connect the nodes with the same high density in the nearest neighbors to form multiple cluster centers; an adaptive subgraph is constructed for the low-density nodes as the input of the graph convolutional neural network to predict the connectivity between nodes; finally, after merging the two parts, pseudo-label propagation is used to cut off the edges that do not meet the requirements to obtain the final clustering result. The present invention divides the data into two parts based on density, and only constructs a subgraph for the low-density part for reasoning, which improves the efficiency of clustering. At the same time, the adaptive subgraph can extract richer context information, making the reasoning of the subgraph more accurate and improving the accuracy of clustering.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of face clustering, and in particular relates to a face clustering method based on a graph convolutional neural network. Background Art

[0002] With the development of science and technology, face recognition technology has gradually become popular and widely used in aerospace, entertainment, education, security and other fields. The performance of face recognition technology has also been gradually improved. However, the high accuracy of current face recognition technology depends on large-scale face annotation data sets. The more annotated face data sets there are, the better the effect. Although unlabeled data can be easily obtained from the Internet, manual annotation is not only time-consuming but also expensive. In order to utilize these unlabeled data, face clustering method is a good choice.

[0003] At present, face clustering and technology are widely used. Some common scenarios include grouping and marking faces in albums, data cleaning when building large-scale data sets, and new retail customer management. Traditional clustering methods have limitations, such as Spectral Clustering is sensitive to parameter selection, K-means requires the data set to be convex, and DBSCAN requires the data distribution to be relatively uniform. They often do not work well when performing face clustering, and the time complexity is too high when facing large-scale face data sets. In contrast, many link prediction methods have no requirements on data structure and can obtain better clustering results. However, link prediction methods also have the problem of high time complexity, especially when facing large-scale face data sets. Link prediction methods extract features from each face image and regard it as a node. For each node, a subgraph is constructed as input. The subgraph is inferred using a graph convolutional neural network to determine the connectivity between nodes, thereby obtaining clusters. However, the constructed subgraphs are highly overlapping, which not only causes redundant computing resource loss, but also affects the efficiency of clustering. Summary of the invention

[0004] The purpose of the present invention is to provide a face clustering method based on graph convolutional neural network.

[0005] A face clustering method based on graph convolutional neural network includes the following steps:

[0006] Step 1: Process the original face data set and extract facial features;

[0007] Step 2: Consider each facial feature as a node, and use nearest neighbor search to calculate the k nearest neighbor nodes of each node;

[0008] Step 3: Calculate the local density value of each node;

[0009] For each node i, traverse its nearest k neighbor nodes and calculate the number of nodes whose Euclidean distance to node i is less than the cutoff distance E0, which is the local density value ρ of node i. i , the process is expressed as:

[0010]

[0011] Where E0 represents the preset cutoff distance; d ij Represents the Euclidean distance between node i and node j;

[0012] Step 3: Sort the nodes in descending order according to their local density values, and divide them into high-density nodes and low-density nodes according to the set threshold;

[0013] Step 4: The high-density nodes are located in the central area of ​​the cluster and have strong connectivity, belonging to the strongly connected area; traverse each node in the strongly connected area, if there is a node in the m nearest neighbor nodes of the node that belongs to the strongly connected area, and the distance between the two nodes is less than the cutoff distance E0, then the two nodes are divided into the same high-density cluster center, and this process continues until all nodes in the strongly connected area have been traversed and the high-density nodes form different cluster centers; m is a pre-set value, and m≤k;

[0014] Step 5: Process low-density nodes, build adaptive subgraphs for low-density nodes, and use graph convolutional neural networks to infer the adaptive subgraphs to obtain the connectivity between nodes;

[0015] Step 5.1: Construct an adaptive subgraph for low-density nodes. After the construction is completed, the feature matrix F = [f1,f2,f3,...,f v ] T and the adjacency matrix A∈R |V|×|V| ; V represents the set of nodes in the adaptive subgraph;

[0016] Step 5.2: Input the feature matrix and adjacency matrix into the graph convolutional neural network. The graph convolutional network aggregates the node context information and can determine whether there is a connection between a node and the hub node. The graph convolutional layer outputs the transformed feature matrix Y.

[0017] Y=σ([F||GF]W)

[0018] Among them, W is the learnable weight parameter matrix; is an aggregation matrix, Λ is a diagonal matrix; the operator || means concatenating the matrix along the feature dimension; σ represents the Relu activation function;

[0019] Step 5.3: Input the feature matrix Y into the fully connected layer of the graph convolutional network for classification;

[0020] y=softmax(Wx+b)

[0021] Among them, y represents the probability of positive and negative samples; b is the bias term; the softmax function performs normalization operation; x represents the feature vector in the feature matrix Y;

[0022] Step 5.4: Create an edge between the central node and its first-order neighbor. The connectivity of the edge is the probability that the neighbor node is a positive sample.

[0023] Step 6: Combine the results obtained in step 4 and step 5, and finally remove the edges that do not meet the requirements through multiple iterations, and the remaining connected parts form the final clustering.

[0024] Furthermore, the method for constructing an adaptive subgraph for low-density nodes in step 5.1 is specifically as follows:

[0025] Step 5.1.1: For a low-density node p, form a node set V with its second-order neighbors;

[0026] Step 5.1.2: Traverse all first-order neighbor nodes of node p, calculate the maximum value of the Euclidean distance between each node and its k nearest neighbor nodes, and then calculate the average value of these maximum values ​​to obtain the adaptive distance threshold dis;

[0027] Step 5.1.3: Add edges to the adaptive subgraph. The resulting adaptive subgraph consists of a node set V and an edge set E, denoted as (V, E).

[0028] For any node q in the node set V, search for potential edges among q's k nearest neighbors; if a node r appears in V and its Euclidean distance to node q is less than dis, add an edge (q, r) to the edge set E; for node pairs whose Euclidean distance between nodes is greater than dis, directly set the value of the corresponding position in the adjacency matrix to 0, and do not add edges.

[0029] Furthermore, in step 6, the results obtained in step 3 and step 4 are combined to obtain a set of edges, and the weight of the edge is the final prediction result, indicating the possibility that a pair of nodes are connected. In order to obtain the final clustering, a pseudo-label propagation strategy is used to cut off the edges with weights less than a threshold in each iteration, and the remaining edge connection instances form clusters. If there is a cluster among these clusters that is larger than the predefined maximum cluster size, it is added to the queue so that it can be processed in the next iteration. During the iteration process, the threshold for cutting the edge will gradually increase, and the iteration will stop when the queue is empty, because it means that the clustering has been completed.

[0030] The beneficial effects of the present invention are:

[0031] The present invention first extracts features from face data, regards features as nodes, and calculates local density values ​​of all nodes; then, based on local density values, the data is divided into high-density nodes and low-density nodes, and high-density nodes connect nodes with the same high density in the nearest neighbors to form multiple cluster centers; an adaptive subgraph is constructed for low-density nodes, and the subgraph is identified as an adjacency matrix as the input of a graph convolutional neural network to predict the connectivity between nodes; finally, the two parts are merged and used as label propagation to remove edges that do not meet the requirements to obtain the final clustering result. The present invention divides data into two parts based on density, and only constructs a subgraph for the low-density part for reasoning, which reduces the input data scale of the graph convolutional neural network, reduces the number of subgraphs that need to be inferred, and improves the efficiency of clustering. At the same time, the adaptive subgraph can extract richer context information, making the reasoning of the subgraph more accurate, and improving the accuracy of clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a construction diagram of the adaptive subgraph in the present invention.

[0033] Figure 2 It is a framework diagram of the present invention.

[0034] Figure 3 It is a flow chart of the present invention.

[0035] Figure 4 It is a pseudo code diagram of the specific algorithm of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described below in conjunction with the accompanying drawings.

[0037] The purpose of the present invention is to address the problem of face clustering, overcome the shortcomings of the prior art, and provide a face clustering method based on a graph convolutional neural network. The present invention extracts features from a face image and regards it as a node, and divides the data into two parts based on the local density value of the node: a high-density part and a low-density part. The high-density part has strong connectivity because it exists near the cluster center and belongs to a strongly connected area. We only need to construct an adaptive subgraph for the low-density part nodes, use a graph convolutional neural network to infer the constructed subgraph, judge the connectivity of the edges, and finally merge the high and low density parts and iterate multiple times to obtain the final clustering.

[0038] A face clustering method based on graph convolutional neural network includes the following steps:

[0039] Step 1: Process the original face data set and extract facial features;

[0040] Step 2: Consider each facial feature as a node, and use nearest neighbor search to calculate the k nearest neighbor nodes of each node;

[0041] Step 3: Calculate the local density value of each node;

[0042] For each node i, traverse its nearest k neighbor nodes and calculate the number of nodes whose Euclidean distance to node i is less than the cutoff distance E0, which is the local density value ρ of node i. i , the process is expressed as:

[0043]

[0044] Where E0 represents the preset cutoff distance; d ij Represents the Euclidean distance between node i and node j;

[0045] Step 3: Sort the nodes in descending order according to their local density values, and divide them into high-density nodes and low-density nodes according to the set threshold;

[0046] Step 4: The high-density nodes are located in the central area of ​​the cluster and have strong connectivity, belonging to the strongly connected area; traverse each node in the strongly connected area, if there is a node in the m nearest neighbor nodes of the node that belongs to the strongly connected area, and the distance between the two nodes is less than the cutoff distance E0, then the two nodes are divided into the same high-density cluster center, and this process continues until all nodes in the strongly connected area have been traversed and the high-density nodes form different cluster centers; m is a pre-set value, and m≤k;

[0047] Step 5: Process low-density nodes, build adaptive subgraphs for low-density nodes, and use graph convolutional neural networks to infer the adaptive subgraphs to obtain the connectivity between nodes;

[0048] Step 5.1: Construct an adaptive subgraph for low-density nodes. After the construction is completed, the feature matrix F = [f1,f2,f3,...,f v ] T and the adjacency matrix A∈R |V|×|V| ; V represents the set of nodes in the adaptive subgraph;

[0049] The method for constructing an adaptive subgraph for low-density nodes is specifically as follows:

[0050] Step 5.1.1: For a low-density node p, form a node set V with its second-order neighbors;

[0051] Step 5.1.2: Traverse all first-order neighbor nodes of node p, calculate the maximum value of the Euclidean distance between each node and its k nearest neighbor nodes, and then calculate the average value of these maximum values ​​to obtain the adaptive distance threshold dis;

[0052] Step 5.1.3: Add edges to the adaptive subgraph. The resulting adaptive subgraph consists of a node set V and an edge set E, denoted as (V, E).

[0053] For any node q in the node set V, search for potential edges among q's k nearest neighbors; if a node r appears in V and its Euclidean distance to node q is less than dis, add an edge (q, r) to the edge set E; for node pairs whose Euclidean distance between nodes is greater than dis, directly set the value of the corresponding position in the adjacency matrix to 0, and do not add edges;

[0054] Step 5.2: Input the feature matrix and adjacency matrix into the graph convolutional neural network. The graph convolutional network aggregates the node context information and can determine whether there is a connection between a node and the hub node. The graph convolutional layer outputs the transformed feature matrix Y.

[0055] Y=σ([F||GF]W)

[0056] Among them, W is the learnable weight parameter matrix; is an aggregation matrix, Λ is a diagonal matrix; the operator || means concatenating the matrix along the feature dimension; σ represents the Relu activation function;

[0057] Step 5.3: Input the feature matrix Y into the fully connected layer of the graph convolutional network for classification;

[0058] y=softmax(Wx+b)

[0059] Among them, y represents the probability of positive and negative samples; b is the bias term; the softmax function performs normalization operation; x represents the feature vector in the feature matrix Y;

[0060] Step 5.4: Create an edge between the central node and its first-order neighbor. The connectivity of the edge is the probability that the neighbor node is a positive sample.

[0061] Step 6: Combine the results obtained in step 4 and step 5, and finally remove the edges that do not meet the requirements through multiple iterations, and the remaining connected parts form the final clustering.

[0062] In step 6, the results obtained in step 3 and step 4 are combined to obtain a set of edges. The weight of the edge is the final prediction result, indicating the possibility that a pair of nodes are connected. In order to obtain the final clustering, a pseudo-label propagation strategy is used to cut off the edges with weights less than the threshold in each iteration, and the remaining edge connection instances form clusters. If there is a cluster among these clusters that is larger than the predefined maximum cluster size, it is added to the queue for processing in the next iteration. During the iteration process, the threshold for cutting edges will gradually increase, and the iteration will stop when the queue is empty, because it means that the clustering has been completed.

[0063] Embodiment 1:

[0064] Step 1. Process the original face dataset, extract face features, and regard each feature as a node. The original face dataset, after extracting features through the convolutional neural network, is represented as X = [x1, x2, ..., x n ] T ∈R N×D , where x i Represents the feature of the i-th node, N represents the number of images, and D represents the dimension of the feature.

[0065] Step 2. Use the nearest neighbor search to calculate the k nearest neighbor nodes of each node based on the features extracted in step 1. After completing the nearest neighbor search for the node, the density partitioning operation can be performed. First, the local density value of each node needs to be calculated. The specific calculation method of the local density value of the node is as follows:

[0066]

[0067] Among them, E0 represents the preset cutoff distance, d ij represents the Euclidean distance between node i and node j, ρ i It represents the local density value of node i, which is a metric function. The specific calculation method of χ(x) is as follows:

[0068]

[0069] For each node, traverse its k nearest neighbor nodes and calculate the number of nodes whose distance to the central node is less than the cutoff distance. After calculating the local density values ​​of all nodes, the node density value is expressed as d = [d1, d2, ..., d n ].

[0070] Step 3. Divide the nodes into two parts according to density: a high-density part and a low-density part, and process the high-density part to form multiple cluster centers.

[0071] Step 4. Process the low-density nodes and construct an adaptive subgraph (ASG) for the low-density nodes. After the construction is completed, the feature matrix F = [f1, f2, f3, ..., f v ] T and the adjacency matrix A∈R |V|×|V| , the feature matrix and adjacency matrix are input into the graph convolutional neural network. The graph convolutional network aggregates node context information and can determine whether there is a connection between a node and a hub node. The input of the graph convolution layer is the feature matrix F and the adjacency matrix A, and the output is the transformed feature matrix Y. The specific calculation method is as follows:

[0072] Y=σ([F||GF]W)

[0073] in, N is the number of nodes, d in represents the feature dimension of the input, d out represents the feature dimension of the output, W is the learnable weight parameter matrix, is an aggregation matrix, A is an adjacency matrix, and Λ is a diagonal matrix. The operator || represents concatenation of matrices along the feature dimension. σ represents the Relu activation function. The fully connected layer is then used to classify the transformed features, and the transformed features Y are used as input. The specific calculation method is as follows:

[0074] y=softmax(Wx+b)

[0075] y represents the probability of positive and negative samples, W is a learnable parameter, b is a bias term, and softmax performs a normalization operation. The specific calculation method is as follows:

[0076]

[0077] Among them, z j represents the probability of the jth category, and k represents the total number of categories. Finally, an edge is established between the central node and its first-order neighbor, and the connectivity of the edge is the probability that the neighbor node is a positive sample.

[0078] Step 5. After merging the results obtained in step 3 and step 4, we get a set of edges. The weight of the edge is the final prediction result, indicating the possibility that a pair of nodes are connected. In order to obtain the final clustering, we use a pseudo-label propagation strategy. In each iteration, the edges with weights less than the threshold are cut off, and the remaining edge connection instances form clusters. If there is a cluster among these clusters that is larger than the predefined maximum cluster size, it is added to the queue for processing in the next iteration. During the iteration process, the threshold used to cut the edge will gradually increase. Stop the iteration when the queue is empty, because it means that the clustering has been completed.

[0079] Furthermore, in step 3, the data is divided into two parts based on density, high density and low density, and the specific process of forming multiple cluster centers in the high density part is as follows:

[0080] Step 1: Sort the nodes in descending order according to the local density value, and divide the data into two parts by the threshold: high-density nodes and low-density nodes. The high-density nodes are located in the central area of ​​the cluster and have strong connectivity, belonging to the strongly connected area.

[0081] Step 2: Traverse each node in the strongly connected area. If there is a node among the m nearest neighbor nodes of the node that belongs to the strongly connected area, and the distance between the two nodes is less than the cutoff distance E0, then the two nodes are divided into the same high-density cluster center. This process continues until all nodes in the strongly connected area have been traversed and the high-density parts form different cluster centers.

[0082] Furthermore, the specific process of constructing an adaptive subgraph for low-density nodes in step 4 is as follows:

[0083] Step 1: Given a node p, its second-order neighbors form a node set V. The adaptive subgraph is composed of the nodes in the node set V. The ASG is represented as G(V, E), where V represents the node set in the subgraph and E represents the edge set in the subgraph.

[0084] Step 2: Calculate the adaptive distance threshold dis, traverse all first-order neighbor nodes of node p, calculate the maximum value of the Euclidean distance between each node and its k nearest neighbor nodes, and then calculate the average value of these maximum values ​​to obtain the final adaptive distance threshold dis;

[0085] Step 3: Add edges to the subgraph. For any node q in the node set V, search for potential edges among q’s k nearest neighbors. If a node r appears in V and its distance from node q is less than dis, add an edge (q, r) to E. For those node pairs whose distance between nodes is greater than dis, set the corresponding position in the adjacency matrix to 0 directly without adding edges.

[0086] Compared with the prior art, the present invention divides the data into two parts based on density, and only constructs a subgraph for reasoning on the low-density part, which reduces the input data scale of the graph convolutional neural network, reduces the number of subgraphs that need to be inferred, and improves the efficiency of clustering. At the same time, the adaptive subgraph can extract richer contextual information, making the reasoning of the subgraph more accurate and improving the accuracy of clustering.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A face clustering method based on graph convolutional neural network, characterized in that: The following steps are involved: Step 1: Process the original face data set and extract facial features; Step 2: Consider each facial feature as a node, and use nearest neighbor search to calculate the k nearest neighbor nodes of each node; Step 3: Calculate the local density value of each node; For each node i, traverse its nearest k neighbor nodes and calculate the number of nodes whose Euclidean distance to node i is less than the cutoff distance E0, which is the local density value ρ of node i. i , the process is expressed as: Where E0 represents the preset cutoff distance; d ij Represents the Euclidean distance between node i and node j; Step 3: Sort the nodes in descending order according to their local density values, and divide them into high-density nodes and low-density nodes according to the set threshold; Step 4: The high-density nodes are located in the central area of ​​the cluster and have strong connectivity, belonging to the strongly connected area; traverse each node in the strongly connected area, if there is a node in the m nearest neighbor nodes of the node that belongs to the strongly connected area, and the distance between the two nodes is less than the cutoff distance E0, then the two nodes are divided into the same high-density cluster center, and this process continues until all nodes in the strongly connected area have been traversed and the high-density nodes form different cluster centers; m is a pre-set value, and m≤k; Step 5: Process low-density nodes, build adaptive subgraphs for low-density nodes, and use graph convolutional neural networks to infer the adaptive subgraphs to obtain the connectivity between nodes; Step 5.1: Construct an adaptive subgraph for low-density nodes. After the construction is completed, the feature matrix F = [f1,f2,f3,...,f v ] T and the adjacency matrix A∈R |V|×|V| ; V represents the set of nodes in the adaptive subgraph; Step 5.2: Input the feature matrix and adjacency matrix into the graph convolutional neural network. The graph convolutional network aggregates the node context information and can determine whether there is a connection between a node and the hub node. The graph convolutional layer outputs the transformed feature matrix Y. Y=σ([F||GF]W) Among them, W is the learnable weight parameter matrix; is an aggregation matrix, Λ is a diagonal matrix; the operator ∥ means concatenating the matrix along the feature dimension; σ represents the Relu activation function; Step 5.3: Input the feature matrix Y into the fully connected layer of the graph convolutional network for classification; y=softmax(Wx+b) Among them, y represents the probability of positive and negative samples; b is the bias term; the softmax function performs normalization operation; x represents the feature vector in the feature matrix Y; Step 5.4: Create an edge between the central node and its first-order neighbor. The connectivity of the edge is the probability that the neighbor node is a positive sample. Step 6: Combine the results obtained in step 4 and step 5, and finally remove the edges that do not meet the requirements through multiple iterations, and the remaining connected parts form the final clustering.

2. The face clustering method based on graph convolutional neural network according to claim 1, characterized in that: The method for constructing an adaptive subgraph for low-density nodes in step 5.1 is specifically as follows: Step 5.1.1: For a low-density node p, form a node set V with its second-order neighbors; Step 5.1.2: Traverse all first-order neighbor nodes of node p, calculate the maximum value of the Euclidean distance between each node and its k nearest neighbor nodes, and then calculate the average value of these maximum values ​​to obtain the adaptive distance threshold dis; Step 5.1.3: Add edges to the adaptive subgraph. The resulting adaptive subgraph consists of a node set V and an edge set E, denoted as (V, E). For any node q in the node set V, search for potential edges among q's k nearest neighbors; if a node r appears in V and its Euclidean distance to node q is less than dis, add an edge (q, r) to the edge set E; for node pairs whose Euclidean distance between nodes is greater than dis, directly set the value of the corresponding position in the adjacency matrix to 0, and do not add edges.

3. The face clustering method based on graph convolutional neural network according to claim 1, characterized in that: In step 6, the results obtained in step 3 and step 4 are combined to obtain a set of edges. The weight of the edge is the final prediction result, indicating the possibility that a pair of nodes are connected. In order to obtain the final clustering, a pseudo-label propagation strategy is used to cut off the edges with weights less than a threshold in each iteration, and the remaining edge connection instances form clusters. If there is a cluster among these clusters that is larger than the predefined maximum cluster size, it is added to the queue for processing in the next iteration. During the iteration process, the threshold for cutting the edge will gradually increase, and the iteration will stop when the queue is empty, because this means that the clustering has been completed.