Face image classification method, face image classification device and storage medium
By combining graph convolutional networks and global-local graph convolutional networks in a face image classification method, the problem of distinguishing noisy images in existing technologies is solved, achieving higher accuracy in face image classification and data cleaning.
Patent Information
- Application Number
- CN201911082553.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-11-07
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-11-07
AI Technical Summary
Existing face image datasets contain a large number of noisy images. Existing data cleaning methods are difficult to effectively distinguish between target face images and noisy images, especially when scene lighting and face clarity are similar. Noisy nodes are easily incorrectly connected to the maximum connected subgraph.
A face image classification method is adopted, which involves receiving an image set, extracting features, calculating similarity, constructing a graph, and using a graph convolutional network to determine the node probability value. The classification nodes are the target face image and the noisy image. A secondary classification is performed by combining global and local graph convolutional networks to improve accuracy.
It improves the accuracy of face image classification, achieves more effective data cleaning, reduces misclassification of noisy images, and enhances the purity of the dataset.
Smart Images

Figure CN112784636B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a face image classification method, a face image classification device and a storage medium. Background Art
[0002] It is known that there are many large-scale face image datasets, such as MS-Celeb-1M, MegaFace2, etc. In these face image datasets, there are millions of face images of people, and there are enough face images for each person. These face image datasets can be used to train face recognition systems based on deep learning. However, these face image datasets contain many noisy images, that is, face images that are incorrectly labeled as belonging to a certain person or non-face images, so automatic and effective data cleaning methods are needed.
[0003] Furthermore, it is known that there are two mainstream data cleaning methods. The first method is a center-based cleaning method, according to which, for a set of face images marked as corresponding to a certain person, the center of the image set is calculated, and the image close to the center is determined as the face image corresponding to this person. The second method is a graph-based connectivity prediction method, according to which, for a set of face images marked as corresponding to a certain person, a graph is constructed and node connectivity is predicted, and the largest connected subgraph is determined as the set of face images corresponding to this person. A face recognition data cleaning method based on a maximum connected subgraph is provided in Chinese invention patent application publication CN107368812A. Fig.17 An example image set is shown in (a) of FIG. 1 , which is marked as corresponding to Einstein. It can be seen that the image set contains several other people's face images. Fig.17 (b) schematically shows a center-based cleaning method, according to which nodes within the judgment boundary are determined to represent the face image of Einstein. Fig.17 (c) schematically shows a graph-based connectivity prediction method, according to which the node of the largest connected subgraph is determined to be a face image representing Einstein. Fig.17 In (a), “face 1” is used to refer to the facial image of Einstein, and “face 2” is used to refer to the facial images of other people. Summary of the invention
[0004] An object of the present disclosure is to provide an improved method and apparatus for classifying facial images.
[0005] The present disclosure proposes a method for classifying facial images. The method for classifying facial images includes the following steps: receiving an image set, wherein the images in the image set are marked as corresponding to the same target person; extracting features of each image in the image set; calculating the similarity between each pair of images in the image set based on the extracted features; constructing a graph based on the calculated similarity, wherein each image in the image set is represented by a node, and the edges connecting the nodes represent the similarity relationship between the nodes; determining a probability value of each node based on the extracted features and the constructed graph by using a first graph convolutional network, wherein the probability value indicates the probability that the node represents a facial image of the target person; and classifying the nodes into nodes representing the facial image of the target person and nodes representing noise images based on the determined probability values, thereby obtaining a node classification result of the graph.
[0006] Other characteristics, features and advantages of the present disclosure will become apparent from the following description with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the specification, are used to explain the principles of the present disclosure without limitation. In the various figures, similar reference numerals are used to represent similar items. The various figures, which are not necessarily drawn to scale, describe selected illustrative embodiments and are not intended to limit the scope of the present disclosure.
[0008] Figure 1 is a block diagram of an exemplary face image classification device according to some embodiments of the present disclosure.
[0009] Figure 2 is a flowchart illustrating an exemplary face image classification method according to some embodiments of the present disclosure.
[0010] Figure 3 is a block diagram of an exemplary face image classification device according to the first embodiment of the present disclosure.
[0011] Figure 4 is a flowchart illustrating an exemplary face image classification method according to the first embodiment of the present disclosure.
[0012] Figure 5 An architectural diagram of a global graph convolutional network according to the first embodiment of the present disclosure is shown.
[0013] Figure 6 A schematic diagram of forward propagation from hidden layer l to hidden layer (l+1) of a global graph convolutional network according to the first embodiment of the present disclosure is shown.
[0014] Figure 7 4 is a block diagram of an exemplary facial image secondary classification device according to the second embodiment of the present disclosure.
[0015] Figure 8 is a flowchart illustrating an exemplary facial image secondary classification method according to the second embodiment of the present disclosure.
[0016] Fig. 9 An architectural diagram of a local graph convolutional network according to a second embodiment of the present disclosure is shown.
[0017] Fig.10 A schematic diagram showing node classification and graph classification using a global graph convolutional network and a local graph convolutional network according to a second embodiment of the present disclosure is shown.
[0018] Fig.11 The interactive training of a global graph convolutional network and a local graph convolutional network according to the second embodiment of the present disclosure is shown.
[0019] Fig.12 is a block diagram of an exemplary facial image classification post-processing device according to the third embodiment of the present disclosure.
[0020] Fig.13 is a flowchart illustrating an exemplary facial image classification post-processing method according to the third embodiment of the present disclosure.
[0021] Fig.14 is a block diagram of an exemplary album image updating device according to a fourth embodiment of the present disclosure.
[0022] Fig.15 is a flowchart illustrating an exemplary album image updating method according to a fourth embodiment of the present disclosure.
[0023] Fig.16 A general hardware environment in which the present disclosure may be applied according to some embodiments of the present disclosure is shown.
[0024] Fig.17 Two mainstream data cleaning methods and the method of the present disclosure are schematically illustrated. DETAILED DESCRIPTION
[0025] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the described exemplary embodiments. However, it is clear to those skilled in the art that the described embodiments may be practiced without some or all of these specific details. In the described exemplary embodiments, well-known structures or processing steps are not described in detail in order to avoid unnecessarily obscuring the concepts of the present disclosure.
[0026] The blocks in each block diagram shown below can be implemented by hardware, software, firmware or any combination thereof to implement the principles of the present disclosure. It should be understood by those skilled in the art that the blocks described in each block diagram can be combined or divided into sub-blocks to implement the principles of the present disclosure.
[0027] The steps of the method presented in this disclosure are intended to be illustrative. In certain embodiments, the method can be completed with one or more additional steps not described and / or completed without the one or more steps discussed. In addition, the order in which the steps of the method are illustrated and described is not intended to be restrictive.
[0028] In the face recognition data cleaning method based on the maximum connected subgraph as described above, there are the following problems. For a group of face images with similar scene lighting and similar face definition (such as face images of multiple players in a football game), there is usually a high similarity between the face images in this group. Considering such a group of face images, noise nodes (i.e., nodes that are not face images representing the target person) are likely to be incorrectly connected to the maximum connected subgraph. In other words, the cleaned face image set will still contain noise images.
[0029] The face classification method and device disclosed herein can improve the classification accuracy of face images, thereby achieving more effective data cleaning.
[0030] Figure 1 is a block diagram of an exemplary face image classification device 100 according to some embodiments of the present disclosure.
[0031] like Figure 1 As shown, the device 100 may include an image set receiving component 110, which is configured to receive an image set, in which images in the image set are marked as corresponding to the same target person; a feature extraction component 120, which is configured to extract features of each image in the image set; a similarity calculation component 130, which is configured to calculate the similarity between each pair of images in the image set based on the extracted features; a graph construction component 140, which is configured to construct a graph based on the calculated similarities, in which each image in the image set is represented by a node, and the edges connecting the nodes represent the similarity relationship between the nodes; a probability value determination component 150, which is configured to determine the probability value of each node based on the extracted features and the constructed graph by using a first graph convolutional network, and the probability value indicates the probability that the node represents the facial image of the target person; a node classification component 160, which is configured to classify the nodes into nodes representing the facial image of the target person and nodes representing noise images based on the determined probability values, thereby obtaining a node classification result of the graph.
[0032] Figure 22 is a flowchart illustrating an exemplary face image classification method 200 according to some embodiments of the present disclosure. The method 200 may include: at step S210, receiving an image set, the images in the image set are marked as corresponding to the same target person; at step S220, extracting features of each image in the image set; at step S230, calculating the similarity between each pair of images in the image set based on the extracted features; at step S240, constructing a graph based on the calculated similarity, in which each image in the image set is represented by a node, and the edges connecting the nodes represent the similarity relationship between the nodes; at step S250, based on the extracted features and the constructed graph, determining a probability value of each node by using a first graph convolutional network, the probability value indicating the probability that the node represents a face image of the target person; and at step S260, classifying the nodes into nodes representing face images of the target person and nodes representing noise images based on the determined probability values, thereby obtaining a node classification result of the graph.
[0033] As we all know, a graph convolutional network is a neural network architecture for performing machine learning on graphs.
[0034] exist Fig.17 (d) schematically shows the classification method according to the present disclosure. In summary, the classification method according to the present disclosure classifies each node in the graph by using a graph convolutional network. The classification method according to the present disclosure is described in detail below with reference to the embodiments.
[0035] First embodiment
[0036] Next, we will refer to Figures 3 to 6 A first embodiment of the present disclosure is described.
[0037] Figure 3 is a block diagram of an exemplary face image classification device 300 according to the first embodiment of the present disclosure.
[0038] like Figure 3As shown, the apparatus 300 may include: an image set receiving component 310, configured to receive an image set, wherein the images in the image set are marked as corresponding to the same target person; a feature extraction component 320, configured to extract features of each image in the image set; a similarity calculation component 330, configured to calculate the similarity between each pair of images in the image set based on the extracted features; a graph construction component 340, configured to construct a graph based on the calculated similarities, in which each image in the image set is represented by a node, and the edges connecting the nodes represent the similarity relationship between the nodes; a probability The value determination component 350 is configured to determine the probability value of each node based on the extracted features and the constructed graph by using a global graph convolutional network (corresponding to the first graph convolutional network), and the probability value indicates the probability that the node represents the facial image of the target person; the node update feature output component 360 is configured to output the updated features of each node of the graph obtained from the global graph convolutional network; the node classification component 370 is configured to classify the nodes into nodes representing the facial image of the target person and nodes representing the noise image based on the determined probability values, thereby obtaining the node classification result of the graph.
[0039] The following will further describe in detail Figure 3 The operation of each component is shown.
[0040] Figure 4 4 is a flowchart illustrating an exemplary face image classification method 400 according to the first embodiment of the present disclosure.
[0041] The method 400 starts from step S410, where the image set receiving component 310 receives an image set, and the images in the image set are marked as corresponding to the same target person. The images in the image set can be marked with the name of the person, a pre-set number of the person, etc. For example, the data set included in MS Celeb 1M, MegaFace2, etc. can be used. In addition to the face image of the target person, this data set may also include face images of other people or non-face images.
[0042] At step S420, the feature extraction component 320 extracts features of each image in the image set. Known face recognition methods can be used to extract features of face images. For example, deep learning features of face images can be generated by using a convolutional neural network (CNN). Typically, the features of a face image are represented by a vector with a predetermined dimension, such as a 512-dimensional vector. The dimension of the feature vector varies depending on the face recognition method used.
[0043] At step S430 , the similarity calculation unit 330 calculates the similarity between each pair of images in the image set based on the extracted features.
[0044] First, the feature vector of each image is L2 regularized to obtain the image feature set X = [x1, x2, ..., x N ], where N is the number of images, x i is the L2-normalized feature vector of the i-th image in the image dataset.
[0045] Then, the pairwise cosine similarity matrix for each image pair in the image dataset is calculated according to the following equation (1):
[0046] S=X·X T (1)
[0047] Among them, the cosine similarity between the i-th image and the j-th image in the image dataset is:
[0048] S(i,j)=x i ·x j .
[0049] It should be noted that other known methods may be used to calculate the similarity between image pairs.
[0050] At step S440, the graph construction component 340 constructs a graph based on the calculated similarity, in which each image in the image set is represented by a node, and the edge connecting the nodes represents the similarity relationship between the nodes. Specifically, constructing the graph may include: representing each image in the image set as a node; for each node, sorting the similarity between the node and each other node; selecting other nodes in the top K positions of the similarity sort, and establishing an edge between the node and each of the other nodes. K is an integer greater than or equal to 1.
[0051] At step S450, the probability value determination component 350 inputs the features extracted at step S420, the similarity calculated at step S430, and the graph constructed at step S440 into the global graph convolution network, wherein the value of each node in the graph constructed at step S440 is set to the extracted feature of the image represented by the node, and the probability value determination component 350 outputs the probability value of each node obtained from the first graph convolution network. The probability value of each node indicates the probability that the node represents the face image of the target person.
[0052] The following references Figure 5 Describe the architecture of the global graph convolutional network. First, describe the forward propagation process of the global graph convolutional network.
[0053] like Figure 5As shown, the global graph convolutional network has an input layer, L hidden layers (L is an integer greater than or equal to 1, such as L = 4), and an output layer.
[0054] As mentioned above, the input layer receives the extracted features, the calculated similarities, and the constructed graph. The value of each node in the constructed graph is set to the extracted features x1, x2, ..., x of the image represented by the node. N Each hidden layer outputs the updated features of each node Here l represents the hidden layer number, and l takes an integer value from 1 to L. The output layer outputs the probability value P1, P2, ..., P of each node. N .
[0055] Reference below Figure 6 Describe the forward propagation process of the global graph convolutional network from hidden layer (l) to (l+1). The forward propagation process includes the following steps (a)-(c).
[0056] (a) Calculate the aggregate features of the neighbors of node i in the hidden layer (l), where i represents the node number and i takes values from 1 to N. This step (a) is composed of Figure 6 The "aggregation" block in .
[0057] The aggregation function is shown in formula (2):
[0058]
[0059] Among them, N i is the set of neighbors of node i (including node i itself), is the feature of a neighbor node of node i, A (l) and b (l) is a learnable parameter in the hidden layer l, and σ is the activation function. The activation function can be the ELU activation function. The dimension of the aggregated feature of node i is d. The dimension of the aggregated feature of node i can be compared with a feature vector x input to the global graph convolutional network. i The dimension of is the same as (eg, 512 dimensions). Here, the neighbors of node i may include at least one of the following: a node directly connected to node i through an edge; a node connected to node i through an edge and one or more other nodes.
[0060] (b) Transform the aggregated features of node i. This step (b) is composed of Figure 6 The “⊕” symbol in .
[0061] The feature transformation formula is shown in formula (3):
[0062]
[0063] in, is the characteristic of the node itself, || is the connection operator, Here, we weight the features of each neighbor node of node i according to its similarity with node i, and the weight is Where S ij is the similarity between node i and node j, D ii is the degree of node i (the number of edges connected to node i in the graph), D jj is the degree of neighbor node j.
[0064] The aggregated features of the neighbors of node i are connected with the features of node i to form the transformed features of node i. The dimension of the transformed features is 2d.
[0065] (c) Calculate the features of node i in the hidden layer (l+1) The step (c) is performed by Figure 6 The "fully connected layer", "batch normalization" and "activation function" blocks in are represented.
[0066] The calculation formula is shown in formula (4):
[0067]
[0068] Among them, W (l) is a learnable parameter in the hidden layer l. Let the transformed features of node i in the hidden layer (l) pass through a fully connected layer, undergo batch normalization and pass into the activation function σ to obtain the features of node i in the hidden layer (l+1) Features of node i in hidden layer (l+1) is d-dimensional. The activation function may be an ELU activation function. It should be noted that the activation functions used in steps (a) and (c) may be different.
[0069] The forward propagation process from the input layer to the hidden layer 1 and from the hidden layer L to the output layer is the same as the forward propagation process from the hidden layer (l) to (l+1), which will not be repeated here. It should be noted that in the forward propagation process from the hidden layer L to the output layer, the activation function used is the Sigmoid function shown in the following formula (5).
[0070]
[0071] By using the Sigmoid function as the activation function, it is possible to ensure that the feature of each node in the output layer is a value between 0 and 1, i.e., a probability value, which indicates the probability that the node represents the face image of the target person. As previously described, the probability value determination component 350 outputs the probability value of each node as the output of the output layer.
[0072] Return to reference Figure 4 At step S460, the node update feature output component 360 outputs the update feature of each node of the graph obtained from the global graph convolution network. Specifically, the update feature of each node of the graph refers to the output of the last hidden layer (hidden layer L) of the global graph convolution network, that is, as shown in Figure 5 As shown in It should be noted that this step S460 is an optional step. In the case where the secondary classification described below is not performed, this step can be omitted.
[0073] At step S470, the node classification component 370 classifies the nodes into nodes representing facial images of the target person (also referred to as "normal nodes") and nodes representing noise images (also referred to as "noise nodes") based on the probability values determined at step S450, thereby obtaining the node classification results of the graph. Here, the noise image refers to facial images or non-facial images of other persons other than the target person. More specifically, component 370 classifies the nodes into normal nodes and noise nodes by comparing the determined probability value with a first threshold. For example, the first threshold can be set to 0.5. Then, if the probability value of a node is greater than or equal to 0.5, component 370 classifies the node as a normal node, otherwise component 370 classifies the node as a noise node. It should be understood that the classification result of the node is the classification result of the image in the image set. Further, "1" can be used to mark normal nodes, and "0" can be used to mark noise nodes. This classification step is Figure 5 and Figure 6 is shown as a "node classification" block.
[0074] Next, refer to Figure 5 Describe the back propagation process of a global graph convolutional network.
[0075] In the process of back propagation, the network model is trained using stochastic gradient descent. For each node, the loss is represented by the binary cross entropy loss function. For a global graph convolutional network containing N nodes, the loss is as follows (6):
[0076]
[0077] Among them, y i is the probability value determined by the i-th node, and the output value range is [0,1], is the ground truth of the i-th node. G is used to train the global graph convolutional network so that the loss LG minimize the value of .
[0078] It should be understood that the back propagation process of the global graph convolutional network is the training process of the global graph convolutional network. The trained global graph convolutional network is used to determine the probability value of the node and the update feature of the node in the above steps S450 and S460.
[0079] In the first embodiment, by using the graph convolutional network to classify nodes, accurate node classification (i.e., image classification) can be achieved. Moreover, compared with the face recognition data cleaning method based on the maximum connected subgraph mentioned above, the classification method according to the first embodiment can more accurately classify face images with high similarity.
[0080] Second embodiment
[0081] As previously mentioned, in the first embodiment, the classification of images in the image set is implemented. In the second embodiment, the nodes whose classification results in the first embodiment may be questionable are subjected to secondary classification to achieve a more accurate classification of the nodes (i.e., images). In other words, the classification method described in the second embodiment is performed after the classification method described in the first embodiment. In this case, the classification described in the first embodiment can be referred to as a primary classification or a rough classification, while the classification described in the second embodiment can be referred to as a secondary classification or a fine classification.
[0082] Next, we will refer to Figures 7 to 11 A second embodiment of the present disclosure is described.
[0083] Figure 7 is a block diagram of an exemplary facial image secondary classification device 700 according to the second embodiment of the present disclosure.
[0084] like Figure 7As shown, the device 700 may include: a suspicious node determination component 710, configured to determine a suspicious node whose probability value is between a second threshold and a third threshold based on the probability value output at step S450; a subgraph construction component 720, configured to construct a subgraph for each suspicious node, the subgraph including at least the suspicious node, the neighboring nodes of the suspicious node, and the edges established between the suspicious node and its neighboring nodes; a subgraph probability value determination component 730, configured to determine the probability value of each node in the subgraph and determine the probability value of the subgraph by using a local graph convolutional network (corresponding to the second graph convolutional network) based on the updated features of each node of the graph output at step S460 and the constructed subgraph, the probability value of the subgraph indicating the probability that the subgraph is a normal subgraph; subgraph node classification Component 740 is configured to classify each node in the constructed subgraph based on the determined node probability value of the subgraph; subgraph classification component 750 is configured to classify the constructed subgraph into a normal subgraph and a noise subgraph based on the determined probability value of the subgraph; classification result fusion component 760 is configured to fuse the node classification results of multiple subgraphs to obtain a fused node classification result, and fuse the subgraph classification results of multiple subgraphs to obtain a graph classification result for the graph, wherein the graph classification result includes a normal graph and a noise graph; classification result fusion component 770 is configured to update the node classification result of the graph by using the fused node classification result and the graph classification result for the graph to obtain an updated node classification result, thereby classifying the nodes again into normal nodes and noise nodes.
[0085] Here, the noise subgraph refers to a subgraph in which all nodes are noise nodes. The normal subgraph refers to a subgraph containing at least one normal node. The noise graph refers to a graph in which all nodes are noise nodes. The normal graph refers to a graph containing at least one normal node. Here, in each constructed subgraph, the suspicious node is the subgraph center.
[0086] The following will further describe in detail Figure 7 The operation of each component is shown.
[0087] Figure 8 is a flowchart illustrating an exemplary facial image secondary classification method 800 according to the second embodiment of the present disclosure.
[0088] Method 800 starts from step S810, where the suspicious node determination component 710 determines a suspicious node whose probability value is between the second threshold and the third threshold based on the probability value output at step S450. As mentioned above, at step S470, for example, if the probability value of a certain node is greater than or equal to 0.5, then component 370 classifies the node as a normal node, otherwise component 370 classifies the node as a noise node. At this step S810, for example, if the probability value of a certain node is greater than 0.2 and less than 0.8, then component 710 classifies the node as a suspicious node. Here, a suspicious node refers to a node whose primary classification result may be doubtful, or a node that is difficult to accurately classify through a primary classification.
[0089] At step S820, the subgraph construction component 720 constructs a subgraph for each suspicious node, and the subgraph includes at least the suspicious node, the neighbor node of the suspicious node, and the edge established between the suspicious node and its neighbor node. Specifically, constructing a subgraph may include: determining the neighbor node of the suspicious node by using the edge between each node, and generating a subgraph of the edge established between at least a portion of the nodes, the suspicious node, the neighbor node of the suspicious node, and the at least one branch node, the suspicious node, and the neighbor node of the suspicious node in the node classification result of the graph. The subgraph constructed at step S820 is a subset of the graph constructed at step S440. The neighbor node of the suspicious node may include its one-hop node and two-hop node. A one-hop node refers to a node directly connected to a suspicious node through an edge. A two-hop node refers to a node that is not directly connected to a suspicious node but is connected to the one-hop node of a suspicious node through an edge. In an example, a subgraph may include a suspicious node, its neighbor node, and an edge. In another example, the subgraph may include at least a portion of nodes, suspicious nodes, neighbor nodes, and edges that are once classified as normal nodes. In yet another example, the subgraph may include all nodes, suspicious nodes, neighbor nodes, and edges that are once classified as normal nodes.
[0090] In one example, if the constructed subgraph does not contain any node classified as normal in one classification, it means that the subgraph is far away from the normal area, and thus the subgraph can be discarded.
[0091] At step S830, the subgraph probability value determination component 730 determines the probability value of each node in the subgraph and determines the probability value of the subgraph by using the local graph convolution network based on the updated features of each node of the graph output at step S460 and the constructed subgraph. Specifically, component 730 inputs the updated features of each node of the graph output at step S460, the similarity calculated at step S430, and the subgraph constructed at step S820 into the local graph convolution network, wherein the value of each node of the subgraph is set to the updated features of the node obtained from the global graph convolution network, and component 730 outputs the probability value of each node in the subgraph obtained from the local graph convolution network. In addition, component 730 also outputs the probability value of the subgraph obtained from the local graph convolution network.
[0092] The following references Fig. 9 Describe the architecture of local graph convolutional networks.
[0093] like Fig. 9 As shown in Figure 1, the local graph convolutional network has one input layer, L hidden layers, and two output layers. The number of hidden layers in the local graph convolutional network is the same as that in the global graph convolutional network. In addition, the forward propagation process from hidden layer (l) to hidden layer (l+1) of the local graph convolutional network is the same as that of the global graph convolutional network. The following describes the differences between the local graph convolutional network and the global graph convolutional network.
[0094] As mentioned above, the input layer of the local graph convolutional network receives the updated features of each node output from the global graph convolutional network. The similarity S of each node pair as the input of the global graph convolutional network ij , and the constructed subgraph. The value of each node in the constructed subgraph is set to the updated feature of each node obtained from the global graph convolutional network More specifically, the value of node i in the constructed subgraph is set to the updated feature of node i
[0095] An output layer of the local graph convolutional network outputs the probability value P1, P2, ..., P of each node in the subgraph N, the probability value indicates the probability that the node represents the face image of the target person. Here, the generation process of the node probability value of the subgraph is the same as the generation process of the node probability value of the graph in the global graph convolutional network. Another output layer of the local graph convolutional network outputs a probability value indicating the probability that the subgraph is a normal subgraph. The process of generating this probability value is described below. The feature vectors of all nodes in the hidden layer L of the local graph convolutional network are averaged, and then the obtained average feature vector is passed through a fully connected layer and a Sigmoid activation function to obtain a probability value indicating the probability that the subgraph is a normal subgraph. This probability value is between 0 and 1.
[0096] It should be noted that the number of hidden layers and the forward propagation process between hidden layers of the local graph convolutional network can be different from that of the global graph convolutional network.
[0097] At step S840, the subgraph node classification component 740 classifies each node in the constructed subgraph based on the node probability value of the subgraph determined at step S830. More specifically, the component 740 classifies the nodes into normal nodes and noise nodes by comparing the determined probability value with a fourth threshold value. The fourth threshold value may be, for example, 0.5.
[0098] At step S850, the sub-graph classification component 750 classifies the constructed sub-graph into a normal sub-graph or a noise sub-graph based on the probability value of the sub-graph determined at step S830. More specifically, the component 750 classifies the sub-graph into a normal sub-graph or a noise sub-graph by comparing the determined probability value with a fifth threshold value. The fifth threshold value may be, for example, 0.5.
[0099] For each suspicious node, the steps S820-S850 as described above are performed. For multiple suspicious nodes, the steps S820-S850 can be performed serially one by one. Alternatively, for multiple suspicious nodes, the steps S820-S850 can be performed in parallel. This can shorten the time for secondary classification.
[0100] At step S860 , the classification result fusion component 760 fuses the node classification results of the multiple subgraphs to obtain a fused node classification result, and fuses the subgraph classification results of the multiple subgraphs to obtain a graph classification result for the graph.
[0101] Fusion of node classification results of multiple subgraphs may include: for at least a portion of all nodes included in the multiple subgraphs, fusing node classification results of multiple subgraphs corresponding to the same node. For example, for each of all nodes included in the multiple subgraphs, node classification results of multiple subgraphs corresponding to the same node may be fused. If there is only a node classification result from one subgraph for a node, then this node classification result is used as the classification result of the node; and if there are node classification results from multiple subgraphs for a node, then the fusion result of the multiple node classification results is used as the classification result of the node. Here, fusion may include voting using the multiple node classification results, and using the classification result with more votes as the classification result of the node. Alternatively, fusion may include using multiple node probability values corresponding to the same node to calculate a fused probability value, and then obtaining a fused node classification result based on the fused probability value. More specifically, a weighted sum of multiple node probability values corresponding to the same node may be calculated to obtain a fused probability value.
[0102] Alternatively, fusing the node classification results of the plurality of subgraphs includes: selecting a classification result for a suspicious node from the node classification results of each subgraph as a fused node classification result.
[0103] Fusion of the subgraph classification results of the multiple subgraphs may include voting using the multiple subgraph classification results, and using the classification result with the most votes as the graph classification result. Alternatively, fusion of the subgraph classification results of the multiple subgraphs may include calculating a fused probability value using the probability values of the multiple subgraphs, and then obtaining the graph classification result based on the fused probability value. More specifically, a weighted sum of the probability values of the multiple subgraphs may be calculated to obtain the fused probability value.
[0104] At step S870, the classification result updating component 770 updates the node classification result of the graph by using the fused node classification result and the graph classification result for the graph to obtain an updated node classification result, thereby classifying the nodes again into normal nodes and noise nodes. More specifically, the fused node classification result can be used to replace the node classification result of the corresponding graph. In one example, the classification result for each suspicious node determined at step S860 can be used to replace the node classification result of the corresponding graph. It should be noted that the present disclosure is not limited to this, and the fused node classification result determined at step S860 can be used to replace the node classification results of more graphs. In addition, in the case where a graph is classified as a noise graph, all nodes in this graph are classified as noise nodes, and all nodes in this graph are marked with "0".
[0105] exist Fig.10A schematic diagram of using a global graph convolutional network and a local graph convolutional network for node classification and graph classification is shown in FIG. Fig.10 , the output results of the subgraph construction step S820, the subgraph node classification step S840, the node classification result fusion step S860, and the updating step S870 as described above are shown. Fig.10 shows the case where local graph convolution operations are performed in parallel. In this case, parameters can be shared between the local graph convolution operations performed in parallel. In particular, Fig.10 The box in the lower right corner of shows the voting process for a suspicious node. In addition, it should be noted that Fig.10 shows the case where all sub-graphs are normal sub-graphs. It should also be noted that Fig.10 The figures and sub-figures shown in the figure are merely illustrative and are not intended to limit the scope of the present disclosure. Fig.10 The graph and subgraph shown in do not fully show the K edges extending from each node.
[0106] As described above, by using the two-stage classification method (i.e., a method including primary classification and secondary classification) as described in the second embodiment, at least suspicious nodes can be classified more accurately, thereby improving the accuracy of node classification. Once again, compared to the face recognition data cleaning method based on the maximum connected subgraph as mentioned above, the classification method according to the second embodiment can excellently classify face images with high similarity.
[0107] Next, refer to Fig. 9 Describe the back-propagation process of a local graph convolutional network.
[0108] In the process of back propagation, as shown in the following formula (7), the loss L L It consists of two parts, L n represents the node classification loss, L g represents the graph classification loss.
[0109] L L =L n +λL g (7)
[0110] Among them, the loss L n and loss L g It is represented by the binary cross entropy loss function as shown in formula (6), where λ is a preset constant. L is used to train the local graph convolutional network so that the loss L LThe value of is minimized. It should be understood that the back propagation process of the local graph convolutional network is the training process of the local graph convolutional network. The trained local graph convolutional network is used to determine the node probability value and the probability value of the subgraph in the above step S830. In addition, as described below, the loss L L It is also used to train global graph convolutional networks.
[0111] Below, reference Fig.11 Describes interactive training of global and local graph convolutional networks.
[0112] like Fig.11 As shown by the dotted line ① in the figure, by using the loss L of the global graph convolutional network G To update the global graph convolutional network. Here, the loss L G Represents the difference between the node classification result of the graph and its true value. More specifically, the global graph convolutional network is updated so that the loss L G The value of is minimized. Fig.11 As shown by the dotted line indicated by ② in the figure, by using the loss L of the local graph convolutional network L To update the local graph convolutional network. Here, the loss L L Represents the difference between the updated node classification results and the graph classification results and their true values. Similarly, the local graph convolutional network is updated so that the loss L L In addition, if Fig.11 As shown by the dotted line ③ in the figure, the loss L L Multiply it by the preset scaling factor α and then pass it back to the global graph convolutional network and pass it through the global loss L G and the scaled local loss L L To update the global graph convolutional network again. And, as mentioned above, the updated features of each node of the graph obtained from the global graph convolutional network are input into the local graph convolutional network, and the value of each node of the subgraph is set to the updated feature of the node obtained from the global graph convolutional network.
[0113] The interactive training strategy is better than training the two graph convolutional networks separately. This is because the local graph convolutional network does not learn difficult examples from the beginning, but learns from easy to difficult according to the output of the global graph convolutional network. On the other hand, the global graph convolutional network also adapts to the local graph convolutional network and updates itself according to the learning direction of the local graph convolutional network.
[0114] Third embodiment
[0115] The third embodiment is devoted to the post-processing of the face image classification method described in the first and second embodiments. In the first and second embodiments, the processing for one data set is described, while in the third embodiment, the processing for multiple data sets for which classification has been completed is described.
[0116] Next, we will refer to Figure 12 to Figure 13 A third embodiment of the present disclosure is described.
[0117] Fig.12 is a block diagram of an exemplary facial image classification post-processing device 1200 according to the third embodiment of the present disclosure.
[0118] like Fig.12 As shown, the device 1200 may include: a noise image deletion component 1210, configured to delete noise images from an image set, thereby updating the image set; an image set merging component 1220, configured to compare and merge multiple updated image sets, thereby obtaining a merged image set, which corresponds to the same target person; an image restoration component 330, configured to restore images erroneously deleted from each image set, thereby expanding the merged image set; and a duplicate image deletion component 1240, configured to delete duplicate images from the merged image set.
[0119] The following will further describe in detail Fig.12 The operation of each component is shown.
[0120] Fig.13 is a flowchart illustrating an exemplary facial image classification post-processing method 1300 according to the third embodiment of the present disclosure.
[0121] At step S1310, the noise image removal component 1210 removes the noise image from the image set, thereby updating the image set. In other words, the noise image removal component 1210 removes the noise node from the graph corresponding to the image set, thereby updating the graph. As a result, a cleaned graph, i.e., a cleaned image set, is obtained.
[0122] At step S1320, the image set merging component 1220 compares and merges multiple updated image sets to obtain a merged image set, which corresponds to the same target person. In one example, the image set merging component 1220 calculates the center of each updated graph, finds graphs with overlapping or close centers, and merges the found graphs. As a result, a merged graph corresponding to the same target person can be obtained. Here, the center is close to the distance between the centers is less than or equal to the preset distance. For example, the image set merging component 1220 can calculate the average value of the values of each node of each updated graph as the center of each updated graph.
[0123] At step S1330, the image restoration component 330 restores the image deleted from each image set by mistake, thereby expanding the merged image set. In one example, for an image deleted from each of the merged multiple image sets, the image restoration component 330 compares the features of the image with the center of the merged image, and restores the image to the image set obtained at step S1320 when the similarity between the image and the center of the merged image is greater than a sixth threshold.
[0124] At step S1340, the duplicate image deleting component 1240 deletes duplicate images from the image set obtained at step S1330. In one example, the duplicate image deleting component 1240 uses a vector of locally aggregated descriptors (VLAD) clustering method to delete duplicate images from the image set.
[0125] By adopting the facial image classification post-processing according to the third embodiment, a high-quality cleaned facial image set can be obtained, and the facial image set corresponds to the same target person.
[0126] Fourth embodiment
[0127] In the fourth embodiment, an apparatus and method for classifying new images in an album are described. Figure 14 to Figure 15 A fourth embodiment of the present disclosure is described.
[0128] Fig.14 is a block diagram of an exemplary album image updating device 1400 according to the fourth embodiment of the present disclosure.
[0129] like Fig.14As shown, the device 1400 may include: a graph and image receiving component 1410, configured to receive a graph corresponding to a set of images in an album, in which each image in the image set is represented by a node, the edges connecting the nodes represent the similarity relationship between the nodes, the value of each node represents the extracted features of the corresponding image, and in the graph, each node has been classified as a normal node or a noise node, and receives a new image input into the image set; a new node creating component 1420, configured to extract features of the new image, calculate the similarity between the new image and each other image in the image set, and create a new node corresponding to the new image and edges connected to the new node; a subgraph constructing component 1430, configured to construct a subgraph, which includes at least a new node, neighboring nodes of the new node, and edges established between the new node and its neighboring nodes; a new image classifying component 1440, configured to determine the probability value of the new node based on the features of the new image and the constructed subgraph by using a graph convolutional network, and, based on the determined probability value, classify the new node as a normal node or a noise node.
[0130] The following will further describe in detail Fig.14 The operation of each component is shown.
[0131] Fig.15 is a flowchart illustrating an exemplary album image updating method 1500 according to the fourth embodiment of the present disclosure.
[0132] At step S1510, the graph and image receiving component 1410 receives the graph as described above corresponding to the image set in the album, and receives a new image input into the image set. The image set in the album may include, for example, a face image of the owner (i.e., the target person), face images of other people, and face images of non-human faces (such as landscape photos, etc.).
[0133] At step S1520, the new node creation component 1420 extracts features of the new image, calculates the similarity between the new image and each other image in the image set, and creates a new node corresponding to the new image and an edge connected to the new node. Here, the operations of extracting features and calculating similarity are similar to those in the reference Figure 4 Here, the steps to create an edge connecting to a new node are the same as those in reference Figure 4 The operations for setting up edges are the same as described.
[0134] At step S1530, the subgraph construction component 1430 constructs a subgraph, which includes at least the new node, the neighbor nodes of the new node, and the edges established between the new node and its neighbor nodes. The neighbor nodes of the new node may include single-hop nodes and double-hop nodes of the new node. In one example, the constructed subgraph includes: at least a portion of the normal nodes in the graph received at step S1510, the new node, the neighbor nodes of the new node, and the edges established between all these nodes.
[0135] At step S1540, the new image classification component 1440 determines the probability value of the new node by using a graph convolution network based on the features of the new image and the constructed subgraph, and, based on the determined probability value, classifies the new node as a normal node or a noise node. More specifically, the new image classification component 1440 inputs the constructed subgraph, the features of the new image, the values of the nodes in the graph (representing the extracted features of the corresponding image) and the similarity between the features into the graph convolution network, and outputs the probability value of the new node obtained from the graph convolution network. Further, the new image classification component 1440 classifies the new node as a normal node representing the owner's face image or a noise node representing a noise image by comparing the probability value of the new node with the seventh threshold. Here, the noise image refers to the face image or non-face image of other people other than the owner. Optionally, the normal node can be marked with the number "1", and the noise node can be marked with the number "0". Here, the seventh threshold can be, for example, 0.5.
[0136] According to the album update method of the fourth embodiment, by using a graph convolutional network to classify new images, the new images can be accurately classified. More specifically, the new images can be accurately classified as the owner's image or the noise image.
[0137] It should be noted that the above describes the situation of classifying a new image as an image of the owner or an image of a non-owner. However, it is clear that the apparatus and method described above can also be used to classify a new image as an image of a target person (such as a relative, friend, etc. of the owner) or an image of a non-target person.
[0138] Hardware Implementation
[0139] Fig.16 A general hardware environment 1600 is shown in which the present disclosure may be applied according to an exemplary embodiment of the present disclosure.
[0140] refer to Fig.16, a computing device 1600 will now be described as an example of a hardware device applicable to various aspects of the present disclosure. The computing device 1600 may be any machine configured to perform processing and / or computing, and may be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smart phone, a portable camera, or any combination thereof. Each of the above-mentioned apparatuses 100, 300, 700, 1200, 1400 may be implemented in whole or at least in part by the computing device 1600 or a similar device or system.
[0141] The computing device 1600 may include elements that can be connected to or communicate with the bus 1602 via one or more interfaces. For example, the computing device 1600 may include a bus 1602, one or more processors 1604, one or more input devices 1606, and one or more output devices 1608. The one or more processors 1604 may be any type of processor, and may include, but are not limited to, one or more general-purpose processors and / or one or more special-purpose processors (such as special-purpose processing chips). The input device 1606 may be any type of device that can input information to the computing device, and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 1608 may be any type of device that can present information, and may include, but are not limited to, a display, a speaker, a video / audio output terminal, and / or a printer. The computing device 1600 may also include or be connected to a non-transient storage device 1610, which may be any storage device that is non-transient and can implement a data repository, and may include but is not limited to a disk drive, an optical storage device, a solid-state storage device, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, a compact disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions and / or code. The non-transient storage device 1610 may be detachable from the interface. The non-transient storage device 1610 may have data / instructions / code for implementing the above methods and steps. The computing device 1600 may also include a communication device 1612. The communication device 1612 may be any type of device or system capable of communicating with an external device and / or with a network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication equipment and / or a communication device such as Bluetooth. TM Chipsets for 1602.11 devices, WiFi devices, WiMax devices, cellular communication facilities, etc.
[0142] The bus 1602 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0143] The computing device 1600 may also include a working memory 1614 , which may be any type of working memory that can store instructions and / or data useful for operation of the processor 1604 , and may include, but is not limited to, random access memory and / or read-only memory devices.
[0144] The software elements may be located in the working memory 1614, including but not limited to an operating system 1616, one or more applications 1618, drivers and / or other data and codes. Instructions for executing the above methods and steps may be included in one or more applications 1618, and the components of each of the above devices 100, 300, 700, 1200, 1400 may be implemented by the processor 1604 reading and executing the instructions of one or more applications 1618. More specifically, the graph construction component 140 may be implemented, for example, by the processor 1604 when executing the application 1618 having instructions to execute step S240. The probability value determination component 150 may be implemented, for example, by the processor 1604 when executing the application 1618 having instructions to execute step S250. The node classification component 160 may be implemented, for example, by the processor 1604 when executing the application 1618 having instructions to execute step S260. And, similarly, the image set receiving component 110, the feature extraction component 120, and the similarity calculation component 130 can be implemented, for example, by the processor 1604 when executing an application 1618 having instructions for respectively executing steps S210, S220, and S230. In addition, similarly, the components of each of the above-mentioned apparatuses 300, 700, 1200, and 1400 can be implemented by the processor 1604 reading and executing an application 1618 having corresponding program instructions. The executable code or source code of the instructions of the software element can be stored in a non-transitory computer-readable storage medium (such as the above-mentioned (one or more) storage devices 1610) and can be read into the working memory 1614 in the case of possible compilation and / or installation. The executable code or source code of the instructions of the software element can also be downloaded from a remote location.
[0145] From the above embodiments, it will be clear to those skilled in the art that the present disclosure can be implemented by software and necessary hardware, or can be implemented by hardware, firmware, etc. Based on this understanding, the embodiments of the present disclosure can be partially implemented in the form of software. Computer software can be stored in a computer-readable storage medium, such as a floppy disk, a hard disk, an optical disk, or a flash memory. Computer software includes a series of instructions that enable a computer (such as a personal computer, a service station, or a network terminal) to run a method according to each embodiment of the present disclosure or a part thereof.
[0146] The disclosure having thus been described, it will be apparent that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the disclosure, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.
Claims
1. A face image classification method, It is characterized in that The following steps are involved: receiving a set of images, wherein images in the set are labeled as corresponding to the same target person, Extract features of each image in the image set, Based on the extracted features, the similarity between each pair of images in the image set is calculated. Based on the calculated similarities, a graph is constructed in which each image in the image set is represented by a node and the edges connecting the nodes represent the similarity relationships between the nodes. Based on the extracted features and the constructed graph, determining a probability value for each node by using a first graph convolutional network, the probability value indicating a probability that the node represents a face image of a target person, and Based on the determined probability values, the nodes are classified into nodes representing the face image of the target person and nodes representing the noise image, thereby obtaining the node classification result of the graph. Wherein, based on the determined probability value, a suspicious node whose probability value is between a first threshold and a second threshold is determined, the probability value of each node in the subgraph including the suspicious node is determined by using a second graph convolutional network, and the node classification result of the graph is updated according to the probability value of each node in the subgraph.
2. The method according to claim 1, in, The steps to construct the graph include: Represent each image in the image set as a node, For each node, rank the similarity between that node and every other node, and Select the top K nodes in similarity ranking, and establish an edge between the node and each of the other nodes, where K is an integer greater than or equal to 1.
3. The method according to claim 1, in, The steps to determine the probability value of each node include: inputting the extracted features, the calculated similarities, and the constructed graph into a first graph convolutional network, wherein the value of each node in the constructed graph is set to the extracted features of the corresponding image, and Output the probability value of each node obtained from the first graph convolutional network.
4. The method according to claim 1, in, The step of determining the probability value of each node also includes: Output the updated features for each node of the graph obtained from the first graph convolutional network.
5. The method according to claim 1, in, The steps to classify images in an image set include: By comparing the determined probability value with a threshold value, the nodes are classified into nodes representing a face image of a target person and nodes representing a noise image.
6. The method according to claim 1, further comprising: include: For each suspicious node, Constructing a subgraph, the subgraph at least including the suspicious node, the neighboring nodes of the suspicious node, and the edges established between the suspicious node and its neighboring nodes, Based on the updated features of each node of the graph obtained from the first graph convolutional network and the constructed subgraph, a probability value of each node in the subgraph is determined by using a second graph convolutional network, Based on the determined node probability value of the subgraph, each node in the constructed subgraph is classified, The node classification results of multiple subgraphs are fused to obtain a fused node classification result, and the node classification result of the graph is updated by using the fused node classification result to obtain an updated node classification result, so that the nodes are reclassified into nodes representing the face image of the target person and nodes representing the noise image.
7. The method according to claim 6, in, For each suspicious node, constructing a subgraph includes: By using the edges between nodes, the neighbor nodes of the suspicious node are determined, and A subgraph is generated, including at least a portion of the nodes classified as representing the facial image of the target person in the node classification result of the graph, the suspicious node, the neighboring nodes of the suspicious node, and the edges established between the at least a portion of the nodes, the suspicious node, and its neighboring nodes.
8. The method according to claim 6, in, Determining the probability value of each node in the subgraph includes: Inputting the updated features of each node of the graph obtained from the first graph convolutional network, the calculated similarity, and the constructed subgraph into the second graph convolutional network, wherein the value of each node of the subgraph is set to the updated feature of the node obtained from the first graph convolutional network, and Output the probability value of each node in the subgraph obtained from the second graph convolutional network.
9. The method according to claim 6, in, The node classification results of the multiple subgraphs are integrated, including: for at least a portion of all nodes included in the multiple subgraphs, the node classification results of the multiple subgraphs corresponding to the same node are integrated.
10. The method according to claim 6, in, Fusion of node classification results of multiple subgraphs includes: selecting a classification result for a suspicious node from the node classification results of each subgraph as a fused node classification result.
11. The method according to claim 6, in, The updating step includes: replacing the node classification results of the corresponding graph with the fused node classification results.
12. The method according to claim 6, further comprising: include: Based on the updated features of each node of the graph obtained from the first graph convolutional network and the constructed subgraph, a probability value of the subgraph is determined by using a second graph convolutional network, the probability value of the subgraph indicating the probability that the subgraph is a normal subgraph, Based on the determined probability values of the sub-graphs, the constructed sub-graphs are classified into normal sub-graphs and noise sub-graphs. The sub-graph classification results of multiple sub-graphs are fused to obtain the graph classification result for the graph, where the graph classification result includes a normal graph and a noise graph, and The node classification result of the graph is updated by using the fused node classification result and the graph classification result for the graph, thereby obtaining an updated node classification result.
13. The method according to claim 12, in, Training the first graph convolutional network and the second graph convolutional network involves iteratively performing the following operations: The first graph convolutional network is updated by using the loss of the first graph convolutional network, where the loss of the first graph convolutional network represents the difference between the node classification result of the graph and the true node classification result, Updating the second graph convolutional network by using the loss of the second graph convolutional network, wherein the loss of the second graph convolutional network represents the difference between the updated node classification results and the graph classification results and the true classification results, and The first graph convolutional network is updated again by using the loss of the first graph convolutional network and the loss of the second graph convolutional network, wherein the updated features of each node of the graph obtained from the first graph convolutional network are input into the second graph convolutional network, and wherein the value of each node of the subgraph is set to the updated feature of the node obtained from the first graph convolutional network.
14. The method according to claim 1 or 6, further comprising: include: Remove the noisy images from the image set, thereby updating the image set, Comparing and merging the multiple updated image sets to obtain a merged image set, the merged image set corresponds to the same target person, Restoring images that were mistakenly deleted from each image set to expand the combined image set, and Remove duplicate images from the merged image set.
15. A method for classifying new images in an album, The following steps are involved: receiving a graph corresponding to an image set in an album, in which each image in the image set is represented by a node, edges connecting the nodes represent similarity relationships between the nodes, a value of each node represents an extracted feature of the corresponding image, and in which each node has been classified as a node representing a face image of a target person or a node representing a noise image, and receiving a new image input into the image set, Extract features of the new image, compute the similarity between the new image and every other image in the image set, and create a new node corresponding to the new image and edges connecting the new nodes, Constructing a first subgraph, the first subgraph including at least the new node, the neighbor nodes of the new node, and the edges established between the new node and its neighbor nodes, and Based on the features of the new image and the constructed first subgraph, a probability value of the new node is determined by using a graph convolutional network, and, based on the determined probability value, the new node is classified as a node representing a face image of a target person or a node representing a noise image, Wherein, based on the determined probability value, a suspicious node whose probability value is between a first threshold and a second threshold is determined, the probability value of each node in the second subgraph including the suspicious node is determined by using a second graph convolutional network, and the classification of the new node is updated according to the probability value of each node in the second subgraph.
16. A facial image classification device, It is characterized in that Comprising means for performing the method according to any one of claims 1-15.
17. A facial image classification device, It is characterized in that include: at least one processor; as well as At least one storage device stores instructions, which, when executed by the at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 15.
18. A non-transitory computer-readable storage medium, Features Instructions are stored which, when executed by a processor, cause the method according to any one of claims 1 to 15 to be performed.
