Electronic device and control method thereof

Abstract

An electronic device and a method for controlling the same are disclosed. A method for controlling an electronic device according to the present disclosure comprises: acquiring a plurality of images for performing clustering; acquiring a plurality of target regions corresponding to each of the plurality of images and acquiring a plurality of feature vectors corresponding to the plurality of target regions; acquiring a plurality of center nodes corresponding to the plurality of feature vectors and acquiring neighbor nodes associated with each of the plurality of center nodes; acquiring a subgraph based on the plurality of center nodes and neighbor nodes; identifying connection probabilities between a plurality of center nodes of the subgraph and neighbor nodes of each of the plurality of center nodes based on a graph convolution network; and clustering a plurality of target regions based on the identified connection probabilities.

Description

Technology Field

[0001] The present disclosure relates to image clustering technology, and in particular to an electronic device for performing image clustering and a method for controlling the same. Background Technology

[0002] Clustering algorithms are a process that uses computer technology, data analysis, and processing techniques to group individual objects with identical physical or abstract properties into the same category. Each category generated by a clustering algorithm possesses similar abstract properties. This is widely used in image processing and other fields.

[0003] Current commercial clustering algorithms generally utilize conventional clustering algorithms, such as those based on similarity measures (K-means) that require determining clustering centers; however, the representativeness of these centers directly impacts the accuracy of subsequent clustering. Therefore, there is a need to optimize existing image clustering methods. The problem to be solved

[0004] The present disclosure was devised in response to the aforementioned necessity and proposes an electronic device and a method for controlling the same, which cluster data by acquiring feature vectors of data and acquiring nodes corresponding to the feature vectors and connection probabilities between nodes. means of solving the problem

[0005] A control method for an electronic device according to one embodiment of the present disclosure for achieving the above objective comprises: acquiring a plurality of images for performing clustering; acquiring a plurality of target regions corresponding to each of the plurality of images and acquiring a plurality of feature vectors corresponding to the plurality of target regions; acquiring a plurality of center nodes corresponding to the plurality of feature vectors and acquiring neighbor nodes associated with each of the plurality of center nodes; acquiring a subgraph based on the plurality of center nodes and the neighbor nodes; identifying connection probabilities between a plurality of center nodes of the subgraph and neighbor nodes of each of the plurality of center nodes based on a graph convolution network; and clustering the plurality of target regions based on the identified connection probabilities.

[0006] And, the step of acquiring the subgraph may include: identifying one of the plurality of feature vectors as the center node and acquiring neighbor nodes of the center node based on feature vectors different from the feature vector corresponding to the center node; and configuring the subgraph according to the center node and the neighbor nodes.

[0007] And, the step of acquiring the neighbor nodes may include the step of acquiring cosine distances between a feature vector corresponding to the center node and feature vectors different from the feature vector corresponding to the center node, respectively; and the step of screening the neighbor nodes from the feature vectors different from the feature vector corresponding to the center node based on the cosine distances.

[0008] And, the step of obtaining the connection probability may include: obtaining a node embedding of the subgraph according to at least one layer of the graph convolution network; and obtaining a connection probability between the central node of the subgraph and its neighbor nodes based on the node embedding.

[0009] And, the step of acquiring the node embedding may include: a step of acquiring an input feature corresponding to the subgraph; and a step of using the output result obtained by inputting the input feature to at least one layer of the graph convolution network for feature extraction as the node embedding.

[0010] And, the step of clustering the target regions according to the identified connection probability may include: for at least one target region to be clustered, obtaining a set of neighbor nodes of the same category according to the connection probability; for at least one neighbor node of the set of neighbor nodes of the same category, adjusting the connection probability between the at least one neighbor node and the central node according to the connection probabilities of each neighbor node of the set of neighbor nodes of the same category; clustering the target regions based on the adjusted connection probability to determine whether the at least one neighbor node is a neighbor node of the same category; and clustering each target region to be clustered according to neighbor nodes of the same category corresponding to each target region to be clustered.

[0011] And, the step of acquiring the set of neighbor nodes may include determining neighbor nodes for the at least one target area to be clustered, wherein the connection probabilities with the central node are greater than or equal to a preset threshold, and forming the determined neighbor nodes as a set of neighbor nodes of the same category.

[0012] And, the step of adjusting the connection probability may include, for at least one neighbor node of the set of neighbor nodes of the same category, determining the average connection probability of each neighbor node with the at least one neighbor node of the set of neighbor nodes of the same category as the connection probability of the at least one neighbor node and the center node.

[0013] And, the step of clustering each of the above target regions may include: determining connection probabilities between central nodes corresponding to each of the target regions to be clustered according to neighbor nodes of the same category corresponding to each of the target regions to be clustered; and clustering each of the target regions to be clustered based on the connection probabilities between each of the central nodes.

[0014] And, the method may further include the steps of: receiving a keyword for image search input by a user; identifying a category associated with the keyword based on the clustering result; and searching for images that match the keyword among the images of the identified category.

[0015] Meanwhile, according to one embodiment of the present disclosure for achieving the above objective, an electronic device comprises: a memory storing at least one instruction; and a processor connected to the memory and controlling the electronic device. By executing the at least one instruction, the processor acquires a plurality of images for performing clustering, acquires a plurality of target regions corresponding to each of the plurality of images, acquires a plurality of feature vectors corresponding to the plurality of target regions, acquires a plurality of center nodes corresponding to the plurality of feature vectors, acquires neighbor nodes associated with each of the plurality of center nodes, acquires a subgraph based on the plurality of center nodes and the neighbor nodes, identifies connection probabilities between the plurality of center nodes of the subgraph and the neighbor nodes of each of the plurality of center nodes based on a graph convolution network, and clusters the plurality of target regions based on the identified connection probabilities.

[0016] And, the processor can identify one of the plurality of feature vectors as the center node, obtain neighbor nodes of the center node based on feature vectors different from the feature vector corresponding to the center node, and construct the subgraph according to the center node and the neighbor nodes.

[0017] And, the processor can obtain cosine distances between a feature vector corresponding to the center node and feature vectors different from the feature vector corresponding to the center node, respectively, and screen neighbor nodes from feature vectors different from the feature vector corresponding to the center node based on the cosine distances.

[0018] And, the processor can obtain a node embedding of the subgraph according to at least one layer of the graph convolution network, and obtain a connection probability between the central node of the subgraph and its neighbor nodes based on the node embedding.

[0019] In addition, the processor may acquire input features corresponding to the subgraph and use the output result obtained by inputting the input features to at least one layer of the graph convolution network for feature extraction as the node embedding.

[0020] And, the processor may, for at least one target area to be clustered, acquire a set of neighbor nodes of the same category according to the connection probability, for at least one neighbor node of the set of neighbor nodes of the same category, adjust the connection probability between the at least one neighbor node and the central node according to the connection probabilities of each neighbor node of the set of neighbor nodes of the same category, cluster the target areas based on the adjusted connection probability to determine whether the at least one neighbor node is a neighbor node of the same category, and cluster each target area to be clustered according to the neighbor nodes of the same category corresponding to each target area to be clustered.

[0021] And, the processor can determine neighbor nodes for the at least one target area to be clustered, wherein the connection probabilities with the central node are greater than or equal to a preset threshold, and form the determined neighbor nodes as a set of neighbor nodes of the same category.

[0022] And, for at least one neighbor node of the set of neighbor nodes of the same category, the processor can determine the average connection probability of each neighbor node with the at least one neighbor node of the set of neighbor nodes of the same category as the connection probability of the at least one neighbor node and the center node.

[0023] And, the processor determines the connection probabilities between central nodes corresponding to each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered, and can cluster each target area to be clustered based on the connection probabilities between each central node.

[0024] And, the processor receives a keyword for image search input by a user, identifies a category associated with the keyword based on the clustering result, and can search for images that match the keyword among the images of the identified category. Brief explanation of the drawing

[0025] To more clearly explain the technical solution of the embodiments of the present disclosure, the drawings used in describing the embodiments of the present disclosure will be briefly described below. Figure 1 is a schematic diagram of structured data in digital graphics. Figure 2 is a schematic diagram of structured data of speech / text. Figure 3 is a schematic diagram of unstructured data in a network structure. Figure 4 is a schematic diagram of a solution for applying a clustering algorithm to a face image in the prior art. Figure 5 is a schematic diagram of a solution that applies a clustering algorithm to a vocabulary in the prior art. Figure 6 is a schematic diagram of a solution for applying a clustering algorithm to voice information in the prior art. Figure 7 is a schematic diagram of a solution for applying a clustering algorithm to network terminal management in the prior art. Figure 8 is a schematic flowchart of the entire clustering solution in the prior art. Figure 9 is a schematic flowchart of a progressive clustering solution in the prior art. Figure 10 is a schematic flowchart of a solution for the k-means clustering algorithm in the prior art. Figure 11 is a schematic diagram of the influence of a threshold on clustering results in the prior art. Figure 12a is a schematic diagram of the clustering error caused by using the existing clustering method in the example. Figure 12b is a schematic diagram of clustering one person into different people using a conventional clustering method in an example. Figure 12c is a schematic diagram of clustering different people into the same person using a conventional clustering method in an example. Figure 13a is a schematic diagram showing the effect of clustering centers using the conventional k-means algorithm in the example. Figure 13b is a schematic diagram of removing noise nodes using an existing clustering method in an example. FIG. 14 is a schematic flowchart of an image clustering method provided by an embodiment of the present disclosure. Figure 15 is a schematic diagram of the subgraph structure of Euclidean space data. Figure 16 is a schematic diagram of the subgraph structure of non-Euclidean space data. FIG. 17 is a schematic structural diagram of subgraph G in an example provided by the present disclosure. Figure 18 is a schematic diagram showing the adjacency matrix of the unweighted subgraph G of Figure 17. Figure 19 is a schematic diagram showing the adjacency matrix of the weighted subgraph G of Figure 17. Figure 20 is a schematic diagram of the degree matrix of the subgraph G of Figure 17. Figure 21 is a schematic diagram of the Laplacian matrix of subgraph G of Figure 17. FIG. 22 is a schematic structural diagram of a subgraph G in another example provided by the present disclosure. FIG. 23 is a schematic diagram of the adjacency matrix of subgraph G of FIG. 22, node features of the first layer, and node features of the (l + 1) layer. FIG. 24 is a schematic diagram of the adjacency matrix of subgraph G of FIG. 22, node features of the (1 + 1) layer, and node features of the (1 + 2) layer. FIG. 25 is a schematic diagram of the adjacency matrix of subgraph G of FIG. 22, node features of the (1 + 2) layer, and node features of the (1 + 3) layer. FIG. 26 is a schematic flowchart of an image clustering method provided by an embodiment of the present disclosure. FIG. 27a is a schematic flowchart of an image clustering method in an example of the present disclosure. Figure 27b is a schematic diagram comparing the Euclidean distance and the cosine distance in an example. Figure 27c is a schematic diagram comparing the Euclidean distance and the cosine distance in an example. FIG. 28a is a schematic flowchart of an image clustering method in an example of the present disclosure. FIG. 28b is a schematic flowchart of feature extraction for data in an example of the present disclosure. Figure 29 is a schematic diagram of the neural network used in Figure 28b. Figure 30 is a schematic diagram of the cosine distance in the example. FIG. 31 is a schematic flowchart for obtaining neighbor nodes in an example of the present disclosure. FIG. 32 is a schematic diagram of a solution for constructing a subgraph in an example of the present disclosure. FIG. 33 is a schematic flowchart for obtaining neighbor nodes in an example of the present disclosure. FIG. 34 is a schematic diagram of a solution for constructing a subgraph in an example of the present disclosure. Figure 35 is a schematic diagram of a comparison of similarity results using Euclidean distance and cosine distance. Figure 36 is a schematic diagram of a graph convolution network in the prior art. FIG. 37a is a schematic diagram of a graph convolution network introducing a shot-cut mechanism provided by an embodiment of the present disclosure. FIG. 37b is a schematic diagram of a graph convolution network introducing a shot-cut mechanism and hierarchical clustering provided by an embodiment of the present disclosure. FIG. 37c is a schematic flowchart of an image clustering method in an example of the present disclosure. FIG. 37d is a schematic diagram of an image clustering solution that introduces a shot-cut mechanism and hierarchical clustering in an example of the present disclosure. FIG. 38 is a schematic diagram of an image clustering solution that introduces a shot-cut mechanism and hierarchical clustering in an example of the present disclosure. FIG. 39a is a schematic diagram of clustering using a graph convolution network that introduces hierarchical clustering behavior in an example of the present disclosure. FIG. 39b is a schematic diagram of clustering using a graph convolution network that introduces hierarchical clustering behavior in an example of the present disclosure. FIG. 40a is a schematic flowchart of an image clustering method in an example of the present disclosure. FIG. 40b is a schematic diagram of a dimensional recovery solution in an example of the present disclosure. FIG. 41 is a schematic diagram of a clustering solution based on connection probability in an example of the present disclosure. FIG. 42a is a schematic diagram of a solution for removing noise nodes according to an average value strategy in an example of the present disclosure. FIG. 42b is a schematic flowchart of a method for removing noise nodes by an average value strategy in an example of the present disclosure. FIG. 43 is a schematic flowchart of clustering new data to be clustered, provided by an embodiment of the present disclosure. FIG. 44 is a schematic flowchart of an image clustering method applied to a mobile terminal provided by an embodiment of the present disclosure. FIG. 45 is a schematic diagram of a neural network structure applied to a mobile terminal in an embodiment of the present disclosure. FIG. 46 is a schematic flowchart of an image clustering method applied to a server provided by an embodiment of the present disclosure. FIG. 47 is a schematic diagram of an image clustering device provided by an embodiment of the present disclosure. FIG. 48 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. FIG. 49 is a flowchart illustrating a method for controlling an electronic device according to the present disclosure. Specific details for implementing the invention

[0026] Embodiments of the present disclosure will be described in detail below. Examples of such embodiments are indicated throughout the drawings by identical or similar reference numerals referring to identical or similar elements or elements having identical or similar functions. The embodiments described below with reference to the drawings are illustrative and are used merely to explain the present disclosure and should not be construed as any limitation thereto.

[0027] It should be understood by those skilled in the art that the singular forms “a,” “an,” “the,” and “said” are intended to include the plural forms unless otherwise noted. It should also be understood that the term “include” as used herein specifies the presence of the mentioned feature, integer, step, action, element, and / or component, but does not exclude the presence or addition of one or more other features, integer, step, action, element, component, and / or combinations thereof. It should be understood that when a component is referred to as “connected” or “coupled” to another component, this may provide an element that is directly connected or coupled to the other component or interposed therebetween. Additionally, “connected” or “coupled” as used herein may include wireless connection or coupling. The term “and / or” as used herein includes all or any of one or more associated enumerated units or a combination thereof.

[0028] To make the purpose, technical solution, and advantages of the present disclosure clearer, embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0029] Clustering algorithms are a process that groups individual items with identical physical or abstract attributes into the same category using computer technology, data analysis, and processing techniques. Each category generated by a clustering algorithm possesses similar abstract attributes. Clustering algorithms are one of the important methods for data analysis and mining, and have been widely used in image processing, text analysis, speech recognition, and network management.

[0030] Currently, all commercial clustering algorithms refer to conventional clustering algorithms such as Density-based Spatial Clustering of Application with Noise (DBSCAN), Incremental Grid Density-Based Clustering Algorithm (IGDCA), clustering algorithms based on similarity measures (K-means), and clustering algorithms based on neighbor relationship measures (Shared Nearest Neighbor). In actual applications and research, the accuracy of existing conventional clustering algorithms has been found to be relatively low, and they have been unable to better meet actual commercial requirements.

[0031] Over the past few years, due to improvements in algorithms, more convenient data acquisition, and increased computer hardware speeds, deep learning has achieved better accuracy and robustness in various tasks and has gradually replaced conventional algorithms to become mainstream. Convolutional Neural Networks (CNNs) perform feature extraction through convolution, which can extract better features than conventional feature extraction algorithms in image processing tasks. Recurrent Neural Networks (RNNs) are deep learning models specifically designed to solve sequential data, enabling them to better capture sequential features. Images and speech belong to European spatial data, characterized by structural rules. CNNs and RNNs are well-suited for processing structured data such as images and speech. Figure 1 is a schematic diagram of structured data in digital graphics. Figure 2 is a schematic diagram of structured data in speech / text. Figure 3 is a schematic diagram of unstructured data in a network structure.

[0032] Digital graphics (also referred to as digital images) shown in Fig. 1 and speech or text shown in Fig. 2 are all structured data; however, CNNs and RNNs cannot process unstructured data such as the social relationship network (also referred to as a social network) shown in Fig. 3. Graph Convolutional Networks (GCNs) are specifically proposed to solve unstructured data.

[0033] Clustering algorithms are widely used in image processing (such as face clustering, animal clustering, and object-scene clustering), text analysis (such as text clustering), speech recognition (such as language analysis), network management (such as intelligent power saving management for 5G terminals), and other fields.

[0034] Face clustering is one of the common applications of album management, and as shown in FIG. 4, after face detection, face alignment, and feature extraction are performed on images in an album, the faces are clustered using a clustering algorithm (an exemplary photo is from the YTB dataset). FIG. 4 is a schematic diagram of a solution for applying a clustering algorithm to face images in the prior art.

[0035] Each category contains its own photos, making it convenient for users to manage specific photos.

[0036] The clustering and face recognition system using a graph convolution network of the present disclosure is a supervised learning system that can be acquired through training based on annotated training data. Meanwhile, it can also implement the management of unlabeled data. In real-world scenarios, face images are very easy to acquire, but labeling them into face categories would require significant manpower and material resources. Unlabeled data is clustered and labeled through a clustering algorithm, which not only saves significant manpower and material resources but also improves the accuracy of face recognition.

[0037] Figure 5 is a schematic diagram of a solution that applies a clustering algorithm to a vocabulary in the prior art.

[0038] As shown in Figure 5, vocabulary clustering extracts specific words from text for feature extraction and clustering, and clusters vocabulary with the same attributes into a single category to realize document / text analysis. For example, in movie reviews, each movie review is clustered, and through the clustering results, the user's movie evaluation can be reflected and the number of positive reviews, negative reviews, and average reviews can be counted, which is convenient for movie scoring, etc.

[0039] Figure 6 is a schematic diagram of a solution for applying a clustering algorithm to voice information in the prior art.

[0040] As shown in Fig. 6, there are many languages ​​around the world, and each language may additionally include different dialects. Acquired speech information can be clustered into multiple subcategories through a clustering algorithm. For example, in speech recognition, each language type needs to be determined first, after which speech recognition can be performed and functions such as machine translation can be completed. Record data may include multiple languages, and the accuracy of speech recognition can be improved by first analyzing and clustering the record data to determine which subcategory each language belongs to, and then performing recognition.

[0041] Figure 7 is a schematic diagram of a solution for applying a clustering algorithm to network terminal management in the prior art.

[0042] As shown in Figure 7, network terminal management is necessary because 5G has been commercialized and is convenient for people's lives, but the power consumption of 5G base stations is 12 times that of 4G, so intelligent power saving management of base stations may be required to effectively reduce 5G service costs. For example, a 5G base station consists of multiple components, and each component has different power consumption. According to a clustering algorithm, devices with the same power consumption attributes are grouped into subcategories, and power consumption attributes are analyzed and managed in a targeted manner to increase node management efficiency and avoid resource waste.

[0043] In face clustering techniques, to address the problem of grouping Category A faces and Category B faces into a single category when Category A face photos include Category B faces, the prior art proposes a two-stage clustering algorithm that combines full clustering and incremental clustering. First, a portion of the face data is selected for full clustering, and then incremental clustering is performed based on the full clustering. The specific implementation is as follows:

[0044] As the name implies, full clustering involves clustering all face data to determine a subset of clustering categories prior to progressive clustering, which provides a category basis for the subsequent clustering of new face data.

[0045] The implementation flowchart of pool clustering is shown in Fig. 8. The specific implementation process is as follows:

[0046] Figure 8 is a schematic flowchart of the entire clustering solution in the prior art.

[0047] 1) N photos are selected for face detection, face alignment, and face feature extraction (S810), and each photo is used as a separate category (S820);

[0048] 2) Calculate the distance between every two face categories among N face photos (S830);

[0049] 3) Determine whether the distance between two categories is smaller than a given threshold (S840); if it is smaller than the given threshold (840-Y), fuse the two categories (S850) so that the two categories are combined into one category; if it is larger than the threshold (840-N), identify the two categories as two separate categories;

[0050] 4) Repeat step 2) until all distances between face categories are greater than a given threshold, and determine whether the distance between all categories is less than the threshold (S860). Then, if the distance between all categories is less than the threshold (S860-Y), terminate, which indicates that full clustering is complete.

[0051] Pool clustering is identical to the initialization of incremental clustering, but pool clustering is used first for face clustering, and subsequent clustering operations are based on incremental clustering. The implementation flowchart of incremental clustering is shown in Fig. 9. The specific implementation is as follows:

[0052] Figure 9 is a schematic flowchart of a progressive clustering solution in the prior art.

[0053] 1) In the final clustering, count the number of categories M, obtain one of the M categories as category A, use the newly added P face feature information as category P, and take one of the P categories as category B; that is, obtain face data of category A and face data of category B (S910).

[0054] 2) Re-cluster facial features in Category A using a density clustering algorithm, and set a lower threshold to obtain one or more categories;

[0055] 3) Count the number of subcategories in category A and select the subcategory with the largest number of faces as the representative of category A; that is, calculate the number of faces in each subcategory and select the subcategory with the largest number as the representative of category A (S930).

[0056] 4) Calculate the distance between the representative subcategory of Category A and the face data of Category B (S940);

[0057] 5) Determine whether the distance between all two is smaller than a given threshold; if it is smaller than the given threshold, all two categories are combined into one category; if it is larger than the threshold, all two categories are separate categories; that is, if it is larger than the threshold, the faces of category A and the faces of category B are clustered into different categories (S950).

[0058] 6) Repeat step 1) until the number of faces in all categories no longer changes, indicating the termination of clustering. That is, determine whether the face data in all categories no longer changes (S960), and if the face data in all categories no longer changes (S960-Y), terminate the process. If at least one face data in all categories changes (S960-N), repeat step 1) (S910). That is, repeat step 1) for all categories M and P (S970).

[0059] The k-means clustering algorithm is a commonly used clustering algorithm, where k refers to clustering data into k categories, and means refers to calculating the mean value of all nodes in each subcategory and using it as the clustering center for that subcategory. To address the problem where the initial clustering centers of conventional clustering algorithms cannot guarantee representativeness, leading to poor clustering algorithms and poor clustering results, the prior art proposes an optimized algorithm for the problem of difficulty in selecting the number of clusters k. As shown in FIG. 10, the specific implementation is as follows:

[0060] FIG. 10 is a schematic flowchart of a solution for a k-means clustering algorithm in the prior art. S1: All nodes to be clustered into a set M are formed, and the density function values ​​of all nodes to be clustered are calculated. That is, a data set is obtained and the density function value of each node is calculated (S1010), nodes whose density function value is greater than or equal to the average density function value of the set are combined to form a dense point set Y; specifically, points with a density function value smaller than the average density function value are removed (S1020).

[0061] S2: Two nodes are selected from a dense point set Y to form a set Q; specifically, two data points with the largest density function values ​​are used as initial cluster centers and placed in the point set Q (S1030).

[0062] S3: Select m nodes from set M by the focus statistics method and form a set C of replacement initial clustering centers by m nodes; that is, select a set C of candidate initial cluster center points (S1040).

[0063] S4: Nodes in a dense point set Y are divided into categories in which each initial clustering center is located in set Q, and the average maximum similarity of the first cluster is obtained; that is, data points are clustered according to the clustering centers of point set Q, and the maximum similarity is calculated (S1050)

[0064] S5: Select a node from set C of candidate initial clustering centers, add the node to set Q as a new initial clustering center from set Q, and simultaneously delete the node from C; steps S4-S5 are repeated, and the minimum value of the average maximum similarity of the clusters is selected by comparing the average maximum similarity of each cluster;

[0065] S6: The clustering center of set Q corresponding to the minimum value of the average maximum similarity of the clusters is used as the initial clustering center with the optimal k-means clustering, and then k-means clustering is performed to obtain the clustering result. That is, it is determined that the maximum similarity is less than the previous maximum similarity (S1060), and if the maximum similarity is less than the previous maximum similarity (S1060-Y), one sample point is selected from the candidate initial cluster centers and placed in point set Q (S1070). Then, if the maximum similarity is greater than or equal to the previous maximum similarity (S1060-N), the point set Q with the smallest maximum similarity is selected as the cluster center (S1080). Then, k-means clustering is performed and the result is output (S1090).

[0066] The problems with existing clustering algorithms are as follows:

[0067] Figure 11 is a schematic diagram of the influence of a threshold on clustering results in the prior art.

[0068] (A) As shown in Fig. 11, a two-stage clustering algorithm combining full clustering and incremental clustering is proposed to solve the problem of clustering A-category faces and B-category faces into a single category when A-category face photos include B-category faces in face clustering. First, a portion of the face data is selected for full clustering, and then incremental clustering is performed based on the full clustering. This method can, to some extent, avoid clustering two different categories into the same category. However, if the threshold is too high, the clustering algorithm discards faces at the edges, which may lead to the problem of one category being clustered into two categories. If the threshold is too low, an error of clustering two categories into the same category may occur. Furthermore, these errors may gradually accumulate during subsequent clustering processes.

[0069] (B) Almost all existing clustering algorithms determine whether two cluster nodes belong to the same category based on the distance between cluster nodes, but this can lead to inaccurate clustering results because there may be situations where "my neighbor's neighbor is not necessarily my neighbor." Figure 12a is a schematic diagram of the clustering error caused by using the existing clustering method in the example. As shown in Figure 12a, Node 12 belongs to the category of Node 1, and Node 6 belongs to the category of Node 8. However, the distance between Node 12 and Node 6 is relatively small. According to a standard distance clustering algorithm, Node 12 and Node 6 are clustered into one category. The distance between Node 6 and Node 7, and between Node 8 and Node 9, is smaller and the relationship is closer, and Node 6 is a neighbor of Node 1 (Node 12), and must belong to the category of Node 8, not the category of Node 1.

[0070] (C) Existing clustering algorithms use original clustering data directly for clustering, which can lead to problems such as poor clustering performance and low accuracy. For example, in face clustering, face detection, face alignment, and face feature extraction are performed on collected images, but the facial features are significantly different due to lighting and differences between frontal and lateral faces. Alternatively, simply clustering using existing facial features or clustering using Euclidean distance can result in clustering one person into two categories or clustering others into one category.

[0071] Figure 12b is a schematic diagram of clustering one person into different people using a conventional clustering method in an example. As shown in Figure 12b, the photos of the circles on the far left belong to the same category, but the background, lighting conditions, image quality, and shooting period of each photo are different, and alternatively, using Euclidean distance or conventional photo features for clustering may result in clustering one person into multiple categories.

[0072] FIG. 12c is a schematic diagram of clustering different people into the same person using a conventional clustering method in an example. As shown in FIG. 12c, different categories can be clustered into one person under the influence of lighting and posture. Category A is a photograph of a young boy, and Category B is a photograph of a young girl, and d1 and d2 represent the Euclidean distances between A3 and Categories A and B, respectively. Due to the influence of the external environment, d2 may result in d1, so the clustering result may be that A and B belong to the same category.

[0073] (D) Conventional k-means clustering algorithms are iterative clustering analysis algorithms that require continuous iterations to select appropriate K values ​​and initial clustering centers, and such algorithms are very complex and inefficient.

[0074] (E) Similar to the conventional k-means algorithm, many clustering algorithms require determining clustering centers, and the representativeness of the clustering centers directly affects the accuracy of subsequent clustering. In particular, in subcategories introduced with negative samples, the presence of negative samples causes the overall clustering centers to be offset, making the clustering algorithm more inaccurate. Figure 13a is a schematic diagram illustrating the effect of clustering centers using the conventional k-means algorithm in an example. As shown in Figure 13a, node 13 represents the clustering center in the category, where the left diagram shows the clustering center when no negative samples are introduced, and if clustering becomes erroneous due to the introduction of the 6th node with negative samples, the overall clustering centers will be offset to the right, which will certainly make the clustering more inaccurate.

[0075] (F) Conventional technology proposes using a simple graph convolutional neural network (GCN) to extract features of a subgraph, and the GCN reduces the difference between features by clustering features between nodes during network transmission. If there are noise nodes in the subgraph, the difference between the noise nodes and other neighboring nodes will become increasingly smaller, and eventually the noise nodes will be clustered into this category, which results in a clustering error.

[0076] (G) Conventional technology proposes using GCN to predict probabilities between a central node and neighboring nodes for clustering, but does not perform any post-processing. If there are noise nodes, it will inevitably lead to errors in neural network prediction, which result in a final clustering error. Figure 13b is a schematic diagram of removing noise nodes using a conventional clustering method in an example. As shown in Figure 13b, A, B, C, D, and E belong to the same category, where the connection probability between F and A is 0.9, but there is no connection between F and other nodes, and such F can consequently be considered a noise node. However, without post-processing, F and other nodes will be clustered into a single category, which results in a clustering error.

[0077] The present disclosure proposes a clustering algorithm with better accuracy and stronger robustness to solve the aforementioned problem. The clustering algorithm proposed by the present disclosure is as follows.

[0078] A) A GCN-based clustering algorithm is proposed, and the accuracy of the algorithm according to the present disclosure is higher than that of existing algorithms, but the complexity is relatively lower;

[0079] B) Regarding the "my neighbor's neighbor is not necessarily my neighbor" problem and poor clustering effects caused by external factors such as lighting, a Codec Graph Neural Network is proposed to map node features to a different dimensional space through encoders within the neural network, and a Hierarchical Clustering Module is proposed in a standard GCN layer to pre-cluster nodes with strong relationships and determine noise nodes by normalizing features; the Hierarchical Clustering Module can extract hierarchical features of subgraphs to increase the difference between noise nodes and other nodes. Therefore, through the Hierarchical Clustering Module, the difference between features of nodes with the same attribute can be reduced, while the difference between nodes with different attributes can be increased;

[0080] C) To improve the accuracy and stability of clustering, an average score-based post-processing algorithm is proposed to determine whether isolated nodes are noise nodes based on neighbor relationships;

[0081] D) Pool clustering involves clustering all nodes simultaneously, and as the number of clustered nodes increases, the complexity of the clustering algorithm increases; to reduce the time complexity of the algorithm, efficient additional logic is proposed that can not only guarantee the accuracy of the algorithm but also reduce the complexity of the algorithm;

[0082] E) Conventional clustering algorithms require determining clustering centers, and the representativeness of the clustering centers directly affects the accuracy of subsequent clustering; the new clustering algorithm proposed in this disclosure does not require determining clustering centers, thereby avoiding the risk of errors in the entire clustering caused by errors in the clustering centers;

[0083] F) A shot-cut mechanism is proposed to maintain differences between features within GCN, which is advantageous for removing noise nodes and improving clustering accuracy, and the shot-cut mechanism can mitigate the excessive smoothing problem of GCN.

[0084] G) Through a process that uses cosine distance instead of Euclidean distance, cosine distance is more accurate than Euclidean distance and can determine the similarity of nodes within the same category.

[0085] The technical solution of the present disclosure and how the technical solution of the present disclosure solves the technical problem described above will be described in detail below with specific embodiments. Some of the following specific embodiments may be combined with one another, and in some embodiments, the same or similar concepts or processes may not be repeated. Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0086] Embodiments of the present disclosure provide possible implementation methods. As shown in FIG. 14, an image clustering method is provided, which may include the following steps: FIG. 14 is a schematic flowchart of an image clustering method provided by an embodiment of the present disclosure.

[0087] Step S401: Obtain the target region of all images to be clustered, and obtain the feature vector of the target region.

[0088] Specifically, the target area may be an area containing the target to be detected, and the target may be a human face, a specific object, etc. in an image.

[0089] According to one embodiment of the present disclosure, for at least one target region to be clustered, the target region can be extracted from an image, and then a feature vector can be extracted from the target region.

[0090] Step S402: Cluster the acquired feature vectors based on a graph convolution network to obtain results.

[0091] Here, the Graph Convolutional Network (GCN) is a neural network architecture that has gained popularity in recent years. Unlike conventional network models such as LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Networks), which can only be used for grid-based data, the Graph Convolutional Network can process data with a generalized topological graph structure and can explore the characteristics and fields of data in depth with a generalized topological graph structure, such as PageRank reference networks, social networks, communication networks, protein molecular structures, and other series of irregular data with spatial topological graph structures. Further details regarding the Graph Convolutional Network will be discussed later.

[0092] Specifically, a subgraph corresponding to at least one feature vector may be obtained first, and then the adjacency matrix, feature matrix, and first-order neighbor vector of the subgraph may be obtained; the adjacency matrix, feature matrix, and first-order neighbor vector of the subgraph may be input into at least one layer of a graph convolution network to perform sequential transformation (which may also be considered as performing feature extraction), and the transformed matrix is ​​input into at least one layer of a fully connected layer for classification to obtain the connection probability between the center node of the subgraph and each neighbor node.

[0093] According to one embodiment of the present disclosure, for at least one subgraph, the subgraph may include a central node and at least one neighbor node, and a connection probability between the central node and each first-order neighbor node is obtained; if the connection probability is greater than a preset threshold, the central node and the first-order neighbor node with a connection probability greater than the preset threshold may be classified into one category.

[0094] Here, the first-degree neighbor nodes are neighbor nodes directly connected to the central node, and the neighbor nodes may also include second-degree neighbor nodes, that is, neighbors adjacent to the neighbors of the central node, rather than neighbor nodes directly connected to the central node.

[0095] In the above embodiment, the target regions of all images are clustered through a graph convolution network without determining the clustering center, thereby avoiding deviations in the overall clustering results that may occur due to deviations in the clustering center, and effectively improving the accuracy of the clustering results.

[0096] In the following, the subgraph and the graph convolution network (also referred to as a graph convolution neural network) will be described in more detail with reference to the attached drawings.

[0097] 1) Definition of a graph

[0098] In this disclosure, the definition of a graph is not the definition of an ordinary image, but the definition of a graph in the mathematical sense.

[0099] Digital images, voice, and text belong to European spatial data and have fixed dimensions, so neural networks such as CNNs or RNNs can be used to extract features. Unstructured data, such as social networks, does not have fixed dimensions, and Graph Convolutional Neural Networks (GCNs) were previously proposed to process this type of unstructured data.

[0100] FIG. 15 is a schematic diagram of a subgraph structure of Euclidean space data according to one embodiment of the present disclosure.

[0101] If each pixel of the image is taken as a node of the graph, it can be seen from FIG. 15 that each node is a node with 8 fields and that a fixed-size convolution kernel can be used to extract features of the image. For example, in FIG. 15, the neighbor nodes of the first node are {2, 3, 4, 5, 6, 7, 8, 9} and a 3 x 3 convolution kernel can be used to extract those features. In FIG. 15, the neighbor nodes of the second node are {1, 3, 4, 8, 9, 10, 14, 15} and a 3 x 3 convolution kernel can be used to extract those features. For the upper points of the boundary, zeros can be filled to satisfy the 8-neighbor feature, such as that of the eighth node.

[0102] FIG. 16 is a diagram illustrating a subgraph structure of non-Euclidean space data according to an embodiment of the present disclosure. In the case of non-Euclidean space data as shown in FIG. 16, the number of neighbor nodes of each node is different, and in this case, a convolution kernel of a fixed size cannot be used to perform feature extraction for the non-Euclidean space data. For example, in FIG. 16, the neighbor nodes of the first node are {0, 3, 4, 5}, and the number of neighbor nodes is 4, which means that it is necessary to use a convolution kernel having a block of size 4 nodes to perform feature extraction. In FIG. 16, the neighbor nodes of the second node are {3}, that is, there is only one neighbor, which means that it is necessary to use a convolution kernel having a block of size 2 nodes to perform feature extraction. In the case of data in non-Euclidean space, due to the different number of neighbors of each node, it is necessary to use a different convolution kernel for each node for feature extraction, and thus it can be seen that conventional CNNs and RNNs cannot be used. Accordingly, a Graph Convolutional Neural Network (GCN) can measure the relationships between non-Euclidean space data nodes through a graph adjacency matrix and a Laplacian matrix.

[0103] A graph G = (V, E) is defined, where V represents a set of nodes and E represents a set of edges. Each node of the graph is typically represented as a vector of dimension D to calculate the result of the graph, which is represented by the adjacency matrix of the graph. FIG. 17 is a diagram illustrating a subgraph G according to an embodiment of the present disclosure. As shown in FIG. 17, the structure can be represented as a graph G, where V represents a set of nodes {0, 1, 2, 3, 4, 5} and E represents a set of edges {a, b, c, e, f, g, h}. Also, {a, b, c, e, f, g, h} can represent the weights of the edges. N indicates that there are 6 nodes, A is a vector representation of node 4, and its dimension D is 4.

[0104] In graph theory, the adjacency matrix and the Laplacian matrix of a graph are typically used to measure the relationships between nodes in a graph. Fig. 18 is a diagram showing the representation of the adjacency matrix of an unweighted graph G. Typically, "1" is used to indicate that there is a connection between two nodes, and "0" is used to indicate that there is no connection between two nodes. Fig. 19 is a schematic diagram showing the adjacency matrix of the weighted subgraph G of Fig. 17.

[0105] As shown in FIG. 19, FIG. 19 is a representation of a weighted adjacency matrix.

[0106] Figure 20 is a schematic diagram of the degree matrix of the subgraph G of Figure 17.

[0107] In graph G, the degree of a node indicates how many paths pass through the node; for example, the degree of node 0 is 3. The degree matrix D of graph G is shown in Figure 20, and the degree matrix has values ​​only on the diagonal, which represents the degree of the node, and the remaining positions are 0.

[0108] Figure 21 is a schematic diagram of the Laplacian matrix of subgraph G of Figure 17. As shown in Figure 21, the Laplacian matrix of graph G is L = D - A.

[0109] 2) Definition of Graph Convolution

[0110]

[0111] Here, represents the node features of the first layer; σ represents the nonlinear transformation; A represents the adjacency matrix; represents the weight of the first layer; represents the intercept of the first layer; represents the node characteristics of the (l + 1)th layer.

[0112] Graph convolution is essentially an operation in which the nodes of a graph are represented by their own characteristics and the characteristics of their neighboring nodes. The result of graph convolution is that the differences between the characteristics of neighboring nodes become progressively smaller, while the differences between unrelated nodes become larger; this implies that graph convolution itself possesses a clustering function.

[0113] FIG. 22 is a schematic structural diagram of a subgraph G in another example according to the present disclosure.

[0114] For example, graph G is shown in FIG. 22, and adjacency matrix A is used to represent graph G (typically, each node is considered to be connected to itself), and X represents the feature vector of each node in graph G. From graph G, it can be seen that nodes 0, 1, 2, and 3 belong to the same category of nodes, and nodes 4 and 5 belong to the same category of nodes.

[0115] The mathematical principles of GCN are explored from the simplest perspective. It is assumed that when the values ​​of all weights W are 1, the values ​​of all intercepts (offsets) b are 0, and the non-linear transformation employs the linear function y = x, the graph convolution formula can be expressed as follows:

[0116]

[0117] In the formula, represents the node characteristics of the first layer; A represents the adjacency matrix; represents the node characteristics of the (l + 1)th layer.

[0118] FIG. 23 is a schematic diagram of the adjacency matrix of subgraph G of FIG. 22, node features of the first layer, and node features of the (l + 1) layer.

[0119] Using graph G of Fig. 22, the adjacency matrix A and the node features of the first layer and node characteristics of the (l + 1)th layer This can be obtained as shown in Figure 23.

[0120] From the above formula (2), the following formula (3) can be derived:

[0121]

[0122] Adjacency matrix A and node characteristics of the (l + 1)th layer shown in Fig. 23 Accordingly, the node characteristics of the (l + 2) layer shown in FIG. 24 This can be obtained;

[0123] Likewise, the following formula can be derived:

[0124]

[0125] Adjacency matrix A and node characteristics of the (l + 2)th layer shown in Fig. 24 Accordingly, the node characteristics of the (l + 3) layer shown in FIG. 25 This can be obtained;

[0126] Through this, we learn that the new feature representation for each node in graph G is the sum of the features of its neighbors and itself. Each represents the layers of the GCN network, and through the above calculations, if there is a relationship between nodes—that is, if these nodes belong to the same category—the difference between similar nodes may become smaller as the GCN network progresses. For example, Node 4 and Node 5 belong to the same category, and the characteristics between these nodes are identical in subsequent GCN calculations; Node 0, Node 1, Node 2, and Node 3 belong to the same category, and the characteristics between these nodes will become closer. The difference in characteristics between nodes of different categories will become larger. Since GCN itself has the function of node clustering, it can be seen that the present disclosure implements a clustering function with a rigorous mathematical and scientific basis using GCN.

[0127] As can be seen from the graph above, the feature values ​​of the nodes increase with the depth of the GCN network. When calculating gradients during backpropagation, the gradients may disappear or explode, which severely impacts the efficiency of the algorithm. Therefore, in practical applications, features are typically normalized, but the characteristics of the GCN remain unchanged.

[0128] The adjacency matrix A is normalized by the degree matrix D of the graph, and , the formula for GCN is as follows:

[0129]

[0130] Is It is converted to:

[0131]

[0132] Here, represents the node features of the first layer; σ represents the nonlinear transformation; D represents the degree matrix; A represents the adjacency matrix; represents the weight of the first layer; represents the intercept of the first layer; represents the node characteristics of the (l + 1)th layer.

[0133] Generally, since each node is considered to have a connection to itself and be connected to a self-circle, the computational formula for GCN is as follows:

[0134]

[0135] Here: represents the node features of the first layer; σ represents the nonlinear transformation; A represents the adjacency matrix corresponding to node i; represents A + I, a self-cycle, and I represents a diagonal matrix with a value of 1; Is Represents the order matrix corresponding to; represents the weight of the first layer; represents the intercept of the first layer; represents the node characteristics of the (l + 1)th layer.

[0136] The above is a detailed introduction to graphs and graph convolution networks, and the specific process for constructing subgraphs will be further explained below with reference to drawings and examples.

[0137] In a possible implementation of an embodiment of the present disclosure, the step of acquiring a feature vector of a target region in step S401 may include the following:

[0138] (1) detecting feature points of a target region, aligning the target region based on the detected feature points, and obtaining a corresponding target transformation region; and

[0139] (2) A step of extracting a vector of the target transformation region to obtain a feature vector of the target region.

[0140] In a specific implementation process, the transformation may be a process of orientation transformation and feature point alignment. For example, in the case of a human face, a target region containing the human face is extracted, but if the human face is not oriented forward, feature points of the target region containing the human face need to be extracted, and the feature points are affine transformed to obtain the front image of the face.

[0141] FIG. 26 is a schematic flowchart of an image clustering method provided by an embodiment of the present disclosure. In a possible implementation of an embodiment of the present disclosure, as shown in FIG. 26, a target region of all images to be clustered is obtained, and a feature vector of the target region is obtained (S401). Then, in step S402, the step of clustering the feature vector based on a graph convolution network to obtain a result may include the following:

[0142] Step S210: A step of constructing a subgraph corresponding to a feature vector, wherein the subgraph includes a central node corresponding to the feature vector and at least one neighbor node corresponding to another feature vector.

[0143] Specifically, a feature vector can be used as a center node, neighbor nodes corresponding to the center node can be obtained, and a subgraph of the feature vector can be constructed.

[0144] In a specific implementation process, similarity between a feature vector and another feature vector can be obtained, where another feature vector refers to a vector other than the feature vector among a plurality of feature vectors corresponding to the target region to be clustered; neighbor nodes can be determined from the other feature vector based on the similarity between the feature vector and the other feature vector.

[0145] Step S220: Based on a graph convolution network, determine the connection probability between the central node of a subgraph and its neighbor nodes, and cluster the target region according to the determined connection probability to obtain the clustering result.

[0146] Here, the neighbor node can be a first-degree neighbor node, that is, a neighbor node directly connected to the central node.

[0147] Specifically, corresponding input features can be obtained based on the subgraph, and after the input features are input into a graph convolution network for transformation, the transformed matrix is ​​input into a fully connected network to obtain the connection probabilities between the central node of the subgraph and each neighbor node.

[0148] In a specific implementation process, a first-order neighbor node with a connection probability greater than a preset threshold can be determined as being in the same category as the central node.

[0149] The specific process for constructing the subgraph will be described in detail below with specific examples.

[0150] In a specific implementation process, the step of constructing a subgraph corresponding to a feature vector in step S210 may include the following:

[0151] (1) A step of obtaining neighbor nodes of the center node from other feature vectors by taking a feature vector as the center node.

[0152] Here, other feature vectors refer to feature vectors corresponding to data other than the central node.

[0153] Specifically, neighboring nodes can be screened from other feature vectors based on the similarity between the feature vector corresponding to the central node and other feature vectors.

[0154] In a specific implementation process, the step of using a feature vector as a center node and obtaining neighbor nodes of the center node from another feature vector may include the following:

[0155] a. A step of obtaining the cosine distance between a feature vector and another feature vector, respectively; and

[0156] b. A step of screening neighbor nodes of a feature vector from another feature vector based on the cosine distance between the feature vector and another feature vector.

[0157] The specific process for screening neighboring nodes will be explained in detail below.

[0158] Here, cosine distance, also called cosine similarity, is an algorithm that evaluates similarity by calculating the cosine value of the angle between two face feature vectors; for two features A and B, the larger the Euclidean distance between them, the greater the difference between them; and the larger the cosine distance, the smaller the difference between them.

[0159] (2) A step of forming a subgraph based on the central node and neighboring nodes.

[0160] Specifically, after neighbor nodes are screened from feature vectors corresponding to other data, a subgraph is constructed according to the center node and at least one neighbor node obtained by the screening.

[0161] The above-described embodiment explains a specific process for constructing a subgraph. Below, a face image will be used as an example to further explain the specific process for the image clustering method and subgraph construction provided in this disclosure.

[0162] FIG. 27a is a schematic flowchart of an image clustering method according to one embodiment of the present disclosure.

[0163] As shown in FIG. 27a, taking a face image as an example, the image clustering method may include the following steps:

[0164] S1: A step of performing face detection and cropping the face image;

[0165] S2: A step of using a CNN to extract feature vectors of a face image, i.e., face features shown in the drawing, and using the extracted face features as nodes;

[0166] S3: A step of using each node as a center node, determining at least one neighbor node based on cosine distance, and constructing a subgraph by connecting the center node and the neighbor node;

[0167] Here, step S3 may include the following:

[0168] (1) Step of calculating the cosine distance between two nodes (S2710);

[0169] (2) A step of using each node as a center node and selecting a first-order neighbor node based on cosine distance (S2720);

[0170] (3) A step of selecting a neighbor node of a first-order neighbor node, i.e., a second-order neighbor node, based on cosine distance (S2730);

[0171] (4) A step (S2740) of selecting k neighbor nodes for each node to form a subgraph, wherein k is a natural number; and

[0172] (5) A step (S2750) of repeating steps (1) through (4) above until a subgraph of each node is formed.

[0173] S4. A step of extracting features of each subgraph by GCN; using a hierarchical clustering module to maintain the difference between noise and other node features, and then determining the connection probability between the central node and its first-order neighbor nodes according to a softmax function; and

[0174] S5. A step of classifying face images based on the connection probability between the central node and the first-degree neighbor nodes.

[0175] FIG. 27a illustrates only steps S2 through S4. When using a conventional clustering method that uses Euclidean distance to calculate the distance between nodes, clustering errors may occur. In this example, cosine distance is used instead of Euclidean distance, and since cosine distance is more accurate than Euclidean distance, the similarity of nodes of the same category can be determined.

[0176] In step S3 above, cosine distance is used instead of Euclidean distance. Numerous experiments and academic studies have proven that cosine distance is more accurate than Euclidean distance and can more precisely determine the similarity of nodes within the same category.

[0177] FIGS. 27b and FIGS. 27c are schematic diagrams comparing Euclidean distance and cosine distance according to the present disclosure. As shown in FIGS. 27b and FIGS. 27c, numbers 0 through 9 represent faces having different faces. A, B, and C in FIG. 27b correspond to A, B, and C in FIG. 27c. LAB represents the Euclidean distance between A and B, and e represents the angle between A and B; LBC represents the Euclidean distance between B and C, and d represents the angle between B and C. A smaller angle corresponds to a larger cosine distance and a higher similarity between two features.

[0178] A and B in Fig. 27b represent the same person, and C and A may not be the same person. Since the facial features of the same person vary depending on posture, lighting, and the moment, clustering errors will occur if Euclidean distance is used. For example, in Fig. 27c, the Euclidean distance between A and B is 1.45, and the Euclidean distance between B and C is 1.22; B and C can be clustered into a single category, and thus, the results of clustering based on Euclidean distance may be incorrect. Accordingly, unlike clustering based on Euclidean distance, cosine distance may not result in errors, and using cosine distance for similarity calculation may be more accurate. Since the angle between features within the same category is very small, cosine distance can be used for more accurate clustering.

[0179] FIG. 28a is a schematic flowchart of an image clustering method according to the present disclosure. As shown in FIG. 28a, taking a face image as an example, the image clustering method may include the following steps:

[0180] S10: A step of performing feature extraction on the collected data, i.e., extracting features of the data to be clustered (S2810);

[0181] S20: A step of calculating the cosine distance between each node (face); that is, calculating the cosine distance between features (S2820).

[0182] S30: A step of using each node as a center node, finding k neighbor nodes, and constructing a subgraph; that is, selecting K neighbor nodes based on cosine distance to construct the subgraph (S2830).

[0183] S40: A step of using a GCN neural network to calculate the connection probability between neighbor nodes and a central node for each subgraph and forming each connection pair; and

[0184] S50: A step of clustering into one category if the connection probability between two nodes is greater than a set threshold; otherwise, clustering into two categories. Specifically, a subgraph is transmitted to the GCN to calculate the connection probability between nodes (S2840), and then it is determined whether it is greater than a set threshold (S2850). If the connection probability between two nodes is greater than the set threshold (S2850-Y), it is clustered into one category (S2860); if the connection probability between two nodes is not greater than the set threshold (S2850-N), it is clustered into two different categories (S2870).

[0185] In the above example, by calculating the cosine distance between nodes, a subgraph is constructed using each node as a center node, and the connection probability between the center node of each subgraph and its first-degree neighbor node is calculated, and it is determined whether the two nodes of each connection pair belong to the same category based on the connection probability, which can improve the accuracy of clustering.

[0186] FIG. 28b is a schematic flowchart of feature extraction for data according to the present disclosure. As shown in FIG. 28b, the step of performing feature extraction for data collected in step S10 includes a data preprocessing submodule and a feature extraction submodule.

[0187] The data preprocessing module may include a face detection module, a face alignment module, and a face feature extraction module.

[0188] The face detection module is an algorithm for finding faces in collected images.

[0189] The original image contains more background information, and when the original image is used for clustering, different people may be clustered as the same person under the same background conditions; however, when a face detection algorithm is used to detect faces in an image and to crop and save the faces, only faces are clustered in subsequent clustering, thereby avoiding clustering errors caused by background interference.

[0190] The cropped face image is normalized to a specific resolution size.

[0191] The face alignment module is an algorithm for correcting tilted face images. The face feature point detection algorithm is used to detect feature points of the face, and the detected feature points are used to perform an affine transformation to obtain a front image of the face.

[0192] The feature extraction module uses a deep learning algorithm to perform feature extraction on aligned faces. As the name implies, feature extraction is the representation of a face image as a 128-dimensional vector. Currently, vectors of other dimensions may also be used in the examples, and this is not specifically limited herein.

[0193] Face feature extraction is based on learning from a large amount of face data. For example, a dataset is constructed containing 10,000 people in three groups: Negroid, Caucasian, and Mongolian, where each face has over 450 data points and each group includes data for children, adolescents, young adults, middle-aged adults, and the elderly. A neural network performs face feature extraction training and learning on this dataset. After training, the deep learning model can realize feature extraction for each face image.

[0194] Figure 29 is a schematic diagram of the neural network used in Figure 28b. As shown in Figure 29, feature extraction is performed on the face image after face detection and face alignment, where Box 1 represents a convolutional neural network layer, Box 2 represents a depth-separable convolutional layer, Box 6 represents a fully connected layer, and the output is a 128-dimensional face feature.

[0195] In step S2 of Fig. 27a, the face is represented as a 128-dimensional vector, and in step S3 of Fig. 27a, the cosine distance between all 2 face nodes is calculated.

[0196] Cosine distance, also known as cosine similarity, is an algorithm that evaluates similarity by calculating the cosine value of the angle between two face feature vectors.

[0197] FIG. 30 is a schematic diagram of the cosine distance according to the present disclosure. As shown in FIG. 30, f1 represents a feature vector of face A, f2 represents a feature vector of face B, and a represents the angle between the vectors f1 and f2. The smaller the angle a, the higher the similarity between f1 and f2 may be. When the angle a is 0, the cosine value is 1, which means that f1 and f2 have the highest similarity; when the angle a is 90, the cosine value is 0, which means that f1 and f2 are independent of each other, that is, f1 and f2 are not the same person. The calculation formula is as follows:

[0198]

[0199] Here, x and y represent two vectors for which similarity is to be calculated, respectively; sim(X, Y) represents the cosine similarity between two vectors.

[0200] In step S3, each node is used as the central node P, and the neighbor node m1 with the largest cosine distance from the central node P, the neighbor node m2 with the smallest cosine distance from the central node P, and the neighbor node m3 with an intermediate random cosine distance from the central node P are searched, where the number of neighbor nodes of P is k1 = m1 + m2 + m3; for each neighbor node of P, the node n with the largest cosine distance from each neighbor node of P is found, where k2 = k1 * n. Through the above steps, it can be determined that the number of first-order neighbor nodes of the central node P is k1, the number of second-order neighbor nodes is k2, and the theoretical total number of neighbor nodes of P is k = k1 + k2. In practice, since there may be identical neighbors between each first-order neighbor node, the number of neighbor nodes of P is less than or equal to k;

[0201] For each central node P, p neighbor nodes are selected to form connections between them to construct a subgraph represented by an adjacency matrix A, and calculations are performed to obtain a degree matrix D.

[0202] The above steps are repeated until all nodes are used as central nodes to complete the subgraph construction.

[0203] In step S4, different GCN neural network models are designed according to different application scenarios and computing platforms, and each subgraph is computed through the GCN neural network to calculate the connection probability between neighbor nodes and the central node, thereby forming each connection pair. Then, the stability of the algorithm is improved through the Average Score algorithm. Finally, each connection pair is traversed to obtain the clustering result.

[0204] From the above formula (7), the characteristics of a node in a graph To extract, it can be seen that it is necessary to calculate the graph's degree matrix D and adjacency matrix A. The weight W and offset b are parameters learned by the GCN through training, and is a representation of the node's characteristics in the current state.

[0205] The present disclosure proposes a new GCN-based clustering algorithm that transforms a clustering algorithm problem into a node connection prediction problem, that is, determines whether there is a connection between two nodes and thereby determines whether the two nodes belong to the same category. If the probability that two nodes are connected is greater than a set threshold, it is considered that there is a connection between the two nodes and they belong to the same category, otherwise they are considered not to belong to the same category. Accordingly, the present disclosure uses each node as a center node, searches for k neighbor nodes with the largest cosine distance to form a subgraph, and then determines the probability that the center node is connected to its first-degree neighbor nodes through GCN.

[0206] The present disclosure proposes two ways of configuring a subgraph: a fixed number of neighbor nodes and an indefinite number of nodes.

[0207] As the name implies, a fixed number of neighbor nodes is selected to form a subgraph. To balance the number of positive and negative samples, the present disclosure finds not only the closest node as a negative sample, but also the farthest node.

[0208] FIG. 31 is a schematic flowchart for obtaining neighbor nodes according to the present disclosure. As shown in FIG. 31, the algorithm flow is as follows:

[0209] S1: Calculate the cosine distance between two nodes (S3110);

[0210] S2: Each node is used as the center node P, and k1 neighbor nodes are selected based on cosine distance (S3120). Specifically, the m1 neighbor node with the largest cosine distance from the center node P, the m2 neighbor node with the smallest cosine distance from the center node P, and the m3 neighbor node with an intermediate random cosine distance from the center node P are searched, where the number of neighbor nodes of P is k1 = m1 + m2 + m3;

[0211] S3: For k1 neighbor nodes, k2 neighbor nodes are selected based on cosine distance (S3130). Specifically, for each neighbor node of P, the n node having the largest cosine distance from each neighbor node of P is searched, where k2 = k1 * n; through the above steps, the number of first-order neighbor nodes of the central node P is k1, the number of second-order neighbors is k2, and the theoretical total number of neighbor nodes of P can be determined to be mp = k1 + k2. In practice, since there may be identical neighbors between each first-order neighbor node, the number of neighbor nodes of P is less than or equal to mp.

[0212] S4: For each node, k neighbors to be connected are selected (S3140). For each central node P, k neighbor nodes are selected to form connections between them to form a subgraph represented by an adjacency matrix A, and the degree matrix D is calculated;

[0213] S5: Repeat until each node completes the subgraph configuration, and terminate when the subgraph configuration is complete (S3150). Repeat steps S2 through S4 above until all nodes are used as center nodes to complete the subgraph configuration.

[0214] FIG. 32 is a schematic diagram of a solution for constructing a subgraph in an example of the present disclosure. As shown in FIG. 32, for example, each node is labeled from 1 to 12, and the cosine distance between all 2 nodes among the 12 nodes is calculated; the neighbor nodes k1 = 2 having the largest cosine distance from the center node are calculated using each node as the center node; then the neighbor nodes k2 = 2 are calculated; and finally, the k nodes closest to each node are selected for connection. For example, the first node is taken as the center node, and then the 2 neighbor nodes having the largest cosine distance, the second node and the twelfth node are searched from the first node; then the neighbor nodes of the second node, i.e., the sixth node and the twelfth node, are calculated and searched, as well as the neighbor nodes of the twelfth node, i.e., the seventh node and the second node, are calculated and searched; and all nodes other than the center node, i.e., the first node, are connected to complete the subgraph construction.

[0215] The method of an unfixed number of nodes is also a method of constructing a subgraph with an unfixed number of selected neighbors. The present disclosure uses a cosine distance radius method to find neighbor nodes of a central node. The cosine distance radius is set to R, and if the distance from the central node is greater than the radius R, that node can be considered a neighbor node of the central node. Most neighbor nodes found in this way are neighbor nodes of the central node. To balance the number of positive and negative samples, a larger radius R1 is reset, and nodes with a cosine distance smaller than R and greater than R1 are found.

[0216] FIG. 33 is a schematic flowchart for obtaining neighbor nodes according to the present disclosure. As shown in FIG. 33, the algorithm flow is as follows:

[0217] S1: Calculate the cosine distance between features (S3310). That is, calculate the cosine distance between two nodes;

[0218] S2: Use each node as a center node P, set the cosine distance radius threshold to R, and if the cosine distance of a node is smaller than the threshold, use the node as a neighbor of the center node P (S3320);

[0219] S3: Expand the search range, reset the cosine distance radius threshold to R1, and select nodes where the cosine distance is greater than R and less than R1 (S3330).

[0220] S4: For each node, k neighbors to be connected are selected (S3340). Specifically, k nodes are selected for each node to be connected and a subgraph is constructed; where the subgraph is represented by an adjacency matrix A and its degree matrix D is calculated;

[0221] S5: Repeat steps S2 through S4 above until all nodes are used as center nodes to complete the subgraph configuration, and terminate when the subgraph configuration is complete (S3350).

[0222] FIG. 34 is a schematic diagram of a solution for constructing a subgraph according to the present disclosure.

[0223] As shown in FIG. 34, for example, nodes to be clustered are numbered from 1 to 10, and the cosine distance between all 2 nodes among the 10 nodes is calculated; each node is used as a center node P, and the cosine distance radius threshold is set to R, and if the value is less than the threshold, the node is considered a neighbor of the center node P; the search range is expanded, the cosine distance radius threshold is reset to R1, and nodes having a cosine distance greater than R and less than R1 are selected; and for each node, k nodes are selected and connected. For example, if the first node is taken as the center node, the cosine distance radius is set to R1, and the second, third, fourth, and fifth nodes are selected as neighbor nodes; the larger radius is reset to R2, and the sixth node is additionally selected as a neighbor node of the first node. However, since the nearest neighbor node of the sixth node is not within the range of k1 neighbor nodes, the sixth node is a negative sample and is not connected to any node.

[0224] Conventional technology also uses a fixed number of neighboring nodes to form a subgraph for each node, but uses Euclidean distance to calculate the distance between nodes, whereas the present disclosure uses cosine similarity to calculate the distance between nodes. Euclidean distance reflects the absolute difference between numerical features and is more often used to reflect differences in dimensional values. Cosine distance represents the difference between features in terms of direction. For example, in the case of the same person, facial features at different poses, different lighting, and different moments may be significantly different, yet the facial features still represent the same person. If Euclidean distance were used, the same person's face would be clustered into two people. Therefore, the use of cosine distance in the present disclosure can effectively improve the accuracy of clustering.

[0225] Euclidean distance measures feature similarity based on the absolute numerical values ​​of features in each dimension. The calculation formula is as follows:

[0226]

[0227] Here, dist(X, Y) represents the Euclidean distance between vectors x and y; x i represents the i-th dimension value of vector x; y i represents the i-th dimension value of vector y.

[0228] For features A and B, a larger Euclidean distance between features A and B corresponds to a larger difference between features A and B; a larger cosine distance corresponds to a smaller difference between features A and B.

[0229] Figure 35 is a schematic diagram of a comparison of similarity results using Euclidean distance and cosine distance. As shown in Figure 35, it can be seen that the cosine distance of a person's face at different poses, different lighting, and different moments is relatively large, indicating that they are all the same person. The Euclidean distance is also relatively large, indicating that the face may not belong to the same person. In Figure 35, O represents the Euclidean distance; and C represents the cosine distance.

[0230] The process of constructing the subgraph of the image clustering method has been explained in detail above with reference to the attached drawings and examples. The process of obtaining the connection probabilities between the central node and each neighboring node based on the subgraph will be further explained below.

[0231] In a possible implementation of an embodiment of the present disclosure, the step of determining the connection probability between a central node of a subgraph and its neighbor nodes based on a graph convolution network in step S220 of FIG. 26 may include the following:

[0232] (1) A step of obtaining node embeddings of a subgraph according to at least one layer of a graph convolution network.

[0233] In the present disclosure, node embeddings may be in the form of a matrix, and node embeddings may also be referred to as feature representations or feature matrices.

[0234] Specifically, input features corresponding to a subgraph can be obtained. The input features may include a feature matrix, an adjacency matrix, and a first-order neighbor vector; and the input features are input to at least one layer of a graph convolution network to obtain node embeddings of the subgraph.

[0235] In a specific embodiment, a feature matrix, an adjacency matrix, and a first-order neighbor vector may be input to at least one layer of a graph convolution network for feature extraction, and the output result of the last layer of the graph convolution network is used as a node embedding.

[0236] Specifically, matrices can be input into multiple layers of a graph convolution network for sequential transformation, and the structure of the graph convolution network can also be enhanced, such as by adding pooling or fusing results between two layers of the graph convolution network, which can be described in detail below for the improvement of the graph convolution network.

[0237] (2) Based on the node embedding (also called the feature representation or feature matrix), the connection probability between the central node of the subgraph and its neighboring nodes is obtained.

[0238] Here, the neighbor node can be a first-degree neighbor node, that is, a neighbor node directly connected to the central node.

[0239] Specifically, the node embeddings can be input into at least one fully connected layer (softmax layer) to obtain the connection probability between the center node of each subgraph and each first-order neighbor node.

[0240] In order to express the structural improvement of a graph convolution network in this disclosure more easily and intuitively, a graph convolution network of the prior art will first be described.

[0241] Figure 36 is a schematic diagram of a graph convolution network in the prior art. In the prior art solution, as shown in Figure 36, a 4-layer GCN pair is used to extract node features. The adjacency matrix, feature matrix, and first-order neighbor vector of the subgraph are passed as inputs to the GCN network layer, and then pass through a 2-layer fully connected layer and a softmax layer to predict whether there is a connection probability between the first-order neighbor node and the central node, where cross-entropy is used as a loss function to train the entire neural network. The network structure is shown in the figure.

[0242] Here, Box 1 represents the adjacency matrix A of the subgraph, where A is an N * N matrix and N represents the number of nodes; Box 2 represents the feature matrix F of the subgraph, where F is an N * d matrix and d represents the dimension of the feature, where d = 512 in the prior art, Box 3 represents the first-order neighbor vector, where the first-order neighbor vector is a 1 * k1 matrix and k1 represents the number of first-order neighbors of the central node; Box 4 represents the GCN network layer for extracting node features; and Box 5 represents the fully connected layer.

[0243] From FIG. 36, when there is a relationship between nodes, that is, when nodes belong to the same category, forward propagation in the GCN network will cause the difference between nodes of the same category to decrease and the difference between nodes of different categories to increase. Due to the use of the Euclidean distance mentioned above or the influence of pose, lighting, and facial photographs from different periods, the characteristics of nodes of two different categories may result in very similarities, and connections exist between the nodes. Forward propagation in the GCN network causes the difference between these two nodes of different categories to decrease, leading to subsequent clustering errors. Therefore, the present disclosure employs the following network structure to reduce the occurrence of such errors.

[0244] Improvements to the graph convolution network will be described in more detail below with reference to drawings and examples.

[0245] In one embodiment of the present disclosure, for at least one layer of a graph convolution network other than the first layer of the graph convolution network, the output result of a previous layer of the graph convolution network is used as an input to at least one layer of the graph convolution network; or the output result of a previous layer of the graph convolution network and the output result of at least one layer of the graph convolution network prior to the previous layer of the graph convolution network are fused as an input to at least one layer of the graph convolution network.

[0246] Specifically, the outputs of any two graph convolution networks can be fused as inputs to the next network, which is then used as a single graph convolution network.

[0247] FIG. 37a is a schematic diagram of a graph convolution network introducing a shot-cut mechanism provided by an embodiment of the present disclosure. In the example, as shown in FIG. 37a, the present disclosure introduces a shot-cut mechanism based on the prior art, which can effectively reduce the occurrence of the aforementioned error and improve the accuracy of clustering. The present disclosure uses a multi-layer GCN (such as a 5-layer) to extract node features. The adjacency matrix, feature matrix, and first-order neighbor vector of the subgraph are passed as inputs to the GCN network layers, and then pass through a 2-layer full-connection layer and a softmax layer to predict whether there is a connection probability between the first-order neighbor node and the central node, where cross-entropy is used as a loss function to train the entire neural network and fuses the feature memory of the back layer with the features of the shallow layer. Therefore, the closer to the back layer of the GCN, the smaller the difference between nodes of the same category becomes, and the features of the shallow layer are fused with the features of the back layer; this allows for the preservation of differences between different nodes and avoids clustering different nodes into the same category. For example, features extracted from the first layer of the GCN and features extracted from the third layer are fused, and then the fused features are input into the fourth layer, and features extracted from the second and fifth layers are fused and output.

[0248] The graph convolution network structure shown in FIG. 37a has five layers, which is understood to be merely for the purpose of describing the structure. That is, according to the present disclosure, the specific number of layers of the graph convolution network structure is not limited, and the output of any 2-layer graph convolution network is fused and used as the input to the next layer network of a subsequent graph convolution network, which is not specifically limited herein.

[0249] As shown in Table 1 below, for graph convolution networks without a shot-cut mechanism, this algorithm can effectively improve clustering accuracy, recall rate, and f1-score.

[0250] Accuracy = Number of accurate clusters / Number of clusters

[0251] Recall rate = Number of correct clusters / Number of positive samples in clusters

[0252] F1-Score = Accuracy * Recall Score * 2 / (Accuracy + Recall Rate)

[0253] method Accuracy Recall rate F1-Score Conventional technology 0.8915 0.7669 0.8245 Introduction of shot-cut mechanism 0.9407 0.8297 0.8812

[0254] [Table 1: Comparison of Effects Between Conventional Technology and the Introduction of Shot-Cut Mechanism]

[0255] In the above embodiments, the present disclosure introduces a shot-cut mechanism into a graph convolution network to fuse the output results of the graph convolution network into different layers, and as one approaches the back layer of the graph convolution network, the difference between nodes of the same category becomes smaller, so that the difference between different nodes can be maintained to some extent, the problem of clustering different nodes into the same category can be avoided, and the accuracy of clustering is further improved. In another embodiment, hierarchical clustering is also introduced to improve the structure of the graph convolution network.

[0256] Specifically, at least two adjacent graph convolution networks include a clustering layer between them, and the clustering layer clusters the feature vectors of each node included in the output result of a valuable layer of the graph convolution network (hereinafter also referred to as hierarchical clustering), and outputs the clustering result to the next layer of the graph convolution network.

[0257] Specifically, the feature vector of each node is the first dimension; the clustering result output by the clustering layer is the second dimension; and the second dimension is smaller than the first dimension.

[0258] Specifically, a dimension recovery layer may also be provided between at least two adjacent graph convolution networks located after the clustering layer, and the dimension recovery layer is used to perform dimension recovery on the output result of the clustering layer. Here, the feature input to the dimension recovery layer is a second dimension; and the input feature of the dimension recovery layer is a first dimension, which ensures that the clustering result of the graph convolution network ultimately matches the dimension input to the graph convolution network, thereby improving the accuracy and efficiency of the entire clustering operation.

[0259] FIG. 37b is a schematic diagram of a graph convolution network introducing a shot-cut mechanism and hierarchical clustering provided by an embodiment of the present disclosure.

[0260] As shown in FIG. 37b, a hierarchical clustering module is introduced after each original layer of the GCN. The present disclosure introduces a differentiateable hierarchical clustering operation that can be used at any layer of the GCN. In the process of GCN delivery, nodes belonging to the same category are fused, which means that the clustering function is completed for the nodes within a single GCN, thereby improving the accuracy and efficiency of the entire clustering operation.

[0261] The graph convolutional network structure having a hierarchical clustering module shown in FIG. 37b is understood to have four layers for the purpose of describing the structure. In a specific implementation process, the specific number of layers of the graph convolutional network structure is not limited, and the output of any 2-layer graph convolutional network is fused and used as the input to the next layer network of a subsequent graph convolutional network, and each layer of the graph convolutional network may have a hierarchical clustering operation, and only one layer or part of the graph convolutional network may have a hierarchical clustering operation, and this is not specifically limited herein.

[0262] Considering the problem of clustering errors that may arise from the fact that a conventional GCN clusters node features and can reduce the difference between a noise node and other nodes, the following improvements are made to the GCN structure in this disclosure:

[0263] 1. Use a shot-cut mechanism to maintain differences between different nodes in a subgraph;

[0264] 2. We propose a hierarchical clustering module in GCN to extract hierarchical features of subgraphs and increase the difference between noise nodes and other nodes.

[0265] As shown in FIGS. 37b and FIGS. 37c, in the example, taking a face image as an example, the image clustering method may include the following steps.

[0266] FIG. 37c is a schematic flowchart of an image clustering method in an example of the present disclosure.

[0267] S1: Perform face detection, and face image cropping and alignment;

[0268] S2: Use a CNN to extract feature vectors of the face image, i.e., the face features shown in the diagram, and use the extracted face features as nodes;

[0269] S3: Use each node as a center node, determine at least one neighbor node based on cosine distance, and form a subgraph by connecting the center node and the neighbor node;

[0270] S4: Features of each subgraph are extracted by GCN; differences between noise and other node features are preserved using a shot-cut mechanism and a hierarchical clustering module, and then connection probabilities between the central node and its first-order neighbor nodes are obtained according to a softmax function; and

[0271] Here, step S4 may include the following:

[0272] (1) Take an adjacency matrix, a feature matrix, and a vector of neighbor nodes (specifically, they may be first-order neighbor nodes) as inputs to GCN (S3710);

[0273] (2) Extract features of the subgraph using the first layer of GCN (S3720);

[0274] (3) A hierarchical clustering module is used for hierarchical clustering and feature extraction (S3730);

[0275] (4) In the second and third layers, hierarchical clustered features of the nodes are extracted, and shallow layer features are fused through a shot-cut mechanism (S3740). (5) The number of nodes is restored to the original dimension through a dimension upgrade module (S3750). That is, the number of nodes is upgraded to the original dimension through a dimension upgrade module;

[0276] (6) Features of the upgraded subgraph are extracted using the 4th layer of GCN, and the features extracted from the 4th layer are fused with the features of the 1st layer of GCN to maintain the differences between nodes (S3760); and

[0277] (7) Use softmax to predict the connection probability between the central node and its first-degree neighbor nodes (S3770).

[0278] S5: Classify face images based on the connection probability between the central node and the first-degree neighbor nodes.

[0279] Based on the connection probability between each node and its neighboring nodes, the connection relationships between all nodes can be obtained, and then face images can be classified according to the connection relationships between all nodes, where the average score can be used to remove noise nodes (described in detail later).

[0280] FIG. 37c shows only steps S3 through S5. The shot-cut mechanism of GCN can fuse features of different layers to preserve the difference between noise and other node features.

[0281] FIG. 37d is a schematic diagram of an image clustering solution introducing a shot-cut mechanism and hierarchical clustering according to the present disclosure. As shown in FIG. 37d, FIG. 37d illustrates a process simulating a GCN in an example, where G represents a subgraph and the second node represents a noise node. A represents a feature matrix extracted from the GCN of the second-to-last layer, and B represents a feature matrix after the shot-cut.

[0282] As shown in Table 2 below, AA represents the cosine distance between features without shot-cut, and BB represents the cosine distance between features after using the shot-cut mechanism. After adding shot-cut, the similarity between the noise node and other nodes decreases, and clustering performance improves.

[0283] [Table 2]

[0284]

[0285] [Table 2: Comparison of Cosine Distances Before and After the Introduction of the Shot-Cut Mechanism]

[0286] As can be seen from Table 1, the shot-cut mechanism can maintain the differences between features of different nodes, effectively remove noise nodes, and improve the accuracy of subsequent clustering.

[0287] FIG. 38 is a schematic diagram of an image clustering solution that introduces a shot-cut mechanism and hierarchical clustering according to the present disclosure. As shown in FIG. 38, in the example, a hierarchical clustering module can be introduced into a conventional GCN layer to cluster nodes with strong relationships and to normalize features to determine noise nodes.

[0288] The hierarchical clustering process of GCN is simply simulated. B represents the result of hierarchical clustering, and C indicates that when nodes belong to the same category, the nodes are marked by the characteristics of the central node. It can be seen that the hierarchical clustering module can effectively remove noise nodes and resolve the problem where the difference between noise nodes and other nodes is reduced by conventional GCNs that cluster node characteristics, ultimately leading to clustering errors.

[0289] FIG. 39a is a schematic diagram of clustering using a graph convolution network that introduces a hierarchical clustering operation according to the present disclosure.

[0290] As shown in FIG. 39a, the nodes in Box 1 and Box 2 belong to Node A of the same category, the node in Box 3 belongs to Node B of the same category, and the node in Box 4 belongs to Node C of the same category. There is a node in the category of Node A, and its distance is close to the node in Box 2, and there is a connection between them. After applying the GCN network, the characteristics of the node in Box 2 and the node in Box 3 will become closer, which may affect the subsequent clustering effect. However, through the hierarchical clustering module of the present disclosure, small clustering is completed for each node within the GCN, similar node characteristics are fused, thereby reducing the influence of a small number of incorrect nodes.

[0291] FIG. 39b is a schematic diagram of clustering using a graph convolution network that introduces a hierarchical clustering operation according to the present disclosure.

[0292] As shown in FIG. 39b, the third node is not in the same category as the 0th, 1st, and 2nd nodes, but the cosine distance between the 0th node and the 3rd node is relatively large due to lighting and orientation. If a conventional clustering algorithm is used, the high similarity between the 0th node and the 3rd node causes two categories to be clustered into one category, resulting in a clustering error. However, in the present disclosure, hierarchical clustering is used to determine whether two nodes belong to the same category by observing information about neighboring nodes. If they belong to the same category, they are clustered into one subcategory in the GCN; otherwise, they are not clustered into one category, while a dimension recovery module increases the category interval using the features of neighboring nodes that represent those features. For example, although the 0th node and the 3rd node have high similarity, it can be determined from the neighbor information of the two nodes that the 0th node and the 3rd node belong to nodes of different categories. Through hierarchical clustering, this is clustered into small categories, and neighbor node information may indicate that the difference between the features of the 0th node and the 3rd node has increased. 0.2 in Fig. 39b represents the average connection probability between the 0th node and the 3rd node, and since this is smaller than the set threshold, the 0th node and the 3rd node may disconnect, thereby avoiding subsequent clustering clustering the 0th node and the 3rd node into a single category.

[0293] Based on the GCN neural network model above, this solution reduces the number of GCN network layers one by one and adds a hierarchical clustering module and a dimension recovery module after each GCN layer, as shown in Table 3, and the F1-score of these modules increased by 0.0126 compared to the improved model above.

[0294] method Accuracy Recall rate F1-Score Conventional technology 0.8915 0.7669 0.8245 Introduction of shot-cut mechanism 0.9407 0.8297 0.8812 Introduction of hierarchical clustering 0.8947 0.8929 0.8938

[0295] [Table 3: Comparison of the effects of conventional technology, hierarchical clustering, and the introduction of shot-cut mechanisms]

[0296] A subgraph G is defined. Each subgraph is represented by an adjacency matrix A (Block 1 in Fig. 37b). A is an N x N matrix in which internal elements relate to whether there is a connection between nodes; for example, since it is 1 if there is a connection between nodes and 0 otherwise, A ∈ {0,1} N×N The feature vector of each node of graph G is used to form the feature matrix F (Block 2 in Fig. 37b), where F is an N * D dimension matrix and D represents the dimension of the feature vector of each node, and The present disclosure performs clustering by predicting the probability between neighbor nodes and a central node, so Node represents a first-degree neighbor node label (Block 3 in FIG. 37b), Node is a 1 * n1 dimensional vector, and n1 represents the number of first-degree neighbor nodes of the central node, and Box 4 in Fig. 37b represents the GCN layer, and the formula is as follows:

[0297] Here, H l+1 represents the characteristics of each node in the next layer of GCN, and H l represents the feature representation of the current layer node, and H in the first layer of GCN l = F and; W l represents the weight matrix of the first layer, which is a D × D1 dimensional matrix, and and; b l represents the offset of the first layer, and M represents the normalized state value of the node adjacency matrix, And, represents the adjacency matrix, and represents the order matrix of the adjacency matrix, σ represents a non-linear activation layer, and the present disclosure uses ReLU as the activation function.

[0298] Since graph structures are unstructured data, pooling operations in digital images cannot be used directly. Therefore, the present disclosure relates to a learnable transformation factor S (1) Implements hierarchical clustering functionality through. S (l) is an N * N1 dimensional matrix, which represents the probability that each node belongs to a different category, and N1 represents the clustering of N nodes in the graph into N1 nodes. S (l) Since it is trained through a single layer of the GCN, it can be directly embedded into any layer of the entire GCN neural network. Z l represents the node clustering factor:

[0299]

[0300] Nodes are clustered by transformation factors:

[0301]

[0302] Therefore, the node feature matrix after GCN hierarchical clustering is X l+1 And this is a matrix of dimension N1 * D. X l+1 Since is the feature matrix of the nodes after clustering, the adjacency matrix A between nodes (l+1) It is necessary to reconstruct:

[0303]

[0304] Here, S (l) represents the transformation matrix and also the hierarchical clustering matrix.

[0305] Adjacency matrix A of nodes after clustering l+1 and feature matrix X l+1 It can be obtained using the above formula.

[0306] In the above embodiment, hierarchical clustering is introduced after each layer of the original graph convolution network, and hierarchical clustering can be used at any layer of the graph convolution network and can fuse nodes belonging to the same category during the transmission process of the graph convolution network; that is, each clustering function is completed for a node within a single graph convolution network, thereby improving the accuracy and efficiency of the overall clustering operation.

[0307] FIG. 40a is a schematic flowchart of an image clustering method according to the present disclosure.

[0308] As shown in FIG. 40a, in the example, the hierarchical clustering and dimension recovery process may include the following steps:

[0309] S1: Takes an adjacency matrix, a feature matrix, and a vector of neighbor nodes (specifically, they may be first-order neighbor nodes) as inputs to the GCN (S4010);

[0310] S2: Use the first layer of GCN to extract features of the subgraph, and use a hierarchical clustering module to perform hierarchical clustering and feature extraction (S4020);

[0311] S3: In the second and third layers of GCN, cluster node features and shallow layer cluster features are extracted through a shot-cut mechanism (S4030);

[0312] S4: Upgrade the number of nodes to the original dimension through the dimension upgrade module (S4040). That is, restore the number of nodes to the original dimension through the dimension upgrade module.

[0313] In S1, the adjacency matrix, feature matrix, and first-order neighbor matrix are used to extract node features through GCN. In S2, the hierarchical clustering module is used to cluster nodes with the same identity into the same category. In S3, the clustered feature nodes are displayed as features of the original nodes. In S4, the dimension recovery module is used to recover the dimensions of the subgraph, and the clustered features are used to represent the features of nodes in the same subcategory.

[0314]

[0315] Here, M (l) It represents the category to which each node belongs after hierarchical clustering.

[0316] In the dimensional recovery module, M (l) and X (l+1) is used to obtain the recovered feature matrix. For example, in FIG. 40a, all nodes (0,1,2,3) are clustered into two categories (categories 0 and 1) through a hierarchical clustering module. In step S2, M (l) = {0,0,1,0} indicates that the 0th, 1st, and 3rd nodes belong to category 0, and the 2nd node belongs to category 1. M (l) Since the 0th node, the 1st node, and the 3rd node have the same identity, it can be seen that in S4, the characteristics of the 4th node represent the 0th node, the 1st node, and the 3rd node.

[0317] FIG. 40b is a schematic diagram of a dimensional recovery solution according to the present disclosure.

[0318] As shown in FIG. 40b, FIG. 40b illustrates the process of dimension recovery. According to Formula 15, the category to which each node belongs can be determined, and the node feature vector at the corresponding location is used as the feature vector of the recovered location when the dimension is recovered. For example, through the hierarchical clustering module, all nodes (0,1,2,3) are clustered into two categories (categories 0 and 1), and the categories obtained by clustering are M (l) = denoted as {0,0,1,0}, which indicates that in step S2, the 0th node, the 1st node, and the 3rd node belong to category 0, and the 2nd node belongs to category 1; the feature vector of the 0th position after clustering is used as the feature vector of the node after dimension recovery, which is the feature vector of the 0th node, the 1st node, and the 3rd node, and the same applies to other nodes. In Formula 15, M (l) It can also be used to calculate the recovered neighbor matrix representation in the dimension recovery module:

[0319]

[0320] Here, A (r) is the sum of the adjacency matrix of the subgraph after the dimension recovery module and the adjacency matrix of the subgraph of the previous layer representing the final adjacency matrix of the dimension recovery module; A (l + 1) is used to represent the adjacency matrix of the subgraph for the previous layer; A (') is used to represent the adjacency matrix of the subgraph after the dimension recovery module; A (') The meaning of [i][j] is A (') It matches the meaning of.

[0321] The above example details improvements to a graph convolution network, including the introduction of a shot-cut mechanism and a hierarchical clustering module. The process of clustering target regions based on connection probabilities will be described below.

[0322] In the implementation process, the step of clustering target regions according to the connection probability determined in step S220 of FIG. 26 may include the following:

[0323] For at least one target region to be clustered, if the probability of connection between a first-degree neighbor node and a central node in a subgraph corresponding to the target region is greater than a preset threshold, then the target region corresponding to the first-degree neighbor node is in the same category as the target region.

[0324] Specifically, if the connection probability between the first-degree neighbor node and the central node is greater than a preset threshold, the target area corresponding to the first-degree neighbor node and the target area corresponding to the central node belong to the same category.

[0325] After the GCN neural network, a pair of probability values ​​between each central node and a first-order neighbor node is obtained; then, each prediction result is traversed, and if it is lower than a set threshold, it means there is no connection between the central node and the neighbor node. Finally, a breadth-first algorithm is used to traverse each edge, nodes with a probability greater than the threshold are connected, and clustering is completed.

[0326] FIG. 41 is a schematic diagram of a clustering solution based on connection probability according to one embodiment of the present disclosure.

[0327] As shown in Fig. 41, each cluster node is used as a central node, and neighbor nodes are found through cosine distance to form a subgraph having the central node as a unit; each subgraph is applied to a GCN neural network to predict the connection probability between the central node and the first-order neighbor node; if the connection probability between two nodes is greater than a threshold, they are connected; and finally, clustering is completed by traversing each connection pair.

[0328] In another implementation process, the step of clustering target regions according to the connection probability determined in step S220 of FIG. 26 may include the following:

[0329] (1) For at least one target area to be clustered, a set of neighbor nodes of the same category is obtained according to the connection probability between the corresponding center node and neighbor nodes.

[0330] Here, the set of neighbor nodes of the same category includes at least one neighbor node determined to be in the same category as the central node.

[0331] Specifically, for at least one target region to be clustered, the step of obtaining a set of neighbor nodes of the same category according to the connection probability between a corresponding central node and neighbor nodes may include the following:

[0332] For at least one target region to be clustered, a neighbor node whose connection probability with the central node is greater than or equal to a preset threshold is determined, and the determined neighbor node is formed as a set of neighbor nodes of the same category.

[0333] (2) For at least one neighbor node among a set of neighbor nodes of the same category, a step of adjusting the connection probability between at least one neighbor node and the central node according to the connection probability between each neighbor node and at least one neighbor node among a set of neighbor nodes of the same category.

[0334] Specifically, for at least one neighbor node among a set of neighbor nodes of the same category, the average connection probability between each neighbor node and at least one neighbor node of the set of neighbor nodes of the same category can be determined, and the average connection probability is used as the connection probability between at least one neighbor node and the central node.

[0335] (3) A step of clustering target regions based on adjusted connection probabilities to determine whether at least one neighbor node is a neighbor node of the same category.

[0336] Specifically, if the adjusted connection probability is greater than or equal to a preset first threshold, the neighbor node corresponding to the adjusted connection probability can be set as a neighbor node of the same category.

[0337] (4) A step of clustering each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered.

[0338] Specifically, the step of clustering each target region to be clustered according to neighbor nodes of the same category corresponding to each target region to be clustered may include the following:

[0339] a: a step of determining the connection probability between central nodes corresponding to each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered; and

[0340] b: A step of clustering each target region to be clustered based on the connection probability between each central node.

[0341] Specifically, if the connection probability between central nodes is greater than or equal to a preset second threshold, the target area corresponding to the central node can be set to the same category.

[0342] Specifically, the present disclosure proposes an average score-based algorithm to enhance the stability of the algorithm. For each subgraph, the probability between a central node and its neighboring nodes is predicted via GCN, and a set C of all nodes connected to A (nodes with a threshold greater than the threshold) is obtained. If the connection probability between A and B is greater than the set threshold, the average value of the probabilities of all nodes connected to set C and set B is calculated as the probability value of two nodes A and B being connected. Finally, clustering is completed by traversing each connection pair. By considering connection information between multiple nodes and B, the stability that A and B are not nodes of the same category can be enhanced.

[0343] FIG. 42a is a schematic diagram of a solution for removing noise nodes according to an average value strategy according to the present disclosure.

[0344] As shown in Figure 42a, for example, A, B, C, D, and E are nodes of the same category, and the connection probability between node F and A is greater than 0.9, but the connection probabilities between F and B, C, D, and E are 0.05, 0.1, 0.2, and 0.15. If the average value of the probability 0.1 is taken as the probability value between node A and F, it can be indicated that the connection probability between A and F is very small, thereby improving the accuracy of the algorithm.

[0345] FIG. 42b is a schematic flowchart of a method for removing noise nodes by an average value strategy according to the present disclosure.

[0346] As shown in FIG. 42b, in the example, the process of determining a noise node may include the following steps:

[0347] S1: A step (S4210) of traversing all nodes and, for any node, setting as isolated nodes a node in which the connection probability is greater than a preset threshold and the number of neighbor nodes of the node is less than a preset number of nodes, wherein the preset number of nodes may be 2;

[0348] S2: A step of sorting according to the connection probability between an isolated node and its second-degree neighbor node (i.e., neighbor of a neighbor node) (S4220);

[0349] S3: A step of using the average of the connection probabilities as the connection probability between an isolated node and its neighbor nodes (S4230);

[0350] S4: If the connection probability is less than the set threshold, the node is a noise node (noise) (S4240).

[0351] To improve the accuracy and stability of clustering, an average algorithm was proposed in the above example to determine whether an isolated node is a noise node based on neighbor relationships. A, B, C, D, and E are nodes of the same category. The connection probability between F and A is greater than 0.9, but the connection probabilities between F and B, C, D, and E are very small, at 0.05, 0.1, 0, and 0, respectively. The average of the connection probabilities between F and A, B, C, D, and E is taken as the connection probability between F and A. Based on this algorithm, F can be determined to be a noise node of category A; that is, F does not belong to category A. The average value algorithm can effectively avoid clustering errors and improve the accuracy and stability of clustering.

[0352] Different clustering methods based on connection probability were described above. The clustering process described above involves clustering all face images to be clustered. However, in small devices such as mobile phones with limited computing resources, initial clustering and post-additional clustering are commonly used. As the number of faces to be clustered increases, the temporal and spatial complexity of the clustering algorithm proposed in this disclosure also increases, requiring the consumption of more computing resources. Meanwhile, when users take photos using a mobile phone, they do not capture all photos at once but gradually accumulate them over time. Therefore, the additional method enables users to quickly check clustering results and improve the user experience.

[0353] In a possible implementation of an embodiment of the present disclosure, the image clustering method further comprises the following:

[0354] (1) A step of acquiring a new target region to be clustered;

[0355] (2) A step of obtaining a preset number of target areas in at least one clustered category;

[0356] (3) A step of clustering the acquired target region clustered based on a graph convolution network and the new target region to be clustered to obtain a new clustering result; and

[0357] (4) A step of determining the category of a new target area to be clustered based on the new clustering result and the category corresponding to the clustered target area.

[0358] Specifically, when clustering a new target area to be clustered based on the acquired clustering results, a preset number of images may be acquired from each clustered category, and new clustering results are acquired according to the clustered target area and the new target area to be clustered in order to determine the category of the new target area to be clustered. This can be applied to small devices, such as mobile phones with limited computing resources, to improve clustering efficiency and allow users to quickly check clustering results, thereby enhancing the user experience.

[0359] FIG. 43 is a schematic flowchart of clustering new data to be clustered, provided by an embodiment of the present disclosure.

[0360] As shown in FIG. 43, in the initial clustering node, a preset number of N faces are selected to obtain n face categories, i.e., clustered categories; when a new photo is detected, face detection, face alignment, and face feature extraction are performed on the new photo to obtain the features of each face (S4320); n2 face images from the n clustered categories are randomly selected to represent the category and fused with the new face images, and simultaneously, the fused images are clustered; the ratio of the new photo of the new clustering result to the existing clustering result is calculated, and if it is greater than a set ratio of 1 (e.g., 0.5), it is fused with the existing clustering result, or if it is less than a set ratio of 2 (e.g., 0.2), it is considered a new category; if it is greater than 0.2 and less than 0.5, no action is performed, and re-clustering can be performed in the next clustering.

[0361] To more clearly understand the application of the image clustering method of the present disclosure, the image clustering method of the present disclosure will be described below with reference to an embodiment.

[0362] In a possible implementation of an embodiment of the present disclosure, the image clustering method may further include the following steps:

[0363] (1) Receive keywords for image search entered by the user;

[0364] (2) Determine the category associated with the keyword based on the clustering results; and

[0365] (3) Search for images that match the keyword among the images in the determined category.

[0366] Specifically, when a user needs to search for images matching a keyword from a large number of images, direct search is inaccurate and time-consuming; therefore, the category associated with the keyword can first be determined from multiple clustered categories.

[0367] For example, if it is necessary to search for a person's name among multiple images of various categories, the category of "female star" can be determined from the clustered categories, and then a search can be performed from the category where the keyword was determined; this process can be effective in improving search efficiency and reducing search time.

[0368] In a possible implementation of an embodiment of the present disclosure, the image clustering method may further include the following steps:

[0369] (1) Obtain multiple tags to be marked; and

[0370] (2) Based on the clustering results, mark the target area of ​​each clustered category with a tag.

[0371] Specifically, in the process of labeling multiple images of various categories, target areas are first clustered to determine the categories to which they can be clustered, and then the closest tag for each label to be labeled can be determined for automatic category labeling. This process can effectively improve labeling efficiency.

[0372] Depending on the requirements of the application scenario and hardware environment, the present disclosure may use other solutions to complete the corresponding tasks.

[0373] 1) High-performance clustering solution for mobile-end

[0374] The mobile-end integration method requires low computing resource consumption and uses low-channel, low-dimensional, and low-input solutions in the network model design. Solutions for initial clustering and additional clustering are used in the clustering logic, and as shown in Fig. 44, the specific workflow is as follows:

[0375] FIG. 44 is a schematic flowchart of an image clustering method applied to a mobile terminal provided by an embodiment of the present disclosure.

[0376] S1: Perform face detection, face alignment, and face feature extraction (128-dimensional face feature extraction) on photos in a mobile album (S4400);

[0377] S2: GCN is used to predict the node connection probabilities for existing face features in order to complete clustering through a post-processing algorithm to obtain initial clustering results;

[0378] S3: The electronic device acquires new features to be clustered (S4405). When new face features to be clustered are acquired, the electronic device uses the features previously extracted in S4400 and the newly acquired features in S4405 to select six face features as representatives from the clustered categories to form a new set to be clustered, and determines whether the number of face features to be clustered is greater than a set threshold (e.g., 20). For example, the electronic device determines whether the number of face features to be clustered is greater than 20 (S4410); if the number of face features to be clustered is less than 20 (S4415-N), the electronic device sets the ID of all faces to -1 and waits for the next clustering (S4415). Then, if the number of face features to be clustered is greater than 20 (S4415-Y), GCN clustering is performed (S4420).

[0379] S4: GCN is used to predict node connection probabilities to complete clustering through a post-processing algorithm;

[0380] S5: Determine whether the clustering category is greater than a set number (such as 5). For example, the electronic device determines whether the output cluster category is greater than 5 (S4425). If it is less than 5 (S4425-N), the electronic device sets the IDs of all faces to -1 and waits for the next clustering (S4430). If the output cluster category is greater than 5 (S4425-Y), proceed to the next step;

[0381] S6: If the output cluster category is greater than 5 (S4425-Y), the electronic device calculates the ratio of the new photo of the new clustering result to the existing clustering result (S4435). Here, if it is greater than a set ratio of 1 (such as 0.5), the electronic device fuses it with the existing clustering result (S4440). If it is less than a set ratio of 2 (such as 0.2), it will be considered as a new category, and the electronic device can use the output cluster category as the new category (S4445). And, if it is greater than 0.2 and less than 0.5, the electronic device does not perform any action, and re-clustering can be performed in the next clustering (S4450).

[0382] Then, the electronic device updates each category corresponding to 20 features (S4455) and can obtain a new clustering result (S4460). Then, the electronic device updates the clustering result (S4465) and can repeat the process described above.

[0383] FIG. 45 is a schematic diagram of a neural network structure applied to a mobile terminal in an embodiment of the present disclosure.

[0384] A neural network model of the mobile end is shown in FIG. 45, where Box 1 represents the adjacency matrix A of the subgraph, where A is an N * N matrix and N = 31 represents the number of nodes; Box 2 represents the feature matrix F of the subgraph, where F is an N * d matrix and d = 128 represents the dimension of the features; Box 3 represents the label "Node" of the first-order neighbors, where "Node" is a 1 * k1 matrix and k1 = 10 represents the number of first-order neighbors of the central node; Boxes 4, 5, and 6 all represent GCN network layers for node feature extraction, where different color shading indicates that the output channels of the GCN network layers are different, and the narrower channels correspond to smaller channels with less corresponding computation; Box 7 represents a fully connected layer for predicting the probability that the central node is connected to the first-order neighbor nodes.

[0385] 2) Server-end high-performance solution

[0386] Since large servers possess powerful computing performance and large memory, large neural network models can be used to improve the accuracy of clustering. A neural network structure introducing hierarchical clustering, namely the neural network structure shown in FIG. 37b, has high F1-scores on multiple test sets and can be used for tasks requiring an F1-score indicator. A neural network introducing a shot-cut mechanism, namely the neural network shown in FIG. 37a, has high accuracy and is suitable for tasks requiring an accuracy indicator. As shown in FIG. 46, the batch flow is as follows:

[0387] FIG. 46 is a schematic flowchart of an image clustering method applied to a server provided by an embodiment of the present disclosure.

[0388] S1: The electronic device performs face detection, face alignment, and face feature extraction on the photos in the album (S4610);

[0389] S2: GCN is used to predict the node connection probability for existing face features in order to complete clustering through a post-processing algorithm to obtain an initial clustering result; that is, the electronic device can obtain an initial clustering result through clustering GCN (S4620).

[0390] S3: The electronic device can identify whether a new face photo is acquired (S4630). If a new face photo is acquired (S4630-Y), the electronic device extracts facial features from the new face photo and fuses them with facial features from the original face photo to perform re-clustering (S4640). Then, the electronic device can repeat the processes described above, S4620, S4630, and S4640.

[0391] If no new face photos are acquired (S4630-N), the electronic device can output a clustering result (S4650).

[0392] In the image clustering method described above, the present disclosure clusters target regions of each image through a graph convolution network without determining a clustering center, thereby avoiding deviations in the overall clustering results caused by deviations in the clustering center. Consequently, the accuracy of the clustering results can be effectively improved.

[0393] Additionally, the distance between nodes can be calculated using cosine similarity. Cosine distance distinguishes differences between features in terms of orientation. Regarding a person's different pose, different lighting, and different moments, it implies that the differences in facial features are relatively large but still represent the same person. Compared to Euclidean distance, which reflects the absolute differences in features regarding values, if Euclidean distance is used, it would cause the same person to be clustered as two people. Therefore, the use of cosine distance in the present disclosure can further improve the accuracy of clustering.

[0394] In addition, the present disclosure introduces a shot-cut mechanism in a graph convolution network to fuse the output results of the graph convolution network for different layers, and as one approaches the back layer of the graph convolution network, the difference between nodes of the same category becomes smaller, so that the difference between different nodes can be maintained to some extent, the problem of clustering different nodes into the same category can be avoided, and the clustering accuracy is further improved.

[0395] In addition, hierarchical clustering is introduced after at least one layer of the original graph convolution network, and hierarchical clustering can be used at any layer of the graph convolution network. Nodes belonging to the same category can be fused during the propagation process of the graph convolution network; that is, a single clustering function is completed for the nodes within the graph convolution network, thereby improving the accuracy and efficiency of the overall clustering operation.

[0396] In addition, when a new target area to be clustered is acquired based on the clustered categories, a preset number of images can be acquired from each clustered category. To determine the category of the new target area to be clustered, a new clustering result is obtained based on the clustered images and the new target area to be clustered. This can be applied to small devices such as mobile phones with limited computing resources, thereby improving clustering efficiency and allowing users to quickly verify clustering results, and thus enhancing the user experience.

[0397] The above-described embodiment introduces an image clustering method in terms of method flow. The following describes the image clustering method in terms of virtual modules, specifically as follows:

[0398] An embodiment of the present disclosure provides an image clustering device (700). FIG. 47 is a schematic diagram of the structure of an image clustering device provided by an embodiment of the present disclosure.

[0399] As shown in FIG. 47, the image clustering device (700) may include an acquisition module (701) and a first clustering module (702).

[0400] The acquisition module (701) may be configured to acquire target regions of all images to be clustered and to acquire feature vectors of the target regions. Then, the first clustering module (702) may be configured to cluster the acquired feature vectors based on a graph convolution network to acquire results.

[0401] In a possible embodiment of the present disclosure, the first clustering module (702) may cluster the acquired feature vectors based on a graph convolution network to obtain results, specifically: the first clustering module (702) may construct a subgraph corresponding to the feature vectors, wherein the subgraph includes a central node corresponding to the feature vector and at least one neighbor node corresponding to another feature vector; determine the connection probability between the central node of the subgraph and its neighbor node based on a graph convolution network, and may be configured to cluster target regions according to the determined connection probability to obtain results.

[0402] In a possible implementation of an embodiment of the present disclosure, when constructing a subgraph corresponding to a feature vector, the first clustering module (702) is specifically configured to take a feature vector as a center node and obtain neighbor nodes of the center node from other feature vectors. Then, the first clustering module (702) constructs a subgraph according to the center node and neighbor nodes.

[0403] In a possible implementation of an embodiment of the present disclosure, when obtaining neighbor nodes of a central node from another feature vector by taking a feature vector as a central node, the first clustering module (702) may be configured to specifically: obtain a cosine distance between a feature vector and another feature vector, respectively; and screen neighbor nodes of a feature vector from another feature vector based on the cosine distance between a feature vector and another feature vector.

[0404] In a possible implementation of an embodiment of the present disclosure, when determining the connection probability between a central node of a subgraph and its neighbor nodes based on a graph convolution network, the first clustering module (702) may be specifically configured to: obtain a node embedding of the subgraph according to at least one layer of the graph convolution network; and obtain a connection probability between a central node of the subgraph and its neighbor nodes based on the node embedding.

[0405] In a possible implementation of an embodiment of the present disclosure, when obtaining a node embedding of a subgraph according to at least one layer of a graph convolution network, the first clustering module (702) specifically: obtains an input feature corresponding to the subgraph; inputs the input feature to at least one layer of the graph convolution network for feature extraction, and may be configured to use the output result of the last layer of the graph convolution network as a node embedding.

[0406] In a possible implementation of an embodiment of the present disclosure, for at least one layer of a graph convolution network other than the first layer of the graph convolution network, the output result of a previous layer of the graph convolution network is used as an input to at least one layer of the graph convolution network; or the output result of a previous layer of the graph convolution network and the output result of at least one layer of the graph convolution network prior to the previous graph convolution network may be fused as an input to at least one layer of the graph convolution network.

[0407] In a possible implementation of an embodiment of the present disclosure, at least two adjacent graph convolution networks include a clustering layer between them, and the clustering layer clusters the feature vectors of each node included in the output result of a valuable layer of the graph convolution network and can input the clustering result to the next layer of the graph convolution network.

[0408] In a possible implementation of an embodiment of the present disclosure, when clustering target regions according to a determined connection probability, the first clustering module (702) specifically: for at least one target region to be clustered, obtains a set of neighbor nodes of the same category according to the connection probability between a corresponding center node and a neighbor node; for at least one neighbor node in the set of neighbor nodes of the same category, adjusts the connection probability between at least one neighbor node and the center node according to the connection probability between each neighbor node and at least one neighbor node in the set of neighbor nodes of the same category; clusters target regions based on the adjusted connection probability to determine whether at least one neighbor node is a neighbor node of the same category; and may be configured to cluster each target region to be clustered according to a neighbor node of the same category corresponding to each target region to be clustered.

[0409] In a possible implementation of an embodiment of the present disclosure, when acquiring a set of neighbor nodes of the same category according to the connection probability between a corresponding center node and neighbor nodes for at least one target area to be clustered, the first clustering module (702) may specifically be configured to: determine a neighbor node whose connection probability with the center node is greater than or equal to a preset threshold for at least one target area to be clustered, and to form the determined neighbor nodes as a set of neighbor nodes of the same category.

[0410] In a possible implementation of an embodiment of the present disclosure, when adjusting the connection probability between at least one neighbor node and a central node according to the connection probability between each neighbor node and at least one neighbor node of the set of neighbor nodes of the same category for at least one neighbor node of the same category, the first clustering module (702) may specifically be configured to determine the average connection probability between each neighbor node and at least one neighbor node of the set of neighbor nodes of the same category for at least one neighbor node of the same category as the connection probability between at least one neighbor node and a central node.

[0411] In a possible implementation of an embodiment of the present disclosure, when clustering each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered, the first clustering module (702) specifically: determines the probability of connection between central nodes corresponding to each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered; and may be configured to cluster each target area to be clustered based on the probability of connection between each central node.

[0412] In a possible implementation of an embodiment of the present disclosure, the image clustering device further comprises a search module, wherein the search module may be configured to: receive a keyword for image search entered by a user; determine a category associated with the keyword based on a clustering result; and search for an image that matches the keyword among images of the determined category.

[0413] In a possible implementation of an embodiment of the present disclosure, the image clustering device further comprises a labeling module, wherein the labeling module may be configured to: acquire a plurality of tags to be marked; and mark the tags respectively in a target area of ​​an image of each clustered category based on the clustering result.

[0414] In a possible implementation of an embodiment of the present disclosure, the image clustering device further comprises a second clustering module, wherein the second clustering module may be configured to: acquire a new target region of an image to be clustered; acquire a preset number of target regions in at least one clustered category; cluster the acquired target region and the new target region to be clustered based on a graph convolution network to obtain a new clustering result; and determine the category of the new target region to be clustered based on the new clustering result and the category corresponding to the clustered target region.

[0415] In the image clustering device described above, the present disclosure clusters target regions of all images through a graph convolution network without determining a clustering center, thereby avoiding deviations in the overall clustering results caused by deviations in the clustering center. As a result, the accuracy of clustering can be effectively improved.

[0416] Additionally, the distance between nodes can be calculated using cosine similarity. Cosine distance distinguishes differences between features based on direction. For a person with different poses, different lighting, and different moments, the differences in facial features are relatively large, yet they still refer to the same person. Compared to Euclidean distance, which reflects the absolute differences in features in terms of values, using Euclidean distance would result in clustering into two people. Therefore, the use of cosine distance in this disclosure can further improve the accuracy of clustering.

[0417] In addition, the present disclosure introduces a shot-cut mechanism in a graph convolution network to fuse the output results of the graph convolution network for different layers, and as one approaches the back layer of the graph convolution network, the difference between nodes of the same category becomes smaller, so that the difference between different nodes can be maintained to some extent, the clustering of different nodes into the same category can be avoided, and the clustering accuracy can be further improved.

[0418] In addition, hierarchical clustering is introduced after at least one layer of the original graph convolution network, and hierarchical clustering can be used at any layer of the graph convolution network and can fuse nodes belonging to the same category during the propagation process of the graph convolution network; that is, a single clustering function is completed for the nodes within the graph convolution network, thereby improving the accuracy and efficiency of the entire clustering operation.

[0419] In addition, based on clustering, when a new target region to be clustered is obtained, a preset number of images can be obtained from each clustered category, and a new clustering result is obtained according to the clustered images and the new target region to be clustered in order to determine the category of the new target region to be clustered. This can be applied to small devices such as mobile phones with limited computing resources, thereby improving clustering efficiency and allowing users to quickly check clustering results, which can thereby enhance the user experience.

[0420] The image clustering device of the embodiments of the present disclosure can execute the image clustering method provided by the embodiments of the present disclosure, and the implementation principle is similar. The action executed by the module of the image clustering device in each embodiment of the present disclosure corresponds to the step of the image clustering method in each embodiment of the present disclosure. For a detailed functional description of the module of the image clustering device, one may refer to the description of the corresponding image clustering method shown in the preceding section, which may not be repeated here.

[0421] In the device provided in the embodiment of the present disclosure, at least one of the plurality of modules may be implemented through an AI (Artificial Intelligence) model. Functions associated with AI may be performed through non-volatile memory, volatile memory, and a processor.

[0422] The processor may include one or more processors. In this case, the one or more processors may be a general-purpose processor (such as a central processing unit (CPU), an application processor (AP), etc.), a pure graphics processing unit (such as a graphics processing unit (GPU), a visual processing unit (VPU), etc.) and / or an AI-dedicated processor (such as, for example, a neural processing unit (NPU)).

[0423] One or more processors control the processing of input data according to predefined behavior rules or artificial intelligence (AI) models stored in non-volatile and volatile memory. Predefined behavior rules or AI models are provided through training or learning.

[0424] Here, "provided by learning" refers to obtaining an AI model having predefined behavioral rules or desired features by applying a learning algorithm to multiple learning data. Depending on the embodiment, learning may be performed on the device itself where the AI ​​is executed and / or realized by a separate server / system.

[0425] AI models can be composed of multiple neural network layers. Each layer has multiple weights, and the computation of one layer is performed by the computation result of the previous layer and the multiple weights of the current layer. Examples of neural networks include, but are not limited to, convolutional neural networks (CNN), deep neural networks (DNN), recurrent neural networks (RNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), bidirectional recurrent deep neural networks (BRDNN), generative adversarial networks (GAN), and deep Q networks.

[0426] A learning algorithm is a method of training a predetermined target device (e.g., a robot) using multiple learning data to enable, allow, or control the target device to make decisions or predictions. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0427] The image clustering device provided by the embodiments of the present disclosure was introduced above in terms of functional modularization. Next, the electronic device provided by the embodiments of the present disclosure will be introduced in terms of hardware implementation, and the computing system of the electronic device will be introduced simultaneously.

[0428] An electronic device is also provided in the embodiments of the present disclosure based on the same principles as the method shown in the embodiments of the present disclosure. The electronic device may include, but is not limited to, a processor and a memory, the memory is for storing computer operation instructions, and the processor is for calling computer operation instructions to execute the image clustering method shown in the embodiments. Compared to the prior art, the image clustering method provided by the embodiments of the present disclosure can avoid deviation in the overall clustering result caused by deviation in the clustering center and can effectively improve the accuracy of the clustering result.

[0429] In an optional embodiment, an electronic device is provided. FIG. 48 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.

[0430] As shown in FIG. 48, the electronic device (1000) shown in FIG. 48 includes a processor (1001) and a memory (1003). Here, the processor (1001) and the memory (1003) are connected, for example, via a bus (1002). Optionally, the electronic device (1000) may further include a transceiver (1004). It should be noted that in actual applications, the transceiver (1004) is not limited to one, and the structure of the electronic device (1000) does not limit the embodiments of the present disclosure.

[0431] The processor (1001) may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Data Signal Processor), an ASIC (Application-Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. These may implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of this disclosure. The processor (1001) may also be a combination that realizes computing functions, for example, a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0432] The bus (1002) may include a path for transmitting information between the components described above. The bus (1002) may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus (1002) may be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, only one thick line is used in FIG. 48, but this does not mean that there is only one bus or one bus category.

[0433] The memory (1003) may be a ROM (read-only memory) or a static storage device of another category capable of storing static information and instructions, a RAM (Random Access Memory) or a dynamic storage device of another category capable of storing information and instructions, and may also be an EEPROM (electrically erasable programmable read-only memory), a CD-ROM (compact disc read-only memory) or other optical disc storage, an optical disc storage (including compressed optical discs, laser discs, optical discs, digital discs, Blu-ray discs, etc.), a disc storage medium or other magnetic storage device, or any other medium used to transmit or store instructions or desired program code in the form of a data structure, and may be accessed by a computer, but is not limited thereto.

[0434] The memory (1003) is configured to store application program code for executing the solution of the present disclosure and is controlled by a processor (1001) to perform execution. The processor (1001) is configured to execute the application program code stored in the memory (1003) to implement the content shown in any one of the above-described method embodiments.

[0435] In particular, according to the present disclosure, the processor (1001) may acquire a plurality of images for performing clustering. The plurality of images may include, for example, face photographs, but the present disclosure is not limited thereto.

[0436] When multiple images are acquired, the processor (1001) can acquire multiple target regions corresponding to each of the multiple images and acquire multiple feature vectors corresponding to the multiple target regions.

[0437] And, the processor (1001) can obtain multiple center nodes corresponding to multiple feature vectors and obtain neighbor nodes associated with each of the multiple center nodes.

[0438] Additionally, the processor (1001) can obtain a subgraph based on a plurality of central nodes and neighbor nodes. Specifically, the processor (1001) identifies one of a plurality of feature vectors as the central node, obtains neighbor nodes of the central node based on feature vectors different from the feature vector corresponding to the central node, and obtains a subgraph based on the central node and neighbor nodes. Furthermore, the processor (1001) can obtain cosine distances between the feature vector corresponding to the central node and feature vectors different from the feature vector corresponding to the central node, respectively, and obtain neighbor nodes by screening neighbor nodes from feature vectors different from the feature vector corresponding to the central node based on the cosine distances.

[0439] Additionally, the processor (1001) can identify the connection probability between a plurality of central nodes of a subgraph and each of the neighbor nodes of the plurality of central nodes based on a graph convolution network. Specifically, the processor (1001) can obtain a node embedding of the subgraph according to at least one layer of the graph convolution network and obtain the connection probability between the central node of the subgraph and its neighbor nodes based on the node embedding. In one embodiment, the processor (1001) can obtain an input feature corresponding to the subgraph and obtain a knob embedding by using the output result obtained by inputting the input feature to at least one layer of the graph convolution network for feature extraction as the node embedding.

[0440] And, the processor (1001) can cluster target regions based on the identified connection probability. Specifically, the processor (1001) obtains a set of neighbor nodes of the same category according to the connection probability for at least one target region to be clustered, and for at least one neighbor node of the set of neighbor nodes of the same category, adjusts the connection probability between at least one neighbor node and the central node according to the connection probabilities of each neighbor node and each neighbor node of the set of neighbor nodes of the same category, and clusters the target regions based on the adjusted connection probability to determine whether at least one neighbor node is a neighbor node of the same category, and clusters each target region to be clustered according to the neighbor nodes of the same category corresponding to each target region to be clustered.

[0441] In one embodiment of the present disclosure, a processor (1001) can obtain a set of neighbor nodes by determining neighbor nodes for at least one target area to be clustered, wherein the connection probabilities with a central node are greater than or equal to a preset threshold, and forming the determined neighbor nodes as a set of neighbor nodes of the same category.

[0442] In one embodiment of the present disclosure, the processor (1001) may determine, for at least one neighbor node of a set of neighbor nodes of the same category, the average connection probability between each neighbor node and at least one neighbor node of the set of neighbor nodes of the same category as the connection probability between at least one neighbor node and a central node. In one embodiment of the present disclosure, the processor (1001) may determine the connection probabilities between central nodes corresponding to each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered, and may cluster each target area to be clustered based on the connection probabilities between each central node.

[0443] Additionally, the processor (1001) receives a keyword for image search entered by a user, identifies a category associated with the keyword based on clustering results, and can search for images that match the keyword among the images of the identified category.

[0444] Here, electronic devices include, but are not limited to, mobile terminals (e.g., mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals)) and fixed terminals such as digital TVs and desktop computers. The electronic devices shown in FIG. 48 are merely examples and should not be used to limit the functions and scope of use of the embodiments of the present disclosure.

[0445] FIG. 49 is a flowchart illustrating a method for controlling an electronic device according to the present disclosure. An electronic device (1000) according to the present disclosure may acquire a plurality of images for performing clustering (S4910). The plurality of images may include, for example, face photographs, but the present disclosure is not limited thereto.

[0446] When multiple images are acquired, the electronic device (1000) can acquire multiple target regions corresponding to each of the multiple images and acquire multiple feature vectors corresponding to the multiple target regions (S4920).

[0447] And, the electronic device (1000) can acquire multiple center nodes corresponding to multiple feature vectors and acquire neighbor nodes associated with each of the multiple center nodes (S4930).

[0448] And, the electronic device (1000) can obtain a subgraph based on a plurality of central nodes and neighbor nodes (S4940). Specifically, the electronic device (1000) identifies one of a plurality of feature vectors as the central node, obtains neighbor nodes of the central node based on feature vectors different from the feature vector corresponding to the central node, and obtains a subgraph based on the central node and neighbor nodes. In addition, the electronic device (1000) can obtain cosine distances between the feature vector corresponding to the central node and feature vectors different from the feature vector corresponding to the central node, respectively, and obtain neighbor nodes by screening neighbor nodes from feature vectors different from the feature vector corresponding to the central node based on the cosine distances.

[0449] And, the electronic device (1000) can identify the connection probability between a plurality of central nodes of a subgraph and each of the neighbor nodes of the plurality of central nodes based on a graph convolution network (S4950). Specifically, the electronic device (1000) can obtain a node embedding of the subgraph according to at least one layer of the graph convolution network and obtain the connection probability between the central node of the subgraph and its neighbor nodes based on the node embedding. In one embodiment, the electronic device (1000) can obtain a knob embedding by obtaining an input feature corresponding to the subgraph and using the output result obtained by inputting the input feature to at least one layer of the graph convolution network for feature extraction as the node embedding.

[0450] And, the electronic device (1000) can cluster target regions based on the identified connection probability (S4960). Specifically, the electronic device (1000) obtains a set of neighbor nodes of the same category according to the connection probability for at least one target region to be clustered, and for at least one neighbor node of the set of neighbor nodes of the same category, adjusts the connection probability between at least one neighbor node and the central node according to the connection probabilities of each neighbor node and at least one neighbor node of the set of neighbor nodes of the same category, and clusters the target regions based on the adjusted connection probability to determine whether at least one neighbor node is a neighbor node of the same category, and clusters each target region to be clustered according to the neighbor nodes of the same category corresponding to each target region to be clustered.

[0451] In one embodiment of the present disclosure, an electronic device (1000) can obtain a set of neighbor nodes by determining neighbor nodes for at least one target area to be clustered, wherein the connection probabilities with a central node are greater than or equal to a preset threshold, and forming the determined neighbor nodes as a set of neighbor nodes of the same category.

[0452] In one embodiment of the present disclosure, an electronic device (1000) may determine, for at least one neighbor node of a set of neighbor nodes of the same category, the average connection probability of each neighbor node with at least one neighbor node of the set of neighbor nodes of the same category as the connection probability of at least one neighbor node and a central node. In one embodiment of the present disclosure, the electronic device (1000) may determine the connection probabilities between central nodes corresponding to each target area to be clustered according to neighbor nodes of the same category corresponding to each target area to be clustered, and may cluster each target area to be clustered based on the connection probabilities between each central node.

[0453] Additionally, the electronic device (1000) receives a keyword for image search input by a user, identifies a category associated with the keyword based on clustering results, and can search for images that match the keyword among the images of the identified category.

[0454] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when executed on a computer, the computer can execute content corresponding to the method embodiment described above. Compared to the prior art, the image clustering method provided by an embodiment of the present disclosure can avoid deviation in the overall clustering result due to deviation in the clustering center and can effectively improve the accuracy of the clustering result.

[0455] In the flowchart of the drawings, various steps are shown sequentially as indicated by arrows, but it should be understood that these steps do not necessarily need to be executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited and may be performed in a different order. Additionally, at least some of the steps in the flowchart of the drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily performed simultaneously and may be performed at different times; they are not necessarily performed sequentially but may be performed in turn or alternately with at least some of the steps, sub-steps, or stages of other steps.

[0456] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of both. A computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof, but is not limited thereto. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any type of medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, device, or device. In the present disclosure, a computer-readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier wave, and may contain computer-readable program code. Such propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium. A computer-readable signal medium may transmit, propagate, or send a program for use by or in combination with an instruction execution system, device, or device.Program code contained in a computer-readable medium may be transmitted by any suitable medium including, but not limited to, wired, optical cable, RF (radio frequency), etc., or any suitable combination thereof.

[0457] The computer-readable medium described above may be included in the electronic device described above; or may exist independently without being assembled into the electronic device.

[0458] The computer-readable medium described above carries one or more programs, and when the one or more programs described above are executed by an electronic device, the electronic device executes the method shown in the embodiment described above.

[0459] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof. The programming languages ​​mentioned above include object-oriented programming languages ​​such as Java, Smalltalk, and C++, and also include ordinary sequential programming languages ​​such as the "C" language or similar programming languages. The program code may be executed wholly on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., using an Internet service provider that provides an Internet connection).

[0460] The flowcharts and block diagrams in the attached drawings illustrate possible implementation architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block of a flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code may contain one or more executable instructions for realizing a specific logical function. Additionally, it should be noted that in some alternative implementations, the functions marked in the blocks may also occur in an order different from the order marked in the drawings. For example, two consecutively marked blocks may actually be executed in essentially parallel, or sometimes in reverse order depending on the related functions. Additionally, it should be noted that each block of the block diagram and / or flowchart, and combinations of blocks of the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs a specific function or operation, or realized by a combination of dedicated hardware and computer instructions.

[0461] The modules included in the embodiments described in this disclosure may be implemented in software or hardware. Herein, the name of the module does not constitute a limitation on the module itself under specific circumstances. For example, the first clustering module may also be described as a "module for clustering images."

[0462] The foregoing description is merely an explanation of preferred embodiments and applied technical principles of the present disclosure. A person skilled in the art should understand that the scope of the disclosure included in the present disclosure is not limited to technical solutions formed by specific combinations of the technical features described above, and should also include other technical solutions formed by any combination of the technical features described above or their equivalent features without departing from the concept disclosed above. For example, the features described above and technical features having similar functions disclosed in the present disclosure (but not limited thereto) may be substituted for each other to form technical solutions. Explanation of the symbols

[0464] 1000: Electronic device 1001: Processor 1002: Bus 1003: Memory 1004: Transceiver

Claims

Claim 1 A method for controlling an electronic device comprises: acquiring a plurality of images for performing clustering; acquiring a plurality of target regions corresponding to each of the plurality of images and acquiring a plurality of feature vectors corresponding to the plurality of target regions; acquiring a plurality of center nodes corresponding to the plurality of feature vectors and acquiring neighbor nodes associated with each of the plurality of center nodes; acquiring a subgraph based on the plurality of center nodes and the neighbor nodes; identifying connection probabilities between a plurality of center nodes of the subgraph and neighbor nodes of each of the plurality of center nodes based on a graph convolution network; and clustering the plurality of target regions based on the identified connection probabilities. Claim 2 A method according to claim 1, wherein the step of obtaining the subgraph comprises: identifying one of the plurality of feature vectors as the center node and obtaining neighbor nodes of the center node based on feature vectors different from the feature vector corresponding to the center node; and configuring the subgraph according to the center node and the neighbor nodes. Claim 3 In claim 2, the step of acquiring neighbor nodes comprises: acquiring cosine distances between a feature vector corresponding to the center node and feature vectors different from the feature vector corresponding to the center node, respectively; and screening neighbor nodes from feature vectors different from the feature vector corresponding to the center node based on the cosine distances. Claim 4 The method according to claim 1, wherein the step of identifying the connection probability comprises: obtaining a node embedding of the subgraph according to at least one layer of the graph convolution network; and obtaining a connection probability between the central node of the subgraph and its neighbor nodes based on the node embedding. Claim 5 A method according to claim 4, wherein the step of obtaining the node embedding comprises: a step of obtaining an input feature corresponding to the subgraph; and a step of using the output result obtained by inputting the input feature to at least one layer of the graph convolution network for feature extraction as the node embedding. Claim 6 A method according to claim 1, wherein the step of clustering the plurality of target regions according to the identified connection probability comprises: for at least one target region to be clustered, obtaining a set of neighbor nodes of the same category according to the connection probability; for at least one neighbor node of the set of neighbor nodes of the same category, adjusting the connection probability between the at least one neighbor node and the central node according to the connection probabilities of each neighbor node of the set of neighbor nodes of the same category; clustering the plurality of target regions based on the adjusted connection probability to determine whether the at least one neighbor node is a neighbor node of the same category; determining the connection probabilities between central nodes corresponding to each target region to be clustered according to neighbor nodes of the same category corresponding to each target region to be clustered; and clustering each target region to be clustered based on the connection probabilities between each central node. Claim 7 In claim 6, the step of obtaining the set of neighbor nodes comprises determining neighbor nodes for the at least one target area to be clustered, wherein the connection probabilities with the central node are greater than or equal to a preset threshold, and forming the determined neighbor nodes as a set of neighbor nodes of the same category. Claim 8 In claim 6, the step of adjusting the connection probability comprises, for at least one neighbor node of a set of neighbor nodes of the same category, determining the average connection probability of each neighbor node with the at least one neighbor node of the set of neighbor nodes of the same category as the connection probability of the at least one neighbor node and the center node. Claim 9 delete Claim 10 A method according to claim 1, further comprising: receiving a keyword for image search input by a user; identifying a category associated with the keyword based on the clustering result; and searching for images that match the keyword among the images of the identified category. Claim 11 An electronic device comprising: a memory for storing at least one instruction; and a processor connected to the memory for controlling the electronic device; wherein the processor, by executing the at least one instruction, acquires a plurality of images for performing clustering, acquires a plurality of target regions corresponding to each of the plurality of images, acquires a plurality of feature vectors corresponding to the plurality of target regions, acquires a plurality of center nodes corresponding to the plurality of feature vectors, acquires neighbor nodes associated with each of the plurality of center nodes, acquires a subgraph based on the plurality of center nodes and the neighbor nodes, identifies connection probabilities between a plurality of center nodes of the subgraph and neighbor nodes of each of the plurality of center nodes based on a graph convolution network, and clusters the plurality of target regions based on the identified connection probabilities. Claim 12 An electronic device according to claim 11, wherein the processor identifies one of the plurality of feature vectors as the center node, obtains neighbor nodes of the center node based on feature vectors different from the feature vector corresponding to the center node, and constructs the subgraph according to the center node and the neighbor nodes. Claim 13 An electronic device according to claim 12, wherein the processor obtains cosine distances between a feature vector corresponding to the center node and feature vectors different from the feature vector corresponding to the center node, respectively, and screens neighbor nodes from feature vectors different from the feature vector corresponding to the center node based on the cosine distances. Claim 14 An electronic device according to claim 11, wherein the processor obtains a node embedding of the subgraph according to at least one layer of the graph convolution network and obtains a connection probability between the central node of the subgraph and its neighbor nodes based on the node embedding. Claim 15 An electronic device according to claim 14, wherein the processor acquires input features corresponding to the subgraph and uses the output result obtained by inputting the input features to at least one layer of the graph convolution network for feature extraction as the node embedding. Claim 16 An electronic device according to claim 11, wherein the processor obtains a set of neighbor nodes of the same category according to the connection probability for at least one target area to be clustered, adjusts the connection probability between the at least one neighbor node and the central node according to the connection probabilities of each neighbor node with the at least one neighbor node of the set of neighbor nodes of the same category for at least one neighbor node of the set of neighbor nodes of the same category, clusters the target area based on the adjusted connection probability to determine whether the at least one neighbor node is a neighbor node of the same category, determines the connection probabilities between central nodes corresponding to each target area to be clustered according to the neighbor nodes of the same category corresponding to each target area to be clustered, and clusters each target area to be clustered based on the connection probabilities between each central node. Claim 17 An electronic device according to claim 16, wherein the processor determines neighbor nodes for the at least one target region to be clustered, wherein the connection probabilities with the central node are greater than or equal to a preset threshold, and forms the determined neighbor nodes as a set of neighbor nodes of the same category. Claim 18 In claim 16, the processor is an electronic device that determines, for at least one neighbor node of a set of neighbor nodes of the same category, the average connection probability of each neighbor node with the at least one neighbor node of the set of neighbor nodes of the same category as the connection probability of the at least one neighbor node and the center node. Claim 19 delete Claim 20 In claim 11, the processor is an electronic device that receives a keyword for image search input by a user, identifies a category associated with the keyword based on the clustering result, and searches for images that match the keyword among the images of the identified category.

Images