Image clustering method, device and computer storage medium

Abstract

The embodiment of the present application provides a kind of image clustering method, device and computer storage medium.The method comprises: obtaining the image set to be clustered;Determine the augmented inter-image relationship feature corresponding to any two images in the image set to be clustered;Determine the image similarity between the two images corresponding to the augmented inter-image relationship feature based on the augmented inter-image relationship feature;According to all image similarities, the image included in the image set to be clustered is clustered, and the clustering result corresponding to the image set to be clustered is obtained.The technical scheme provided by the embodiment, by determining the augmented inter-image relationship feature of any two images, since the augmented inter-image relationship feature is related to the image feature and context feature of any two images, when determining the image similarity between the two images based on the augmented inter-image relationship feature, the accuracy and reliability of determining the image similarity are effectively improved, and a more accurate clustering result can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image clustering method and device and computer storage medium. BACKGROUND

[0002] With the rapid development of image application technology, people can use image acquisition equipment to record portrait scenes anytime and anywhere, and high-performance automatically generated face albums have become a must-have function to meet the public demand. With the rapid development of deep neural networks, face recognition technology has reached a high level.

[0003] In the process of clustering face images, due to the complexity of the distribution of face images, the image features extracted from the face images have high uncertainty. Therefore, when clustering is performed based on the above-mentioned extracted image features, the images of the same person are easily divided into multiple clusters, or the images of different persons are clustered into one cluster, thereby reducing the accuracy of the clustering result. SUMMARY

[0004] The embodiments of the present application provide an image clustering method, device and computer storage medium, which are used to solve the problem that the accuracy of the clustering result is reduced due to the high uncertainty of the image features extracted from the face images in the prior art.

[0005] In a first aspect, the embodiments of the present application provide an image clustering method, comprising:

[0006] obtaining a set of images to be clustered;

[0007] determining an augmented inter-image relationship feature corresponding to any two images in the set of images to be clustered;

[0008] determining an image similarity between the two images corresponding to the augmented inter-image relationship feature based on the augmented inter-image relationship feature;

[0009] performing clustering processing on the images included in the set of images to be clustered according to all the image similarities, and obtaining a clustering result corresponding to the set of images to be clustered.

[0010] In a second aspect, the embodiments of the present application provide an image clustering device, comprising:

[0011] a first obtaining module configured to obtain a set of images to be clustered;

[0012] The first determining module is configured to determine an augmented inter-image relationship feature corresponding to any two images in the image set to be clustered, the augmented inter-image relationship feature being related to at least one of the following: image features corresponding to the images in any two images, context features corresponding to the images in any two images.

[0013] The first determining module is further configured to determine, based on the augmented inter-image relationship feature, an image similarity between the two images corresponding to the augmented inter-image relationship feature.

[0014] The first processing module is configured to perform clustering processing on the images included in the image set to be clustered according to all the image similarities, to obtain a clustering result corresponding to the image set to be clustered.

[0015] In a third aspect, an electronic device is provided, including a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions, when executed by the processor, implement the image clustering method in the first aspect.

[0016] In a fourth aspect, a computer storage medium is provided, configured to store a computer program, and the computer program causes a computer to implement the image clustering method in the first aspect when executed.

[0017] In a fifth aspect, an image clustering method is provided, applied to a data processing platform, and the data processing platform is configured to allow at least one user to perform data processing operations, and the method includes the following steps.

[0018] Obtaining a plurality of images to be clustered uploaded by at least one user to the data processing platform;

[0019] Determining an image similarity between any two images in the plurality of images to be clustered, the image similarity being determined by an augmented inter-image relationship feature between the two images;

[0020] Performing clustering processing on the plurality of images to be clustered according to all the image similarities;

[0021] Displaying a clustering result corresponding to the plurality of images to be clustered.

[0022] In a sixth aspect, an image clustering device is provided, applied to a data processing platform, and the data processing platform is configured to allow at least one user to perform data processing operations, and the device includes the following modules.

[0023] A second obtaining module is configured to obtain a plurality of images to be clustered uploaded by at least one user to the data processing platform;

[0024] a second determining module, configured to determine image similarity between any two images in the plurality of images to be clustered, the image similarity being determined by the augmented inter-image relationship feature between the any two images;

[0025] a second processing module, configured to perform clustering processing on the plurality of images to be clustered according to all the image similarity;

[0026] a second display module, configured to display a clustering result corresponding to the plurality of images to be clustered.

[0027] In a seventh aspect, an embodiment of the present application provides an electronic device, including a memory and a processor; the memory is used to store one or more computer instructions; when the one or more computer instructions are executed by the processor, the image clustering method in the fifth aspect is implemented.

[0028] In an eighth aspect, an embodiment of the present application provides a computer storage medium, used to store a computer program; when the computer program is executed by a computer, the image clustering method in the fifth aspect is implemented.

[0029] In a ninth aspect, an embodiment of the present application provides an image clustering method, applied to a data communication device, the data communication device being used for data communication of at least one user, and the method includes the following steps:

[0030] acquiring a plurality of images to be clustered transmitted by the at least one user through the data communication device;

[0031] determining image similarity between any two images in the plurality of images to be clustered, the image similarity being determined by an augmented inter-image relationship feature between the any two images;

[0032] performing clustering processing on the plurality of images to be clustered according to all the image similarity;

[0033] displaying a clustering result corresponding to the plurality of images to be clustered.

[0034] In a tenth aspect, an embodiment of the present application provides an image clustering device, applied to a data communication device, the data communication device being used for data communication of at least one user, and the device includes the following steps:

[0035] a third acquiring module, configured to acquire a plurality of images to be clustered transmitted by the at least one user through the data communication device;

[0036] a third determining module, configured to determine image similarity between any two images in the plurality of images to be clustered, the image similarity being determined by an augmented inter-image relationship feature between the any two images;

[0037] The third processing module is configured to perform clustering processing on the plurality of images to be clustered according to all the image similarities.

[0038] The third display module is configured to display a clustering result corresponding to the plurality of images to be clustered.

[0039] In a eleventh aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are configured to implement the image clustering method in the ninth aspect when executed by the processor.

[0040] In a twelfth aspect, an embodiment of the present application provides a computer storage medium configured to store a computer program, and the computer program is configured to implement the image clustering method in the ninth aspect when executed by a computer.

[0041] The technical scheme provided by the embodiment includes the following steps: obtaining a set of images to be clustered, and determining an augmented inter-image relationship feature corresponding to any two images in the set of images to be clustered. Since the augmented inter-image relationship feature is related to at least one of the following: image features corresponding to images in any two images, and context features corresponding to images in any two images, when the image similarity between the two images corresponding to the augmented inter-image relationship feature is determined based on the augmented inter-image relationship feature, the accuracy and reliability of determining the image similarity between the two images can be effectively improved. Therefore, when the images included in the set of images to be clustered are clustered based on all the image similarities, a clustering result with higher accuracy can be obtained, the accuracy and reliability of using the image clustering method are further improved, the practicability of the method is ensured, and the market promotion and application are facilitated. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0043] Figure 1 A flowchart of an image clustering method provided by an embodiment of the present application;

[0044] Figure 2 An application scenario diagram of an image clustering method provided by an embodiment of the present application;

[0045] Figure 3A flowchart for determining augmented inter-image relationship features corresponding to any two images in the image set to be clustered is provided for an embodiment of the present application.

[0046] Figure 4 A flowchart for determining proximity features corresponding to the image features in the image set to be clustered is provided for an embodiment of the present application.

[0047] Figure 5 A flowchart for augmenting the image features and proximity features using a first machine learning model to obtain augmented image features corresponding to each image in the image set to be clustered is provided for an embodiment of the present application.

[0048] Figure 5a A schematic diagram for obtaining augmented image features corresponding to each image in the image set to be clustered is provided for an embodiment of the present application.

[0049] Figure 6 A flowchart for determining augmented inter-image relationship features between any two images in the image set to be clustered based on the augmented image features corresponding to each image in the image set to be clustered is provided for an embodiment of the present application.

[0050] Figure 7 A flowchart for another image clustering method is provided for an embodiment of the present application.

[0051] Figure 8 A flowchart for determining augmented inter-image relationship features corresponding to any two images in the image set to be clustered is provided for another embodiment of the present application.

[0052] Figure 9 A schematic diagram of an image clustering method is provided for an application embodiment of the present application.

[0053] Figure 10 A schematic diagram for obtaining augmented image features corresponding to each image in the image set to be clustered is provided for an application embodiment of the present application.

[0054] Figure 11 A schematic diagram for obtaining augmented inter-image relationship features between the image and the proximity image in the image set to be clustered is provided for an application embodiment of the present application.

[0055] Figure 12 A comparative schematic diagram of the effects of an image clustering method is provided for an application embodiment of the present application.

[0056] Figure 13 A structural schematic diagram of an image clustering device is provided for an embodiment of the present application.

[0057] Figure 14 A structural schematic diagram of an image clustering device is provided for an embodiment of the present application.Figure 13 A structure diagram of an electronic device corresponding to the image clustering apparatus provided by the embodiment shown;

[0058] Figure 15 A flow diagram of another image clustering method provided by the embodiment of the present application;

[0059] Figure 16 A structure diagram of another image clustering apparatus provided by the embodiment of the present application;

[0060] Figure 17 A structure diagram of an electronic device corresponding to the image clustering apparatus provided by the embodiment shown; Figure 16 A structure diagram of an electronic device corresponding to the image clustering apparatus provided by the embodiment shown;

[0061] Figure 18 A flow diagram of another image clustering method provided by the embodiment of the present application;

[0062] Figure 19 A structure diagram of another image clustering apparatus provided by the embodiment of the present application;

[0063] Figure 20 A structure diagram of an electronic device corresponding to the image clustering apparatus provided by the embodiment shown. Figure 19 A structure diagram of an electronic device corresponding to the image clustering apparatus provided by the embodiment shown. DETAILED DESCRIPTION

[0064] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0065] The terms used in the embodiments of the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two, but does not exclude the case of including at least one.

[0066] It should be understood that the term "and / or" used herein is merely to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0067] Depending on the context, the word "if" as used herein can be interpreted to mean "upon determining," or "in response to a determination" or "when it is determined" or "upon the occurrence of." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "upon determining" or "in response to a determination" or "when [a stated condition or event] is detected" or "upon the occurrence of [a stated condition or event]."

[0068] It should also be noted that the terms "comprising," "including," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0069] In addition, the sequence of steps in the following method embodiments is only an example, not a strict limitation.

[0070] In order to facilitate the understanding of the technical solutions of the present application, the prior art is briefly described as follows:

[0071] With the rapid development of image technology, people increasingly use images to record portraits anytime and anywhere, and high-performance automatically generated face albums have become a must-have function that can cater to the needs of the public. With the rapid development of deep neural networks and metric learning, face recognition technology has reached a high level. In the process of clustering face images, due to the complexity of the distribution of face images, the face features extracted from face images will have uncertainty. At this time, when clustering is performed based on the above-mentioned extracted face features, the same person's image is easily divided into multiple clusters, the clustering result will have a split file, the same person's picture will easily get multiple clusters, or different people's images will be clustered into a cluster, thereby reducing the accuracy of the clustering result.

[0072] For example, the traditional method of clustering images mainly includes k-means clustering algorithm (KMeans clustering algorithm), density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, referred to as DBSCAN), hierarchical agglomerative clustering algorithm (Hierarchical Agglomerative Clustering, referred to as HAC), spectral clustering algorithm (Spectral Clustering, referred to as SC) and other algorithms.

[0073] However, the above clustering algorithm has poor clustering effect because it cannot mine the context information of the image, and in addition, a supervised clustering algorithm can also be used for image clustering operation. Specifically, the current supervised clustering method based on deep neural network can include the following categories: Linkage-Graph Convolutional Network (Linkage-GCN) containing incremental clustering, Learning to cluster faces on an affinity graph (DS-GCN) based on detection and semantic annotation framework, Learning to Cluster Faces via Confidence and Connectivity Estimation (VE-GCN) based on quality metric, Density-Aware Feature Embedding for Face Clustering (DA-NET) based on density-sensitive features, etc. When the above different clustering methods are used for image clustering operation, the accuracy of the obtained clustering result is not very high.

[0074] In order to solve the problem that the image features extracted from the face image have high uncertainty, thereby reducing the accuracy of the clustering result in the prior art, the embodiment provides an image clustering method, device and computer storage medium. The clustering method obtains a set of images to be clustered, and determines an augmented inter-image relationship feature corresponding to any two images in the set of images to be clustered. Since the augmented inter-image relationship feature is related to at least one of the following: image features corresponding to images in any two images, context features corresponding to images in any two images, when determining the image similarity between the two images corresponding to the augmented inter-image relationship feature based on the augmented inter-image relationship feature, the accuracy and reliability of determining the image similarity between the two images can be effectively improved, thereby obtaining a clustering result with higher accuracy when clustering the images included in the set of images to be clustered according to all image similarities, further improving the accuracy and reliability of using the image clustering method, ensuring the practicality of the method, and facilitating market promotion and application.

[0075] The various specific implementation modes and implementation effects of the image clustering method in the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features in the embodiments can be combined with each other without conflict between the embodiments.

[0076] Figure 1 A flowchart of an image clustering method provided by the embodiment of the present application is shown in the accompanying drawings.Figure 1 As shown, the embodiment provides an image clustering method, an execution subject of the method can be an image clustering device, and it can be understood that the image clustering device can be implemented as software or a combination of software and hardware. Specifically, the image clustering method can include:

[0077] Step S101: Obtain a set of images to be clustered.

[0078] Step S102: Determine an augmented inter-image relationship feature corresponding to any two images in the set of images to be clustered.

[0079] Step S103: Determine an image similarity between the two images corresponding to the augmented inter-image relationship feature based on the augmented inter-image relationship feature.

[0080] Step S104: Cluster the images included in the set of images to be clustered according to all the image similarities, and obtain a clustering result corresponding to the set of images to be clustered.

[0081] The following will be described in detail with respect to each of the above steps:

[0082] Step S101: Obtain a set of images to be clustered.

[0083] The set of images to be clustered includes a plurality of images that need to be clustered, and it can be understood that the plurality of images can vary depending on the specific application scenario, for example: the set of images to be clustered can include biological face images to be clustered, product images to be clustered, landscape images to be clustered, etc., wherein the biological face images can be human face images, cat face images, dog face images, or other biological face images, etc.

[0084] In addition, the images in the set of images to be clustered can include at least one of the following: image information obtained by a shooting device, image information in video information, synthetic images, etc. Furthermore, the embodiment does not limit the specific implementation of obtaining the set of images to be clustered, and a person skilled in the art can set it according to the specific application requirements and design requirements, for example: the set of images to be clustered can be stored in a preset area, and the set of images to be clustered can be obtained by accessing the preset area. Or, the set of images to be clustered can be stored in a preset device, and the image clustering device is communicatively connected with the preset device, so that the image clustering device can actively or passively obtain the set of images to be clustered.

[0085] Of course, a person skilled in the art can also use other ways to obtain the set of images to be clustered, as long as the accuracy and reliability of obtaining the set of images to be clustered can be guaranteed, which will not be described here.

[0086] Step S102: determining an augmented inter-image relationship feature corresponding to any two images in the image set to be clustered.

[0087] After obtaining the image set to be clustered, any two images in the image set to be clustered can be analyzed and processed to obtain an augmented inter-image relationship feature corresponding to the two images. The augmented inter-image relationship feature refers to a feature obtained after augmenting or enhancing the inter-image relationship feature. The inter-image relationship feature can refer to the correlation feature of any two images in terms of color feature, texture feature, shape feature, or spatial relationship feature.

[0088] In some examples, the augmented inter-image relationship feature can be related to the image feature and / or context feature corresponding to the images in any two images. Preferably, the augmented inter-image relationship feature includes the correlation between the image feature and the context feature of any two images, so that the accuracy and reliability of clustering the images in the image set to be clustered can be improved based on the augmented inter-image relationship feature.

[0089] In addition, the specific determination method of the augmented inter-image relationship feature is not limited in this embodiment, and can be set according to specific application requirements and design requirements by those skilled in the art. For example, a machine learning model is pre-trained, which is used to analyze and process the image feature and the context feature of any two images to obtain the augmented inter-image relationship feature corresponding to the two images. Specifically, the image feature and the context feature corresponding to each of the two images can be obtained first, and then the image feature and the context feature corresponding to the images in the two images are input into the machine learning model, so that the augmented inter-image relationship feature corresponding to the two images can be obtained.

[0090] Of course, those skilled in the art can also determine the augmented inter-image relationship feature corresponding to any two images in the image set to be clustered in other ways, as long as the accuracy and reliability of obtaining the augmented inter-image relationship feature can be ensured, which will not be described here.

[0091] Step S103: determining the image similarity between the two images corresponding to the augmented inter-image relationship feature based on the augmented inter-image relationship feature.

[0092] After obtaining the relationship features between augmented images, these features can be analyzed and processed to determine the image similarity between two images corresponding to the augmented image relationship features. Specifically, determining the image similarity between two images corresponding to the augmented image relationship features based on them can include: using a classifier to analyze and process the augmented image relationship features to obtain the image similarity between the two images corresponding to the augmented image relationship features.

[0093] Specifically, a pre-trained classifier is used to determine the image similarity between two images. After obtaining the augmented image relationship features, these features can be input into the classifier to obtain the similarity between the two images corresponding to the augmented image relationship features. The pre-trained classifier can be constructed using a multilayer perceptron (MLP) network.

[0094] For example, the augmented inter-image relationship feature between two images is e. q，k When inputting the augmented image relationship features mentioned above into the classifier, the image similarity between two images corresponding to the augmented image relationship features can be obtained using the following formula: p q，k =ω(MLP(e) q，k ), where p q，k ω is the image similarity between two images corresponding to the relationship features between augmented images. ω() is a preset activation function. In this embodiment, the activation function can be implemented using a Parametric Rectified Linear Unit (PReLU). MLP() is a multilayer perceptron network.

[0095] Of course, those skilled in the art can also use other methods to determine the image similarity between two images corresponding to the relationship features between augmented images, as long as the accuracy and reliability of determining the image similarity between the two images can be guaranteed, which will not be elaborated here.

[0096] Step S104: Perform clustering processing on the images included in the image set to be clustered according to all image similarities to obtain the clustering results corresponding to the image set to be clustered.

[0097] Wherein, for any two images included in the image set to be clustered, the image similarity between the two images can be obtained based on the above method. For example, when the image set to be clustered includes image 1, image 2 and image 3, the image similarity a between image 1 and image 2, the image similarity b between image 2 and image 3, and the image similarity c between image 1 and image 3 can be obtained using the above method.

[0098] After obtaining all the image similarities corresponding to the image set to be clustered, the images included in the image set to be clustered can be clustered based on all the image similarities, so that the clustering result corresponding to the image set to be clustered can be obtained. In some examples, clustering the images included in the image set to be clustered based on all the image similarities to obtain the clustering result corresponding to the image set to be clustered can include: obtaining a similarity threshold for clustering the images included in the image set to be clustered; when the image similarity is greater than or equal to the similarity threshold, the two images corresponding to the image similarity are clustered to obtain the clustering cluster corresponding to the image set to be clustered.

[0099] Specifically, a similarity threshold for clustering the images included in the image set to be clustered is set in advance, wherein different similarity thresholds can correspond to clustering the images in different image sets to be clustered. The similarity threshold can be stored in a preset area, and after obtaining the image set to be clustered, the preset area can be accessed based on the image set to be clustered, so that the similarity threshold for clustering the images included in the image set to be clustered can be obtained.

[0100] After obtaining the similarity threshold, all the image similarities can be compared with the similarity threshold, and when the image similarity is greater than or equal to the similarity threshold, the two images corresponding to the image similarity can be clustered, so that the clustering cluster corresponding to the image set to be clustered can be obtained. Correspondingly, when the image similarity is less than the similarity threshold, the two images corresponding to the image similarity can be divided into different clustering clusters.

[0101] In one application scenario, referring to FIG. 1, Figure 2 As shown in the figure, the image clustering method can include the following steps: obtaining an image set to be clustered, which can include a plurality of images that need to be clustered. Taking any two images included in the image set to be clustered as an example, the image a and the image b, the image a and the image b can be analyzed to obtain the augmented inter-image relationship feature e q，1 Then, the classifier is used to analyze the augmented inter-image relationship to obtain the image similarity p q，1 between the image a and the image b, and pq，1 similarity threshold p th analysis and comparison, when p q，1 > p th When p > p, it indicates that the similarity between image a and image b is high, that is, image a and image b are the face images of the same user, and then image a and image b can be divided into a cluster, realizing the clustering operation of the face image a and the face image b.

[0102] The image clustering method provided by the embodiment comprises the following steps: obtaining a to-be-clustered image set; determining an augmented inter-image relationship feature corresponding to any two images in the to-be-clustered image set; determining an image similarity between the two images corresponding to the augmented inter-image relationship feature based on the augmented inter-image relationship feature; and clustering the images included in the to-be-clustered image set according to all the image similarities. Since the augmented inter-image relationship feature is related to at least one of the following: image features corresponding to the images in any two images, and context features corresponding to the images in any two images, the accuracy and reliability of determining the image similarity between the two images can be effectively improved when the image similarity between the two images corresponding to the augmented inter-image relationship feature is determined based on the augmented inter-image relationship feature. Therefore, when the images included in the to-be-clustered image set are clustered according to all the image similarities, a clustering result with higher accuracy can be obtained, the accuracy and reliability of using the image clustering method are further improved, the practicability of the method is ensured, and the market promotion and application are facilitated.

[0103] Figure 3 The flowchart for determining the augmented inter-image relationship feature corresponding to any two images in the to-be-clustered image set is provided for the embodiment of the application. On the basis of the above embodiment, with reference to FIG. 3, the specific determination manner of the augmented inter-image relationship feature is not limited in the embodiment, and a person skilled in the art can set it according to specific application requirements and design requirements. More preferably, the determination of the augmented inter-image relationship feature corresponding to any two images in the to-be-clustered image set in the embodiment can comprise the following steps. Figure 3

[0104] Step S301: acquiring an augmented image feature corresponding to each image in the to-be-clustered image set, wherein the augmented image feature comprises an image feature corresponding to the image and a context feature.

[0105] Step S302: determining an augmented inter-image relationship feature between any two images in the to-be-clustered image set based on the augmented image feature corresponding to each image in the to-be-clustered image set.

[0106] ​In order to ensure the accuracy and reliability of obtaining the augmented image relationship features, after obtaining the image set to be clustered, feature extraction can be performed on each image in the image set to be clustered to obtain image features and context features corresponding to the image, and then the image features and context features can be analyzed and processed to obtain augmented image features corresponding to each image. The augmented image features refer to the features obtained after the image features are augmented or enhanced, which can accurately represent the image.

[0107] In some examples, in order to improve the accuracy and reliability of obtaining the augmented image features corresponding to each image, obtaining the augmented image features corresponding to each image in the image set to be clustered in this embodiment can include: extracting image features corresponding to each image in the image set to be clustered; determining neighboring features corresponding to the image features in the image set to be clustered; and using a first machine learning model to augment the image features and the neighboring features to obtain the augmented image features corresponding to each image in the image set to be clustered, wherein the first machine learning model is trained to augment the image features.

[0108] Specifically, after obtaining the image set to be clustered, a pre-set feature extractor can be used to perform feature extraction on each image in the image set to be clustered to obtain image features corresponding to each image. The image features can include at least one of the following: color features, texture features, shape features, spatial relationship features, etc.

[0109] After obtaining the image features corresponding to each image in the image set to be clustered, all image features can be analyzed and processed to determine neighboring features corresponding to the image features in the image set to be clustered, wherein the image features and the neighboring features refer to feature information corresponding to neighboring images corresponding to the image, and the neighboring images refer to images associated with the image. For example, the neighboring images can be images obtained after using the nearest neighbor algorithm to analyze and process the images in the image set to be clustered. The number of obtained neighboring images can be one or more, and therefore the number of neighboring features corresponding to the image features can be one or more.

[0110] After the image feature and the adjacent feature are acquired, the first machine learning model can be used to augment the image feature and the adjacent feature, so that the augmented image feature corresponding to each image in the image set to be clustered can be obtained. The first machine learning model is trained in advance to augment the image feature. Specifically, the first machine learning model can be generated by learning and training a deep neural network. Of course, the skilled person in the art can also use other ways to obtain the augmented image feature, as long as the accuracy and reliability of obtaining the augmented image feature can be ensured, and details are not repeated here.

[0111] After the augmented image feature corresponding to each image is acquired, the augmented image feature corresponding to each image can be analyzed to determine the augmented image relationship feature between any two images in the image set to be clustered, which effectively improves the accuracy and reliability of obtaining the augmented image relationship feature, and further ensures the practicability of the method.

[0112] Figure 4 The flowchart for determining the adjacent feature corresponding to the image feature in the image set to be clustered is provided for the embodiment of the application. Based on the above embodiment, the adjacent feature corresponding to the image feature in the image set to be clustered is determined, and the adjacent feature corresponding to the image feature in the image set to be clustered is determined. Figure 4 In the process of obtaining the augmented image feature corresponding to each image in the image set to be clustered, in order to ensure the accuracy and reliability of determining the adjacent feature corresponding to the image feature, the adjacent feature corresponding to the image feature in the image set to be clustered in the embodiment can include:

[0113] Step S401: based on the image feature, the nearest neighbor search is performed in the image set to be clustered, and the adjacent feature set corresponding to the image feature is obtained, the adjacent feature set including at least one adjacent feature corresponding to the image feature.

[0114] Step S402: in the adjacent feature set, the target adjacent feature corresponding to the image feature is determined.

[0115] After the image feature corresponding to each image in the image set to be clustered is acquired, the nearest neighbor search can be performed in the image set to be clustered by using the adjacent search algorithm. The adjacent search algorithm can include at least one of the following: nearest neighbor search algorithm, approximate nearest neighbor search algorithm or other algorithms that can realize the nearest neighbor search, so that the adjacent feature set corresponding to the image feature can be obtained, the adjacent feature set including at least one adjacent feature corresponding to the image feature, each adjacent feature corresponding to an adjacent image.

[0116] Specifically, after obtaining the neighboring feature set, since the neighboring feature set can include at least one neighboring feature, the number of the neighboring features that can be applied in the process of obtaining the augmented image feature corresponding to each image in the image set to be clustered can be the same as or different from the number of the neighboring features included in the neighboring feature set. Therefore, in order to ensure the accuracy and reliability of determining the neighboring feature corresponding to the image feature in the process of obtaining the augmented image feature, after obtaining the neighboring feature set, the target neighboring feature corresponding to the image feature can be determined in the neighboring feature set.

[0117] In some examples, determining the target neighboring feature corresponding to the image feature in the neighboring feature set can include: obtaining parameter information for defining the number of target neighboring features; and determining the target neighboring feature corresponding to the image feature based on the neighboring feature set and the parameter information.

[0118] The parameter information for defining the number of target neighboring features is pre-configured, which can be adjusted according to different application scenarios or application requirements. In addition, the parameter information can be stored in a preset area, and the parameter information for defining the number of target neighboring features can be obtained by accessing the preset area. After obtaining the parameter information, the target neighboring feature corresponding to the image feature can be determined based on the neighboring feature set and the parameter information.

[0119] In some examples, determining the target neighboring feature corresponding to the image feature based on the neighboring feature set and the parameter information can include: obtaining the number of set features of the neighboring features included in the neighboring feature set, and then analyzing and comparing the number of set features with the parameter information, and determining the target neighboring feature corresponding to the image feature based on the analysis and comparison result.

[0120] It should be noted that in the process of image clustering for an image set to be clustered, the image set to be clustered can be searched for one or more times, and the obtained neighboring feature set can correspond to a larger range of neighboring feature search, so that a neighboring feature set including a larger number of neighboring features can be obtained. In some examples, after obtaining the above neighboring feature set, in the process of image clustering operation, the neighboring features included in the neighboring feature set can be directly utilized or reused without further or multiple times of neighboring feature search operation.

[0121] For example, the n neighboring features are included in the historical neighboring feature set, the parameter information for limiting the number of target neighboring features is m1, and n < m1, then the neighboring search can be performed based on the parameter information m1, the m1 neighboring features can be included in the neighboring feature set, and then the m1 neighboring features included in the neighboring feature set can be determined as the target neighboring features.

[0122] If the parameter information for limiting the number of target neighboring features is m2, and m2 < n, that is, the number of neighboring features included in the neighboring feature set is greater than the number of neighboring features required in the process of obtaining the augmented image feature, then m2 neighboring features can be extracted from the neighboring feature set, and then the m2 neighboring features can be determined as the target neighboring features corresponding to the image feature, thereby effectively ensuring the accuracy and reliability of determining the target neighboring features.

[0123] In this embodiment, the neighboring feature set corresponding to the image feature is obtained by performing the nearest neighbor search based on the image feature in the image set to be clustered, and then the target neighboring features corresponding to the image feature are determined in the neighboring feature set, thereby effectively ensuring the quality and efficiency of determining the target neighboring features, further improving the accuracy and reliability of obtaining the augmented image feature corresponding to each image based on the target neighboring features, and being beneficial to improving the accuracy of image clustering processing.

[0124] Figure 5 The flowchart for obtaining the augmented image feature corresponding to each image in the image set to be clustered by using the first machine learning model to augment the image feature and the neighboring feature is provided in the embodiment of the application. Based on the above embodiment, with reference to the accompanying drawings, the specific implementation mode of using the first machine learning model to augment the image feature and the neighboring feature is not limited in the embodiment, and more preferably, the using the first machine learning model to augment the image feature and the neighboring feature in the embodiment to obtain the augmented image feature corresponding to each image in the image set to be clustered can include: Figure 5

[0125] Step S501: processing the neighboring features by using the self-attention network layer and the common attention layer included in the first machine learning model to obtain the context features corresponding to the image features.

[0126] Step S502: obtaining the augmented image features corresponding to each image in the image set to be clustered based on the image features and the context features.

[0127] ​In the established first machine learning model, the self-attention network layer and the common attention layer can be included. After obtaining the neighboring feature, the self-attention network layer and the common attention layer can be used to analyze and process the neighboring feature to obtain the context feature corresponding to the image feature. Specifically, when the self-attention network layer and the common attention layer are used to analyze and process the neighboring feature, the following formula can be used to implement:

[0128]

[0129] wherein LN is a graph layer normalization algorithm, MH is a multi-head attention mechanism algorithm, is the feature output after the l-1 layer self-attention network layer analyzes and processes the neighboring feature f q，k , is the feature obtained after the l layer self-attention network layer analyzes and processes the feature output by the l-1 layer self-attention network layer, K l-1 is the key feature corresponding to the multi-head attention mechanism algorithm, and V l-1 is the value feature corresponding to the multi-head attention mechanism algorithm.

[0130] In short, after the multi-layer self-attention network layer is used to analyze and process the neighboring feature, the context feature output by the last layer network layer can be obtained, which can be implemented by the following formula: wherein is the context feature corresponding to the image feature, MH is the multi-head attention mechanism algorithm, f ，k is the neighboring feature, is the key feature corresponding to the multi-head attention mechanism algorithm, is the value feature corresponding to the multi-head attention mechanism algorithm, thereby effectively ensuring the accuracy and reliability of obtaining the context feature corresponding to the image feature.

[0131] After obtaining the context feature corresponding to the image feature, the image feature and the context feature can be analyzed and processed to obtain the augmented image feature corresponding to each image in the image cluster to be clustered. In some examples, based on the image feature and the context feature, the augmented image feature corresponding to each image in the image cluster to be clustered can include: fusing the image feature and the context feature to obtain a fused image feature; and normalizing the fused image feature to obtain the augmented image feature corresponding to each image in the image cluster to be clustered.

[0132] After obtaining the image features and context features, the image features and context features can be fused. Specifically, operations such as overlay processing, splicing processing, or weighted summation processing can be performed on the image features and context features to obtain the fused image features. After obtaining the fused image features, the fused image features can be normalized to obtain the augmented image features corresponding to each image in each image set to be clustered.

[0133] For example, see attached document. Figure 5a As shown, the image set to be clustered includes images 1, 2, 3, 4, and 5. In this set, a nearest neighbor search is performed on image 1 to obtain its neighboring images. These neighboring images are assumed to include images 1, 2, and 3. Then, the image features f corresponding to images 1, 2, 3, and 4 can be obtained respectively. q Image features f q，1 Image features f q，2 and image features f q，3 Then, the first machine learning model is used to analyze and process the above image features to obtain the context image features corresponding to image 1 to be clustered.

[0134] Next, the fusion process will be illustrated using a weighted summation of image features and contextual features as an example. This is done after obtaining the image features f. q and context features Then, the image features f can be analyzed. q and context features By performing fusion processing, the features of the fused image can be obtained. Then, the augmented image feature g corresponding to the image can be obtained using the following formula. q , Wherein, LN is the layer normalization algorithm.

[0135] In this embodiment, by using the self-attention network layer and the common attention layer included in the first machine learning model to process the neighboring features, contextual features corresponding to the image features are obtained. Then, based on the image features and contextual features, augmented image features corresponding to each image in the image set to be clustered are obtained, thereby effectively ensuring the accuracy and reliability of the acquisition of augmented image features.

[0136] Figure 6This is a flowchart illustrating the process of determining the augmented image relationship features between any two images in a set of images to be clustered, based on augmented image features corresponding to each image in the set. Based on the above embodiments, refer to the appendix... Figure 6 As shown, this embodiment does not limit the specific implementation method for determining the augmented image relationship features between any two images in the image set to be clustered. Those skilled in the art can set it according to specific application and design requirements. Preferably, in this embodiment, determining the augmented image relationship features between any two images in the image set to be clustered based on the augmented image features corresponding to each image in the image set to be clustered may include:

[0137] Step S601: Obtain the first augmented image features corresponding to each image in the image set to be clustered, and the second augmented image features corresponding to the neighboring images of the image.

[0138] Step S602: Use the second machine learning model to augment the first augmented image features and the second augmented image features to obtain augmented image relationship features between the images in the image set to be clustered and the neighboring images. The second machine learning model is trained to augment the image relationship features between two images based on the image features.

[0139] For each image in the image set to be clustered, the image set may include neighboring images corresponding to the image. In order to perform accurate clustering processing on each image in the image set to be clustered, the method described in the above embodiments can be used to obtain the first augmented image features corresponding to each image and the second augmented image features of the neighboring images corresponding to that image.

[0140] After obtaining the first augmented image features corresponding to each image and the second augmented image features of the neighboring images corresponding to the image, the first and second augmented image features can be augmented using a second machine learning model. This allows the augmented image relationship features between the images in the image set to be clustered and their neighboring images to be obtained. The second machine learning model is trained to augment the image relationship features between two images based on image features. Specifically, the second machine learning model can be generated by learning and training a deep neural network, thereby effectively ensuring the quality and efficiency of obtaining the augmented image relationship features.

[0141] In some examples, the obtaining, by using the second machine learning model, the augmented image features of the first augmented image features and the second augmented image features can include: processing, by using a self-attention network layer in the second machine learning model, the second augmented image features to obtain augmented neighboring image features corresponding to the neighboring image; and obtaining, based on the augmented neighboring image features and the first augmented image features, the augmented inter-image relationship features between the image in the image set to be clustered and the neighboring image.

[0142] Specifically, the first augmented image features corresponding to each image in the image set to be clustered can be denoted as g q , the second augmented image features of the neighboring image can be denoted as g q，1 , g q，2 , g q，3 , and g q，k , and the like. After obtaining the second augmented image features, the self-attention network layer in the second machine learning model can be used to process the second augmented image features. Specifically, the following formula can be used to implement the processing: wherein, is the second augmented image features obtained after the l-th layer network processing, LN is a layer normalization algorithm, MH is a multi-head attention mechanism algorithm, K is a key feature corresponding to the multi-head attention mechanism algorithm, V is a value feature corresponding to the multi-head attention mechanism algorithm, is the second augmented image features obtained after the (l-1)-th layer network processing, so that the augmented neighboring image features corresponding to the neighboring image can be accurately obtained.

[0143] After obtaining the augmented neighboring image features and the first augmented image features, the augmented neighboring image features and the first augmented image features can be analyzed and processed to obtain the augmented inter-image relationship features between the image in the image set to be clustered and the neighboring image. Specifically, based on the augmented neighboring image features and the first augmented image features, the augmented inter-image relationship features between the image in the image set to be clustered and the neighboring image can be obtained by: performing relationship extraction processing on the augmented neighboring image features and the first augmented image features to obtain the augmented inter-image relationship features between the image in the image set to be clustered and the neighboring image.

[0144] wherein, the implementation manner of the relationship extraction processing can include splicing, difference, dot product, or weighted summation processing of the augmented neighboring image features and the first augmented image features, and the like. Of course, those skilled in the art can also use other manners to implement the relationship extraction operation, as long as the accuracy and reliability of the relationship extraction operation on the augmented neighboring image features and the first augmented image features can be ensured, so as to facilitate ensuring the accuracy and reliability of the obtaining of the augmented inter-image relationship features.

[0145] Specifically, after obtaining the augmented neighbor image features and the first augmented image features, these features can be analyzed and processed. In some instances, the following formula can be used to perform the processing: Among them, e q，k For augmented image-to-image relationship features between an image and its neighboring images, `cat` is a function used to join arrays, and `g`... k For the first augmented image feature, To augment neighbor image features, this method effectively achieves stable acquisition of augmented image relationship features between images and neighboring images in the image set to be clustered by concatenating the augmented neighbor image features and the first augmented image features.

[0146] Figure 7 This is a flowchart illustrating another image clustering method provided in an embodiment of the present invention; based on the above embodiments, refer to the appendix. Figure 7 As shown, after obtaining the clusters corresponding to the set of images to be clustered, in one application scenario, the image clustering device can also receive newly added images that need to be clustered. In this case, to improve the quality and efficiency of clustering the newly added images, the association between the newly added images and the clusters can be identified to determine whether the newly added images can be classified into the clusters corresponding to the images to be clustered. Specifically, the method in this embodiment may further include:

[0147] Step S701: Obtain the image to be processed and the first image features corresponding to the image to be processed.

[0148] In this process, when the image clustering device acquires an image to be processed transmitted from another device, or an image to be processed generated in a specific application scenario, it can analyze and process the image to obtain a first image feature corresponding to the image to be processed. This first image feature may include color features, texture features, shape features, or spatial relationship features, etc., used to characterize the image to be processed. It should be noted that there can be one or more images to be processed, and when there are multiple images to be processed, there are also multiple first image features.

[0149] Step S702: Determine the augmented class features corresponding to the clusters. The augmented class features include the class features corresponding to the clusters and contextual information.

[0150] In order to improve the quality and efficiency of clustering processing on the to-be-processed image, after obtaining the clustering cluster corresponding to the to-be-clustered image set, a feature extraction operation can be performed on the clustering cluster to determine an augmented class feature corresponding to the clustering cluster, the augmented class feature including a class feature corresponding to the to-be-clustered cluster and context information.

[0151] In some examples, determining the augmented class feature corresponding to the clustering cluster can include: obtaining the class feature corresponding to the clustering cluster and a neighboring class feature; and performing augmented processing on the class feature and the neighboring class feature by using a third machine learning model to obtain the augmented class feature corresponding to the clustering cluster, wherein the third machine learning model is trained to perform augmented processing on the class feature of the clustering cluster.

[0152] Specifically, after obtaining the clustering cluster, an analysis processing can be performed on the clustering cluster to obtain a class feature corresponding to the clustering cluster and a neighboring class feature. It can be understood that the neighboring class feature is a feature corresponding to a neighboring clustering cluster associated with the current clustering cluster. After obtaining the class feature corresponding to the clustering cluster and the neighboring class feature, the class feature and the neighboring class feature can be augmented by using a third machine learning model, that is, the augmented class feature corresponding to the clustering cluster can be obtained. Through the above-mentioned augmented class feature, the accuracy and reliability of expressing the clustering cluster can be effectively enhanced, thereby facilitating to improve the accuracy of clustering processing on the image.

[0153] Step S703: determining whether to cluster the to-be-processed image to the clustering cluster based on the first image feature and the augmented class feature.

[0154] After obtaining the first image feature and the augmented class feature, an analysis processing can be performed on the first image feature and the augmented class feature to determine whether the to-be-processed image can be clustered into the clustering cluster corresponding to the augmented class feature according to the analysis processing result. In some examples, determining whether to cluster the to-be-processed image to the clustering cluster based on the first image feature and the augmented class feature can include: obtaining a matching degree between the to-be-processed image and the clustering cluster based on the first image feature and the augmented class feature; when the matching degree is greater than or equal to a preset threshold, clustering the to-be-processed image to the clustering cluster; or when the matching degree is less than the preset threshold, prohibiting the to-be-processed image from being clustered to the clustering cluster.

[0155] Wherein, after obtaining the first image feature and the augmented class feature, the first image feature and the augmented class feature can be analyzed and processed to obtain the matching degree between the to-be-processed image and the clustering cluster. Specifically, in the analysis and processing of the first image feature and the augmented class feature, one implementation manner is that a classifier for obtaining the matching degree between the image feature and the class feature is pre-trained, and after obtaining the first image feature and the augmented class feature, the first image feature and the augmented class feature can be input into the classifier to obtain the matching degree between the to-be-processed image and the clustering cluster. Another implementation manner is that after obtaining the first image feature and the augmented class feature, the Euclidean distance, the cosine distance, etc. between the first image feature and the augmented class feature can be obtained, and then the matching degree between the to-be-processed image and the clustering cluster can be determined based on the obtained Euclidean distance, cosine distance, etc.

[0156] Of course, the skilled in the art can also use other ways to obtain the matching degree between the to-be-processed image and the clustering cluster, as long as the accuracy and reliability of obtaining the matching degree between the to-be-processed image and the clustering cluster can be ensured, which will not be described here.

[0157] After obtaining the matching degree between the to-be-processed image and the clustering cluster, the matching degree can be analyzed and compared with the preset threshold, and when the matching degree is greater than or equal to the preset threshold, it indicates that the similarity between the to-be-processed image and the image in the current clustering cluster is high, and then the to-be-processed image can be clustered to the clustering cluster; correspondingly, when the matching degree is less than the preset threshold, it indicates that the similarity between the to-be-processed image and the image in the current clustering cluster is low, and then the to-be-processed image is prohibited from being clustered to the clustering cluster.

[0158] In the embodiment, by obtaining the to-be-processed image and the first image feature corresponding to the to-be-processed image, the augmented class feature corresponding to the clustering cluster is determined, and then whether the to-be-processed image is clustered to the clustering cluster is judged based on the first image feature and the augmented class feature. Specifically, when the similarity between the to-be-processed image and the image in the clustering cluster is high, the to-be-processed image can be divided into the current clustering cluster, and when the similarity between the to-be-processed image and the image in the clustering cluster is low, the to-be-processed image is prohibited from being divided into the current clustering cluster, thereby not only realizing the clustering processing of the new image, but also effectively improving the quality and efficiency of the clustering processing operation of the new image, further improving the flexible reliability of the method.

[0159] Figure 8 The flowchart for determining the augmented image relationship feature corresponding to any two images in the to-be-clustered image set is provided for another embodiment of the application. Based on the above embodiment, reference is made to the accompanying Figure 8As shown, the embodiment provides another implementation manner of determining the augmented inter-image relationship feature corresponding to any two images in the image set to be clustered. Specifically, the determination of the augmented inter-image relationship feature corresponding to any two images in the image set to be clustered in the embodiment can include:

[0160] Step S801: Obtain the image feature corresponding to each image in the image set to be clustered.

[0161] Step S802: Determine the augmented inter-image relationship feature between any two images in the image set to be clustered based on the image feature corresponding to each image in the image set to be clustered.

[0162] After obtaining the image set to be clustered, a feature extraction operation can be performed on each image in the image set to be clustered, that is, the image feature corresponding to each image can be obtained. The image feature can include color feature, texture feature, shape feature, spatial relationship feature, etc. for representing the image.

[0163] After obtaining the image feature corresponding to each image in the image set to be clustered, the image feature can be analyzed and processed to determine the augmented inter-image relationship feature between any two images in the image set to be clustered. Specifically, based on the image feature corresponding to each image in the image set to be clustered, the augmented inter-image relationship feature between any two images in the image set to be clustered can be determined, which can include: determining the adjacent image corresponding to the image in the image set to be clustered; performing augmentation processing on the image feature and the adjacent feature corresponding to the adjacent image by using the first machine learning model to obtain the augmented image feature corresponding to each image in the image set to be clustered, wherein the first machine learning model is trained to perform augmentation processing on the image feature.

[0164] Specifically, the specific implementation manner and implementation effect of determining the adjacent image corresponding to the image and performing augmentation processing on the image feature and the adjacent feature corresponding to the adjacent image by using the first machine learning model in the embodiment are similar to those of the above Figure 3 The specific implementation manner and implementation effect of the corresponding embodiment are similar to those of the above

[0165] In a specific application, reference is made to the accompanying drawings Figure 9 As shown, taking the plurality of face images to be clustered in the image set to be clustered as an example, the application embodiment provides an image clustering method, which can include the following steps:

[0166] Step 1: Obtain the image set to be clustered.

[0167] The image set to be clustered includes images 1, 2, 3, 4, 5, and 6.

[0168] Step 2: Extract the image features corresponding to each image in the above image set to be clustered.

[0169] Specifically, a deep neural network is used to cluster all input images {X1,X2,...,X} in the image set to be clustered. n By performing feature extraction operations, we can obtain all image features {f1, f2, ..., f} corresponding to all input images. n}, where N represents the number of images in the image set to be clustered. For example, the image features f corresponding to image 1 can be obtained. q Image features f1 corresponding to image 2, image features f2 corresponding to image 3, and image features f3 corresponding to image 4, etc.

[0170] Step 3: Based on the image features f corresponding to image 1 q Nearest neighbor retrieval is performed on the set of images to be clustered to obtain a set of neighboring features corresponding to the image features.

[0171] The neighbor feature set may include at least one neighbor feature, which corresponds to the neighboring images corresponding to image 1. For example, the neighboring images corresponding to image 1 include image 2, image 3, and image 4. The neighbor feature set may include neighbor feature f. q，1 Neighborhood features f q，2 Neighborhood features f q，3 .

[0172] Step 4: Extract hop2 neighboring features from the neighboring feature set as target neighboring features corresponding to the image features.

[0173] Step 5: Process the target proximity features mentioned above using the self-attention network layer and the common attention layer included in the first machine learning model to obtain contextual features corresponding to the image features.

[0174] Specifically, the self-attention mechanism in the first machine learning model is used to focus on the target's neighboring features (f q，1 f q，2 f q，3 The image features are processed to obtain contextual features corresponding to the image features.

[0175] Step 6: Based on image features and contextual features Obtain augmented image features corresponding to each image in the image set to be clustered.

[0176] Specifically, referring to FIG. 6, the processing of the image features and the context features by using the first machine learning model to obtain the augmented image features corresponding to each image in the image set to be clustered can include the following steps: Figure 10

[0177] Step 6.1: Fuse the image features fqand the context features by using a neural network Transformer with a self-attention mechanism to obtain fused image features.

[0178] Step 6.2: Normalize the fused image features to obtain augmented image features corresponding to each image in the image set to be clustered.

[0179] Step 7: In the neighboring feature set, obtain hop1 neighboring features, and obtain hop1 nearest neighboring features of image 1: {f q，1 , f q，2 , f q，3 ..., f q，hop1}.

[0180] Step 8: Obtain the augmented image features corresponding to the above neighboring features, so as to obtain the augmented image features g_q(corresponding to image 1, hereinafter referred to as the first augmented image features) and g q，1 , g q，2 ,..., g q，hop1 (corresponding to the second neighboring image, hereinafter referred to as the second augmented image features).

[0181] Step 9: Perform augmentation processing on the first augmented image features and the second augmented image features by using the second machine learning model to obtain the augmented inter-image relationship features between image 1 and the neighboring images.

[0182] Specifically, referring to FIG. 6, the processing of the image features and the context features by using the first machine learning model to obtain the augmented image features corresponding to each image in the image set to be clustered can include the following steps: Figure 11

[0183] Step 9.1: Process the second augmented image features by using the self-attention network layer in the second machine learning model to obtain augmented neighboring image features corresponding to the neighboring images.

[0184] Step 9.2: Obtain the augmented inter-image relationship features between the images in the image set to be clustered and the neighboring images based on the augmented neighboring image features and the first augmented image features.

[0185] ​​​Specifically, the implementation method and process of the above steps in this embodiment are similar to the implementation method of obtaining the relationship features between augmented images in the above embodiments. For details, please refer to the above description, which will not be repeated here.

[0186] Step 10: Utilize a classifier to analyze the relationship features between augmented images (e... q，k The image is processed to obtain the image similarity (p) between two images corresponding to the relationship features between augmented images. q，k ).

[0187] Step 11: Cluster the images in the image set to be clustered based on all image similarities to obtain the clustering results corresponding to the image set to be clustered.

[0188] Image similarity is analyzed and compared with a preset threshold. At image similarity p... q，k When the similarity is greater than or equal to a preset threshold, the image similarity can be identified as p, which indicates a positive correlation between two images. q，1 The two images corresponding to the image similarity are grouped into the same cluster; when the image similarity is less than a preset threshold, the image similarity can be identified as p, which indicates that the two images satisfy a negative correlation. q，2 This involves assigning two images with similarity scores to different clusters.

[0189] For details, please refer to the appendix. Figure 12 As shown, when using the above image clustering method to cluster large-scale face images, compared with image clustering through the first or second machine learning model, more accurate clustering results can be obtained, which is beneficial to improving the image processing performance of the image clustering method.

[0190] The image clustering method provided in this application embodiment processes images based on a hierarchical machine learning model to obtain augmented image features. Then, based on the augmented image features, it obtains augmented image relationship features between any two images. By using the augmented image relationship features to characterize the association between images, it effectively increases the accuracy and reliability of image processing. Thus, when performing image clustering based on augmented image relationship features, it is beneficial to improve the accuracy and reliability of image clustering, further ensuring the accuracy and reliability of the method and facilitating its market promotion and application.

[0191] Figure 13 This is a schematic diagram of an image clustering device provided in an embodiment of the present invention; see attached diagram. Figure 13 As shown, this embodiment provides an image clustering device that can perform the above-described... Figure 1The corresponding image clustering method, the image clustering apparatus can include a first acquisition module 11, a first determination module 12 and a first processing module 13; specifically,

[0192] The first acquisition module 11 is configured to acquire a set of images to be clustered;

[0193] The first determination module 12 is configured to determine an augmented inter-image relationship feature corresponding to any two images in the set of images to be clustered, the augmented inter-image relationship feature being related to at least one of: an image feature corresponding to an image in the any two images, a context feature corresponding to the image in the any two images;

[0194] The first determination module 12 is further configured to determine, based on the augmented inter-image relationship feature, an image similarity between the two images corresponding to the augmented inter-image relationship feature;

[0195] The first processing module 13 is configured to perform clustering processing on the images included in the set of images to be clustered according to all the image similarities, and obtain a clustering result corresponding to the set of images to be clustered.

[0196] In some examples, the augmented inter-image relationship feature is related to at least one of: an image feature corresponding to an image in the any two images, a context feature corresponding to the image in the any two images.

[0197] In some examples, when the first determination module 12 determines the augmented inter-image relationship feature corresponding to any two images in the set of images to be clustered, the first determination module 12 is configured to perform: acquiring an augmented image feature corresponding to each image in the set of images to be clustered, the augmented image feature including: an image feature corresponding to the image and a context feature; determining the augmented inter-image relationship feature between any two images in the set of images to be clustered based on the augmented image feature corresponding to each image in the set of images to be clustered.

[0198] In some examples, when the first determination module 12 acquires the augmented image feature corresponding to each image in the set of images to be clustered, the first determination module 12 is configured to perform: extracting an image feature corresponding to each image in the set of images to be clustered; determining a neighboring feature corresponding to the image feature in the set of images to be clustered; performing augmented processing on the image feature and the neighboring feature using a first machine learning model to obtain the augmented image feature corresponding to each image in the set of images to be clustered, wherein the first machine learning model is trained to perform augmented processing on the image feature.

[0199] In some examples, when the first determining module 12 determines the neighboring features corresponding to the image feature in the image set to be clustered, the first determining module 12 is configured to perform: performing a nearest neighbor search based on the image feature in the image set to be clustered to obtain a set of neighboring features corresponding to the image feature, the set of neighboring features including at least one neighboring feature corresponding to the image feature; and determining a target neighboring feature corresponding to the image feature in the set of neighboring features.

[0200] In some examples, when the first determining module 12 determines the target neighboring feature corresponding to the image feature in the set of neighboring features, the first determining module 12 is configured to perform: obtaining parameter information for defining the number of target neighboring features; and determining the target neighboring feature corresponding to the image feature based on the set of neighboring features and the parameter information.

[0201] In some examples, when the first determining module 12 augments the image feature and the neighboring features using the first machine learning model to obtain augmented image features corresponding to each image in the image set to be clustered, the first determining module 12 is configured to perform: processing the neighboring features using a self-attention network layer and a co-attention layer included in the first machine learning model to obtain context features corresponding to the image feature; and obtaining the augmented image features corresponding to each image in the image set to be clustered based on the image feature and the context features.

[0202] In some examples, when the first determining module 12 obtains the augmented image features corresponding to each image in the image set to be clustered based on the image feature and the context features, the first determining module 12 is configured to perform: fusing the image feature and the context features to obtain fused image features; and performing normalization processing on the fused image features to obtain the augmented image features corresponding to each image in the image set to be clustered.

[0203] In some examples, when the first determining module 12 determines augmented inter-image relationship features between any two images in the image set to be clustered based on the augmented image features corresponding to each image in the image set to be clustered, the first determining module 12 is configured to perform: obtaining first augmented image features corresponding to each image in the image set to be clustered and second augmented image features corresponding to neighboring images of the images; and augmenting the first augmented image features and the second augmented image features using a second machine learning model to obtain augmented inter-image relationship features between the images and the neighboring images in the image set to be clustered, wherein the second machine learning model is trained to augment inter-image relationship features between two images based on image features.

[0204] In some examples, when the first determining module 12 obtains the augmented inter-image relationship features between the images in the image set to be clustered and the neighboring images by using the second machine learning model to augment the first augmented image features and the second augmented image features, the first determining module 12 is configured to perform the following: processing the second augmented image features by using a self-attention network layer in the second machine learning model to obtain augmented neighboring image features corresponding to the neighboring images; and obtaining the augmented inter-image relationship features between the images in the image set to be clustered and the neighboring images based on the augmented neighboring image features and the first augmented image features.

[0205] In some examples, when the first determining module 12 obtains the augmented inter-image relationship features between the images in the image set to be clustered and the neighboring images based on the augmented neighboring image features and the first augmented image features, the first determining module 12 is configured to perform the following: performing relationship extraction processing on the augmented neighboring image features and the first augmented image features to obtain the augmented inter-image relationship features between the images in the image set to be clustered and the neighboring images.

[0206] In some examples, when the first determining module 12 determines the image similarity between the two images corresponding to the augmented inter-image relationship features based on the augmented inter-image relationship features, the first determining module 12 is configured to perform the following: performing analysis processing on the augmented inter-image relationship features by using a classifier to obtain the image similarity between the two images corresponding to the augmented inter-image relationship features.

[0207] In some examples, when the first processing module 13 performs clustering processing on the images included in the image set to be clustered based on all the image similarities to obtain the clustering result corresponding to the image set to be clustered, the first processing module 13 is configured to perform the following: obtaining a similarity threshold used for clustering processing on the images in the image set to be clustered; and when the image similarity is greater than or equal to the similarity threshold, clustering the two images corresponding to the image similarity to obtain the clustering cluster corresponding to the image set to be clustered.

[0208] In some examples, after obtaining the clustering cluster corresponding to the image set to be clustered, the first obtaining module 11, the first determining module 12, and the first processing module 13 in the embodiment are configured to perform the following steps:

[0209] The first obtaining module 11 is configured to obtain the to-be-processed image and the first image features corresponding to the to-be-processed image.

[0210] The first determining module 12 is configured to determine the augmented class features corresponding to the clustering cluster, wherein the augmented class features include the class features corresponding to the clustering cluster and the context information.

[0211] The first processing module 13 is configured to determine whether to cluster the to-be-processed image into the clustering cluster based on the first image feature and the augmented class feature.

[0212] In some examples, when the first determining module 12 determines the augmented class feature corresponding to the clustering cluster, the first determining module 12 is configured to perform: obtaining a class feature corresponding to the clustering cluster and a neighboring class feature; and performing augmented processing on the class feature and the neighboring class feature by using a third machine learning model to obtain the augmented class feature corresponding to the clustering cluster, wherein the third machine learning model is trained to perform augmented processing on the class feature of the clustering cluster.

[0213] In some examples, when the first processing module 13 determines whether to cluster the to-be-processed image into the clustering cluster based on the first image feature and the augmented class feature, the first processing module 13 is configured to perform: obtaining a matching degree between the to-be-processed image and the clustering cluster based on the first image feature and the augmented class feature; and when the matching degree is greater than or equal to a preset threshold, clustering the to-be-processed image into the clustering cluster; or when the matching degree is less than the preset threshold, prohibiting the to-be-processed image from being clustered into the clustering cluster.

[0214] In some examples, when the first determining module 12 determines the augmented image-to-image relationship feature corresponding to any two images in the to-be-clustered image set, the first determining module 12 is configured to perform: obtaining an image feature corresponding to each image in the to-be-clustered image set; and determining the augmented image-to-image relationship feature between any two images in the to-be-clustered image set based on the image feature corresponding to each image in the to-be-clustered image set.

[0215] In some examples, when the first determining module 12 determines the augmented image-to-image relationship feature between any two images in the to-be-clustered image set based on the image feature corresponding to each image in the to-be-clustered image set, the first determining module 12 is configured to perform: determining a neighboring image corresponding to the image in the to-be-clustered image set; and performing augmented processing on the image feature and a neighboring feature corresponding to the neighboring image by using a first machine learning model to obtain the augmented image feature corresponding to each image in the to-be-clustered image set, wherein the first machine learning model is trained to perform augmented processing on the image feature.

[0216] In some examples, the to-be-clustered image set includes to-be-clustered face images.

[0217] Figure 13 The apparatus can perform the method of the embodiments Figures 1-12 The method of the embodiments is not described in detail, and reference can be made to the related description of the Figures 1-12 embodiments. The execution process and technical effects of the technical solution are described in the Figures 1-12 embodiments, which will not be repeated here.

[0218] In one possible design, Figure 13 The structure of the image clustering apparatus shown can be implemented as an electronic device, which can be a mobile phone, a tablet computer, a server, or various devices. As shown in the figure, Figure 14 The electronic device can include a first processor 21 and a first memory 22. The first memory 22 is configured to store programs for the electronic device to perform the above-mentioned Figures 1-12 The first processor 21 is configured to execute the programs stored in the first memory 22.

[0219] The programs include one or more computer instructions, and the one or more computer instructions can implement the following steps when executed by the first processor 21:

[0220] Obtain a set of images to be clustered;

[0221] Determine augmented inter-image relationship features corresponding to any two images in the set of images to be clustered;

[0222] Determine image similarity between the two images corresponding to the augmented inter-image relationship features based on the augmented inter-image relationship features;

[0223] Cluster the images included in the set of images to be clustered according to all the image similarities, and obtain a clustering result corresponding to the set of images to be clustered.

[0224] Further, the first processor 21 is further configured to execute all or part of the steps in the above-mentioned Figures 1-12 embodiments.

[0225] The structure of the electronic device can further include a first communication interface 23 for communication between the electronic device and other devices or communication networks.

[0226] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for the electronic device, which includes programs for performing the image clustering method in the above-mentioned Figures 1-12 embodiments.

[0227] Figure 15 Another flowchart of the image clustering method provided by the embodiment of the present application; with reference to the accompanying Figure 15 The embodiment provides another image clustering method, and the execution subject of the method can be an image clustering apparatus. It can be understood that the image clustering apparatus can be implemented as software or a combination of software and hardware, and the image clustering apparatus can be applied to a data processing platform. The data processing platform is used for at least one user to perform data processing operations.

[0228] In different application scenarios, the data processing platform can be used by a user to implement different data processing operations, which can include at least one of data transmission operations, data storage operations, data editing operations, and the like. When the data processing platform is used by a user to perform data transmission operations, the data processing platform can be implemented as a data transmission device with image clustering function, a data transmission server, and the like. When the data processing platform is used by a user to perform data storage operations, the data processing platform can be implemented as a data storage device with image clustering function, a cloud storage, a network hard disk, and the like. When the data processing platform is used by a user to perform data editing operations, the data processing platform can be implemented as a data editing device with image clustering function.

[0229] Specifically, the image clustering method can include:

[0230] Step S1501: Obtain a plurality of to-be-clustered images uploaded by at least one user to the data processing platform.

[0231] Step S1502: Determine the image similarity between any two images in the plurality of to-be-clustered images, the image similarity being determined by the augmented inter-image relationship feature between any two images.

[0232] Step S1503: Cluster the plurality of to-be-clustered images according to all image similarities.

[0233] Step S1504: Display the clustering result corresponding to the plurality of to-be-clustered images.

[0234] When the data processing platform with image clustering function is applied, one or more users can synchronously or asynchronously transmit to-be-clustered images to the data processing platform, so that the data processing platform can obtain a plurality of to-be-clustered images uploaded by at least one user to the data processing platform. In some examples, the to-be-clustered images can include to-be-clustered face images.

[0235] After obtaining the plurality of to-be-clustered images, the plurality of to-be-clustered images can be analyzed and processed to obtain the image similarity between any two images in the plurality of to-be-clustered images. The image similarity can be determined by the augmented inter-image relationship feature between any two images. Specifically, the specific implementation manner and implementation effect of determining the image similarity in this embodiment are the same as those of the corresponding embodiment described above, and details are not repeated here. Figure 1 The specific implementation manner and implementation effect of determining the image similarity in this embodiment are the same as those of the corresponding embodiment described above, and details are not repeated here.

[0236] After obtaining all image similarities between any two images in the plurality of images to be clustered, all image similarities can be analyzed and processed to cluster the plurality of images to be clustered according to the analysis and processing result, so as to obtain the clustering result corresponding to the plurality of images to be clustered. Specifically, the specific implementation and implementation effect of determining the clustering result corresponding to the plurality of images to be clustered in the embodiment are the same as those of the above Figure 1 The specific implementation and implementation effect of the corresponding embodiment are the same, and specific reference can be made to the above statements, which will not be repeated here.

[0237] After obtaining the clustering result corresponding to the plurality of images to be clustered, in order to facilitate at least one user to know the clustering result in time and accurately, the display module of the data processing platform can be used to display the clustering result corresponding to the plurality of images to be clustered.

[0238] For example, when the data transmission platform is a cloud server or a network hard disk, the users with access rights to the cloud server can include user A, user B and user C. User A, user B and user C can synchronously or asynchronously upload a plurality of face images to be clustered to the cloud server. It should be noted that when user A, user B and user C asynchronously upload a plurality of face images to be clustered to the cloud server, in order to improve the speed and efficiency of analyzing and processing the plurality of face images to be clustered, the cloud server can be used to obtain the plurality of face images to be clustered within a preset time period (for example, 1 min, 5 min or 10 min, etc.).

[0239] After the cloud server obtains the plurality of face images to be clustered, the image similarity between any two face images can be obtained, which can be determined by the augmented image relationship feature between any two images. After obtaining all image similarities, the plurality of face images to be clustered can be clustered based on all image similarities to obtain the clustering result corresponding to the plurality of images to be clustered, for example, the clustering result includes image a1, image a2 and image a3 corresponding to person a, image b1 and image b2 corresponding to person b, and image c1, image c2, image c3 and image c4 corresponding to person c, etc. Then the clustering result can be displayed in groups, so that user A, user B and user C can know the clustering result in time and accurately.

[0240] The image clustering method provided in this embodiment obtains at least one set of images to be clustered uploaded by a user to the data processing platform, and determines the image similarity between any two images among the multiple images to be clustered. Since the image similarity is determined by the augmented image relationship features between any two images, the accuracy and reliability of determining the image similarity between two images can be effectively improved when determining the image similarity between two images based on the augmented image relationship features. Therefore, when clustering the images included in the image set to be clustered according to all image similarities, more accurate clustering results can be obtained and displayed, further improving the accuracy and reliability of using the image clustering method, ensuring the practicality of the method, and facilitating its market promotion and application.

[0241] Figure 16 This is a schematic diagram of another image clustering device provided in an embodiment of the present invention; see attached diagram. Figure 16 As shown, this embodiment provides another image clustering device that can perform the above-described... Figure 15 The corresponding image clustering method, and the image clustering device is applied to a data processing platform, which is used for at least one user to perform data processing operations. Specifically, the image clustering device may include a second acquisition module 31, a second determination module 32, a second processing module 33, and a second display module 34; specifically,

[0242] The second acquisition module 31 acquires at least one user's uploaded multiple images to be clustered to the data processing platform;

[0243] The second determining module 32 is used to determine the image similarity between any two images among the plurality of images to be clustered, wherein the image similarity is determined by augmented image relationship features between any two images;

[0244] The second processing module 33 is used to perform clustering processing on the plurality of images to be clustered based on all image similarities;

[0245] The second display module 34 is used to display the clustering results corresponding to the plurality of images to be clustered.

[0246] Figure 16 The device shown can perform Figure 15 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 15 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 15 The descriptions in the illustrated embodiments will not be repeated here.

[0247] In one possible design,Figure 16 The structure of the image clustering apparatus shown can be implemented as an electronic device, which can be a mobile phone, a tablet computer, a server, or various devices. As shown in the figure, Figure 17 The electronic device can include a second processor 41 and a second memory 42. The second memory 42 is configured to store programs for the electronic device to perform the above-mentioned Figure 15 The programs of the image clustering method provided in the embodiment shown are stored in the second memory 42, and the second processor 41 is configured to execute the programs.

[0248] The programs include one or more computer instructions, and the one or more computer instructions can implement the following steps when executed by the second processor 41:

[0249] Obtain a plurality of images to be clustered uploaded by at least one user to the data processing platform;

[0250] Determine the image similarity between any two images in the plurality of images to be clustered, the image similarity being determined by the augmented image relationship feature between any two images;

[0251] Cluster the plurality of images to be clustered according to all the image similarities;

[0252] Display the clustering results corresponding to the plurality of images to be clustered.

[0253] Further, the second processor 41 is further configured to execute all or part of the above-mentioned Figure 15 steps in the embodiment shown.

[0254] The structure of the electronic device can further include a second communication interface 43 for communication between the electronic device and other devices or communication networks.

[0255] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for the electronic device, which includes programs for executing the above-mentioned Figure 15 programs involved in the image clustering method in the method embodiment shown.

[0256] Figure 18 Another flowchart of the image clustering method provided by the embodiment of the present application; with reference to the accompanying Figure 18 The embodiment provides another image clustering method, and the execution subject of the method can be an image clustering apparatus. It can be understood that the image clustering apparatus can be implemented as software or a combination of software and hardware, and the image clustering apparatus can be applied to a data communication apparatus for data communication by at least one user.

[0257] In different application scenarios, the data communication device can be used by a user to implement different data communication operations, which can include at least one of the following: instant communication operation and non-instant communication operation, etc. When the data communication device is used by a user to perform instant communication operation, the data communication device can be an instant communication device, such as DingTalk application, etc. When the data communication device is used by a user to perform asynchronous communication operation, the data communication device can be a non-instant communication device, such as a mail application.

[0258] Specifically, the image clustering method can include:

[0259] Step S1801: Obtain a plurality of to-be-clustered images transmitted by at least one user through the data communication device.

[0260] Step S1802: Determine the image similarity between any two images in the plurality of to-be-clustered images, the image similarity being determined by the augmented inter-image relationship feature between any two images.

[0261] Step S1803: Cluster the plurality of to-be-clustered images according to all image similarities.

[0262] Step S1804: Display the clustering result corresponding to the plurality of to-be-clustered images.

[0263] In some examples, the user corresponding to the plurality of to-be-clustered images obtained by the data communication device can be located in a communication group.

[0264] In this embodiment, the specific implementation and implementation effects of the above steps are similar to those of the corresponding embodiments, and specific reference can be made to the above statements, which will not be repeated here. Figure 15

[0265] For ease of understanding, the DingTalk program capable of realizing instant communication is taken as an example to illustrate the data communication device, the users who can access the DingTalk program can include user A, user B and user C, wherein the user A, user B and user C can be located in a communication group, the user A, user B and user C can synchronously or asynchronously communicate data in the communication group, and a plurality of to-be-clustered images can be uploaded in the communication group.

[0266] ​In order to improve the quality and efficiency of image clustering operation, when a plurality of to-be-clustered images transmitted by at least one user in a communication group are acquired, the plurality of to-be-clustered images acquired in a preset time period (for example, 1 min, 5 min or 10 min, etc.) for the communication group can be counted. After acquiring the plurality of to-be-clustered images for the communication group, the image similarity between any two images can be acquired, which can be determined by the augmented inter-image relationship feature between the two images. After acquiring all the image similarities, the plurality of to-be-clustered images can be clustered based on all the image similarities to obtain a clustering result corresponding to the plurality of to-be-clustered images, for example, the clustering result includes image a1, image a2 and image a3 corresponding to person a, image b1 and image b2 corresponding to person b, and image c1, image c2, image c3 and image c4 corresponding to person c, etc. Then the above clustering result can be displayed in groups in the communication group, so that user A, user B and user C can timely and accurately know the clustering result.

[0267] The image clustering method provided by the embodiment can acquire a plurality of to-be-clustered images transmitted by at least one user through the data communication device, and determine the image similarity between any two images in the plurality of to-be-clustered images. Since the image similarity is determined by the augmented inter-image relationship feature between the two images, the accuracy and reliability of determining the image similarity between the two images can be effectively improved when the image similarity between the two images corresponding to the augmented inter-image relationship feature is determined based on the augmented inter-image relationship feature. Therefore, when the images included in the to-be-clustered image set are clustered based on all the image similarities, a clustering result with higher accuracy can be obtained and displayed, the accuracy and reliability of using the image clustering method are further improved, the practicability of the method is ensured, and the market promotion and application are facilitated.

[0268] Figure 19 A structural schematic diagram of another image clustering device provided by the embodiment of the application is shown in FIG. 5. Figure 19 The embodiment provides another image clustering device, which can perform the image clustering method corresponding to the above-mentioned image clustering method. Figure 18 The image clustering device is applied to a data communication device for data communication of at least one user. Specifically, the image clustering device can include a third acquisition module 51, a third determination module 52, a third processing module 53 and a third display module 54.

[0269] The third acquisition module 51 is configured to acquire a plurality of to-be-clustered images transmitted by at least one user through the data communication device.

[0270] The third determining module 52 is used to determine the image similarity between any two images among the plurality of images to be clustered, wherein the image similarity is determined by augmented image relationship features between any two images;

[0271] The third processing module 53 is used to perform clustering processing on the plurality of images to be clustered based on all image similarities;

[0272] The third display module 54 is used to display the clustering results corresponding to the plurality of images to be clustered.

[0273] Figure 19 The device shown can perform Figure 18 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 18 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 18 The descriptions in the illustrated embodiments will not be repeated here.

[0274] In one possible design, Figure 19 The structure of the image clustering device shown can be implemented as an electronic device, which can be various devices such as mobile phones, tablets, and servers. Figure 20 As shown, the electronic device may include a third processor 61 and a third memory 62. The third memory 62 is used to store data executed by the corresponding electronic device. Figure 18 In the illustrated embodiment, the program for the image clustering method is configured such that the third processor 61 is configured to execute the program stored in the third memory 62.

[0275] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the third processor 61, can perform the following steps:

[0276] Acquire multiple images to be clustered transmitted by at least one user through the data communication device;

[0277] Determine the image similarity between any two images among the plurality of images to be clustered, wherein the image similarity is determined by augmented image relationship features between any two images;

[0278] Clustering is performed on the multiple images to be clustered based on all image similarities;

[0279] Display the clustering results corresponding to the plurality of images to be clustered.

[0280] Furthermore, the third processor 61 is also used to perform the aforementioned... Figure 18 All or part of the steps in the illustrated embodiments.

[0281] The structure of the electronic device can further include a third communication interface 63 for communication between the electronic device and other devices or communication networks.

[0282] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for the electronic device, which includes the computer software instructions for executing the above-mentioned Figure 18 The program involved in the image clustering method in the method embodiment.

[0283] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0284] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of a general hardware platform as necessary, and of course can also be realized by means of combination of hardware and software. Based on such understanding, the above technical solutions can be embodied in the form of a computer product, and the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0285] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable devices to produce a machine, so that the instructions executed by the computer or other programmable devices produce a device implemented in the flowcharts and / or block diagrams. Figure One The device for performing the function specified in one flow or multiple flows and / or blocks. Figure One The device for performing the function specified in one flow or multiple flows and / or blocks.

[0286] These computer program instructions can also be stored in a computer readable storage medium capable of guiding the computer or other programmable devices to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which implement the flowcharts and / or block diagrams. Figure One The device for performing the function specified in one flow or multiple flows and / or blocks.Figure One the function specified in one or more blocks.

[0287] These computer program instructions can also be loaded into computer or other programmable devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer-implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart block or blocks. Figure One the flowchart block or blocks. Figure One the function specified in one or more blocks.

[0288] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0289] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about the operating environment. This memory is an example of computer readable media. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), programmable gate array (PGA), or flash memory. While the memory is computer readable media, it is not transitory.

[0290] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible to computing devices. According to the definition herein, computer readable media does not include transitory media such as modulated data signals and carrier waves.

[0291] Finally, it should be noted that the above-mentioned embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image clustering method characterized by, The method comprises: obtaining a set of images to be clustered; determining augmented inter-image relationship features corresponding to any two images in the set of images to be clustered, the augmented inter-image relationship features being obtained by augmenting the inter-image relationship features of any two images; determining image similarity between the two images corresponding to the augmented inter-image relationship features based on the augmented inter-image relationship features; performing clustering processing on the images included in the set of images to be clustered according to all the image similarity, and obtaining a clustering result corresponding to the set of images to be clustered.

2. The method of claim 1, wherein, The augmented inter-image relationship features are related to at least one of the following: image features corresponding to the images in any two images, context features corresponding to the images in any two images.

3. The method of claim 1, wherein, The method comprises: obtaining augmented image features corresponding to each image in the set of images to be clustered, the augmented image features being obtained by augmenting image features, the augmented image features including image features and context features corresponding to the image; determining augmented inter-image relationship features between any two images in the set of images to be clustered based on the augmented image features corresponding to each image in the set of images to be clustered.

4. The method of claim 3, wherein, The method comprises: extracting image features corresponding to each image in the set of images to be clustered; determining neighboring features corresponding to the image features in the set of images to be clustered; augmenting the image features and neighboring features using a first machine learning model to obtain augmented image features corresponding to each image in the set of images to be clustered, wherein the first machine learning model is trained to augment image features.

5. The method of claim 4, wherein, The method comprises: performing a nearest neighbor search based on the image features in the set of images to be clustered to obtain a set of neighboring features corresponding to the image features, the set of neighboring features including at least one neighboring feature corresponding to the image features; determining a target neighboring feature corresponding to the image features in the set of neighboring features.

6. The method of claim 5, wherein, The method comprises: obtaining parameter information for limiting the number of target neighboring features; determining a target neighboring feature corresponding to the image features based on the set of neighboring features and the parameter information.

7. The method of claim 4, wherein, The method comprises: processing the neighboring features using a self-attention network layer and a common attention layer included in the first machine learning model to obtain context features corresponding to the image features; obtaining augmented image features corresponding to each image in the set of images to be clustered based on the image features and the context features.

8. The method of claim 7, wherein, obtaining augmented image features corresponding to each image in the image set to be clustered based on the image features and the context features, comprising: performing fusion processing on the image features and the context features to obtain fused image features; performing normalization processing on the fused image features to obtain augmented image features corresponding to each image in the image set to be clustered.

9. The method of claim 3, wherein, determining augmented inter-image relationship features between any two images in the image set to be clustered based on the augmented image features corresponding to each image in the image set to be clustered, comprising: obtaining first augmented image features corresponding to each image in the image set to be clustered, and second augmented image features corresponding to adjacent images of the image; performing augmented processing on the first augmented image features and the second augmented image features by using a second machine learning model to obtain augmented inter-image relationship features between the image and the adjacent images in the image set to be clustered, wherein the second machine learning model is trained to perform augmented processing on inter-image relationship features between two images based on image features.

10. The method of claim 9, wherein, performing augmented processing on the first augmented image features and the second augmented image features by using a second machine learning model to obtain augmented inter-image relationship features between the image and the adjacent images in the image set to be clustered, comprising: processing the second augmented image features by using a self-attention network layer in the second machine learning model to obtain augmented adjacent image features corresponding to the adjacent images; obtaining augmented inter-image relationship features between the image and the adjacent images in the image set to be clustered based on the augmented adjacent image features and the first augmented image features.

11. The method of claim 10, wherein, obtaining augmented inter-image relationship features between the image and the adjacent images in the image set to be clustered based on the augmented adjacent image features and the first augmented image features, comprising: performing extraction relationship processing on the augmented adjacent image features and the first augmented image features to obtain augmented inter-image relationship features between the image and the adjacent images in the image set to be clustered.

12. The method of claim 1, wherein, determining image similarity between two images corresponding to the augmented inter-image relationship features based on the augmented inter-image relationship features, comprising: performing analysis processing on the augmented inter-image relationship features by using a classifier to obtain image similarity between two images corresponding to the augmented inter-image relationship features.

13. The method of claim 1, wherein, performing clustering processing on images included in the image set to be clustered according to all image similarities to obtain a clustering result corresponding to the image set to be clustered, comprising: obtaining a similarity threshold for clustering processing on images in the image set to be clustered; when the image similarity is greater than or equal to the similarity threshold, clustering processing is performed on two images corresponding to the image similarity to obtain a clustering cluster corresponding to the image set to be clustered.

14. The method of claim 13, wherein, After obtaining the clustering cluster corresponding to the image set to be clustered, the method further comprises: obtaining a to-be-processed image and first image features corresponding to the to-be-processed image; determine an augmented class feature corresponding to the cluster, the augmented class feature comprising a class feature corresponding to the cluster and context information; determine whether to cluster the to-be-processed image to the cluster based on the first image feature and the augmented class feature.

15. The method of claim 14, wherein, determining an augmented class feature corresponding to the cluster comprises: obtaining a class feature corresponding to the cluster and a neighboring class feature; augmenting the class feature and the neighboring class feature by using a third machine learning model to obtain an augmented class feature corresponding to the cluster, wherein the third machine learning model is trained to be used for augmenting a class feature of a cluster.

16. The method of claim 14, wherein, determining whether to cluster the to-be-processed image to the cluster based on the first image feature and the augmented class feature comprises: obtaining a matching degree between the to-be-processed image and the cluster based on the first image feature and the augmented class feature; when the matching degree is greater than or equal to a preset threshold, clustering the to-be-processed image to the cluster; or, when the matching degree is less than the preset threshold, prohibiting clustering the to-be-processed image to the cluster.

17. The method of claim 1, wherein, determining an augmented image-to-image relationship feature corresponding to any two images in the to-be-clustered image set comprises: obtaining an image feature corresponding to each image in the to-be-clustered image set; determining an augmented image-to-image relationship feature between any two images in the to-be-clustered image set based on the image feature corresponding to each image in the to-be-clustered image set.

18. The method of claim 17, wherein, determining an augmented image-to-image relationship feature between any two images in the to-be-clustered image set based on the image feature corresponding to each image in the to-be-clustered image set comprises: determining a neighboring image corresponding to the image in the to-be-clustered image set; augmenting the image feature and a neighboring feature corresponding to the neighboring image by using a first machine learning model to obtain an augmented image feature corresponding to each image in the to-be-clustered image set, wherein the first machine learning model is trained to be used for augmenting an image feature.

19. The method of any of claims 1-18, wherein, The to-be-clustered image set comprises to-be-clustered face images.

20. An image clustering apparatus, characterized by comprising: comprises: a first obtaining module configured to obtain a to-be-clustered image set; a first determining module configured to determine an augmented image-to-image relationship feature corresponding to any two images in the to-be-clustered image set, the augmented image-to-image relationship feature being obtained by augmenting an image-to-image relationship feature of any two images; the first determining module is further configured to determine an image similarity between two images corresponding to the augmented image-to-image relationship feature based on the augmented image-to-image relationship feature; a first processing module configured to cluster images included in the to-be-clustered image set according to all the image similarities to obtain a clustering result corresponding to the to-be-clustered image set.

21. An electronic device, comprising: comprises: a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the image clustering method in any one of claims 1-19.

22. An image clustering method, characterized by, The method is applied to a data processing platform, the data processing platform is used for data processing operation of at least one user, and the method comprises the following steps: Obtain a plurality of images to be clustered uploaded by at least one user to the data processing platform; Determine the image similarity between any two images in the plurality of images to be clustered, the image similarity is determined by the augmented inter-image relationship feature between any two images, and the augmented inter-image relationship feature is obtained by augmenting the inter-image relationship feature of any two images; Cluster the plurality of images to be clustered according to all the image similarities; Display the clustering result corresponding to the plurality of images to be clustered.

23. An image clustering apparatus, characterized by comprising: The device is applied to a data processing platform, the data processing platform is used for data processing operation of at least one user, and the device comprises the following: A second acquisition module is configured to obtain a plurality of images to be clustered uploaded by at least one user to the data processing platform; A second determination module is configured to determine the image similarity between any two images in the plurality of images to be clustered, the image similarity is determined by the augmented inter-image relationship feature between any two images, and the augmented inter-image relationship feature is obtained by augmenting the inter-image relationship feature of any two images; A second processing module is configured to cluster the plurality of images to be clustered according to all the image similarities; A second display module is configured to display the clustering result corresponding to the plurality of images to be clustered.

24. An electronic device, comprising: The device comprises: A memory and a processor; wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the image clustering method of claim 22.

25. A method of image clustering, the method comprising: The method is applied to a data communication device, the data communication device is used for data communication of at least one user, and the method comprises the following steps: Obtain a plurality of images to be clustered transmitted by at least one user through the data communication device; Determine the image similarity between any two images in the plurality of images to be clustered, the image similarity is determined by the augmented inter-image relationship feature between any two images, and the augmented inter-image relationship feature is obtained by augmenting the inter-image relationship feature of any two images; Cluster the plurality of images to be clustered according to all the image similarities; Display the clustering result corresponding to the plurality of images to be clustered.

26. An image clustering apparatus, characterized by comprising: The device is applied to a data communication device, the data communication device is used for data communication of at least one user, and the device comprises the following: A third acquisition module is configured to obtain a plurality of images to be clustered transmitted by at least one user through the data communication device; A third determination module is configured to determine the image similarity between any two images in the plurality of images to be clustered, the image similarity is determined by the augmented inter-image relationship feature between any two images, and the augmented inter-image relationship feature is obtained by augmenting the inter-image relationship feature of any two images; A third processing module is configured to cluster the plurality of images to be clustered according to all the image similarities; A third display module is configured to display a clustering result corresponding to the plurality of images to be clustered.

27. An electronic device, comprising: The method comprises the following steps: A memory and a processor are included, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the image clustering method in claim 25.

Images