Clustering method and device, electronic device and storage medium

Abstract

The disclosed embodiment proposes a clustering method and device, an electronic device and a storage medium, the method comprising: obtaining a neural network for correcting the similarity of facial features, the neural network being trained based on difficult samples; the difficult samples being samples in at least one category of face data set; processing images to be clustered based on the neural network to obtain similarity correction values ​​of facial features of the images to be clustered; correcting feature similarity using the similarity correction values ​​to obtain corrected feature similarity; the feature similarity represents the similarity of facial features between the images to be clustered and images of the corresponding category; clustering the images to be clustered based on the corrected feature similarity to obtain clustering results of the images to be clustered and the images of the corresponding category.

Description

Technical Field

[0001] The present disclosure relates to computer vision processing technology, and in particular to a clustering method and device, an electronic device and a storage medium. Background Art

[0002] In related technologies, clustering is one of the important steps in data mining. Through clustering, the structure in the data can be discovered and effective information can be extracted for further analysis.

[0003] However, in the related art, due to the complex distribution of facial features, the face clustering model may not be able to adapt to the clustering of various types of faces (for example, the elderly, children, black people, etc.), that is, there is a problem of low clustering accuracy of the face clustering model for some types of faces. Summary of the invention

[0004] The embodiments of the present disclosure are intended to provide a technical solution for clustering processing.

[0005] The present disclosure provides a clustering method, the method comprising:

[0006] Obtaining a neural network for correcting similarity of facial features, wherein the neural network is trained based on difficult samples; the difficult samples are samples in at least one category of face data set;

[0007] Processing the images to be clustered based on the neural network to obtain similarity correction values ​​of facial features of the images to be clustered;

[0008] Correcting the feature similarity using the similarity correction value to obtain a corrected feature similarity; the feature similarity represents the similarity of the facial features between the image to be clustered and the image of the corresponding category;

[0009] Based on the corrected feature similarity, clustering processing is performed on the image to be clustered to obtain a clustering result of the image to be clustered and the corresponding category image.

[0010] In some embodiments, the method further comprises:

[0011] Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets;

[0012] Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index;

[0013] Based on the first similarity threshold, clustering the samples to obtain clustering results of the samples of the multiple categories of face data sets;

[0014] Based on the clustering results of the samples of the multiple categories of face data sets, samples that meet the set conditions are determined, and the difficult samples are determined from the samples that meet the set conditions. The set conditions include: the clustering results of the samples are different from the labeling information of the samples.

[0015] In some embodiments, determining the first similarity threshold comprises:

[0016] Based on each similarity threshold of a plurality of predetermined similarity thresholds, clustering is performed on the samples to obtain a clustering result corresponding to each similarity threshold;

[0017] Based on the clustering results corresponding to each similarity threshold, determining that the clustering accuracy reaches multiple second similarity thresholds of the set accuracy index;

[0018] Among the multiple second similarity thresholds, the largest similarity threshold is determined as the first similarity threshold.

[0019] In some embodiments, the step of determining the difficult sample from samples that meet a set condition includes:

[0020] The difficult samples are determined from the samples that meet the set conditions in a positive-negative sample balance manner, wherein the difficult samples include positive samples and negative samples, the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces.

[0021] In some embodiments, the neural network is trained based on difficult samples, including:

[0022] Inputting the difficult sample into the neural network;

[0023] Processing the difficult sample using the neural network to obtain a processing result of the difficult sample, wherein the processing result of the difficult sample represents a similarity correction value of a facial feature in the difficult sample;

[0024] According to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, the network parameter value of the neural network is adjusted.

[0025] In some embodiments, adjusting the network parameter value of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample includes:

[0026] Determining a hinge loss of the neural network according to a processing result of the difficult sample and pre-acquired labeling information of the difficult sample;

[0027] According to the hinge loss, the network parameter values ​​of the neural network are adjusted.

[0028] In some embodiments, the method further comprises:

[0029] Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets;

[0030] Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index;

[0031] The determining the hinge loss of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample includes:

[0032] The hinge loss of the neural network is determined according to the processing result of the difficult sample, the pre-acquired labeling information of the difficult sample and the first similarity threshold.

[0033] In some embodiments, the clustering process is performed on the to-be-clustered images based on the corrected feature similarity to obtain the clustering results of the to-be-clustered images, including:

[0034] Based on the corrected feature similarity, a face clustering method is used to perform clustering processing on the image to be clustered to obtain a clustering result of the image to be clustered and images of corresponding categories, wherein the corrected feature similarities corresponding to images to be clustered of different categories are different.

[0035] The present disclosure provides another clustering method, which includes:

[0036] Obtaining a neural network for obtaining feature similarity, wherein the neural network is obtained by training based on difficult samples, and the difficult samples are samples in at least one category of face data set; the feature similarity represents the similarity of face features between the image to be clustered and the image of the corresponding category;

[0037] Processing the to-be-clustered images based on the neural network to obtain the feature similarity; wherein the feature similarity is obtained based on similarity correction values ​​of facial features of different categories;

[0038] Based on the feature similarity, clustering processing is performed on the images to be clustered to obtain clustering results of the images to be clustered and images of corresponding categories.

[0039] The present disclosure also provides a clustering device, the device comprising:

[0040] A first acquisition module is used to acquire a neural network for correcting the similarity of facial features, wherein the neural network is trained based on difficult samples; the difficult samples are samples in at least one category of face data set;

[0041] A first processing module, used for processing the images to be clustered based on the neural network to obtain similarity correction values ​​of facial features of the images to be clustered;

[0042] A second processing module is used to correct the feature similarity using the similarity correction value to obtain the corrected feature similarity; the feature similarity represents the similarity of the facial features between the image to be clustered and the image of the corresponding category;

[0043] The third processing module is used to perform clustering processing on the images to be clustered based on the corrected feature similarity to obtain clustering results of the images to be clustered and the corresponding category images.

[0044] The present disclosure also provides another clustering device, the device comprising:

[0045] A second acquisition module is used to acquire a neural network for obtaining feature similarity, wherein the neural network is obtained by training based on difficult samples, and the difficult samples are samples in at least one category of face data set; the feature similarity represents the similarity of face features between the image to be clustered and the image of the corresponding category;

[0046] A fourth processing module, used for processing the to-be-clustered images based on the neural network to obtain the feature similarity, wherein the feature similarity is obtained based on similarity correction values ​​of facial features of different categories;

[0047] The fifth processing module is used to perform clustering processing on the images to be clustered based on the feature similarity to obtain clustering results of the images to be clustered and the images of the corresponding category.

[0048] The present disclosure also provides an electronic device, including a processor and a memory for storing a computer program that can be run on the processor; wherein:

[0049] The processor is used to run the computer program to execute any one of the above clustering methods.

[0050] The embodiment of the present disclosure further provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, any one of the above clustering methods is implemented.

[0051] In the clustering method, device, electronic device and computer storage medium proposed in the embodiment of the present disclosure, a neural network for correcting the similarity of facial features is obtained, and the neural network is obtained by training based on difficult samples; the difficult samples are samples in at least one category of face data set; the image to be clustered is processed based on the neural network to obtain a similarity correction value of the facial features of the image to be clustered; the feature similarity is corrected using the similarity correction value to obtain the corrected feature similarity; the feature similarity represents the similarity of the facial features of the image to be clustered and the corresponding category image; based on the corrected feature similarity, the image to be clustered is clustered to obtain the clustering result of the image to be clustered and the corresponding category image. In this way, in the embodiment of the present disclosure, the similarity of the facial features of the face images to be clustered can be corrected based on the neural network for correcting the similarity of facial features, and the clustered images are clustered based on the corrected similarity; that is, for various types of face images to be clustered, the facial feature similarity can be corrected based on the neural network, and the correction of the facial feature similarity is conducive to improving the clustering accuracy of face clustering.

[0052] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0054] Figure 1 A flow chart of a clustering method according to an embodiment of the present disclosure;

[0055] Figure 2A A schematic diagram of the structure of a neural network in an embodiment of the present disclosure;

[0056] Figure 2B Schematic diagram of the structure of the convolutional layer in the embodiment of the present disclosure;

[0057] Figure 3 is a flowchart of a neural network training method according to an embodiment of the present disclosure;

[0058] Figure 4 A flowchart of another clustering method according to an embodiment of the present disclosure;

[0059] Figure 5 A schematic diagram of the structure of a clustering device according to an embodiment of the present disclosure;

[0060] Figure 6 A schematic diagram of the composition structure of another clustering device according to an embodiment of the present disclosure;

[0061] Figure 7 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0062] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are only used to explain the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are partial embodiments for implementing the present disclosure, rather than providing all embodiments for implementing the present disclosure. In the absence of conflict, the technical solutions recorded in the embodiments of the present disclosure can be implemented in any combination.

[0063] It should be noted that, in the embodiments of the present disclosure, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or apparatus. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements (such as steps in a method or units in a device, for example, a unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.) in the method or apparatus including the element.

[0064] For example, the clustering method provided by the embodiment of the present disclosure includes a series of steps, but the clustering method provided by the embodiment of the present disclosure is not limited to the recorded steps. Similarly, the clustering device provided by the embodiment of the present disclosure includes a series of modules, but the device provided by the embodiment of the present disclosure is not limited to including the modules explicitly recorded, and may also include modules required to obtain relevant information or perform processing based on information.

[0065] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0066] The disclosed embodiments can be applied to a computer system consisting of a terminal and / or a server, and can operate with many other general or special computing system environments or configurations. Here, the terminal can be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a network personal computer, a small computer system, etc., and the server can be a server computer system, a small computer system, a large computer system, and a distributed cloud computing technology environment including any of the above systems, etc.

[0067] Electronic devices such as terminals and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0068] In the related art, due to the imbalance of training data and the complex distribution of facial features, the clustering accuracy of the face clustering method may be low. In the related art, two clustering methods are mainly used to implement face clustering.

[0069] The first clustering method: a clustering method represented by the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) method.

[0070] The main process of clustering by the DBSCAN method is: first, select a feature point at random, and then find all the features whose distance to this feature point is less than or equal to the scanning radius (eps). If the number of feature points within the distance eps from the feature point is less than the minimum number of included points (min_samples), then this feature point is marked as noise. If the number of feature points within the distance eps is greater than min_samples, this point is marked as a core sample and assigned a new cluster label; then visit all neighbors of the feature point (within the distance eps). If they have not been assigned a cluster, then assign the newly created cluster label to them. If they are core samples, then visit their neighbors in turn, and so on. The cluster gradually increases until there are no more core samples within the eps distance of the cluster.

[0071] In practical applications, the distance between two feature points can be calculated based on the similarity between the two feature points. The similarity between the two feature points is negatively correlated with the distance between the two feature points. For example, the distance between the two feature points can be obtained by referring to formula (1):

[0072] dist=1-2*sim (1)

[0073] Among them, sim represents the similarity between two feature points, and dist represents the distance between two feature points.

[0074] When applying the DBSCAN method for face clustering, eps can be used as dist, and sim can be calculated according to the above formula (1), and the calculated sim can be used as the similarity threshold; then, it can be judged whether two feature points belong to the same category based on the similarity threshold; when the similarity threshold is high, if the facial features of the same person are distributed more dispersedly (that is, the distance between the feature points is far), the same facial features cannot be classified into the same category; when the similarity threshold is low, if the facial features of different people are close in distance, the facial features of different people may be classified into the same category; in short, the DBSCAN method cannot be applied to face clustering in various scenarios.

[0075] It can be seen that when applying the DBSCAN method for face clustering, the clustering accuracy of the DBSCAN method depends on the hyperparameter of the similarity threshold. For a certain type of face, if the similarity threshold is not set accurately enough, the clustering accuracy of the BSCAN method will be low.

[0076] The second clustering method: clustering method based on graph convolution, represented by Learnt Clustering.

[0077] Graph convolution-based clustering methods, represented by Learnt Clustering, have the problems of long time consumption and low clustering accuracy.

[0078] In view of the above technical problems, in some embodiments of the present disclosure, a clustering method is proposed, and the embodiments of the present disclosure can be applied to the fields of video surveillance, etc. By using the clustering method of the embodiments of the present disclosure, faces in various scenes can be accurately clustered.

[0079] Figure 1 is a flow chart of a clustering method according to an embodiment of the present disclosure, such as Figure 1 As shown, the process may include:

[0080] Step 101: Obtain a neural network for correcting similarity of facial features, wherein the neural network is trained based on difficult samples; the difficult samples are samples in at least one category of face data set.

[0081] In the embodiment of the present disclosure, difficult samples are represented as samples that meet the set conditions, and the set conditions include that the clustering result of the sample is different from the annotation information of the sample, that is, the difficult samples are samples that cannot obtain the correct clustering result, and the difficult samples can be samples selected from at least one preset category of face data set. Exemplarily, the at least one category of face data set can be a child face data set, an adult face data set, a black face data set, etc., and the embodiment of the present disclosure does not limit this.

[0082] In some embodiments of the present disclosure, difficult samples may include positive samples and / or negative samples, wherein the positive sample represents a pair of sample images labeled as the same face, and after clustering the positive samples, a clustering result of the positive sample can be obtained, and the clustering result of the positive sample indicates that a pair of sample images in the positive sample are not classified into the same category, and it can be seen that the clustering result of the above positive sample is different from the labeling information of the above positive sample, that is, the above positive sample meets the set conditions; the negative sample represents a pair of sample images labeled as different faces, and after clustering the negative samples, a clustering result of the negative sample can be obtained, and the clustering result of the negative sample indicates that a pair of sample images in the negative sample are classified into the same category; it can be seen that the clustering result of the above negative sample is different from the labeling information of the above negative sample, that is, the above negative sample meets the set conditions.

[0083] In some embodiments of the present disclosure, the difficult samples may be samples used in a face clustering method, and the face clustering method may be a density-based clustering method or a graph convolution-based clustering method.

[0084] In the disclosed embodiment, the neural network may be a network based on a deep learning model. Here, the structure of the neural network is not limited. For example, the neural network may be a neural network with two convolutional layers.

[0085] Figure 2A is a schematic diagram of the structure of a neural network in an embodiment of the present disclosure, such as Figure 2A As shown, features of difficult samples can be extracted in advance, the extracted features can be transformed to a fixed size through a resize operation, and the features after the resize operation can be processed in sequence through the first convolutional layer, the second convolutional layer, the first linear layer, the activation layer, and the second linear layer to obtain the processing results of the difficult samples.

[0086] Figure 2B Schematic diagram of the structure of the convolution layer in the embodiment of the present disclosure. Figure 2BAs shown, the input data of the convolutional layer can be processed by convolution, batch normalization (BN) and activation in sequence to obtain the output data of the convolutional layer; here, the role of convolution processing is to perceive the relationship between different dimensions of features, and the role of batch normalization processing is to normalize the distribution of each dimension of the feature.

[0087] In the disclosed embodiment, the difficult sample is processed using the above neural network to obtain a processing result of the difficult sample; the above neural network may be a neural network for obtaining a similarity correction value of facial features, and the processing result of the difficult sample represents a similarity correction value of facial features in the difficult sample.

[0088] Step 102: Processing the images to be clustered based on the neural network to obtain similarity correction values ​​of facial features of the images to be clustered.

[0089] In the embodiments of the present disclosure, the images to be clustered may include at least one face image; in actual applications, the images to be clustered may be obtained from a local storage area or a network, for example, the images to be clustered may be images taken by a mobile terminal or a camera, etc.; the format of the images to be clustered may be Joint Photographic Experts Group (JPEG), bitmap (BMP), portable network graphics (PNG) or other formats; it should be noted that only examples are given here to illustrate the format and source of the images to be clustered, and the embodiments of the present disclosure do not limit the format and source of the images to be clustered.

[0090] Step 103: Correct the feature similarity using the similarity correction value to obtain a corrected feature similarity; the feature similarity represents the similarity between the facial features of the image to be clustered and the corresponding category image.

[0091] In the disclosed embodiment, the face clustering method may be used to process the clustered image to obtain the above-mentioned feature similarity.

[0092] Step 104: Based on the corrected feature similarity, clustering processing is performed on the image to be clustered to obtain a clustering result of the image to be clustered and the corresponding category image.

[0093] In the disclosed embodiment, based on the corrected feature similarity, a face clustering method can be used to perform clustering processing on the image to be clustered to obtain a clustering result of the image to be clustered and the corresponding category image, wherein the corrected feature similarities corresponding to different categories of images to be clustered are different.

[0094] In practical applications, steps 101 to 104 can be implemented using a processor in an electronic device, and the processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor; in some embodiments, steps 101 to 104 can also be implemented using the processor in combination with a graphics processing unit (GPU), so that the speed of clustering processing can be further improved.

[0095] It can be seen that in the disclosed embodiment, the similarity of the facial features of the face images to be clustered can be corrected based on the neural network for correcting the similarity of facial features, and the clustered images can be clustered based on the corrected similarity; that is, for various types of face images to be clustered, the similarity of facial features can be corrected based on the neural network, and the correction of the similarity of facial features is conducive to improving the clustering accuracy of face clustering. Furthermore, the disclosed embodiment can be implemented in combination with the existing face clustering method, and has the characteristics of scalability and easy implementation.

[0096] In some embodiments, difficult samples may be obtained by:

[0097] Obtain samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of multiple categories of face data sets.

[0098] A first similarity threshold is determined, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of clustering results of samples of multiple categories of face data sets to reach a set accuracy index.

[0099] Based on the first similarity threshold, clustering the samples of the multiple categories of face data sets to obtain clustering results of the samples of the multiple categories of face data sets;

[0100] Based on the clustering results of the samples of the multiple categories of face data sets, samples that meet the set conditions are determined, and the difficult samples are determined from the samples that meet the set conditions. The set conditions include: the clustering results of the samples are different from the labeling information of the samples.

[0101] In the disclosed embodiment, after obtaining samples of multiple categories of face data sets, each sample of the samples of the multiple categories of face data sets can be labeled to obtain the labeling information of each sample; the labeling information of each sample indicates the face identification corresponding to each sample; here, the face identification can be the identity information corresponding to the face. It can be seen that based on the labeling information of each sample, it can be determined whether any two samples of the samples of the multiple categories of face data sets are face images of the same person.

[0102] In the disclosed embodiment, the accuracy of the clustering result can be determined based on the difference between the clustering result and the corresponding annotation information; the smaller the difference between the clustering result and the corresponding annotation information, the higher the accuracy of the clustering result.

[0103] In the embodiment of the present disclosure, the accuracy index is set as a pre-set index, which can be set specifically according to actual application requirements.

[0104] In one embodiment, determining the first similarity threshold may include: based on each similarity threshold of a plurality of pre-determined similarity thresholds, clustering the samples to obtain a clustering result corresponding to each similarity threshold; based on the clustering results corresponding to each similarity threshold, determining a plurality of second similarity thresholds whose clustering accuracy reaches a set accuracy index; and among the plurality of second similarity thresholds, determining the maximum similarity threshold as the first similarity threshold.

[0105] In some embodiments, the predetermined multiple similarity thresholds may include similarity thresholds arranged from small to large, where adjacent intervals of the similarity thresholds arranged from small to large may be equal or unequal; the minimum value and the maximum value of the predetermined multiple similarity thresholds may be determined according to actual application requirements, and the adjacent intervals of the similarity thresholds arranged from small to large may also be pre-set.

[0106] For example, the minimum value of the predetermined multiple similarity thresholds is 0.1, the maximum value of the predetermined multiple similarity thresholds is 0.9, and the adjacent intervals of the similarity thresholds arranged from small to large are all 0.01; that is, in the embodiment of the present disclosure, multiple similarity thresholds can be traversed and selected from 0.1 to 0.9 with a step size of 0.01.

[0107] In the embodiment of the present disclosure, after obtaining a plurality of predetermined similarity thresholds, clustering may be performed for each of the plurality of similarity thresholds. It is understandable that different clustering results may be obtained for different similarity thresholds.

[0108] In the embodiment of the present disclosure, the clustering accuracy corresponding to each similarity threshold can be determined based on the clustering results corresponding to each similarity threshold, and then, based on the clustering accuracy corresponding to each similarity threshold, multiple second similarity thresholds for which the clustering accuracy reaches the set accuracy index can be determined; after obtaining multiple second similarity thresholds, the largest similarity threshold can be selected as the first similarity threshold.

[0109] It can be seen that the embodiments of the present disclosure can realize the search for difficult samples from any data set; further, since the first similarity threshold is used to cluster faces, the clustering accuracy can reach the set accuracy index. Therefore, the clustering results of multiple samples obtained based on the first similarity threshold are clustering results that reach the set accuracy index. On this basis, the difficult samples determined based on the clustering results of multiple samples can better reflect the defects of the face clustering method. Furthermore, training a neural network based on the difficult samples determined based on the clustering results of multiple samples can enable the trained neural network to more accurately correct the similarity of facial features.

[0110] In the disclosed embodiment, since the first similarity threshold is the largest similarity threshold among multiple second similarity thresholds, and the second similarity threshold is used to perform face clustering, the clustering accuracy can reach the set accuracy index. Therefore, the clustering results of multiple samples obtained based on the first similarity threshold are clustering results that reach the set accuracy index. On this basis, the difficult samples determined based on the clustering results of multiple samples can better reflect the defects of the face clustering method. Furthermore, training the neural network based on the difficult samples determined based on the clustering results of multiple samples can enable the trained neural network to more accurately correct the similarity of facial features.

[0111] In some embodiments, determining the difficult sample from the samples that meet the set conditions may include: determining the difficult sample from the samples that meet the set conditions in a positive-negative sample balance manner, wherein the difficult sample includes a positive sample and a negative sample.

[0112] In the disclosed embodiment, the positive and negative sample balance means that the ratio of positive samples to negative samples is within a set ratio range, where the set ratio range may be 40% to 60%; that is, in difficult samples, the number of positive samples and negative samples is relatively close.

[0113] It can be understood that if the number of positive samples in the difficult samples is much larger than the number of negative samples, the trained neural network is more inclined to correct the facial feature similarity of a pair of images labeled as the same face, and the ability to correct the facial feature similarity of a pair of images labeled as different faces may be insufficient; conversely, if the number of negative samples in the difficult samples is much larger than the number of positive samples, the trained neural network is more inclined to correct the facial feature similarity of a pair of images labeled as different faces, and the ability to correct the facial feature similarity of a pair of images labeled as the same face may be insufficient; relatively, the embodiment of the present disclosure adopts a positive and negative sample balance method to determine the difficult samples from the samples that meet the set conditions, so that the trained neural network has a higher ability to correct the facial feature similarity of a pair of images of different faces, as well as a pair of images of the same face.

[0114] The following is an illustrative description of the training process of the neural network.

[0115] Figure 3 is a flowchart of a neural network training method according to an embodiment of the present disclosure, such as Figure 3 As shown, the process may include:

[0116] Step 301: Obtain difficult samples.

[0117] In the embodiment of the present disclosure, the implementation method of this step has been explained in the above-mentioned embodiment and will not be repeated here.

[0118] Step 302: input the difficult sample into the neural network; use the neural network to process the difficult sample to obtain the processing result of the difficult sample.

[0119] Step 303: adjusting the network parameter values ​​of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample.

[0120] Here, the loss of the neural network can be determined according to the processing results of the difficult samples and the pre-acquired labeling information of the difficult samples, and then the network parameter values ​​of the neural network can be adjusted according to the loss of the neural network.

[0121] In some embodiments, the hinge loss of the neural network can be determined based on the processing results of the difficult samples and the pre-acquired labeling information of the difficult samples; then, the network parameter values ​​of the neural network can be adjusted based on the hinge loss of the neural network.

[0122] In some embodiments, the hinge loss of the neural network may be determined by:

[0123] The hinge loss of the neural network is determined according to the processing result of the difficult sample, the pre-acquired labeling information of the difficult sample and the first similarity threshold.

[0124] In some embodiments, the processing result of the difficult sample represents the similarity correction value of the facial features in the difficult sample. In this case, the hinge loss can be calculated by formula (2).

[0125] L = max(0,margin-y ij (x i T x j -σ i -σ j -bias)) (2)

[0126] Wherein, L represents hinge loss, and margin can be an empirically set value, for example, margin = 0.1; x i and x j Represents the facial features of a pair of images in a difficult sample, x i T x j Represents x i and x j The similarity between ij Represents x i and x j The corresponding annotation information is in x i and x j When y is a feature belonging to the same face, ij =1; at x i and x j When y is a feature belonging to different faces, ij = -1; σ i Represents x i The corresponding similarity correction value, σ j Represents x j The corresponding similarity correction value, σ i With σ j The sum represents the similarity correction value of the facial features of a pair of images in the difficult sample; i and j are both integers greater than or equal to 1, and i is not equal to j; bias represents the above-mentioned first similarity threshold.

[0127] Step 304: determine whether the neural network after the network parameter values ​​are adjusted meets the training end condition. If not, re-execute steps 301 to 304; if yes, execute step 305.

[0128] In the disclosed embodiment, the training end condition may be that the number of iterations in training the neural network reaches a set number, or that the loss of the neural network after the network parameter value is adjusted is less than the set loss; here, the set number and the set loss may be pre-set.

[0129] In some embodiments, when the loss of the neural network is the hinge loss, and the processing result of the difficult sample represents the similarity correction value of the facial features in the difficult sample, when x i and x j When y is a feature belonging to different faces, ij = -1, so at x i T x j -σ i -σ j When -bias is less than -margin, the value of the hinge loss is 0, that is, there is no need to adjust σ i With σ j After optimization, it can be considered that the neural network after the network parameter value is adjusted meets the training end condition; when x i and x j When y is a feature belonging to the same face, ij =1, so at x i T x j -σ i -σ j When -bias is greater than margin, the value of the hinge loss is 0, that is, there is no need to adjust σ i With σ j After optimization, it can be considered that the neural network after the network parameter values ​​are adjusted meets the training end conditions.

[0130] Step 305: The neural network after adjusting the network parameter values ​​is used as the trained neural network.

[0131] In practical applications, steps 301 to 305 may be implemented using a processor in an electronic device, and the processor may be at least one of an ASIC, a DSP, a DSPD, a PLD, a FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

[0132] It can be seen that in the disclosed embodiment, by pre-training the neural network, the trained neural network can more accurately correct the similarity of facial features, and then, on the basis of correcting the similarity of facial features, it is helpful to improve the clustering accuracy of subsequent face clustering.

[0133] In some embodiments, the similarity of facial features in the images to be clustered can be calculated using formula (3).

[0134]

[0135] Among them, s ij represents feature similarity, Represents the corrected feature similarity.

[0136] It can be seen that in the embodiments of the present disclosure, no matter what type of facial image to be clustered belongs to, the pre-trained neural network can be used to correct the similarity of facial features in the image to be clustered, and perform face clustering based on the corrected feature similarity. That is to say, the embodiments of the present disclosure can adaptively correct the similarity of facial features based on the facial features in the image to be clustered, so that for various types of facial images, the face clustering accuracy can be improved by correcting the similarity of facial features. That is, the embodiments of the present disclosure can be adapted to the clustering of various types of faces.

[0137] Based on the contents recorded in the foregoing embodiments, the technical solution of the embodiments of the present disclosure is further illustrated by an example.

[0138] In some embodiments, the face clustering method trained with a large number of images of adult yellow people can achieve good face clustering of adult yellow people, but cannot achieve good face clustering of the elderly, children and black people.

[0139] To solve this technical problem, in the embodiment of the present disclosure, a data set D with annotation information can be obtained in advance. b , then we can get the b Difficult samples are selected to train the neural network; the difficult samples are samples in at least one category of face data set.

[0140] In some embodiments, from the dataset D b Selecting difficult samples can be achieved in the following ways:

[0141] From 0.1 to 0.9, multiple similarity thresholds are selected with a step size of 0.01; for the data set D b, feature pairs of face images can be selected based on the DBSCAN clustering method; for each similarity threshold, the DBSCAN clustering method can be used for clustering processing to obtain the prediction results of the feature pairs. Here, if the similarity of the feature pairs is greater than or equal to the similarity threshold, the prediction result of the feature pairs is a positive pair (features belonging to the same face); conversely, if the similarity of the feature pairs is less than the similarity threshold, the prediction result of the feature pairs is a negative pair (features belonging to different faces); then, the error rate of face clustering of the DBSCAN clustering method can be determined based on the difference between the annotation information of the feature pairs and the prediction results. Here, the lower the error rate of face clustering of the DBSCAN clustering method, the higher the accuracy of face clustering of the DBSCAN clustering method.

[0142] Among the various similarity thresholds, the similarity threshold that makes the error rate of face clustering of the DBSCAN method the lowest is determined as the above-mentioned first similarity threshold.

[0143] Based on the first similarity threshold, a difficult sample is obtained according to the method described in the above embodiment.

[0144] In some embodiments, a dataset D of elderly face images may be obtained. old , a dataset of children’s face images D child , dataset D of black face images black And a dataset D of common face images (which can be face images of various types of faces) general After obtaining the trained neural network based on the difficult samples, the trained neural network can be used to train the dataset D old , Dataset D child , Dataset D black and dataset D general The facial feature similarity of the facial images in the image is corrected, and DBSCAN clustering is performed based on the corrected feature similarity; for example, when judging whether two facial images of black people belong to the same person, the facial feature similarity of the two black people can be corrected based on the method of the embodiment of the present disclosure, and DBSCAN clustering is performed based on the corrected facial feature similarity.

[0145] Based on the contents recorded in the above embodiments, the present disclosure also proposes another image processing method: Figure 4 FIG. 1 is a flow chart of another clustering method according to an embodiment of the present disclosure. Figure 4 As shown, the process may include:

[0146] Step 401: Obtain a neural network for obtaining feature similarity, wherein the neural network is trained based on difficult samples, and the difficult samples are samples in at least one category of face data set; the feature similarity represents the similarity of facial features between the image to be clustered and the corresponding category image.

[0147] In the disclosed embodiment, the difficult sample is processed using the above neural network to obtain a processing result of the difficult sample; the above neural network may be a neural network for obtaining the similarity of facial features, and the processing result of the difficult sample represents the similarity of facial features in the difficult sample.

[0148] Step 402: Processing the to-be-clustered images based on the neural network to obtain the feature similarity, wherein the feature similarity is obtained based on similarity correction values ​​of facial features of different categories.

[0149] In the disclosed embodiment, similarity correction values ​​of different types of facial features can be obtained according to the contents recorded in the above embodiments. Then, a neural network can be used to process the clustered images and the similarity correction values ​​corresponding to the clustered images to obtain the above feature similarity.

[0150] Step 403: Based on the feature similarity, clustering processing is performed on the images to be clustered to obtain clustering results of the images to be clustered and images of corresponding categories.

[0151] In the embodiments of the present disclosure, a face clustering method can be used to perform clustering processing on the image to be clustered, and the initial similarity of the facial features in the image to be clustered can be obtained; then, the above feature similarity can be used to replace the initial similarity of the facial features in the image to be clustered; then, based on the above feature similarity, a face clustering method can be used to perform clustering processing on the image to be clustered, and the clustering results of the image to be clustered and the corresponding category image can be obtained.

[0152] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0153] Based on the clustering method proposed in the above-mentioned embodiment, the embodiment of the present disclosure proposes a clustering device.

[0154] Figure 5 FIG. 1 is a schematic diagram of a clustering device according to an embodiment of the present disclosure. Figure 5 As shown, the device may include a first acquisition module 500, a first processing module 501, a second processing module 502 and a third processing module 503, wherein:

[0155] A first acquisition module 500 is used to acquire a neural network for correcting similarity of facial features, wherein the neural network is trained based on difficult samples; the difficult samples are samples in at least one category of face data set;

[0156] A first processing module 501 is used to process the image to be clustered based on the neural network to obtain a similarity correction value of the facial features of the image to be clustered;

[0157] The second processing module 502 is used to correct the feature similarity using the similarity correction value to obtain the corrected feature similarity; the feature similarity represents the similarity of the facial features between the image to be clustered and the image of the corresponding category;

[0158] The third processing module 503 is used to perform clustering processing on the images to be clustered based on the corrected feature similarity to obtain clustering results of the images to be clustered and the corresponding category images.

[0159] In some embodiments, the first acquisition module 500 is further used to:

[0160] Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets;

[0161] Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index;

[0162] Based on the first similarity threshold, clustering the samples of the multiple categories of face data sets to obtain clustering results of the samples of the multiple categories of face data sets;

[0163] Based on the clustering results of the samples of the multiple categories of face data sets, samples that meet the set conditions are determined, and the difficult samples are determined from the samples that meet the set conditions. The set conditions include: the clustering results of the samples are different from the labeling information of the samples.

[0164] In some embodiments, the first acquisition module 500 is used to determine the first similarity threshold, including:

[0165] Based on each similarity threshold of a plurality of predetermined similarity thresholds, clustering the samples to obtain a clustering result corresponding to each similarity threshold;

[0166] Based on the clustering results corresponding to each similarity threshold, determining that the clustering accuracy reaches multiple second similarity thresholds of the set accuracy index;

[0167] Among the multiple second similarity thresholds, the largest similarity threshold is determined as the first similarity threshold.

[0168] In some embodiments, the first acquisition module 500 is used to determine the difficult sample from the samples that meet the set conditions, including:

[0169] The difficult samples are determined from the samples that meet the set conditions in a positive-negative sample balance manner, wherein the difficult samples include positive samples and negative samples, the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces.

[0170] In some embodiments, the apparatus further comprises a first training module, wherein the first training module is configured to:

[0171] Inputting the difficult sample into the neural network;

[0172] Processing the difficult sample using the neural network to obtain a processing result of the difficult sample, wherein the processing result of the difficult sample represents a similarity correction value of a facial feature in the difficult sample;

[0173] According to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, the network parameter value of the neural network is adjusted.

[0174] In some embodiments, the first training module is used to adjust the network parameter value of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, including:

[0175] Determining a hinge loss of the neural network according to a processing result of the difficult sample and pre-acquired labeling information of the difficult sample;

[0176] According to the hinge loss, the network parameter values ​​of the neural network are adjusted.

[0177] In some embodiments, the first training module is further used to:

[0178] Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets;

[0179] Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index;

[0180] In some embodiments, the first training module is used to determine the hinge loss of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, including:

[0181] The hinge loss of the neural network is determined according to the processing result of the difficult sample, the pre-acquired labeling information of the difficult sample and the first similarity threshold.

[0182] In some embodiments, the third processing module 503 is used to perform clustering processing on the image to be clustered based on the corrected feature similarity to obtain a clustering result of the image to be clustered and the corresponding category image, including:

[0183] Based on the corrected feature similarity, the face clustering method is used to perform clustering processing on the image to be clustered to obtain a clustering result of the image to be clustered and the corresponding category image, wherein the corrected feature similarities corresponding to the images to be clustered of different categories are different.

[0184] Based on the clustering method proposed in the above-mentioned embodiment, another clustering device is proposed in the embodiment of the present disclosure.

[0185] Figure 6 FIG. 1 is a schematic diagram of the composition structure of another clustering device according to an embodiment of the present disclosure. Figure 6 As shown, the device may include a second acquisition module 600, a fourth processing module 601 and a fifth processing module 602, wherein:

[0186] The second acquisition module 600 is used to acquire a neural network for obtaining feature similarity, wherein the neural network is obtained by training based on difficult samples, and the difficult samples are samples in at least one category of face data set; the feature similarity represents the similarity of face features between the image to be clustered and the image of the corresponding category;

[0187] The fourth processing module 601 is used to process the to-be-clustered images based on the neural network to obtain the feature similarity, wherein the feature similarity is obtained based on similarity correction values ​​of facial features of different categories.

[0188] The fifth processing module 602 is used to perform clustering processing on the image to be clustered based on the feature similarity to obtain a clustering result of the image to be clustered and the corresponding category image.

[0189] In some embodiments, the second acquisition module 600 is further used to:

[0190] Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets;

[0191] Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index;

[0192] Based on the first similarity threshold, clustering the samples of the multiple categories of face data sets to obtain clustering results of the samples of the multiple categories of face data sets;

[0193] Based on the clustering results of the samples of the multiple categories of face data sets, samples that meet the set conditions are determined, and the difficult samples are determined from the samples that meet the set conditions. The set conditions include: the clustering results of the samples are different from the labeling information of the samples.

[0194] In some embodiments, the second acquisition module 600 is used to determine the first similarity threshold, including:

[0195] Based on each similarity threshold of a plurality of predetermined similarity thresholds, clustering the samples to obtain a clustering result corresponding to each similarity threshold;

[0196] Based on the clustering results corresponding to each similarity threshold, determining that the clustering accuracy reaches multiple second similarity thresholds of the set accuracy index;

[0197] Among the multiple second similarity thresholds, the largest similarity threshold is determined as the first similarity threshold.

[0198] In some embodiments, the second acquisition module 600 is used to determine the difficult sample from the samples that meet the set conditions, including:

[0199] The difficult samples are determined from the samples that meet the set conditions in a positive-negative sample balance manner, wherein the difficult samples include positive samples and negative samples, the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces.

[0200] In some embodiments, the apparatus further comprises a second training module, wherein the second training module is configured to:

[0201] Inputting the difficult sample into the neural network;

[0202] Processing the difficult sample using the neural network to obtain a processing result of the difficult sample;

[0203] According to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, the network parameter value of the neural network is adjusted.

[0204] In some embodiments, the second training module is used to adjust the network parameter value of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, including:

[0205] Determining a hinge loss of the neural network according to a processing result of the difficult sample and pre-acquired labeling information of the difficult sample;

[0206] According to the hinge loss, the network parameter values ​​of the neural network are adjusted.

[0207] In some embodiments, the second training module is further used to:

[0208] Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets;

[0209] Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index;

[0210] In some embodiments, the second training module is used to determine the hinge loss of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, including:

[0211] The hinge loss of the neural network is determined according to the processing result of the difficult sample, the pre-acquired labeling information of the difficult sample and the first similarity threshold.

[0212] In actual applications, the first acquisition module 500, the first processing module 501, the second processing module 502, the third processing module 503, the second acquisition module 600, the fourth processing module 601 and the fifth processing module 602 can all be implemented using a processor in an electronic device, and the above processor can be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

[0213] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

[0214] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0215] Specifically, a computer program instruction corresponding to a clustering method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the computer program instruction corresponding to a clustering method in the storage medium is read or executed by an electronic device, any clustering method of the aforementioned embodiments is implemented.

[0216] Based on the same technical concept as the above embodiments, see Figure 7 , which shows an electronic device 7 provided by an embodiment of the present disclosure, which may include: a memory 701 and a processor 702; wherein,

[0217] The memory 701 is used to store computer programs and data;

[0218] The processor 702 is configured to execute the computer program stored in the memory to implement any one of the clustering methods in the foregoing embodiments.

[0219] In practical applications, the memory 701 may be a volatile memory, such as RAM; or a non-volatile memory, such as ROM, flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 702.

[0220] The processor 702 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understandable that for different devices, the electronic device used to implement the processor function may also be other, which is not specifically limited in the embodiments of the present disclosure.

[0221] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0222] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0223] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0224] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0225] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0226] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0227] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A clustering method, characterized in that: The method comprises: Obtain a neural network for correcting the similarity of facial features, wherein the neural network is obtained by training based on difficult samples, the difficult samples are samples in at least one category of face data set; the difficult samples are determined according to clustering results of samples in multiple categories of face data sets, and the accuracy of the clustering results meets a set accuracy index; the difficult samples represent samples for which a correct clustering result cannot be obtained; the difficult samples include positive samples and / or negative samples, the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces; Processing the images to be clustered based on the neural network to obtain similarity correction values ​​of facial features of the images to be clustered; Correcting the feature similarity using the similarity correction value to obtain a corrected feature similarity, wherein the feature similarity represents the similarity of the facial features between the image to be clustered and the image of the corresponding category; Based on the corrected feature similarity, clustering processing is performed on the images to be clustered to obtain clustering results of the images to be clustered and the corresponding category images.

2. The method according to claim 1, characterized in that The method further comprises: Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets; Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index; Based on the first similarity threshold, clustering the samples of the multiple categories of face data sets to obtain clustering results of the samples of the multiple categories of face data sets; Based on the clustering results of the samples of the multiple categories of face data sets, samples that meet the set conditions are determined, and the difficult samples are determined from the samples that meet the set conditions. The set conditions include: the clustering results of the samples are different from the labeling information of the samples.

3. The method according to claim 2, characterized in that The determining of the first similarity threshold comprises: Based on each similarity threshold of a plurality of predetermined similarity thresholds, clustering the samples to obtain a clustering result corresponding to each similarity threshold; Based on the clustering results corresponding to each similarity threshold, determining that the clustering accuracy reaches multiple second similarity thresholds of the set accuracy index; Among the multiple second similarity thresholds, the largest similarity threshold is determined as the first similarity threshold.

4. The method according to claim 2, characterized in that: The step of determining the difficult sample from the samples satisfying the set condition comprises: The difficult samples are determined from the samples that meet the set conditions by balancing positive and negative samples.

5. The method according to claim 1, characterized in that The neural network is trained based on difficult samples and includes: Inputting the difficult sample into the neural network; Processing the difficult sample using the neural network to obtain a processing result of the difficult sample, wherein the processing result of the difficult sample represents a similarity correction value of a facial feature in the difficult sample; According to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample, the network parameter value of the neural network is adjusted.

6. The method according to claim 5, characterized in that The adjusting the network parameter value of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample includes: Determining a hinge loss of the neural network according to a processing result of the difficult sample and pre-acquired labeling information of the difficult sample; According to the hinge loss, the network parameter values ​​of the neural network are adjusted.

7. The method according to claim 6, characterized in that The method further comprises: Acquire samples of multiple categories of face data sets that need to be clustered and labeling information of each sample in the samples of the multiple categories of face data sets; Determine a first similarity threshold, wherein the first similarity threshold represents a similarity threshold that enables the accuracy of the clustering results of the samples of the plurality of categories of face data sets to reach a set accuracy index; The determining the hinge loss of the neural network according to the processing result of the difficult sample and the pre-acquired labeling information of the difficult sample includes: The hinge loss of the neural network is determined according to the processing result of the difficult sample, the pre-acquired labeling information of the difficult sample and the first similarity threshold.

8. The method according to claim 1, characterized in that The clustering process is performed on the to-be-clustered images based on the corrected feature similarity to obtain clustering results of the to-be-clustered images, including: Based on the corrected feature similarity, a face clustering method is used to perform clustering processing on the images to be clustered to obtain clustering results of the images to be clustered and the corresponding category images, wherein the corrected feature similarities corresponding to the images to be clustered of different categories are different.

9. A clustering method, characterized in that: The method comprises: Obtain a neural network for obtaining feature similarity, wherein the neural network is obtained by training based on difficult samples, the difficult samples are samples in at least one category of face data set; the difficult samples are determined according to clustering results of samples in multiple categories of face data sets, and the accuracy of the clustering results meets a set accuracy index; the difficult samples represent samples for which a correct clustering result cannot be obtained; the difficult samples include positive samples and / or negative samples, the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces; the feature similarity represents the similarity of facial features between the image to be clustered and the image of the corresponding category; Processing the to-be-clustered images based on the neural network to obtain the feature similarity, wherein the feature similarity is obtained based on similarity correction values ​​of facial features of different categories; Based on the feature similarity, clustering processing is performed on the images to be clustered to obtain clustering results of the images to be clustered and the corresponding category images.

10. A clustering device, characterized in that: The device comprises: A first acquisition module is used to acquire a neural network for correcting the similarity of facial features, wherein the neural network is obtained by training based on difficult samples; the difficult samples are samples in at least one category of face data set; the difficult samples are determined based on clustering results of samples in multiple categories of face data sets, and the accuracy of the clustering results meets a set accuracy index; the difficult samples represent samples for which a correct clustering result cannot be obtained; the difficult samples include positive samples and / or negative samples, wherein the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces; A first processing module, used for processing the images to be clustered based on the neural network to obtain similarity correction values ​​of facial features of the images to be clustered; A second processing module is used to correct the feature similarity using the similarity correction value to obtain the corrected feature similarity; the feature similarity represents the similarity of the facial features between the image to be clustered and the image of the corresponding category; The third processing module is used to perform clustering processing on the images to be clustered based on the corrected feature similarity to obtain clustering results of the images to be clustered and the corresponding category images.

11. A clustering device, characterized in that: The device comprises: A second acquisition module is used to acquire a neural network for obtaining feature similarity, wherein the neural network is obtained by training based on difficult samples; the difficult samples are samples in at least one category of face data set; the difficult samples are determined based on clustering results of samples in multiple categories of face data sets, and the accuracy of the clustering results meets the set accuracy index; the difficult samples represent samples for which a correct clustering result cannot be obtained; the difficult samples include positive samples and / or negative samples, the positive samples represent a pair of sample images labeled as the same face, and the negative samples represent a pair of sample images labeled as different faces; the feature similarity represents the similarity of facial features between the image to be clustered and the image of the corresponding category; A fourth processing module, used for processing the to-be-clustered images based on the neural network to obtain the feature similarity; wherein the feature similarity is obtained based on the similarity correction values ​​of facial features of different categories; The fifth processing module is used to perform clustering processing on the images to be clustered based on the feature similarity to obtain clustering results of the images to be clustered and the images of the corresponding category.

12. An electronic device, characterized in that: comprising a processor and a memory for storing a computer program capable of running on the processor; wherein, The processor is configured to run the computer program to execute the clustering method according to any one of claims 1 to 9.

13. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the clustering method according to any one of claims 1 to 9 is implemented.

Images