Image cluster centroid generation method, device and storage medium

Abstract

This application discloses an image clustering centroid generation method, device, and storage medium. The image clustering centroid generation method includes: extracting image features from each sample image in a sample image set to obtain original features; masking the image features corresponding to some style sample images in the original features to obtain masked image features; inputting the masked image features into a model to be trained to obtain sample cluster centroids output by the model; reconstructing image features of each style sample image based on the sample cluster centroids to obtain reconstructed features; and optimizing the model parameters of the model to be trained based on the error between the original features and the reconstructed features to obtain a trained clustering centroid generation model. This improves the information completion capability of the clustering centroid generation model for images with unknown styles. Even if the image set to be processed contains images with few styles, it can still generate image cluster centroids with the ability to recall images of all styles, thus improving the quality of the generated image cluster centroids.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for generating centroids of image clustering. Background Technology

[0002] Image clustering can divide images of different target objects into different image sets, and the core of image clustering algorithms lies in the selection of cluster centroids.

[0003] Because the robustness of features is affected by factors such as image lighting conditions and the pose of the target object, the similarity between different images corresponding to the same target object can vary greatly. If the generated cluster centroids are inaccurate, images of the same target object can easily be classified into multiple different image sets. Existing technologies also use all images as cluster centroids, which can improve clustering accuracy, but the computational cost is enormous, and some images are very similar, so repeatedly calculating on similar images is a waste of resources.

[0004] Therefore, how to efficiently generate the most suitable and accurate image cluster centroids is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides at least one method, apparatus, and storage medium for generating image cluster centroids.

[0006] The first aspect of this application provides a method for generating image cluster centroids. The method includes: extracting image features from each sample image in a sample image set to obtain original features; wherein the sample image set contains multiple sample images of different styles corresponding to the same target object; masking the image features corresponding to some style sample images in the original features to obtain masked image features; inputting the masked image features into a model to be trained to obtain sample cluster centroids output by the model to be trained; reconstructing the image features of each style sample image based on the sample cluster centroids to obtain reconstructed features; optimizing the model parameters of the model to be trained based on the error between the original features and the reconstructed features to obtain a trained cluster centroid generation model; and inputting the image set to be processed into the cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model.

[0007] In one embodiment, the method further includes: splitting multiple sample image sets into multiple parts, with each sample image set being split into multiple sub-sample sets; using the sub-sample sets obtained from splitting the same sample image set as positive samples of the merged business, and using the sub-sample sets obtained from splitting different sample image sets as negative samples of the merged business; and using the positive and negative samples of the merged business to train a cluster centroid generation model to obtain a cluster centroid generation model corresponding to the merged business.

[0008] In one embodiment, inputting a set of images to be processed into a cluster centroid generation model to obtain image cluster centroids output by the cluster centroid generation model includes: inputting multiple sets of images to be processed into a cluster centroid generation model corresponding to a merging service to obtain image cluster centroids corresponding to the multiple sets of images to be processed output by the cluster centroid generation model corresponding to the merging service; the method further includes: calculating the similarity between the image cluster centroids of the multiple sets of images to be processed to obtain a first similarity; and merging the sets of images to be processed whose first similarity satisfies a preset condition.

[0009] In one embodiment, the method further includes: randomly selecting a sample image from the sample image set to obtain a sample image of the same set corresponding to the sample image set; and randomly selecting a sample image from other sample image sets to obtain a sample image of a different set corresponding to the sample image set; using the sample image of the same set and the sample image set as positive samples for retrieval services, and using the sample image of a different set and the sample image set as negative samples for retrieval services; and using the positive samples and negative samples for retrieval services to train a cluster centroid generation model to obtain a cluster centroid generation model corresponding to the retrieval service.

[0010] In one embodiment, the method of inputting the set of images to be processed into a cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model includes: inputting the set of images to be processed into a cluster centroid generation model corresponding to the retrieval service to obtain the image cluster centroids output by the cluster centroid generation model corresponding to the retrieval service; the method further includes: acquiring the image to be retrieved, extracting the image features of the image to be retrieved to obtain the retrieved image features; calculating the similarity between the retrieved image features and the image cluster centroids to obtain a second similarity; if the second similarity meets a preset condition, then the set of images to be processed is used as the retrieval result of the image to be retrieved.

[0011] In one embodiment, masking the image features corresponding to a portion of the style sample images in the original features to obtain masked image features includes: obtaining the training progress of the model to be trained; determining the number of styles to be masked based on the training progress; and masking the image features of the number of styles to be masked in the original features to obtain masked image features.

[0012] In one embodiment, the image features corresponding to some style sample images in the original features are masked to obtain masked image features, including: calculating the misclassification rate corresponding to different style images to obtain the image misclassification rate corresponding to each style; wherein, the misclassification rate refers to the probability that the image of the target object is incorrectly assigned to the image set of other target objects; determining the style to be masked based on the image misclassification rate corresponding to each style; and masking the image features of the style to be masked in the original features to obtain masked image features.

[0013] In one embodiment, the method further includes: obtaining image features corresponding to images of different styles in the image set to be processed, and obtaining style image features; calculating the similarity between the image cluster centroids output by the cluster centroid generation model and the image features of each style, and obtaining multiple style similarities; if any style similarity is lower than a preset similarity threshold, then generating an image cluster centroid corresponding to the style based on the image of the style corresponding to any style similarity.

[0014] A second aspect of this application provides an image clustering centroid generation apparatus, comprising: a feature extraction module for extracting image features from each sample image in a sample image set to obtain original features; wherein the sample image set contains multiple sample images of different styles corresponding to the same target object; a feature masking module for masking the image features corresponding to some style sample images in the original features to obtain masked image features; a sample centroid generation module for inputting the masked image features into a model to be trained to obtain sample clustering centroids output by the model to be trained; a model optimization module for reconstructing image features of each style sample image based on the sample clustering centroids to obtain reconstructed features, and optimizing the model parameters of the model to be trained based on the error between the original features and the reconstructed features to obtain a trained clustering centroid generation model; and a centroid extraction module for inputting the image set to be processed into the clustering centroid generation model to obtain the image clustering centroids output by the clustering centroid generation model.

[0015] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the above-described image clustering centroid generation method.

[0016] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described image clustering centroid generation method.

[0017] The above scheme extracts image features from each sample image in the sample image set to obtain original features. It then masks the image features corresponding to some style sample images in the original features to obtain masked image features. These masked image features are input into the model to be trained to obtain the sample cluster centroids output by the model. Based on these centroids, the image features of each style sample image are reconstructed to obtain reconstructed features. The model parameters of the model to be trained are optimized based on the error between the original features and the reconstructed features, resulting in a trained centroid generation model. This improves the centroid generation model's ability to complete information for images of unknown styles. The image set to be processed is input into the centroid generation model to obtain the image cluster centroids output by the model. Even if the image set contains images with few styles, it can generate image cluster centroids with the ability to recall images of all styles, thus improving the quality of the generated image cluster centroids.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0020] Figure 1 This is a schematic diagram illustrating the implementation environment of the solution in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating an exemplary embodiment of the image clustering centroid generation method of this application; Figure 3 This is a schematic diagram illustrating a training model for generating cluster centroids, as shown in an exemplary embodiment of this application. Figure 4 This is a block diagram illustrating an image clustering centroid generation apparatus according to an exemplary embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application; Figure 6 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0021] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0022] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0023] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0024] Each image set contains at least one image feature, which is called a sub-feature. Correspondingly, there are one or more principal features. The principal features need to represent the image set well, and the principal features are also called cluster centroids.

[0025] Image clustering generally has two processing modes: offline mode and online mode.

[0026] Offline mode processes a batch of image data at a time, which has high accuracy and computation time. It can generally adopt clustering methods such as hierarchical clustering, spectral clustering, density clustering, and graph neural network clustering.

[0027] The online mode compares a set of images with an existing set of images to obtain the merged result, which is highly real-time.

[0028] For large-scale image clustering, online mode is indispensable. If only offline mode is used, the computational load will be huge due to the gradual increase in the number of accumulated images, and the clustering timeliness will be poor, and the clustering results cannot be obtained in a timely manner.

[0029] Therefore, a combination of offline and online modes is used. For example, after performing offline clustering on a day's image data to form a new set of images, online clustering is then used to merge the new set of images with the existing historical set of images to reduce computation and ensure timeliness.

[0030] In addition, image set retrieval can be performed, which searches for a cluster of images based on a specific image. Of course, many other business analyses can also be conducted.

[0031] Image set merging mainly involves calculating the similarity between cluster centroids, while image set retrieval mainly involves calculating the similarity between image features and cluster centroids. Therefore, for these clustering services, generating the most accurate and appropriate cluster centroids is a key point.

[0032] In existing technologies, image cluster centroids are generally generated based on prior knowledge and the mean of image features. However, their accuracy in specific business applications is poor. In order to efficiently generate the most suitable and accurate image cluster centroids, this application provides an image cluster centroid generation method, device and storage medium.

[0033] The image clustering centroid generation method provided in the embodiments of this application will be described below.

[0034] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating an implementation environment of the scheme according to an exemplary embodiment of this application. The implementation environment may include an image provider 110 and a server 120, which are interconnected.

[0035] The number of image providers 110 can be one or more. Image providers 110 can be cameras, smartphones, tablets, laptops, desktop computers, etc., but are not limited to these.

[0036] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0037] In one example, server 120 can train a cluster centroid generation model based on the sample image set obtained from image provider 110. Of course, server 120 can deploy the cluster centroid generation model locally, to image provider 110, or to other terminals or servers to generate image cluster centroids for the image set to be processed based on the cluster centroid generation model.

[0038] The image clustering centroid generation method provided in this application embodiment can be executed by an image provider 110, a server 120, or by the image provider 110 and the server 120 in an interactive manner. That is, some steps of the method are executed by the image provider 110 and other steps are executed by the server 120. This application does not limit the execution subject.

[0039] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the image clustering centroid generation method of this application, as shown below. Figure 2 As shown, the image clustering centroid generation method includes at least steps S210 to S250, which are described in detail below: Step S210: Extract the image features of each sample image in the sample image set to obtain the original features; wherein, the sample image set contains multiple sample images of different styles corresponding to the same target object.

[0040] Obtain multiple sample image sets, each containing multiple sample images belonging to the same target object and including different styles.

[0041] The target objects include, but are not limited to, people, and / or cats, and / or buildings.

[0042] Image style refers to the unique visual expression of a type of image. For example, image style includes: clear, blurry, first-angle shot, second-angle shot, subject wearing a mask, subject wearing glasses, etc.

[0043] To improve the model training effect, a diverse collection of images with different styles is selected as training samples to ensure that the cluster centroids generated by the trained model can recall images of various styles.

[0044] Image features are extracted from each sample image in the sample image set, and the original features are obtained by combining the extracted image features from each sample image.

[0045] Image features refer to mathematical vectors extracted from an image that can characterize the image content (such as shape, and / or texture, and / or structure, and / or color distribution).

[0046] Among them, the original features refer to image features that contain information about all style sample images in the sample image set.

[0047] Step S220: Mask the image features corresponding to some style sample images in the original features to obtain masked image features.

[0048] The image features of sample images with certain styles in the original features are masked, that is, the image features of sample images corresponding to certain styles in the original features are removed, while the image features of sample images corresponding to other styles are retained.

[0049] For example, the original features contain image features corresponding to sample images of the target object from multiple styles, including first angle, second angle, wearing a mask, and wearing glasses. The image features of the style type of wearing a mask and wearing glasses in the original features are masked, and only the image features of the style type of first angle and second angle are retained, resulting in masked image features.

[0050] It should be noted that the image features of all sample images corresponding to a certain style can be masked. However, considering that in practical application scenarios, there may be a situation where the number of images corresponding to a certain style in an image set is small, the image features of some sample images corresponding to a certain style can also be masked.

[0051] By masking the image features corresponding to some style sample images, we can simulate the situation where some image sets have low style richness in real-world application scenarios.

[0052] Step S230: Input the mask image features into the model to be trained to obtain the sample cluster centroids output by the model to be trained.

[0053] The model to be trained can be a pre-trained neural network model or an initialized neural network model; this application does not impose any restrictions on this.

[0054] The model to be trained can be constructed based on Convolutional Neural Network (CNN), and / or Transformer network, and / or Multilayer Perceptron (MLP), etc. The specific network structure can be flexibly selected according to the actual application scenario, and this application does not limit it.

[0055] The model to be trained outputs sample cluster centroids based on the features of the input mask image. The number of cluster centroids output by the model can be one or more, and the specific number of output centroids can be flexibly selected according to the actual application scenario. This application does not limit this number.

[0056] Step S240: Reconstruct the image features of each style of sample image based on the sample cluster centroid to obtain the reconstructed features. Optimize the model parameters of the model to be trained based on the error between the original features and the reconstructed features to obtain the trained cluster centroid generation model.

[0057] After obtaining the sample cluster centroids, the image features of each style of sample image are reconstructed based on the sample cluster centroids to reflect the quality of the sample cluster centroids.

[0058] It should be noted that each style here refers to all styles contained in the original features before the mask.

[0059] The larger the error between the original features and the reconstructed features, the worse the quality of the sample cluster centroids, meaning the worse the centroid extraction ability of the current model to be trained. Conversely, the smaller the error between the original features and the reconstructed features, the better the quality of the sample cluster centroids, meaning the better the centroid extraction ability of the current model to be trained.

[0060] The training loss of the model can be calculated based on the error between the original features and the reconstructed features. The model parameters of the model to be trained can be optimized by gradient descent, adaptive learning rate optimization algorithm, or second-order optimization method based on the training loss, until the training loss of the model converges or the number of iterations reaches the preset threshold, and the trained cluster centroid generation model is obtained.

[0061] Because the training process requires the cluster centroids output by the model to be trained to fit all styles, the model learns this ability, and can still obtain centroids containing information about all styles of images even if there are only a few image features with limited styles.

[0062] Step S250: Input the set of images to be processed into the cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model.

[0063] The image set to be processed refers to the set of images for which centroid generation is required. This image set can be obtained by clustering based on hierarchical clustering, spectral clustering, density clustering, graph neural network clustering, and other clustering methods.

[0064] The set of images to be processed is input into the cluster centroid generation model to obtain the cluster centroids of the images generated by the cluster centroid generation model.

[0065] The number of generated image cluster centroids can be flexibly determined according to the actual application scenario, and this application does not impose any limitation on this.

[0066] For an example application scenario, please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram illustrating a training cluster centroid generation model, as shown in an exemplary embodiment of this application. Figure 3 As shown, a transformer-based autoencoder is constructed, which includes a transformer encoder network and a transformer decoder network.

[0067] The sample image set contains N sample images, and the input to the transformer encoder network is a D1-dimensional raw feature composed of the image features of the N sample images. The original features are compressed into M D2-dimensional sample cluster centroids through a transformer encoder network. .

[0068] Then, the sample cluster centroid Y is reconstructed to [the correct value] using a transformer decoder network. .

[0069] During the training of the transformer decoder network, it can be used... The training objective is and If the error is sufficiently small, the training loss can be calculated using loss functions such as Mean Absolute Error (MAE) or Mean Square Error (MSE) during training; this application does not impose any restrictions on this. Taking MAE loss as an example: , Let represent the original features of the i-th sample image set, where Encode represents the transformer encoder network and Decode represents the transformer decoder network.

[0070] The number of cluster centroids M and the cluster centroid dimension D2 mentioned above can be adjusted according to actual computing resources and other circumstances, and this application does not impose any restrictions on them.

[0071] The original feature X is masked to obtain the masked image feature. This masked image feature is then input into the transformer encoder network for compression to obtain the sample cluster centroids. These centroids are then input into the transformer decoder network to reconstruct the reconstructed features. The training loss is calculated based on the error between the reconstructed and original features to optimize the transformer encoder network. Again, using MAE loss as an example: , mask refers to the masking operation.

[0072] The trained transformer encoder network is used as a cluster centroid generation model.

[0073] Of course, in addition to the examples mentioned above, other network structures can also be used to generate centroid clusters, and this application does not limit this to any particular model.

[0074] This application extracts image features from each sample image in a sample image set to obtain original features. It then masks the image features corresponding to some style sample images within the original features to obtain masked image features. These masked image features are input into a training model to obtain the sample cluster centroids output by the training model. Based on these sample cluster centroids, the image features of each style sample image are reconstructed to obtain reconstructed features. The model parameters of the training model are optimized based on the error between the original features and the reconstructed features to obtain a trained cluster centroid generation model. This improves the information completion capability of the cluster centroid generation model for images with unknown styles. The image set to be processed is input into the cluster centroid generation model to obtain the image cluster centroids output by the model. Even if the image set contains images with few styles, it can generate image cluster centroids with the ability to recall images of all styles, thus improving the quality of the generated image cluster centroids.

[0075] The following describes some embodiments of this application in detail.

[0076] In some implementations, step S220 involves masking the image features corresponding to a portion of the style sample images in the original features to obtain masked image features, including steps S2211 to S2213 below.

[0077] Step S2211: Obtain the training progress of the model to be trained.

[0078] For example, you can obtain the current convergence level of the model training loss and / or the number of iterations of the model to be trained, and thus obtain the training progress of the model to be trained.

[0079] Step S2212: Determine the number of styles to be masked based on the training progress.

[0080] For example, the smaller the number of iterations, the fewer the number of masking styles; conversely, the larger the number of iterations, the more masking styles.

[0081] For example, the greater the model training loss, the fewer styles to be masked; conversely, the smaller the model training loss, the more styles to be masked.

[0082] Step S2213: Mask the image features of the number of styles to be masked in the original features to obtain masked image features.

[0083] Select image features of a certain number of styles to be masked and mask them to obtain masked image features.

[0084] In the above embodiments, by determining the number of styles to be masked according to the training progress, the model can be trained progressively to improve the model training effect.

[0085] In some implementations, step S220 involves masking the image features corresponding to a portion of the style sample images in the original features to obtain masked image features, including steps S2221 to S2223 below.

[0086] Step S2221: Calculate the misclassification rate of images with different styles to obtain the misclassification rate of images with each style.

[0087] The false positive rate refers to the probability that an image of a target object is incorrectly assigned to an image set of other target objects.

[0088] For example, multiple image sets that have already been clustered can be obtained, and the number of misassigned images in the multiple image sets can be counted. The proportion of images of each style among the misassigned images can be obtained, and the misclassification rate of the image corresponding to each style can be obtained. If the proportion of misassigned images of a certain style is higher, the misclassification rate of the image of that style is higher; if the proportion of misassigned images of a certain style is lower, the misclassification rate of the image of that style is lower. In this case, the similarity between each image in the image set and the cluster centroid of the image set can be calculated to determine whether the image has been misassigned. The cluster centroid can be generated by the cluster centroid generation model trained by the above-mentioned other embodiments, and then further optimized and trained by combining the masking method of this embodiment; or it can be generated by other centroid generation methods. The generation method of cluster centroid used for similarity calculation in this application is limited.

[0089] For example, one could obtain multiple image sets that have already been clustered, count the number of images of each style and / or the difficulty of image recognition in the multiple image sets, estimate the misclassification rate of the corresponding style of images based on the number of images and / or the difficulty of image recognition, and obtain the misclassification rate of the images corresponding to that style. For example, if the number of images corresponding to a certain style is small and / or the difficulty of image recognition is high, then the misclassification rate of the images of that style is higher; if the number of images corresponding to a certain style is large and / or the difficulty of image recognition is low, then the misclassification rate of the images of that style is lower.

[0090] The specific method for recognizing the image false positive rate can be flexibly set according to the actual application scenario, and this application does not limit it.

[0091] Step S2222: Determine the style to be masked based on the image misclassification rate corresponding to each style.

[0092] For example, select a style whose image misclassification rate is higher than a preset misclassification rate threshold to obtain the style to be masked.

[0093] For example, select a preset number of styles with the highest image misclassification rate to obtain the style to be masked.

[0094] Step S2223: Mask the image features of the style to be masked in the original features to obtain masked image features.

[0095] After determining the style to be masked, the image features of the sample images of the corresponding style are masked to obtain the masked image features.

[0096] It should be noted that, in order to ensure that the cluster centroid generation model has recall capability for all styles, the image features of each style should be masked before model training. In the above implementation, after determining the style to be masked, the masking ratio of the image features of the style to be masked can be increased, that is, the original features are still used to mask other styles; or a random masking method can be used to mask the image features of each style in turn, and an initial cluster centroid generation model is pre-trained based on the masked image features. Then, by masking the image features of the style to be masked, the initial cluster centroid generation model is further enhanced and trained to obtain the final trained cluster centroid generation model, which enhances the recall capability for styles with high misclassification rates.

[0097] In some implementations, steps S2221 to S2223 and steps S2221 to S2223 above can be combined, that is, the number of styles to be masked and the styles to be masked are determined respectively, and the original features are masked to obtain masked image features.

[0098] In some implementations, the cluster centroid generation model can be further trained for specific business operations to obtain cluster centroid generation models adapted to different business operations.

[0099] Taking the merging of image sets as an example, the method further includes: splitting multiple sample image sets into multiple parts, and each sample image set is split into multiple sub-sample sets; using the sub-sample sets obtained from splitting the same sample image set as positive samples for the merging business, and using the sub-sample sets obtained from splitting different sample image sets as negative samples for the merging business; using the positive and negative samples for the merging business to train a cluster centroid generation model, and obtaining the cluster centroid generation model corresponding to the merging business.

[0100] Specifically, for positive samples in the merged business: the same set of sample images is randomly split into two sub-sample sets, and two input features are obtained for each sub-sample set. The two input features are then input into the cluster centroid generation model. The cluster centroids of the samples corresponding to the two input features output by the cluster centroid generation model are compared to obtain the similarity. The higher the similarity, the higher the probability of merging the same target object and the higher the recall rate.

[0101] For negative samples of merged business: the processing steps are the same as for positive samples of merged business. The difference is that the lower the similarity, the lower the probability of merging different target objects and the higher the accuracy.

[0102] It should be noted that the image features of the two subsample sets mentioned above are processed by two cluster centroid generation models. These two cluster centroid generation models share weights and belong to Siamese networks.

[0103] The above embodiment uses two sub-sample sets as an example, but in actual application scenarios, more sub-sample sets can be divided.

[0104] By combining positive and negative samples of merged business data, a cluster centroid generation model is trained to adapt to the merged business and obtain the cluster centroid generation model corresponding to the merged business.

[0105] In conjunction with the above embodiments, inputting a set of images to be processed into a cluster centroid generation model to obtain image cluster centroids output by the cluster centroid generation model includes: inputting multiple sets of images to be processed into a cluster centroid generation model corresponding to a merging service to obtain image cluster centroids corresponding to the multiple sets of images to be processed output by the cluster centroid generation model corresponding to the merging service; the method further includes: calculating the similarity between the image cluster centroids of multiple sets of images to be processed to obtain a first similarity; and merging the sets of images to be processed that satisfy the first similarity condition.

[0106] For example, the two sets of images to be processed with the highest first similarity are merged; or, multiple sets of images to be processed with a first similarity greater than a preset similarity threshold are merged.

[0107] Taking image set retrieval as an example, the method further includes: randomly selecting a sample image from the sample image set to obtain the same set sample image corresponding to the sample image set; and randomly selecting a sample image from other sample image sets to obtain the different set sample image corresponding to the sample image set; using the same set sample image and the sample image set as positive samples for retrieval, and using the different set sample image and the sample image set as negative samples for retrieval; and using the positive and negative samples for retrieval to train a cluster centroid generation model to obtain the cluster centroid generation model corresponding to the retrieval.

[0108] Specifically, for positive samples in retrieval operations: An image is randomly selected from the sample image set, and its image features are obtained from the same set of sample images. These image features are then fed into a cluster centroid generation model to obtain the sample cluster centroids. The image features of the same set of sample images are compared with the sample cluster centroids to obtain a similarity score. A higher similarity score indicates a higher probability of hitting the same target object and a higher recall rate.

[0109] For negative samples in retrieval operations: Randomly select an image from other sample image sets to obtain image features from the dissimilar sample images. The remaining steps are the same as above. The lower the similarity obtained, the lower the probability of retrieval hitting different target objects, and the higher the accuracy.

[0110] A cluster centroid generation model is trained by combining positive and negative samples from retrieval services to adapt to the retrieval services, thus obtaining the cluster centroid generation model corresponding to the retrieval services.

[0111] In conjunction with the above embodiments, the method of inputting the set of images to be processed into a cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model includes: inputting the set of images to be processed into a cluster centroid generation model corresponding to the retrieval service to obtain the image cluster centroids output by the cluster centroid generation model corresponding to the retrieval service; the method further includes: acquiring the image to be retrieved, extracting the image features of the image to be retrieved to obtain the retrieved image features; calculating the similarity between the retrieved image features and the image cluster centroids to obtain a second similarity; if the second similarity meets a preset condition, then the set of images to be processed is used as the retrieval result of the image to be retrieved.

[0112] For example, obtain the set of images to be processed with the highest second similarity to get the search results; or, obtain the set of images to be processed with a second similarity greater than a preset similarity threshold to get the search results.

[0113] Of course, in addition to the business adaptation training shown in the above embodiments, other businesses can also be adapted and trained. In addition, multiple business adaptation training steps can be combined to train the cluster centroid generation model together. This application does not limit this.

[0114] In some implementations, the method further includes: obtaining image features corresponding to images of different styles in the image set to be processed, and obtaining style image features; calculating the similarity between the image cluster centroids output by the cluster centroid generation model and the image features of each style, and obtaining multiple style similarities; if any style similarity is lower than a preset similarity threshold, then generating an image cluster centroid corresponding to the style based on the image of the style corresponding to any style similarity.

[0115] In addition to generating virtual centroids through a clustering centroid generation model, real centroids can also be generated based on specific images. Virtual centroids are those that cannot be mapped to a specific image, while real centroids are those that can be mapped to a specific image.

[0116] Specifically, the images in the image set to be processed are labeled with corresponding style types. The image features corresponding to different styles of images in the image set to be processed are obtained to obtain style image features.

[0117] For example, images of different styles can be randomly selected and their features extracted to obtain style image features.

[0118] For example, images of different styles with image quality scores greater than a preset score threshold can be selected, and image features can be extracted from the selected images to obtain style image features. The image quality score can be determined based on image sharpness, and / or image brightness, and / or image resolution, and / or the degree of occlusion of the target object, etc., and this application does not impose any limitations on this.

[0119] Then, the image cluster centroids of the image set to be processed are obtained from the output of the cluster centroid generation model. The similarity between the image cluster centroids and the features of each style image is calculated to obtain multiple style similarities.

[0120] The system detects whether there is a style similarity lower than a preset similarity threshold. If so, it indicates that the current image cluster centroid is significantly different from the style. Based on the image of the style with the corresponding style similarity, the system can generate an image cluster centroid corresponding to that style to ensure the comprehensiveness of the cluster centroids.

[0121] It should be noted that if the cluster centroid generation model generates multiple image cluster centroids, then when the similarity between any style image feature and all image cluster centroids is lower than the preset similarity threshold, the image cluster centroid corresponding to that style will be generated based on the image of that style.

[0122] The method for generating the image cluster centroids of all images of the corresponding style can be flexibly selected according to the actual application scenario. For example, it can be to calculate the mean of the image features of each image, or to select the image features of any image, or to generate the image cluster centroids of the corresponding style by a single-style cluster centroid generation model trained separately. This application does not limit this.

[0123] The above embodiments can ensure that all image cluster centroids can fully represent images of each style, while reducing the required computing power.

[0124] Figure 4 This is a block diagram illustrating an image clustering centroid generation apparatus according to an exemplary embodiment of this application. Figure 4 As shown, the exemplary image clustering centroid generation apparatus 400 includes: The feature extraction module 410 is used to extract the image features of each sample image in the sample image set to obtain the original features; wherein, the sample image set contains multiple sample images of different styles corresponding to the same target object; The feature masking module 420 is used to mask the image features corresponding to some style sample images in the original features to obtain masked image features; The sample centroid generation module 430 is used to input the mask image features into the model to be trained and obtain the sample cluster centroids output by the model to be trained. The model optimization module 440 is used to reconstruct the image features of each style of sample image based on the sample cluster centroid, obtain the reconstructed features, optimize the model parameters of the model to be trained based on the error between the original features and the reconstructed features, and obtain the trained cluster centroid generation model. The centroid extraction module 450 is used to input the set of images to be processed into the cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model.

[0125] It should be noted that the image clustering centroid generation device and the image clustering centroid generation method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the image clustering centroid generation device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0126] Please see Figure 5 , Figure 5This is a schematic diagram illustrating the structure of an electronic device in an exemplary embodiment of this application. The electronic device 500 includes a memory 510 and a processor 520. The processor 520 executes program instructions stored in the memory 510 to implement the steps in any of the above-described embodiments of the image clustering centroid generation method. In a specific implementation scenario, the electronic device 500 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 500 may also include mobile devices such as laptops and tablets, without limitation.

[0127] Specifically, processor 520 controls itself and memory 510 to implement the steps in any of the above-described image clustering centroid generation method embodiments. Processor 520 can also be referred to as a Central Processing Unit (CPU). Processor 520 may be an integrated circuit chip with signal processing capabilities. Processor 520 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 520 can be implemented using integrated circuit chips.

[0128] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the structure of a computer-readable storage medium in an exemplary embodiment of this application. The computer-readable storage medium 600 stores program instructions 610 executable by a processor, which are used to implement the steps in any of the above-described embodiments of the image clustering centroid generation method.

[0129] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0130] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0132] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for generating centroids in image clustering, characterized in that, The method includes: Extract image features from each sample image in the sample image set to obtain the original features; wherein, the sample image set contains multiple sample images of different styles corresponding to the same target object; Masking the image features corresponding to a portion of the style sample images in the original features yields masked image features. The mask image features are input into the model to be trained to obtain the sample cluster centroids output by the model to be trained; Based on the sample cluster centroids, the image features of each style of sample image are reconstructed to obtain reconstructed features. Based on the error between the original features and the reconstructed features, the model parameters of the model to be trained are optimized to obtain the trained cluster centroid generation model. The set of images to be processed is input into the cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model.

2. The method according to claim 1, characterized in that, The method further includes: Multiple sample image sets are split into multiple parts, and each sample image set is split into multiple sub-sample sets; The sub-sample sets obtained by splitting the same set of sample images are used as positive samples for the merged business, and the sub-sample sets obtained by splitting different sets of sample images are used as negative samples for the merged business. The cluster centroid generation model is trained using the positive and negative samples of the merged business to obtain the cluster centroid generation model corresponding to the merged business.

3. The method according to claim 2, characterized in that, The step of inputting the set of images to be processed into the cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model includes: Multiple sets of images to be processed are input into the cluster centroid generation model corresponding to the merged service to obtain the image cluster centroids corresponding to the multiple sets of images to be processed, respectively, output by the cluster centroid generation model corresponding to the merged service. The method further includes: Calculate the similarity between the centroids of the image clusters of the multiple sets of images to be processed to obtain the first similarity. The set of images to be processed that meet the preset conditions for the first similarity are merged.

4. The method according to claim 1, characterized in that, The method further includes: Randomly select a sample image from the sample image set to obtain a sample image of the same set corresponding to the sample image set; and randomly select a sample image from other sample image sets to obtain a sample image of a different set corresponding to the sample image set. The same set of sample images and the sample image set are used as positive samples for retrieval services, and the different set of sample images and the sample image set are used as negative samples for retrieval services. The cluster centroid generation model is trained using the positive and negative samples of the retrieval service to obtain the cluster centroid generation model corresponding to the retrieval service.

5. The method according to claim 4, characterized in that, The step of inputting the set of images to be processed into the cluster centroid generation model to obtain the image cluster centroids output by the cluster centroid generation model includes: The set of images to be processed is input into the cluster centroid generation model corresponding to the retrieval service to obtain the image cluster centroids output by the cluster centroid generation model corresponding to the retrieval service. The method further includes: Obtain the image to be retrieved, extract the image features of the image to be retrieved, and obtain the retrieved image features; Calculate the similarity between the retrieved image features and the image cluster centroid to obtain a second similarity. If the second similarity meets the preset conditions, then the set of images to be processed will be used as the retrieval result of the image to be retrieved.

6. The method according to claim 1, characterized in that, The step of masking the image features corresponding to a portion of the style sample images in the original features to obtain masked image features includes: Obtain the training progress of the model to be trained; The number of styles to be masked is determined based on the training progress. The image features of the number of styles to be masked in the original features are masked to obtain masked image features.

7. The method according to claim 1, characterized in that, The step of masking the image features corresponding to a portion of the style sample images in the original features to obtain masked image features includes: The false positive rate for images of different styles is calculated to obtain the false positive rate for each style; wherein, the false positive rate refers to the probability that an image of a target object is incorrectly assigned to an image set of other target objects; Based on the image misjudgment rate corresponding to each style, the style to be masked is determined. The image features of the style to be masked in the original features are masked to obtain masked image features.

8. The method according to claim 1, characterized in that, The method further includes: Obtain the image features corresponding to images of different styles in the set of images to be processed, and obtain the style image features; Calculate the similarity between the image cluster centroids output by the cluster centroid generation model and the features of each style image to obtain multiple style similarities; If the similarity of any style is lower than a preset similarity threshold, then the image cluster centroid corresponding to the style is generated based on the image of the style corresponding to the style of any style similarity.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-8.

Images