Method for generating feature extraction model, image feature extraction method and device

Abstract

The present disclosure relates to a method for generating a feature extraction model, an image feature extraction method and apparatus. The method includes: obtaining a plurality of candidate images; determining a clustering tree corresponding to the plurality of candidate images based on the feature extraction model and the plurality of candidate images, wherein the clustering tree includes clusters at multiple levels; generating target sample pairs based on the plurality of candidate images and the clustering tree, wherein the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters; training the feature extraction model based on the target sample pairs. Thereby, when performing classification based on contrastive learning, local semantics at different granularities can be learned, improving the accuracy and stability of the feature extraction model.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to a method for generating a feature extraction model, an image feature extraction method, and an apparatus. Background Art

[0002] There is a hierarchical semantic structure in natural image datasets. Taking animal image classification as an example, for instance, images corresponding to Labrador retrievers and poodles can be classified into the category of "dogs", and categories such as "dogs", "cats", and "monkeys" can be further classified into the category of "mammals".

[0003] However, in the related art, self-supervised learning based on contrastive learning cannot model the above hierarchical semantic structure. Therefore, when classifying based on contrastive learning, only local semantics of different sample images at the same granularity can be learned, making the features of different images far away from each other in the feature space, and the processing of each sample image with different differences is the same, which is difficult to be applicable to the above scenarios with higher levels. Summary of the Invention

[0004] This Summary of the Invention section is provided to introduce concepts in a concise form, which will be described in detail in the following Detailed Description section. This Summary of the Invention section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a method for generating a feature extraction model, the method including:

[0006] Obtain a plurality of candidate images;

[0007] Based on the feature extraction model and the plurality of candidate images, determine a clustering tree corresponding to the plurality of candidate images, where the clustering tree includes clusters at multiple levels;

[0008] Based on the plurality of candidate images and the clustering tree, generate target sample pairs, where the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters;

[0009] Train the feature extraction model based on the target sample pairs.

[0010] In a second aspect, the present disclosure provides an image feature extraction method, the method including:

[0011] Receive an image to be processed;

[0012] Input the image to be processed into the feature extraction model to obtain the feature image output by the feature extraction model, where the feature extraction model is generated by the generation method of the feature extraction model described in the first aspect.

[0013] In a third aspect, the present disclosure provides a generating device for a feature extraction model, the device includes:

[0014] An obtaining module, configured to obtain a plurality of candidate images;

[0015] A determining module, configured to determine a clustering tree corresponding to the plurality of candidate images based on the feature extraction model and the plurality of candidate images, where the clustering tree includes clusters at multiple levels;

[0016] A generating module, configured to generate target sample pairs based on the plurality of candidate images and the clustering tree, where the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters;

[0017] A training module, configured to train the feature extraction model based on the target sample pairs.

[0018] In a fourth aspect, there is provided an image feature extraction device, the device includes:

[0019] A receiving module, configured to receive an image to be processed;

[0020] An extraction module, configured to input the image to be processed into the feature extraction model to obtain the feature image output by the feature extraction model, where the feature extraction model is generated by the generation method of the feature extraction model described in the first aspect.

[0021] In a fifth aspect, there is provided a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of the method described in the first aspect are implemented.

[0022] In a sixth aspect, there is provided an electronic device, including:

[0023] A storage device, on which at least one computer program is stored;

[0024] At least one processing device, configured to execute the at least one computer program in the storage device to implement the steps of the method described in the first aspect.

[0025] Through the above technical solution, when generating a feature extraction model based on candidate images, instead of directly determining sample pairs based on the candidate images, a clustering tree with multiple levels is generated based on the current feature extraction model and the candidate images, so as to represent the hierarchical semantics of the candidate images. When performing classification based on contrast learning, local semantics at different granularities can be learned, improving the accuracy of the feature extraction model and providing accurate data support for subsequent image classification, image recognition, etc. Moreover, in the solution of the present disclosure, when generating sample pairs, they can be determined based on the clustering tree and the candidate images, and the obtained sample pairs can include image sample pairs and feature sample pairs, thereby further improving the accuracy, comprehensiveness, and diversity of the sample pairs used for generating the feature extraction model, improving the accuracy and effectiveness of the training data of the feature extraction model to a certain extent, thus improving the training efficiency and stability of the feature extraction model and expanding the application scope of the feature extraction model.

[0026] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Combined with the drawings and referring to the following specific implementation manners, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale. In the drawings:

[0028] Figure 1 is a flowchart of a method for generating a feature extraction model provided according to an embodiment of the present disclosure;

[0029] Figure 2 is a schematic diagram of a clustering tree provided according to an embodiment of the present disclosure;

[0030] Figure 3 is a block diagram of a device for generating a feature extraction model provided according to an embodiment of the present disclosure;

[0031] Figure 4 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0033] It should be understood that the steps described in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0034] As used herein, the term "comprising" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0035] It should be noted that the concepts such as "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0036] It should be noted that the modifications of "one" and "plural" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0037] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0038] Figure 1 As shown, it is a flowchart of a method for generating a feature extraction model provided according to an embodiment of the present disclosure. The method may include:

[0039] In step 11, a plurality of candidate images are obtained. Among them, the candidate images may be all the images in the dataset for training the feature extraction model.

[0040] In step 12, based on the feature extraction model and the plurality of candidate images, a clustering tree corresponding to the plurality of candidate images is determined, where the clustering tree contains clusters at multiple levels.

[0041] In this step, the candidate images can be hierarchically clustered, so that the features of the candidate images can be clustered at different granularities, facilitating the division of the candidate images at different classification levels.

[0042] Exemplarily, the exemplary implementation manner of determining the clustering tree corresponding to the plurality of candidate images based on the feature extraction model and the plurality of candidate images may include:

[0043] Feature extraction is performed on the multiple candidate images based on the feature extraction model to obtain sample features corresponding to each of the candidate images. Exemplarily, each candidate image can be input into the current feature extraction model, so that the sample features corresponding to each candidate image can be output through the feature extraction model.

[0044] Clustering is performed based on each target feature to generate multiple clusters and the cluster center of each cluster, and the target feature is initially the sample feature.

[0045] As an example, clustering can be performed based on a bottom-up clustering method. If the target feature is initially the sample features corresponding to each candidate image, clustering can be performed based on the distance between the sample features. The calculation of this distance can be performed using Manhattan distance, Euclidean distance, or Pearson similarity, and the present disclosure does not limit this. Thus, clustering can be performed based on the sample features corresponding to the candidate image to obtain S cluster centers, where S is used to represent the number of clusters obtained by clustering based on the sample features. For ease of description, this layer can be denoted as the S layer.

[0046] After that, the cluster center of each cluster is used as a new target feature, and hierarchical clustering is performed until the level of the obtained clustering tree reaches the preset level.

[0047] Correspondingly, clustering can be continued based on the S cluster centers to obtain M new cluster centers, where M is less than S. For ease of description, the level containing the M new cluster centers can be denoted as the M layer. At this time, the depth of the formed clustering tree is 2, that is, the bottom layer contains S cluster centers, and the upper layer contains M cluster centers. If the preset level is 3, clustering can be performed again based on the M cluster centers at this time to obtain L higher-level cluster centers, where L is less than M. At this time, the level of the clustering tree reaches the preset level, and the clustering is ended to obtain the clustering tree with clustering completed. For ease of description, the level containing the L higher-level cluster centers can be denoted as the L layer, as Figure 2 shown.

[0048] As can be seen from the above, clustering can be performed on the features of multiple candidate images for training to obtain a clustering tree including multiple levels, and then the hierarchical semantics of the candidate images can be characterized based on this clustering tree, which is convenient for improving the comprehensiveness and accuracy of the features of the candidate images.

[0049] In step 13, based on the multiple candidate images and the clustering tree, target sample pairs are generated, where the target sample pairs include image sample pairs and feature sample pairs. The image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters.

[0050] Among them, during the learning process of contrastive learning, there is no need for users to perform annotations. Instead, sample pairs are constructed based on sample images, and learning is carried out based on each sample pair. In this embodiment, when determining sample pairs based on candidate images, on the one hand, sample pairs can be constructed based on the candidate images themselves. On the other hand, as described above, the clustering tree contains clusters at multiple levels, and the cluster centers at each level can represent the classification features of that level. Then, sample pairs can be further determined based on the features of the candidate images and the features of the cluster centers of the clusters, improving the accuracy and diversity of the sample pairs.

[0051] In step 14, based on the target sample pairs, the feature extraction model is trained.

[0052] Exemplarily, the feature extraction model can be trained based on the target sample pairs using the common contrastive learning method in the art to obtain a trained feature extraction model.

[0053] Thus, through the above technical solution, when generating a feature extraction model based on candidate images, sample pairs are not directly determined based on the candidate images. Instead, a clustering tree with multiple levels is generated based on the current feature extraction model and candidate images, so that the hierarchical semantics of the candidate images can be represented. When performing classification based on contrastive learning, local semantics at different granularities can be learned, improving the accuracy of the feature extraction model and providing accurate data support for subsequent image classification, image recognition, etc. Moreover, in the solution of the present disclosure, when generating sample pairs, they can be determined based on the clustering tree and candidate images. The obtained sample pairs can include image sample pairs and feature sample pairs, thereby further improving the accuracy, comprehensiveness, and diversity of the sample pairs used to generate the feature extraction model, improving the accuracy and effectiveness of the training data of the feature extraction model to a certain extent, thus improving the training efficiency and stability of the generated feature extraction model and expanding the application scope of the feature extraction model.

[0054] In a possible embodiment, an exemplary implementation manner of generating target sample pairs based on multiple candidate images and a clustering tree in step 13 is as follows. This step may include:

[0055] Generate a sample image set and a candidate contrast image set based on the candidate images, where the images in the candidate contrast image set are different from the images in the sample image set, and the number of images corresponding to the candidate contrast image set is greater than the number of images corresponding to the sample image set.

[0056] Among them, W images can be randomly selected from the candidate images as sample images and added to the sample image set X, and Q images can be selected from the images other than the W images in the candidate images as candidate contrast images and added to the candidate contrast image set H, where W and Q are positive integers and W is less than Q.

[0057] For each sample image in the sample image set, based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree, determine the comparison sample set of the sample image at each level, where the comparison samples in the comparison sample set include the features of the target comparison image and the target comparison cluster center.

[0058] As described above, when generating the feature extraction model, it is necessary to construct sample pairs. Each sample pair can include two images or two feature vectors. Each sample image in the sample image set is used to generate a target sample pair. Since the granularity of the feature semantics at each level of the clustering tree is different, in the embodiment, when generating the target sample pair based on the sample image, for each level in the clustering tree, the corresponding comparison image and comparison cluster center of the sample image at that level can be determined, so that individual analysis and determination can be performed for each level, improving the accuracy of the comparison sample set corresponding to the sample image.

[0059] After that, for each sample image in the sample image set, according to the sample image and the comparison sample set of the sample image at each level, generate negative sample pairs corresponding to the sample image at each level, and determine the negative sample pairs as the target sample pairs.

[0060] Exemplarily, for each sample image in the sample image set, according to the sample image and the comparison sample set of the sample image at each level, negative sample pairs can be formed by the sample image and multiple comparison samples in the comparison sample set at that level. The multiple comparison samples can be some or all of the samples in the comparison sample set. Exemplarily, for sample image X1, the comparison sample set DS at layer S contains K comparison samples, then some or all of the K comparison samples can be selected to form negative sample pairs with the sample image X1. Exemplarily, (X1, DS1), (X1, DS2), ……, (X1, DS K ) and other K negative sample pairs. For sample image X1, the comparison sample set DM at layer M contains J comparison samples, then some or all of the J comparison samples can be selected to form negative sample pairs with the sample image X1. Exemplarily, (X1, DM1), (X1, DM2), ……, (X1, DM J ) and other J negative sample pairs. The method of generating target sample pairs for other levels and other sample images is similar to that described above and will not be elaborated here.

[0061] Thus, through the above technical solution, after determining the sample image set and the candidate comparison image set, the comparison samples corresponding to the sample image set can be selected from the candidate comparison image set at each level based on the clustering tree, and the comparison clustering centers corresponding to the sample image can be determined at each level based on the clustering tree. Therefore, while accurately screening the comparison samples of the sample image, the comparison samples can be determined for each level, so that when training the feature extraction model based on the target sample, the semantic features at different levels can be learned, and the features of different images can be distributed differently in the feature spaces of different levels when performing feature extraction based on the feature extraction model, improving the accuracy of the feature extraction model.

[0062] In a possible embodiment, the exemplary implementation manner of determining the comparison sample set of the sample image at each level based on the candidate comparison images in the candidate comparison image set and the clustering centers of the clusters at each level in the clustering tree is as follows. This step includes:

[0063] Determine the hierarchical clustering center to which the sample image belongs at each level in the clustering tree.

[0064] As an example, the feature extraction model can be used to extract features from the sample image, so that for each level in the clustering tree, the distance between the features of the sample image and each clustering center at that level can be calculated, and the clustering center with the minimum corresponding distance can be determined as the hierarchical clustering center to which the sample image belongs at that level.

[0065] For each level in the clustering tree, based on the hierarchical clustering center to which the sample image belongs at that level and the candidate comparison images in the candidate comparison image set, determine the target comparison image corresponding to the sample image at that level.

[0066] As an example, this step can be implemented in the following manner:

[0067] For each level in the clustering tree, determine the image similarity between the hierarchical clustering center of the sample image at that level and each candidate comparison image in the candidate comparison image set. Among them, each candidate comparison image can be input into the feature extraction model to obtain the features of the candidate comparison image. Then, based on the features of the candidate comparison image and the hierarchical clustering center, calculate the image similarity, which can be calculated based on the cosine similarity. Among them, this image similarity can represent the probability that the candidate comparison image and the sample image belong to the same cluster at that level.

[0068] After that, if the image similarity is greater than the first threshold, it means that the similarity between the sample image and the candidate comparison image is relatively high, and the probability that the sample image and the candidate comparison image belong to the same cluster at this level is relatively high. At this time, it is not appropriate to use the candidate comparison image as the comparison sample of the sample image. Therefore, in this embodiment, the candidate comparison images with image similarity greater than the first threshold can be ignored, and the candidate comparison images with image similarity less than or equal to the first threshold can be used as the target comparison images.

[0069] As another example, for each level in the clustering tree, based on the level clustering center to which the sample image belongs at this level and the candidate comparison images in the candidate comparison image set, the implementation manner of the step of determining the target comparison image corresponding to the sample image at this level can be as follows, and may include:

[0070] For each level in the clustering tree, determine the image similarity between the level clustering center of the sample image at this level and each candidate comparison image in the candidate comparison image set. Among them, the method of calculating the image similarity has been described above and will not be elaborated here.

[0071] After determining the image similarity, random sampling can be performed based on the image similarity. When the result of the random sampling is rejection, the candidate comparison image is determined as the target comparison image of the sample image at this level.

[0072] Exemplarily, the image similarity is p. Random sampling can use p as the input, so as to output 1 with a probability of p and output 0 with a probability of 1 - p. It is determined that when the output is 1, it means acceptance, and when the output is 0, it means rejection. Among them, when the result of the random sampling is acceptance, it means that the sample image and the candidate comparison image belong to the same cluster at this level. When the result of the random sampling is rejection, it means that the sample image and the candidate comparison image do not belong to the same cluster at this level. In this embodiment, to determine the comparison sample corresponding to the sample image, that is, to determine a sample different from the sample image, the candidate comparison image with the result of random sampling being rejection can be determined as the target comparison image. Exemplarily, based on the image similarity p for random sampling, if the sampling result is 0, the candidate comparison image is determined as the target comparison image of the sample image at this level.

[0073] Thus, through the above technical solution, for each level in the clustering tree, by determining the similarity between the candidate comparison image and the level clustering center to which the sample image belongs at this level, the target comparison image corresponding to the sample image at this level can be determined, thereby avoiding using the candidate comparison image belonging to the same cluster as the sample image as its comparison sample, ensuring the accuracy of the determined comparison sample, and thus improving the accuracy of the feature extraction model. Moreover, by using the image similarity as the probability for a single random sampling to determine the comparison sample according to the sampling result, the flexibility of the comparison sample selection can also be increased by introducing random sampling.

[0074] For each level in the clustering tree, based on the level clustering center to which the sample image belongs at this level and each clustering center in the clustering tree, determine the target comparison clustering center corresponding to the sample image at this level.

[0075] As an example, this step can be implemented in the following manner:

[0076] Exemplarily, for each level in the clustering tree, if the other clustering centers except the level clustering center to which the sample image belongs at this level are all different from this level clustering center, then the other clustering centers except this level clustering center at this level can be directly determined as the target clustering centers corresponding to this level.

[0077] Exemplarily, as Figure 2 shown in the clustering tree, the level clustering center to which the sample image X1 belongs at the S level is S1, the level clustering center to which the sample image X1 belongs at the M level is M1, and the level clustering center to which the sample image X1 belongs at the L level is L1. Then, for the S level, the other clustering centers in the S level except S1 (i.e., S2 - S17) can be used as the target comparison clustering centers corresponding to the S level; for the M level, the other clustering centers in the M level except M1 (i.e., M2 - M7) can be used as the target comparison clustering centers corresponding to the M level; for the L level, the other clustering centers in the L level except L1 (i.e., L2 and L3) can be used as the target comparison clustering centers corresponding to the L level.

[0078] As another example, the exemplary implementation manner of determining the target comparison clustering center corresponding to the sample image at this level for each level in the clustering tree based on the level clustering center to which the sample image belongs at this level and each clustering center in the clustering tree is as follows. This step may include:

[0079] For each level in the clustering tree except the highest level, determine the feature similarity between the parent clustering center corresponding to the level clustering center of the sample image at this level and the candidate clustering centers except this level clustering center at this level.

[0080] As for Figure 2 the clustering tree shown, if the highest level is the L level, taking the S level as an example, the hierarchical clustering center to which the sample image X1 belongs at the S level is S1, the parent clustering center corresponding to the hierarchical clustering center S1 is M1, and the candidate clustering centers at the S level are S2 - S17. Then, the similarity between the features of each center in S2 - S17 and M1 can be calculated. The similarity can be cosine similarity, and this is not limited. Among them, the higher the determined feature similarity, the greater the probability that the hierarchical clustering center to which the sample image belongs and the candidate clustering center belong to the same parent clustering.

[0081] Based on the feature similarity, random sampling is performed. When the result of the random sampling is rejection, the candidate clustering center is determined as the target comparison clustering center corresponding to the sample image at the said level.

[0082] Among them, the way of random sampling is similar to that described above and will not be elaborated here. When the result of the random sampling is acceptance, it means that the hierarchical clustering center to which the sample image belongs and the candidate clustering center belong to the same parent clustering. When the result of the random sampling is rejection, it means that the hierarchical clustering center to which the sample image belongs and the candidate clustering center do not belong to the same parent clustering. In this embodiment, the comparison sample corresponding to the sample image is determined, that is, the sample different from the sample image is determined. That is, the candidate clustering center with the result of rejection in the random sampling can be used as the target comparison clustering center. For example, based on the feature similarity p', random sampling is performed. If the sampling result is 0, the candidate comparison clustering center is determined as the target comparison clustering center of the sample image at the said level.

[0083] Thus, through the above technical solution, the candidate comparison clustering centers other than the hierarchical clustering center to which the sample image belongs at each level can be further analyzed to determine the target comparison clustering center from the candidate comparison clustering centers, thereby effectively increasing the accuracy and quantity of the comparison samples corresponding to the sample image and improving the accuracy of the negative sample pairs in the contrast learning process. In addition, using the feature similarity as the probability for one random sampling to determine the comparison sample according to the sampling result can increase the flexibility of the comparison sample selection by introducing random sampling and further improve the comprehensiveness of the comparison samples.

[0084] For the highest level in the clustering tree, sampling is performed on the candidate clustering centers other than the hierarchical clustering center corresponding to the sample image at the highest level. When the result of the sampling is rejection, the candidate clustering center is determined as the target comparison clustering center corresponding to the sample image at the said level.

[0085] Continuing the above example, as Figure 2As shown, for the highest level L, the candidate cluster centers other than the cluster center L1 corresponding to the sample image X1 under the highest level can be uniformly sampled. When the result of the sampling is rejection, the candidate cluster center is determined as the target contrast cluster center corresponding to the sample image at the level. Exemplarily, uniform sampling can be performed with a probability of 0.5, so as to select the corresponding target contrast cluster center from the candidate cluster centers, thereby ensuring the flexibility of determining the target contrast cluster center while ensuring the accuracy of the target cluster center, so as to improve the reliability and effectiveness of the determined negative sample pairs.

[0086] Optionally, the exemplary implementation manner of generating the target sample pairs based on the multiple candidate images and the cluster tree may further include:

[0087] For each sample image in the sample image set, based on the transformed image corresponding to the sample image and the hierarchical cluster center to which the sample image belongs at each level of the cluster tree, a positive sample pair corresponding to the sample image is generated.

[0088] As an example, for each sample image, different transformed images of the sample image under different perspectives can be obtained through different random transformations, so that the sample pairs corresponding to the transformed image and the sample image can be used as positive sample pairs.

[0089] As another example, for each sample image in the sample image set, the hierarchical cluster center to which the sample image belongs at each level in the cluster tree is determined. Among them, the method for determining each hierarchical cluster center has been described in detail above and will not be elaborated here. Then, the feature of the sample image and the central feature of S1 can be used as a positive sample pair, the feature of the sample image and the central feature of M1 can be used as a positive sample pair, and the feature of the sample image and the central feature of L1 can be used as a positive sample pair, that is, the feature positive sample pairs corresponding to the sample image at each level are determined.

[0090] Thus, through the above technical solution, the positive sample pairs corresponding to the sample images can be quickly generated, and in the process of generating the positive sample pairs, different samples can also be generated based on the cluster centers at different levels, and the hierarchical semantics in the positive sample pairs are represented and constructed, improving the feature accuracy in the positive sample pairs in the contrast learning process, and providing data support for improving the learning accuracy of the feature extraction model.

[0091] In a possible embodiment, the exemplary implementation manner of training the feature extraction model based on the target sample pairs is as follows, and this step may include:

[0092] Based on the target sample pairs and the sample labels of the target sample pairs, the target loss of the feature extraction model is determined.

[0093] As an example, based on the positive sample pairs and negative sample pairs in the target sample pair, the target loss can be determined through the contrastive learning loss function commonly used in the art. As another example, when determining the target loss, the loss calculation can be performed separately for each level, so as to determine the final target loss. The target loss Loss can be determined by the following formula:

[0094]

[0095] where n is used to represent the number of levels in the clustering tree;

[0096] Ni(q) is used to represent the contrast sample set of the sample image q at the i-th level;

[0097] k is used to represent the contrast sample in the contrast sample set corresponding to the sample image q;

[0098] sim() is used to represent similarity calculation; exp() is used to represent exponential calculation;

[0099] k + is used to represent the transformed image in the positive sample pair corresponding to the sample image q;

[0100] k i + is used to represent the hierarchical clustering center in the positive sample pair of the sample image q at the i-th level.

[0101] If the iteration end condition is not satisfied, the parameters of the feature extraction model are updated based on the target loss to obtain an updated feature extraction model.

[0102] The iteration end condition is that the target loss is less than or equal to the loss threshold, or the number of iterations of the feature extraction model is greater than the number threshold. The loss threshold and the number threshold can be set based on the actual application scenario, and the present disclosure does not limit this. Among them, the parameters of the feature extraction model can be updated based on the target loss by the gradient descent method, which will not be elaborated here.

[0103] Based on the updated feature extraction model and the multiple candidate images, determine the clustering tree corresponding to the multiple candidate images;

[0104] Based on the multiple candidate images and the newly determined clustering tree, generate a new target sample pair, so as to train the feature extraction model based on the new target sample pair until the iteration end condition is satisfied, and obtain a trained feature extraction model.

[0105] Among them, the specific implementation manners of the above steps have all been described above and will not be elaborated here. In this embodiment, after the feature extraction model is updated, the updated feature extraction model can be used to extract features from multiple candidate images to determine a new clustering tree, so as to determine the training samples for the next round. Thus, during the training process of the feature extraction model, by continuously updating the feature extraction model, regenerating the clustering tree based on the updated feature extraction model, and determining new training samples, the accuracy of the clustering tree and the training samples can be improved, and the training efficiency of the feature extraction model can be effectively improved.

[0106] The present disclosure also provides an image feature extraction method, which may include:

[0107] Receiving an image to be processed;

[0108] Inputting the image to be processed into a feature extraction model to obtain a feature image output by the feature extraction model, where the feature extraction model is generated based on the generation method of any of the above-mentioned feature extraction models. The feature image can be used for image recognition, image classification, etc. of the image to be processed. Thus, through the above technical solution, during the generation process of the feature extraction model, local semantics at different granularities can be learned, ensuring the accuracy of the feature extraction model, thereby improving the accuracy and comprehensiveness of the features in the feature image, and providing accurate data support for subsequent image classification, image recognition, etc.

[0109] The present disclosure also provides a generating device for a feature extraction model, as Figure 3 shown, the device includes:

[0110] An obtaining module 100, configured to obtain multiple candidate images;

[0111] A determining module 200, configured to determine a clustering tree corresponding to the multiple candidate images based on the feature extraction model and the multiple candidate images, where the clustering tree includes clusters at multiple levels;

[0112] A generating module 300, configured to generate target sample pairs based on the multiple candidate images and the clustering tree, where the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters;

[0113] A training module 400, configured to train the feature extraction model based on the target sample pairs.

[0114] Optionally, the generating module includes:

[0115] A first generation sub-module for generating a sample image set and a candidate comparison image set based on the candidate images, wherein the images in the candidate comparison image set are different from the images in the sample image set, and the number of images corresponding to the candidate comparison image set is greater than the number of images corresponding to the sample image set;

[0116] A first determination sub-module for, for each sample image in the sample image set, determining a comparison sample set of the sample image at each level based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree, wherein the comparison samples in the comparison sample set include the features of the target comparison image and the target comparison cluster center;

[0117] A second generation sub-module for, for each sample image in the sample image set, generating a negative sample pair corresponding to the sample image at each level according to the sample image and the comparison sample set of the sample image at each level, and determining the negative sample pair as the target sample pair.

[0118] Optionally, the first determination sub-module includes:

[0119] A second determination sub-module for determining the level cluster center to which the sample image belongs at each level in the clustering tree;

[0120] A third determination sub-module for, for each level in the clustering tree, determining the target comparison image corresponding to the sample image at the level based on the level cluster center to which the sample image belongs at the level and the candidate comparison images in the candidate comparison image set;

[0121] A fourth determination sub-module for, for each level in the clustering tree, determining the target comparison cluster center corresponding to the sample image at the level based on the level cluster center to which the sample image belongs at the level and the respective cluster centers in the clustering tree.

[0122] Optionally, the third determination sub-module includes:

[0123] A fifth determination sub-module for, for each level in the clustering tree, determining the image similarity between the level cluster center of the sample image at the level and each candidate comparison image in the candidate comparison image set;

[0124] A sixth determination sub-module for performing random sampling based on the image similarity, and when the result of the random sampling is rejection, determining the candidate comparison image as the target comparison image of the sample image at the level.

[0125] Optionally, the fourth determination sub-module includes:

[0126] A seventh determination sub-module, configured to, for each level in the clustering tree except the highest level, determine the feature similarity between the parent clustering center corresponding to the level clustering center of the sample image at the level and the candidate clustering centers other than the level clustering center at the level;

[0127] An eighth determination sub-module, configured to perform random sampling based on the feature similarity, and when the result of the random sampling is rejection, determine the candidate clustering center as the target comparison clustering center corresponding to the sample image at the level.

[0128] Optionally, the generation module further includes:

[0129] A third generation sub-module, configured to, for each sample image in the sample image set, generate a positive sample pair corresponding to the sample image based on the transformed image corresponding to the sample image and the level clustering centers to which the sample image belongs at each level of the clustering tree.

[0130] Optionally, the training module includes:

[0131] A ninth determination sub-module, configured to determine the target loss of the feature extraction model based on the target sample pair and the sample label of the target sample pair;

[0132] An update sub-module, configured to, if the iteration end condition is not satisfied, update the parameters of the feature extraction model based on the target loss to obtain an updated feature extraction model;

[0133] A tenth determination sub-module, configured to determine the clustering tree corresponding to the multiple candidate images based on the updated feature extraction model and the multiple candidate images;

[0134] A fourth generation sub-module, configured to generate new target sample pairs based on the multiple candidate images and the newly determined clustering tree, so as to train the feature extraction model based on the new target sample pairs until the iteration end condition is satisfied, and obtain a trained feature extraction model, where the iteration end condition is that the target loss is less than or equal to a loss threshold, or the number of iterations of the feature extraction model is greater than a number threshold.

[0135] The present disclosure further provides an image feature extraction device, and the device includes:

[0136] A receiving module, configured to receive an image to be processed;

[0137] An extraction module is configured to input the image to be processed into a feature extraction model to obtain a feature image output by the feature extraction model, where the feature extraction model is generated based on the feature extraction model generation method described above.

[0138] Reference is made below Figure 4 to FIG. 600, which shows a schematic structural diagram of an electronic device 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0139] As Figure 4 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0140] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 FIG. 600 shows an electronic device 600 having various devices, it should be understood that it is not required to implement or include all the shown devices. More or fewer devices may be alternatively implemented or included.

[0141] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0142] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. And in the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0143] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0144] The above computer-readable medium can be included in the above electronic device; or it can exist separately and not be assembled into the electronic device.

[0145] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: obtain a plurality of candidate images; determine a clustering tree corresponding to the plurality of candidate images based on a feature extraction model and the plurality of candidate images, wherein the clustering tree includes clusters at multiple levels; generate target sample pairs based on the plurality of candidate images and the clustering tree, wherein the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters; and train the feature extraction model based on the target sample pairs.

[0146] Alternatively, the above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: receive an image to be processed; input the image to be processed into a feature extraction model to obtain a feature image output by the feature extraction model, wherein the feature extraction model is generated based on a generation method of the feature extraction model.

[0147] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0149] The modules described in the embodiments of the present disclosure may be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the module itself in some cases. For example, the acquisition module may also be described as "the module for acquiring multiple candidate images".

[0150] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, by way of non-limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and so on.

[0151] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] According to one or more embodiments of the present disclosure, Example 1 provides a method for generating a feature extraction model, wherein the method includes:

[0153] Obtain a plurality of candidate images;

[0154] Based on the feature extraction model and the plurality of candidate images, determine a clustering tree corresponding to the plurality of candidate images, wherein the clustering tree contains clusters at multiple levels;

[0155] Based on the plurality of candidate images and the clustering tree, generate target sample pairs, wherein the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters;

[0156] Train the feature extraction model based on the target sample pairs.

[0157] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the generating the target sample pairs based on the plurality of candidate images and the clustering tree includes:

[0158] Generate a sample image set and a candidate comparison image set based on the candidate images, wherein the images in the candidate comparison image set are different from the images in the sample image set, and the number of images corresponding to the candidate comparison image set is greater than the number of images corresponding to the sample image set;

[0159] For each sample image in the sample image set, based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree, determine the comparison sample set of the sample image at each level, where the comparison samples in the comparison sample set include the features of the target comparison image and the target comparison cluster center;

[0160] For each sample image in the sample image set, according to the sample image and the comparison sample set of the sample image at each level, generate negative sample pairs corresponding to the sample image at each level, and determine the negative sample pairs as the target sample pairs.

[0161] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the determining the comparison sample set of the sample image at each level based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree includes:

[0162] Determine the hierarchical cluster center to which the sample image belongs at each level in the clustering tree;

[0163] For each level in the clustering tree, based on the hierarchical cluster center to which the sample image belongs at the level and the candidate comparison images in the candidate comparison image set, determine the target comparison image corresponding to the sample image at the level;

[0164] For each level in the clustering tree, based on the hierarchical cluster center to which the sample image belongs at the level and the cluster centers in the clustering tree, determine the target comparison cluster center corresponding to the sample image at the level.

[0165] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 3, wherein the for each level in the clustering tree, based on the hierarchical cluster center to which the sample image belongs at the level and the candidate comparison images in the candidate comparison image set, determining the target comparison image corresponding to the sample image at the level includes:

[0166] For each level in the clustering tree, determine the image similarity between the hierarchical cluster center of the sample image at the level and each candidate comparison image in the candidate comparison image set;

[0167] Based on the image similarity, perform random sampling, and when the result of the random sampling is rejection, determine the candidate comparison image as the target comparison image of the sample image at the level.

[0168] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 3, wherein, for each level in the clustering tree, based on the level clustering center to which the sample image belongs at the level and each clustering center in the clustering tree, determining the target comparison clustering center corresponding to the sample image at the level includes:

[0169] For each level in the clustering tree except the highest level, determining the feature similarity between the parent clustering center corresponding to the level clustering center of the sample image at the level and the candidate clustering centers other than the level clustering center at the level;

[0170] Based on the feature similarity for random sampling, when the result of the random sampling is rejection, determining the candidate clustering center as the target comparison clustering center corresponding to the sample image at the level.

[0171] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 2, wherein, generating the target sample pairs based on the multiple candidate images and the clustering tree further includes:

[0172] For each sample image in the sample image set, generating the positive sample pair corresponding to the sample image based on the transformed image corresponding to the sample image and the level clustering center to which the sample image belongs at each level of the clustering tree.

[0173] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, wherein, training the feature extraction model based on the target sample pairs includes:

[0174] Based on the target sample pairs and the sample labels of the target sample pairs, determining the target loss of the feature extraction model;

[0175] If the iteration end condition is not satisfied, then updating the parameters of the feature extraction model based on the target loss to obtain an updated feature extraction model;

[0176] Based on the updated feature extraction model and the multiple candidate images, determining the clustering tree corresponding to the multiple candidate images;

[0177] Based on the multiple candidate images and the newly determined clustering tree, generating new target sample pairs to train the feature extraction model based on the new target sample pairs until the iteration end condition is satisfied, and obtaining the trained feature extraction model, where the iteration end condition is that the target loss is less than or equal to the loss threshold, or the iteration times of the feature extraction model are greater than the times threshold.

[0178] According to one or more embodiments of the present disclosure, Example 8 provides an image feature extraction method, the method comprising:

[0179] Receiving an image to be processed;

[0180] Inputting the image to be processed into a feature extraction model to obtain a feature image output by the feature extraction model, wherein the feature extraction model is generated by the generation method of the feature extraction model described in any one of Examples 1-7.

[0181] According to one or more embodiments of the present disclosure, Example 9 provides a generating device for a feature extraction model, the device comprising:

[0182] An obtaining module, configured to obtain a plurality of candidate images;

[0183] A determining module, configured to determine a clustering tree corresponding to the plurality of candidate images based on the feature extraction model and the plurality of candidate images, wherein the clustering tree includes clusters at multiple levels;

[0184] A generating module, configured to generate target sample pairs based on the plurality of candidate images and the clustering tree, wherein the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters;

[0185] A training module, configured to train the feature extraction model based on the target sample pairs.

[0186] According to one or more embodiments of the present disclosure, Example 10 provides an image feature extraction device, the device comprising:

[0187] A receiving module, configured to receive an image to be processed;

[0188] An extraction module, configured to input the image to be processed into a feature extraction model to obtain a feature image output by the feature extraction model, wherein the feature extraction model is generated by the generation method of the feature extraction model described in any one of Examples 1-7.

[0189] According to one or more embodiments of the present disclosure, Example 11 provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of the method described in any one of Examples 1-8 are implemented.

[0190] According to one or more embodiments of the present disclosure, Example 12 provides an electronic device, comprising:

[0191] A storage device, on which at least one computer program is stored;

[0192] At least one processing device for executing the at least one computer program in the storage device to implement the steps of the method according to any one of Examples 1-8.

[0193] The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0194] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0195] Although the subject matter has been described in language specific to structural features and / or methodological act logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms for implementing the claims. Regarding the devices in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

Claims

1. A method for generating a feature extraction model, characterized in that, The method includes: Obtaining a plurality of candidate images; Based on a feature extraction model and the plurality of candidate images, determining a clustering tree corresponding to the plurality of candidate images, wherein the clustering tree includes clusters at multiple levels; Based on the plurality of candidate images and the clustering tree, generating target sample pairs, wherein the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters; Training the feature extraction model based on the target sample pairs; Wherein, the determining the clustering tree corresponding to the plurality of candidate images based on the feature extraction model and the plurality of candidate images includes: Performing feature extraction on the plurality of candidate images based on the feature extraction model to obtain sample features corresponding to each candidate image; performing clustering on each target feature to generate a plurality of clusters and the cluster center of each cluster, the target feature being initially the sample feature; using the cluster center of each cluster as the new target feature and performing hierarchical clustering until the level of the obtained clustering tree reaches a preset level; The generating the target sample pairs based on the plurality of candidate images and the clustering tree includes: Generating a sample image set and a candidate comparison image set based on the candidate images, wherein the images in the candidate comparison image set are different from the images in the sample image set, and the number of images corresponding to the candidate comparison image set is greater than the number of images corresponding to the sample image set; For each sample image in the sample image set, based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree, determining a comparison sample set of the sample image at each level, wherein the comparison samples in the comparison sample set include the features of the target comparison images and the target comparison cluster centers; For each sample image in the sample image set, generating negative sample pairs corresponding to the sample image at each level according to the sample image and the comparison sample set of the sample image at each level, and determining the negative sample pairs as the target sample pairs.

2. The method according to claim 1, wherein The determining the comparison sample set of the sample image at each level based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree includes: Determining the hierarchical cluster center to which the sample image belongs at each level in the clustering tree; For each level in the clustering tree, based on the hierarchical cluster center to which the sample image belongs at the level and the candidate comparison images in the candidate comparison image set, determining the target comparison image corresponding to the sample image at the level; For each level in the clustering tree, based on the hierarchical cluster center to which the sample image belongs at the level and the cluster centers in the clustering tree, determining the target comparison cluster center corresponding to the sample image at the level.

3. The method according to claim 2, characterized in that, For each level in the clustering tree, determining the target comparison image corresponding to the sample image at the level based on the level clustering center to which the sample image belongs at the level and the candidate comparison images in the candidate comparison image set, includes: For each level in the clustering tree, determining the image similarity between the level clustering center of the sample image at the level and each candidate comparison image in the candidate comparison image set; Based on the image similarity, performing random sampling, and when the result of the random sampling is rejection, determining the candidate comparison image as the target comparison image of the sample image at the level.

4. The method according to claim 2, characterized in that, For each level in the clustering tree, determining the target comparison clustering center corresponding to the sample image at the level based on the level clustering center to which the sample image belongs at the level and each clustering center in the clustering tree, includes: For each level in the clustering tree except the highest level, determining the feature similarity between the parent clustering center corresponding to the level clustering center of the sample image at the level and the candidate clustering centers other than the level clustering center at the level; Based on the feature similarity, performing random sampling, and when the result of the random sampling is rejection, determining the candidate clustering center as the target comparison clustering center corresponding to the sample image at the level.

5. The method according to claim 1, characterized in that, Generating the target sample pairs based on the multiple candidate images and the clustering tree further includes: For each sample image in the sample image set, generating the positive sample pair corresponding to the sample image based on the transformed image corresponding to the sample image and the level clustering center to which the sample image belongs at each level of the clustering tree.

6. The method according to claim 1, wherein Training the feature extraction model based on the target sample pairs includes: Based on the target sample pairs and the sample labels of the target sample pairs, determining the target loss of the feature extraction model; If the iteration end condition is not satisfied, updating the parameters of the feature extraction model based on the target loss to obtain the updated feature extraction model; Based on the updated feature extraction model and the multiple candidate images, determining the clustering tree corresponding to the multiple candidate images; Based on the multiple candidate images and the newly determined clustering tree, generating new target sample pairs to train the feature extraction model based on the new target sample pairs until the iteration end condition is satisfied to obtain the trained feature extraction model, where the iteration end condition is that the target loss is less than or equal to the loss threshold, or the number of iterations of the feature extraction model is greater than the number threshold.

7. An image feature extraction method, characterized in that, The method includes: Receiving the image to be processed; Inputting the image to be processed into the feature extraction model to obtain the feature image output by the feature extraction model, where the feature extraction model is generated by the generation method of the feature extraction model according to any one of claims 1-6.

8. A generating device for a feature extraction model, characterized in that The device includes: An acquisition module, configured to acquire multiple candidate images; A determination module, configured to determine a clustering tree corresponding to the multiple candidate images based on a feature extraction model and the multiple candidate images, where the clustering tree includes clusters at multiple levels; A generation module, configured to generate target sample pairs based on the multiple candidate images and the clustering tree, where the target sample pairs include image sample pairs and feature sample pairs, the image sample pairs are formed based on two different candidate images, and the feature sample pairs are formed based on the features of the candidate images and the features of the cluster centers of the clusters; A training module, configured to train the feature extraction model based on the target sample pairs; Wherein, the determination module is configured to: extract features of the multiple candidate images based on the feature extraction model to obtain sample features corresponding to each candidate image; perform clustering on each target feature to generate multiple clusters and the cluster centers of each cluster, where the target feature is initially the sample feature; use the cluster center of each cluster as a new target feature and perform hierarchical clustering until the level of the obtained clustering tree reaches a preset level; The generation module includes: A first generation sub-module, configured to generate a sample image set and a candidate comparison image set based on the candidate images, where the images in the candidate comparison image set are different from the images in the sample image set, and the number of images corresponding to the candidate comparison image set is greater than the number of images corresponding to the sample image set; A first determination sub-module, configured to, for each sample image in the sample image set, determine a comparison sample set of the sample image at each level based on the candidate comparison images in the candidate comparison image set and the cluster centers of the clusters at each level in the clustering tree, where the comparison samples in the comparison sample set include the features of the target comparison images and the target comparison cluster centers; A second generation sub-module, configured to, for each sample image in the sample image set, generate negative sample pairs corresponding to the sample image at each level according to the sample image and the comparison sample set of the sample image at each level, and determine the negative sample pairs as the target sample pairs.

9. An image feature extraction device, characterized in that, The apparatus includes: A receiving module, configured to receive an image to be processed; An extraction module, configured to input the image to be processed into the feature extraction model to obtain a feature image output by the feature extraction model, where the feature extraction model is generated by the generation method of the feature extraction model according to any one of claims 1-6.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by a processing device, it implements the steps of the method according to any one of claims 1-7.

11. An electronic device, characterized in that, Including: A storage device, on which at least one computer program is stored; At least one processing device, configured to execute the at least one computer program in the storage device to implement the steps of the method according to any one of claims 1-7.

Images