Image processing method and apparatus, device, medium, and program product

Abstract

The present application discloses an image processing method and apparatus, and a storage medium. The method comprises: acquiring a reference library and a retrieval library; inputting reference images in the reference library and prompts into a diffusion model to obtain estimated noise; merging the estimated noise to obtain reference noise features; determining a plurality of retrieval noise features corresponding to retrieval images; and on the basis of the similarity between the plurality of retrieval noise features and the reference noise features, determining target labels corresponding to the retrieval images, thereby achieving a label configuration process that does not require training. Because the difference in noise between images in the reference library and the retrieval library obtained by the diffusion model is used to indicate an image similarity matching process, corresponding labels can be configured for the retrieval library without training, thereby improving the image processing efficiency in a training data configuration process.

Description

Image processing method, device, equipment, medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 30, 2023, with application number 2023107953163 and application name “A method, device and storage medium for processing an image”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to image processing. Background Art

[0003] With the rapid development of artificial intelligence technology, deep learning models are widely used in tasks such as image classification and detection. However, as training data is continuously updated, how to adapt the added training data to the deep learning model is a difficult problem.

[0004] Generally, both the existing training data and the training data to be added can be input into the deep learning model for training to complete the adaptation of the added training data.

[0005] However, since the added training data are sporadic samples, it is difficult to quickly complete the model adaptation through data accumulation. The training process takes a lot of time, which affects the efficiency of image processing during the training data configuration process.

[0006] Summary of the Invention

[0007] In view of this, the present application provides an image processing method that can effectively improve the efficiency of image processing during the training data configuration process.

[0008] In a first aspect, the present application provides an image processing method, which can be applied to a system or program including an image processing function in a terminal device, and specifically includes:

[0009] Acquire a reference library and a search library, wherein reference images in the reference library are configured with image labels;

[0010] Inputting reference images in the reference library and prompts corresponding to the reference images into a diffusion model to obtain estimated noise corresponding to each reference image, wherein the prompts are determined according to the image labels;

[0011] Merging the estimated noises corresponding to the reference images based on the image labels corresponding to the reference images to obtain reference noise features corresponding to the image labels;

[0012] Combining the search image in the search library with each of the image labels to obtain a plurality of search combinations, and inputting the plurality of search combinations into the diffusion model to obtain a plurality of search noise features corresponding to the search image;

[0013] A target label corresponding to the retrieval image is determined according to feature similarities between the plurality of retrieval noise features and the reference noise features.

[0014] A second aspect of the present application provides an image processing device, comprising:

[0015] an acquisition unit, configured to acquire a reference library and a search library, wherein the reference images in the reference library are configured with image tags;

[0016] an estimating unit, configured to input reference images in the reference library and prompts corresponding to the reference images into a diffusion model to obtain estimated noise corresponding to each reference image, wherein the prompts are determined according to the image labels;

[0017] a processing unit, configured to combine the estimated noises corresponding to the reference images based on the image labels corresponding to the reference images, so as to obtain reference noise features corresponding to the image labels;

[0018] The processing unit is further configured to combine the search image in the search library with each of the image labels to obtain a plurality of search combinations, and input the plurality of search combinations into the diffusion model to obtain a plurality of search noise features corresponding to the search image;

[0019] The processing unit is further configured to determine a target label corresponding to the retrieval image according to feature similarities between the plurality of retrieval noise features and the reference noise features.

[0020] The third aspect of the present application provides a computer device, comprising: a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the image processing method described in the first aspect or any one of the first aspects according to the instructions in the program code.

[0021] On the other hand, an embodiment of the present application provides a storage medium, which is used to store a computer program, and the computer program is used to execute the method of the above aspect.

[0022] In another aspect, the present invention provides a computer program product comprising a computer program, which, when executed on a computer, causes the computer to perform the above method. As can be seen from the above technical embodiments, the present invention has the following advantages:

[0023] The proposed method involves obtaining a reference library and a search library, with reference images in the reference library assigned image labels. The reference images and their corresponding prompts are then fed into a diffusion model to obtain estimated noise for each reference image, where the prompt is determined based on the image label. The estimated noise for each reference image is then merged based on the image label to obtain reference noise features for each image label. The search images in the search library are then combined with the image labels to obtain multiple search combinations, which are then fed into the diffusion model to obtain multiple search noise features corresponding to the search images. The target label for the search image is then determined based on the feature similarity between the multiple search noise features and the reference noise features, thereby achieving a training-free label assignment process. Because the noise difference between the images in the reference library and the search library, as determined by the diffusion model, is used to indicate image similarity matching, labels can be assigned to the search library without training, improving image processing efficiency during the training data assignment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG1 is a network architecture diagram of an image processing system;

[0025] FIG2 is a flow chart of an image processing process provided by an embodiment of the present application;

[0026] FIG3 is a flowchart of an image processing method provided in an embodiment of the present application;

[0027] FIG4 is a schematic diagram of a scenario of an image processing method provided in an embodiment of the present application;

[0028] FIG5 is a schematic diagram of another image processing method provided in an embodiment of the present application;

[0029] FIG6 is a schematic diagram of another image processing method according to an embodiment of the present application;

[0030] FIG7 is a schematic diagram of another scene of an image processing method provided in an embodiment of the present application;

[0031] FIG8 is a schematic diagram of another image processing method provided in an embodiment of the present application;

[0032] FIG9 is a schematic diagram of another image processing method according to an embodiment of the present application;

[0033] FIG10 is a flowchart of another image processing method provided in an embodiment of the present application;

[0034] FIG11 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0035] FIG12 is a schematic structural diagram of a terminal device provided in an embodiment of the present application;

[0036] FIG13 is a schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application provide an image processing method and related devices, which can be applied to a system or program that includes an image processing function in a terminal device. Corresponding labels can be configured for the retrieval library without training, thereby improving the image processing efficiency during the training data configuration process.

[0038] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] First, some terms that may appear in the embodiments of this application are explained.

[0040] Stable diffusion model (SD model): a conditional latent space diffusion model.

[0041] ChatGPT: A large language model that can be used to generate common sentence conversations and other information.

[0042] Prompt: The content of the prompt text in the SD model, which is basically a short sentence.

[0043] Vincent image: The input is prompt and the output is the generated image.

[0044] Picture-to-text: The input is a picture, and the output is a description of the picture, that is, a prompt.

[0045] Reference library (gallery library): The base library used to find relevant images during the search. It has labels. That is, when searching, find the image with the most similar features in the gallery library and return its corresponding label.

[0046] Retrieval library (query library): images used for retrieval.

[0047] Free training: means that you can get the results directly without separate training.

[0048] It should be understood that the image processing method provided in the present application can be applied to a system or program in a terminal device that includes an image processing function, such as a content generation application. Specifically, the image processing system can operate in a network architecture as shown in Figure 1. As shown in Figure 1, it is a network architecture diagram of the image processing system. As can be seen from the figure, the image processing system can provide an image processing process with multiple information sources, that is, through the data configuration operation of the terminal, the server is triggered to perform label configuration operations on the added data and add it to the existing reference library to support the operation of the diffusion model; it can be understood that Figure 1 shows a variety of terminal devices, and the terminal devices can be computer devices. In actual scenarios, there can be more or fewer types of terminal devices participating in the image processing process. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 shows one server, but in actual scenarios, there can also be multiple servers involved, and the specific number of servers depends on the actual scenario.

[0049] In this embodiment, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be, but is not limited to, a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, intelligent voice interaction device, smart home appliance, or in-vehicle terminal. The terminal and server can be connected directly or indirectly via wired or wireless communication, and the terminal and server can be connected to form a blockchain network, which is not limited in this application.

[0050] It can be understood that the above-mentioned image processing system can run on a personal mobile terminal, for example: as an application such as a content generation application, it can also run on a server, and it can also run on a third-party device to provide image processing to obtain the image processing results of the information source; the specific image processing system can be run in the above-mentioned device in the form of a program, or it can be run as a system component in the above-mentioned device, or it can be used as a cloud service program. This embodiment can be applied to cloud technology, autonomous driving and other scenarios. The specific operation mode depends on the actual scenario and is not limited here.

[0051] With the rapid development of artificial intelligence technology, deep learning models are widely used in tasks such as image classification and detection. However, as training data is continuously updated, how to adapt the added training data to the deep learning model is a difficult problem.

[0052] Generally, both the existing training data and the training data to be added can be input into the deep learning model for training to complete the adaptation of the added training data.

[0053] However, since the added training data are sporadic samples, it is difficult to quickly complete the model adaptation through data accumulation. The training process takes a lot of time, which affects the efficiency of image processing during the training data configuration process.

[0054] In order to solve the above problems, the present application proposes an image processing method, which is applied to the process framework of image processing shown in Figure 2. As shown in Figure 2, a process architecture diagram of image processing provided by an embodiment of the present application is provided, which converts the difference in prediction noise with and without text condition guidance in the diffusion model into the difference in noise between the reference library and the retrieval library, and then calculates the difference in similarity between the two. Moreover, this scheme does not require re-training, thereby efficiently performing label configuration for newly added data.

[0055] It is understood that the method provided in this application can be written as a program to serve as a processing logic in a hardware system, or as an image processing device that implements the aforementioned processing logic in an integrated or external manner. As an implementation method, the image processing device obtains a reference library and a search library, wherein the reference images in the reference library are configured with image labels; then, the reference images in the reference library and the corresponding prompts are input into a diffusion model to obtain an estimated noise corresponding to each reference image, wherein the prompts are determined based on the image labels; further, the estimated noise corresponding to each reference image is merged based on the image labels corresponding to each reference image to obtain a reference noise feature corresponding to each image label; then, the search images in the search library are combined with the image labels to obtain multiple search combinations, which are then input into the diffusion model to obtain multiple search noise features corresponding to the search images; and then, based on the feature similarity between the multiple search noise features and the reference noise features, a target label corresponding to the search image is determined, thereby achieving a label configuration process that does not require training. Because the noise difference between the images in the reference library and the search library obtained by the diffusion model is used to indicate the image similarity matching process, the corresponding labels can be configured for the search library without training, thereby improving the image processing efficiency during the training data configuration process.

[0056] The embodiments provided in the present application relate to artificial intelligence computer vision technology, which is specifically described through the following embodiments:

[0057] In conjunction with the above process architecture, the image processing method in this application will be introduced below. Please refer to Figure 3, which is a flowchart of an image processing method provided in an embodiment of the present application. The processing method can be executed by a server or a terminal. The embodiment of the present application includes at least the following steps:

[0058] 301. Obtain reference library and search library.

[0059] In this embodiment, the reference library is the base library used to search for relevant images in the retrieval task. By executing the retrieval task, the reference image with the most similar features in the reference library is found during the retrieval, and its corresponding image label is returned as the target expression of the retrieval image. Therefore, the reference images in the reference library are configured with image labels, and the retrieval library is a collection of images used to retrieve labels. This collection may include training data for indicating that the deep learning model needs to be trained (corresponding target labels need to be added). The retrieval library can be related to different scenarios, that is, to achieve the purpose of adding scenarios to the deep learning model in use, so that the deep learning model can adapt to the scenario corresponding to the retrieval library.

[0060] Optionally, considering the huge amount of data in the training set where the reference images in the reference library are located, when adding a reference library, reference images with the same or similar categories as the retrieval library can be selected from the training set to generate the reference library required for this retrieval task, that is, first obtain the retrieval library associated with the retrieval task; then determine the category information corresponding to the retrieval library; and then perform image calls on the reference images associated with the retrieval task in the training set based on the category information to obtain the reference library.

[0061] In one possible scenario, the retrieval task can be based on a diffusion model (as an example of the aforementioned deep learning model), where the reference library is training data for the diffusion model, which attenuates information due to noise, and then uses the learned information to generate images. This embodiment uses the SD model as an example. The SD model was trained using LAION-5B and was trained on billions of data sets. This application can fully utilize the capabilities of the SD model trained on large-scale datasets. This embodiment points out that the image-image matching process is equivalent to matching the mean features of the gallery library noise with the mean features of the query library. A method for measuring image-image similarity is then designed, so that matching results can be obtained without additional model training.

[0062] The reference library is constructed by determining the reference images with category information corresponding to the retrieval library from the training set. This not only effectively controls the data volume of the reference library, but also makes it more relevant to the retrieval task, effectively improving the matching efficiency and quality of the target labels of the retrieval library.

[0063] By configuring labels for the search library, the diffusion model can be migrated to more commonly used tasks in real-world applications, such as classification, detection, and matching. This example uses the SD model as an example to illustrate the migration process for matching. This migration process is adaptable to various data types, quickly configuring labels by comparing the similarity of noise features, and enabling rapid data uploading.

[0064] Specifically, the comparison process of the reference library and the search library in this embodiment is shown in Figure 4, which is a scene schematic diagram of an image processing method provided by an embodiment of the present application; the figure shows that the difference in prediction noise with and without text condition guidance in the SD model is converted into the noise difference between the gallery library and the query library, and then the similarity difference between the two is calculated, and this embodiment does not require re-training.

[0065] It is understandable that the purpose of this embodiment is to add a small amount of data, along with the corresponding target tags, directly to the gallery library. Thus, when used online, for the query library as online data, it only needs to determine whether it matches the reference image in the gallery library. If it matches, the corresponding target tag is returned.

[0066] 302. Input the reference images in the reference library and the prompts corresponding to the reference images into the diffusion model to obtain the estimated noise corresponding to each reference image.

[0067] In this embodiment, the prompt is determined according to the image label. For example, if the image label is knife, then prompt is knife. The diffusion model is a pre-trained model, which can be trained based on a reference image, that is, the reference image is the training data of the SD model.

[0068] Specifically, the process of determining the estimated noise corresponding to the reference image can be a processing process based on the SD model. The structure of the SD model is shown in Figure 5, which is a scene diagram of another image processing method provided by an embodiment of the present application; the figure shows that the SD model includes an encoder, a decoder, a text image matching network, and a semantic segmentation network; specifically, first, the reference image in the reference library is input into the encoder in the diffusion model to obtain the potential vector corresponding to the reference image; then the diffusion process is performed, that is, noise is added to the potential vector to obtain a noisy vector. Further, the prompt corresponding to the reference image is input into the text image matching network (CLIP model) in the diffusion model to obtain a text vector; then a denoising process is performed, that is, the noisy vector is input into the semantic segmentation network (U-net model) in the diffusion model, and further, through the semantic segmentation network in the diffusion model, the text vector and the noisy vector are used to predict the estimated noise corresponding to the reference image.

[0069] It should be noted that in related technologies, a diffusion model is used to perform multiple denoising and multiple denoising operations to generate an image, wherein a certain amount of information is removed during each denoising process; however, in this embodiment, the semantic segmentation network in the diffusion model is used to predict the estimated noise in one go using the text vector and the noisy vector, and the denoising process may not be involved; the estimated noise obtained by the one-time prediction will contain more information, so that the reference noise feature will contain more information. Similarly, the retrieval noise feature will contain more information, thereby improving the accuracy of determining the target label using the similarity between the reference noise feature and the retrieval noise feature.

[0070] It can be understood that, for the determination process of the estimated noise, as shown in Figure 6, Figure 6 is a scene diagram of another image processing method provided by an embodiment of the present application; first, for the diffusion process in the above-mentioned SD model, different noise values ​​(Noise amount) are controlled at different times t, and different times t correspond to different noise estimation stages, thereby obtaining noisy images at different noise estimation stages. The model calculates the difference between the noise value calculated by the conditional U-net model and the assigned noise, so this embodiment can also combine multiple noise addition and multiple denoising processes.

[0071] Specifically, for combining prompts across multiple processing steps, for example, a description (prompt), the text encoder in CLIP can be used to compress it into an embedding vector. During the u-net denoising process, the attention mechanism can be used to continuously inject embedding vectors into the denoising process. Each Resnet is no longer directly connected to the adjacent Resnet, but instead an Attention module is added in between. The semantic embeddings obtained by CLIP are then fed into the Attention module for further processing. This method continuously injects semantic information, combining the text vector with the noisy vector.

[0072] In a possible scenario, in order to extract accurate prompts, the data set format used for training the SD model can be parsed. The data set format used for training the SD model is shown in Figure 7. Figure 7 is a scene diagram of another image processing method provided in an embodiment of the present application; the boxed content A1 in the figure is the prompt configured in the SD model, and its format can be aphoto of{class}.

[0073] In addition, considering that the denoising process is carried out step by step, as shown in Figure 8, Figure 8 is a scene diagram of another image processing method provided by an embodiment of the present application; each noise estimation stage (timestep) is carried out gradually over time, so in addition to selecting the output of the last noise estimation stage as the estimated noise, the output of any noise estimation stage in the noise estimation stage can also be selected as the estimated noise to characterize the noise of the reference library. Therefore, the target stage can be determined from the noise estimation stage, and then the target stage in the noise estimation stage can be determined based on the scene information.

[0074] By selecting the target stage for the noise estimation segment in a targeted manner, the computational burden can be effectively reduced and the efficiency of determining the estimated noise can be improved.

[0075] In one possible implementation, scene information corresponding to the retrieval task related to the reference library and the retrieval library is first obtained, and the target stage is determined based on the scene information. Different scene information marks the target stages of different retrieval tasks (such as the scenes corresponding to the retrieval library); then, through the semantic segmentation network in the diffusion model, the text vector and the noisy vector are used to predict the estimated noise corresponding to the reference image at the target stage, thereby adapting to the sensitivity of noise in different scenes.

[0076] In addition, the determination of the target stage (the timestep for extracting estimated noise) in different scenarios can be obtained by performing performance statistics after experiments, that is, first, through the semantic segmentation network in the diffusion model, the text vector and the noisy vector are used to predict the test noise of the reference image in each noise estimation stage; and the retrieval task is performed according to the test noise (that is, the retrieval task is performed after the target label is annotated in the subsequent embodiments) to obtain the effect information corresponding to each noise estimation stage (such as accuracy, conversion rate, etc.); then the target stage is determined based on the performance parameters indicated in the effect information, thereby improving the adaptability of different scenarios.

[0077] It is understandable that to accelerate the image generation process, the SD model does not choose to run the diffusion process on the pixel image itself, but instead chooses to run it on a compressed version of the image. Therefore, the input reference image needs to be resized. That is, first, the size information of the latent space adapted to the diffusion model is obtained; then, based on the size information, the reference image in the reference library is adjusted; and then, the adjusted reference image is input into the encoder of the diffusion model to obtain the latent vector corresponding to the reference image, thereby matching the diffusion process of the diffusion model. This latent space can be a model space used to express and generate latent vectors.

[0078] In combination with the description of the above embodiment, to determine the estimated noise, the reference images in the gallery library can be uniformly resized to 512*512. Through the existing SD model, the prompt used is the original image label (for example, if the image label is knife, the prompt is knife); then the estimated noise obtained by each category after inputting the SD model is obtained (for example, taking the last timestep of the SD model), and the obtained estimated noise is also 512*512.

[0079] In the architecture shown in Figure 5, the Contrastive Language-Image Pre-Training (CLIP) model used can quickly match text and images. This is because the CLIP model consists of an image encoder and a text encoder. The training process begins by randomly selecting an image and a piece of text from the training set. The text and image may not necessarily match, so the CLIP model's task is to predict whether the image and text match, thus starting training.

[0080] Specifically, the training process of the CLIP model is shown in Figure 9, which is a scenario diagram of another image processing method provided by an embodiment of the present application. The figure shows that after randomly extracting text and images, they can be compressed into two embedding vectors using an image encoder and a text encoder, respectively, referred to as the image embedding and the text embedding, i.e., the two 3*1 vectors in the figure. The cosine similarity can then be used to compare the similarity of the two embedding vectors to determine whether the randomly extracted text and image match.

[0081] It's important to note that during the initial training phase, even if the image and text are well-matched, the two encoders are newly initialized and their parameters are chaotic. Consequently, the two embedding vectors are also chaotic, and the calculated similarity is often close to 0. This can lead to situations where, even though the image and text are paired and their image labels indicate similarity, the cosine similarity predictions indicate dissimilarity. In this case, the mismatch between the image labels (similar) and the predictions (not similar) can be used to perform backpropagation based on this matching result to update the parameters of the two encoders. By repeatedly repeating this backpropagation process, the two encoders can be trained effectively. For paired images and text, the two encoders will eventually output similar embedding vectors, resulting in a cosine similarity close to 1. However, for mismatched images and text, the two encoders will output very different embedding vectors, causing the cosine similarity to be close to 0.

[0082] In one possible scenario, if a CLIP model is fed a picture of a puppy and a caption like "puppy photo," the model will generate two similar embedding vectors, thus determining that the text and image match. This unified mathematical representation of image and text allows for the conversion of text into image information using a Text Encoder, or into language information using an Image Encoder. These two can then be used in conjunction with the SD model to improve the matching between text and image.

[0083] Based on the above description of the SD model, the predicted value of the calculated label should be the difference between the estimated noise of different text (label) conditional guidance and the estimated noise of unconditional guidance. The smaller the difference, the more likely it belongs to the corresponding text or label. Therefore, we can set the estimated noise obtained by a picture in the gallery library (under any label) as A, the estimated noise obtained by a picture in the query library (under any label) as B, and the estimated noise obtained by the provided unconditional guidance picture (no label) as C. Then the similarity between any two pictures in the gallery library and the query library is:

[0084] |AC|-|BC|=|AB|

[0085] Therefore, the problem of calculating similarity can be summarized as directly calculating the difference between the estimated noise of the gallery library images and the estimated noise of the query library images. The calculation process of this estimated noise difference is described below.

[0086] 303. Based on the image labels corresponding to the reference images, the estimated noises corresponding to the reference images are merged to obtain reference noise features corresponding to the image labels.

[0087] In this embodiment, the process of merging estimated noise based on image labels is the process of merging the estimated noise of the image category indicated by the image label. Since the same image label corresponds to multiple noise estimates of different images, noise merging can be performed. That is, assuming that the training set has N labels, and each label has {n1, n2, ..nn} pictures, it is necessary to merge the noise features obtained for each category under the N labels. Among them, assuming that the image label corresponds to n1 pictures, n1 512*512 estimated noises will be obtained.

[0088] Specifically, these estimated noises are summed and averaged pixel by pixel to obtain the final estimated noise of the first category. The same method is used to obtain the estimated noise of N categories. For this process, the estimated noise corresponding to each reference image under the image label can be first obtained to obtain a noise set; then the estimated noise in the noise set is summed to obtain the total noise; and then the total noise is averaged according to the number of estimated noises in the noise set to obtain the reference noise feature corresponding to the image label.

[0089] In addition, since the images within the same label are similar, the characteristics of the estimated noise can be determined by the median, that is, the merging process can also perform numerical statistics on the values ​​of the estimated noise at each pixel point in the noise set to obtain statistical results; then determine the median of the values ​​at each pixel point in the statistical results to obtain the reference noise characteristics corresponding to the image label.

[0090] Since the mean or median of the reference images under the image label can reflect the distribution of the estimated noise from the overall level, the reference noise characteristics determined by the mean or median of the estimated noise can more comprehensively reflect the noise characteristics of each reference image under the image label.

[0091] It is understandable that the above merging method is only an example, and the specific merging method depends on the actual scenario.

[0092] 304. Combining the retrieval images in the retrieval library with the respective image labels to obtain a plurality of retrieval combinations, and inputting the plurality of retrieval combinations into a diffusion model to obtain a plurality of retrieval noise features corresponding to the retrieval images.

[0093] In this example, multiple search combinations are input into the diffusion model to obtain features from the query library. The acquisition process is similar to that in the gallery library. That is, the images in the query library are first resized to 512*521 and then input one by one into the frozen SD model. However, the input prompt is the label system set of the current reference library. The following search combination input can be obtained:

[0094] picture+label1;

[0095] picture+label2;

[0096] …

[0097] Picture+labeln.

[0098] Therefore, n 512*512 feature sets can be obtained.

[0099] It should be noted that the above embodiment introduces a process of determining the estimated noise based on different noise estimation stages. At this time, the target stage (timestep) used in the process of determining the retrieval noise feature needs to correspond, that is, the target stage in the noise estimation stage used when first obtaining the estimated noise corresponding to the reference image; then, the retrieval image in the retrieval library is combined with each image label to obtain multiple retrieval combinations; and the multiple retrieval combinations are input into the diffusion model to obtain multiple retrieval noise features corresponding to the retrieval image based on the target stage, thereby improving the effectiveness of the feature comparison process.

[0100] 305. Determine a target label corresponding to the retrieval image based on feature similarities between the multiple retrieval noise features and the reference noise features.

[0101] In this embodiment, the feature similarity comparison process may be performed based on a comparison unit, such as a single pixel or a pixel unit of other sizes; and the feature similarity calculation process may be performed by calculating cosine similarity.

[0102] In one possible scenario, the cosine similarity of the n 512*512 dimensional features obtained for each image in the query library can be calculated pixel by pixel with the N estimated noises in the gallery library. This is as follows:

[0103] Where x1 is the comparison unit in the reference noise feature, x2 is the comparison unit in the search noise feature, and k is the number of pixels, that is, the number of 512*512 points. The reference label in the gallery corresponding to the minimum cosine similarity is the image category corresponding to the search image, and the corresponding target label is determined.

[0104] In addition, to improve the matching efficiency, similarity calculation can also be performed based on the matching window. The matching window can accurately identify the feature sampling range used for feature matching. That is, multiple adjacent pixels are used as matching units for similarity calculation. Specifically, you can first obtain a matching window configured for the retrieval task; then, based on the matching window, sample the retrieval noise features to obtain the retrieval window features; and based on the matching window, sample the reference noise features to obtain the reference window features; then, perform cosine similarity calculation on the retrieval window features and the reference window features to obtain feature similarity; and then determine the target label corresponding to the retrieval image based on the feature similarity, thereby achieving an efficient matching process.

[0105] In addition, since the retrieval library of this embodiment is used to add training data, it is necessary to consider the online cycle of the corresponding product. Therefore, the comparison window can be configured according to different efficiency requirements, that is, first obtain the time limit information configured for the retrieval task; then determine the quantity information corresponding to the retrieval image in the retrieval library; and determine the efficiency parameter based on the time limit information and quantity information; then compare the efficiency parameter with the preset efficiency to obtain the acceleration ratio (for example, if the efficiency parameter is four times the preset efficiency, the acceleration ratio is 4); and then configure the comparison window according to the acceleration ratio, for example, configure a 2*2 pixel window for similarity comparison to adapt to the model online requirements.

[0106] Through the above labeling process, we can quickly match similar images in the gallery based on a small amount of data and return their labels. This process can be completed efficiently because it does not require training.

[0107] In conjunction with the above embodiments, it can be seen that by obtaining a reference library and a search library, the reference images in the reference library are assigned image labels. The reference images in the reference library and the corresponding prompts are then input into a diffusion model to obtain estimated noise corresponding to each reference image, where the prompts are determined based on the image labels. The estimated noise corresponding to each reference image is further merged based on the image labels corresponding to each reference image to obtain reference noise features corresponding to each image label. The search images in the search library are then combined with the image labels to obtain multiple search combinations, which are then input into the diffusion model to obtain multiple search noise features corresponding to the search images. The target label corresponding to the search image is then determined based on the feature similarity between the multiple search noise features and the reference noise features, thereby achieving a label assignment process that does not require training. Because the noise difference between the images in the reference library and the search library, obtained by the diffusion model, is used to indicate the image similarity matching process, the corresponding labels can be assigned to the search library without training, thereby improving the image processing efficiency during the training data assignment process.

[0108] The above embodiment describes the process of configuring labels for a search library. After configuring the labels, they can be quickly added to the reference library, thereby executing related tasks based on the diffusion model. The following describes this scenario. Please refer to Figure 10, which is a flowchart of another image processing method provided by an embodiment of the present application. This embodiment of the present application includes at least the following steps:

[0109] 1001. In response to a search operation of a target object in a target application, determine an input image and input text.

[0110] In this embodiment, the target application is associated with the retrieval library, that is, the capabilities of the diffusion model corresponding to the reference library are transferred to the corresponding field of the target application, such as classification, detection, matching and other tasks. The specific task form depends on the actual scenario.

[0111] 1002. Configuring the search library marked with the target tag in the diffusion model;

[0112] In this embodiment, by configuring a search library tagged with target tags within the diffusion model, the diffusion model's capabilities are migrated. Specifically, a small amount of customer data, along with the configured target tags, can be directly added to the reference library. Thus, when used online, the online data, acting as a search library, only needs to determine whether it matches images in the reference library. If so, the corresponding tags are returned.

[0113] Specifically, the configuration process of the target tag refers to steps 301 to 305 of the embodiment shown in FIG. 3 , which will not be described in detail here.

[0114] 1003. Input the input image and input text into the configured diffusion model to obtain a generated result.

[0115] In this embodiment, the diffusion model can be a large-scale image generation model in the CV field represented by SD; in addition, the diffusion model can also be a large-scale text question-answering model in the NLP field represented by the GPT series.

[0116] By migrating the diffusion model generation capability obtained from a training set of hundreds of millions of units to the application corresponding to the retrieval library, the application development cycle can be shortened and the application performance can be quickly improved. For example, if the target application is image generation in AI-generated content, and a new style needs to be added to the existing image style (reference library), the retrieval library can be configured according to the new style. After performing the above-mentioned label annotation, it can be quickly adapted to the existing generation model without training, realizing an efficient image processing process.

[0117] In order to better implement the above embodiments of the present application, the following also provides related devices for implementing the above embodiments. Please refer to Figure 11, which is a structural diagram of an image processing device provided by the embodiment of the present application. The processing device 1100 includes:

[0118] An acquisition unit 1101 is configured to acquire a reference library and a search library, wherein the reference images in the reference library are configured with image tags;

[0119] an estimating unit 1102, configured to input reference images in the reference library and prompts corresponding to the reference images into a diffusion model to obtain estimated noise corresponding to each reference image, wherein the prompt is determined based on the image label;

[0120] The processing unit 1103 is configured to combine the estimated noises corresponding to the reference images based on the image labels corresponding to the reference images to obtain reference noise features corresponding to the image labels.

[0121] The processing unit 1103 is further configured to combine the search image in the search library with each of the image labels to obtain a plurality of search combinations, and input the plurality of search combinations into the diffusion model to obtain a plurality of search noise features corresponding to the search image;

[0122] The processing unit 1103 is further configured to determine a target label corresponding to the retrieval image according to feature similarities between the plurality of retrieval noise features and the reference noise features.

[0123] Optionally, in some possible implementations of the present application, the estimating unit 1102 is specifically configured to input a reference image in the reference library into an encoder in the diffusion model to obtain a latent vector corresponding to the reference image;

[0124] The estimation unit 1102 is specifically configured to add noise to the latent vector to obtain a noisy vector;

[0125] The estimation unit 1102 is specifically configured to input the prompt corresponding to the reference image into the text image matching network in the diffusion model to obtain a text vector;

[0126] The estimation unit 1102 is specifically configured to predict the estimated noise corresponding to the reference image using the text vector and the noisy vector through the semantic segmentation network in the diffusion model.

[0127] Optionally, in some possible implementations of the present application, the estimation unit 1102 is specifically configured to determine a target stage in the noise estimation stage;

[0128] The estimation unit 1102 is specifically configured to predict the estimated noise corresponding to the reference image at the target stage using the text vector and the noisy vector through the semantic segmentation network in the diffusion model.

[0129] Optionally, in some possible implementations of the present application, the estimation unit 1102 is specifically configured to obtain scene information corresponding to the retrieval task related to the reference library and the retrieval library; and determine a target stage in the noise estimation stage according to the scene information;

[0130] Alternatively, the estimation unit 1102 is specifically configured to predict the test noise corresponding to the reference image in each of the noise estimation stages using the text vector and the noisy vector through a semantic segmentation network in the diffusion model;

[0131] The estimation unit 1102 is specifically configured to perform the retrieval task according to the test noise to obtain effect information corresponding to each noise estimation stage;

[0132] The estimation unit 1102 is specifically configured to determine the target stage based on the performance parameters indicated in the effect information.

[0133] Optionally, in some possible implementations of the present application, the estimating unit 1102 is specifically configured to obtain a target stage in a noise estimation stage used when determining the corresponding estimated noise from the reference image;

[0134] The estimation unit 1102 is specifically configured to execute a process of determining retrieval noise features in the diffusion model by performing multiple retrieval combinations based on the target stage.

[0135] Optionally, in some possible implementations of the present application, the estimating unit 1102 is specifically configured to obtain size information adapted by the diffusion model corresponding to the latent space;

[0136] The estimation unit 1102 is specifically configured to adjust the reference image in the reference library based on the size information;

[0137] The estimation unit 1102 is specifically configured to input the adjusted reference image into an encoder in the diffusion model to obtain a latent vector corresponding to the reference image.

[0138] Optionally, in some possible implementations of the present application, the acquiring unit 1101 is specifically configured to acquire the search library associated with the search task;

[0139] The acquisition unit 1101 is specifically configured to determine category information corresponding to the search library;

[0140] The acquisition unit 1101 is specifically configured to perform image call on the training set associated with the retrieval task based on the category information to obtain the reference library.

[0141] Optionally, in some possible implementations of the present application, the acquiring unit 1101 is specifically configured to acquire estimated noise corresponding to each reference image under the image label to obtain a noise set;

[0142] The acquiring unit 1101 is specifically configured to sum the estimated noise in the noise set to obtain a total noise amount;

[0143] The acquisition unit 1101 is specifically configured to average the total amount of noise according to the amount of estimated noise in the noise set, so as to obtain a reference noise feature corresponding to the image label.

[0144] Optionally, in some possible implementations of the present application, the processing unit 1103 is specifically configured to obtain estimated noise corresponding to each reference image under the image label to obtain a noise set;

[0145] The processing unit 1103 is specifically configured to perform numerical statistics on the value of the estimated noise in the noise set at each pixel point to obtain a statistical result;

[0146] The processing unit 1103 is specifically configured to determine the median of the values ​​at each pixel point in the statistical result to obtain the reference noise feature corresponding to the image label.

[0147] Optionally, in some possible implementations of the present application, the processing unit 1103 is specifically configured to obtain a comparison window configured for a search task, where the search task is associated with the search library;

[0148] The processing unit 1103 is specifically configured to sample the retrieval noise feature based on the comparison window to obtain a retrieval window feature;

[0149] The processing unit 1103 is specifically configured to sample the reference noise feature based on the comparison window to obtain a reference window feature;

[0150] The processing unit 1103 is specifically configured to perform cosine similarity calculation on the retrieval window feature and the reference window feature to obtain the feature similarity;

[0151] The processing unit 1103 is specifically configured to determine a target label corresponding to the search image according to the feature similarity.

[0152] Optionally, in some possible implementations of the present application, the processing unit 1103 is specifically configured to obtain time limit information configured for the retrieval task;

[0153] The processing unit 1103 is specifically configured to determine quantity information corresponding to the search image in the search library;

[0154] The processing unit 1103 is specifically configured to determine an efficiency parameter based on the time limit information and the quantity information;

[0155] The processing unit 1103 is specifically configured to compare the efficiency parameter with a preset efficiency to obtain an acceleration ratio;

[0156] The processing unit 1103 is specifically configured to configure the comparison window according to the acceleration ratio.

[0157] Optionally, in some possible implementations of the present application, the processing unit 1103 is specifically configured to determine the input image and the input text in response to a search operation of the target object in a target application, the target application being associated with a search library;

[0158] The processing unit 1103 is specifically configured to configure the search library marked with the target tag in the diffusion model;

[0159] The processing unit 1103 is specifically configured to input the input image and the input text into the configured diffusion model to obtain a generation result.

[0160] The proposed method involves obtaining a reference library and a search library, with reference images in the reference library assigned image labels. The reference images and their corresponding prompts are then fed into a diffusion model to obtain estimated noise for each reference image, where the prompt is determined based on the image label. The estimated noise for each reference image is then merged based on the image label to obtain reference noise features for each image label. The search images in the search library are then combined with the image labels to obtain multiple search combinations, which are then fed into the diffusion model to obtain multiple search noise features corresponding to the search images. The target label for the search image is then determined based on the feature similarity between the multiple search noise features and the reference noise features, thereby achieving a training-free label assignment process. Because the noise difference between the images in the reference library and the search library, as determined by the diffusion model, is used to indicate image similarity matching, labels can be assigned to the search library without training, improving image processing efficiency during the training data assignment process.

[0161] The present application also provides a terminal device, as shown in FIG12 , which is a schematic diagram of the structure of another terminal device provided by the present application. For ease of explanation, only the parts related to the present application are shown. For specific technical details not disclosed, please refer to the method section of the present application. The terminal device can be any terminal device including a mobile phone, tablet computer, personal digital assistant (PDA), point of sales (POS), car computer, etc. Taking the mobile phone as an example:

[0162] FIG12 is a block diagram illustrating a portion of the structure of a mobile phone related to a terminal provided in an embodiment of the present application. Referring to FIG12 , the mobile phone includes components such as a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290. Those skilled in the art will appreciate that the mobile phone structure shown in FIG12 does not limit the mobile phone and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0163] The following is a detailed introduction to the various components of the mobile phone in conjunction with Figure 12:

[0164] The RF circuit 1210 may be used for sending and receiving information or receiving and sending signals during a call.

[0165] The memory 1220 may be used to store software programs and modules. The processor 1280 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1220 .

[0166] The input unit 1230 may be configured to receive input digital or character information and generate key signal input related to user settings and function control of the mobile phone. Specifically, the input unit 1230 may include a touch panel 1231 and other input devices 1232 .

[0167] The display unit 1240 may be configured to display information input by a user or information provided to a user, as well as various menus of the mobile phone. The display unit 1240 may include a display panel 1241 .

[0168] The mobile phone may further include at least one sensor 1250 .

[0169] The audio circuit 1260 , the speaker 1261 , and the microphone 1262 can provide an audio interface between the user and the mobile phone.

[0170] The processor 1280 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 1220 and calling data stored in the memory 1220, it executes various functions of the mobile phone and processes data, thereby performing overall detection of the mobile phone.

[0171] The mobile phone also includes a power supply 1290 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 1280 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0172] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0173] In the embodiment of the present application, the processor 1280 included in the terminal also has the function of executing each step of the above-mentioned page processing method.

[0174] The embodiment of the present application also provides a server. Please refer to Figure 13. Figure 13 is a structural diagram of a server provided by an embodiment of the present application. The server 1300 may have relatively large differences due to different configurations or performances. It may include one or more central processing units (CPUs) 1322 (for example, one or more processors) and a memory 1332, and one or more storage media 1330 (for example, one or more mass storage devices) for storing application programs 1342 or data 1344. Among them, the memory 1332 and the storage medium 1330 can be temporary storage or permanent storage. The program stored in the storage medium 1330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1322 can be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the server 1300.

[0175] The server 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358, and / or one or more operating systems 1341, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0176] The steps executed by the management device in the above embodiment can be based on the server structure shown in FIG13 .

[0177] In addition, an embodiment of the present application further provides a storage medium, which is used to store a computer program, and the computer program is used to execute the method provided by the above embodiment.

[0178] An embodiment of the present application further provides a computer program product including a computer program, which, when executed on a computer, enables the computer to execute the method provided in the above embodiment.

[0179] An embodiment of the present application also provides an image processing system, which may include the image processing device in the embodiment described in Figure 11, or the terminal device in the embodiment described in Figure 12, or the server described in Figure 13.

[0180] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present application.

[0183] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical embodiment can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, an image processing device, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0185] As described above, the above embodiments are only used to illustrate the technical embodiments of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in this field should understand that they can still modify the technical embodiments recorded in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not cause the essence of the corresponding technical embodiments to deviate from the spirit and scope of the technical embodiments of the various embodiments of the present application.

Claims

1. A method for processing an image, the method being executed by a computer device, the method comprising: Acquire a reference library and a search library, wherein reference images in the reference library are configured with image tags; Inputting reference images in the reference library and prompts corresponding to the reference images into a diffusion model to obtain estimated noise corresponding to each of the reference images, wherein the prompts are determined according to the image labels; Based on the image labels corresponding to the reference images, the estimated noises corresponding to the reference images are merged to obtain reference noise features corresponding to the image labels; Combining the search images in the search library with the image labels to obtain a plurality of search combinations, and inputting the plurality of search combinations into the diffusion model to obtain a plurality of search noise features corresponding to the search images; According to the feature similarities between the plurality of retrieval noise features and the reference noise features, a target label corresponding to the retrieval image is determined.

2. The method according to claim 1, wherein the inputting the reference images in the reference library and the prompts corresponding to the reference images into the diffusion model to obtain the estimated noise corresponding to each of the reference images comprises: Inputting a reference image in the reference library into an encoder in the diffusion model to obtain a latent vector corresponding to the reference image; Adding noise to the latent vector to obtain a noisy vector; Inputting the prompt corresponding to the reference image into the text image matching network in the diffusion model to obtain a text vector; The estimated noise corresponding to the reference image is predicted using the text vector and the noisy vector through a semantic segmentation network in the diffusion model.

3. The method according to claim 2, wherein the step of predicting the estimated noise corresponding to the reference image by using the text vector and the noisy vector through the semantic segmentation network in the diffusion model comprises: determining a target stage in a noise estimation stage; The estimated noise corresponding to the reference image at the target stage is predicted by using the text vector and the noisy vector through the semantic segmentation network in the diffusion model.

4. The method according to claim 3, wherein determining the target stage in the noise estimation stage comprises any one of the following methods: Method 1: Acquire scene information corresponding to the retrieval task related to the reference library and the retrieval library; Determining a target stage in a noise estimation stage according to the scene information; Method 2: Using the semantic segmentation network in the diffusion model, predicting the test noise corresponding to the reference image in each of the noise estimation stages using the text vector and the noisy vector; Executing the retrieval task according to the test noise to obtain effect information corresponding to each of the noise estimation stages; The target stage is determined based on the performance parameters indicated in the effect information.

5. The method according to claim 3, further comprising: Obtaining a target stage in a noise estimation stage used when determining a corresponding estimated noise from the reference image; A process of determining a retrieval noise feature in the diffusion model is performed based on the target stage using a plurality of retrieval combinations.

6. The method according to claim 2, wherein inputting the reference image in the reference library into the encoder in the diffusion model to obtain the latent vector corresponding to the reference image comprises: Obtaining size information adapted to the latent space corresponding to the diffusion model; Adjusting the reference image in the reference library based on the size information; The adjusted reference image is input into an encoder in the diffusion model to obtain a latent vector corresponding to the reference image.

7. The method according to claim 1, wherein obtaining a reference library and a search library comprises: Acquire the search library associated with the search task; Determining category information corresponding to the search library; Based on the category information, an image call is performed on the training set associated with the retrieval task to obtain the reference library.

8. The method according to claim 1, wherein the estimated noises corresponding to the reference images are combined based on the image labels corresponding to the reference images to obtain reference noise features corresponding to the image labels, comprising: Obtaining estimated noise corresponding to each of the reference images under the image label to obtain a noise set; summing the estimated noises in the noise set to obtain a total noise amount; The total amount of noise is averaged according to the amount of estimated noise in the noise set to obtain a reference noise feature corresponding to the image label.

9. The method according to claim 1, wherein the estimated noises corresponding to the reference images are combined based on the image labels corresponding to the reference images to obtain reference noise features corresponding to the image labels, comprising: Obtaining estimated noise corresponding to each of the reference images under the image label to obtain a noise set; Performing numerical statistics on the value of the estimated noise in the noise set at each pixel point to obtain a statistical result; The median of the values ​​at each pixel in the statistical result is determined to obtain a reference noise feature corresponding to the image label.

10. The method according to claim 1, wherein determining the target label corresponding to the retrieval image according to the feature similarity between the plurality of retrieval noise features and the reference noise features comprises: Obtaining a comparison window configured for a search task, wherein the search task is associated with the search library; Sampling the retrieval noise feature based on the comparison window to obtain a retrieval window feature; Sampling the reference noise feature based on the comparison window to obtain a reference window feature; Performing cosine similarity calculation on the retrieval window feature and the reference window feature to obtain the feature similarity; The target label corresponding to the retrieval image is determined according to the feature similarity.

11. The method according to claim 10, wherein obtaining a comparison window configured for a search task comprises: Obtaining time limit information configured for the retrieval task; Determine the quantity information corresponding to the searched images in the search library; determining an efficiency parameter based on the time limit information and the quantity information; Comparing the efficiency parameter with a preset efficiency to obtain an acceleration ratio; The comparison window is configured according to the acceleration ratio.

12. The method according to any one of claims 1 to 11, further comprising: determining an input image and an input text in response to a search operation of a target object in a target application, the target application being associated with a search repository; The search library marked with target tags is configured in the diffusion model; The input image and the input text are input into the configured diffusion model to obtain a generation result.

13. An image processing device, comprising: An acquisition unit, used for acquiring a reference library and a search library, wherein the reference images in the reference library are configured with image tags; an estimation unit, configured to input reference images in the reference library and prompts corresponding to the reference images into a diffusion model to obtain estimated noise corresponding to each of the reference images, wherein the prompts are determined according to the image labels; a processing unit, configured to merge the estimated noises corresponding to the reference images based on the image labels corresponding to the reference images, so as to obtain reference noise features corresponding to the image labels; The processing unit is further used to respectively combine the search image in the search library with each of the image labels to obtain a plurality of search combinations, so as to input the plurality of search combinations into the diffusion model to obtain a plurality of search noise features corresponding to the search image; The processing unit is further used to determine the target label corresponding to the retrieval image according to the feature similarity between the plurality of retrieval noise features and the reference noise feature.

14. A computer device, comprising a processor and a memory: The memory is used to store program codes; the processor is used to execute the image processing method according to any one of claims 1 to 12 according to instructions in the program codes.

15. A storage medium, the storage medium being used to store a computer program, the computer program being used to execute the image processing method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program, which, when executed on a computer, enables the computer to execute the image processing method according to any one of claims 1 to 12.

Images