Generated image evaluation method and device, computer equipment and storage medium

Abstract

The invention relates to a generated image evaluation method and device, computer equipment and a storage medium. The method comprises the following steps: establishing a multi-dimensional evaluation index of a generated image; acquiring image configuration information, inputting the image configuration information into a to-be-evaluated diffusion model, and outputting a plurality of generated images corresponding to the image configuration information through the to-be-evaluated diffusion model; target image configuration information and a generated image corresponding to the target image configuration information are obtained according to a target evaluation index, the target evaluation index is evaluated according to a preset rule, an evaluation result of the target evaluation index is obtained, and the evaluation index comprises image generation quality, preference, style compatibility and LoRA compatibility; and obtaining an evaluation result of the multi-dimensional evaluation index, and evaluating the to-be-evaluated diffusion model. By adopting the method, the quality of the image generated by the diffusion model can be evaluated from the dimensions of image generation quality, preference, style compatibility and LoRA compatibility.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment and storage medium for generating image evaluation. Background Art

[0002] With the widespread application of text-generated image algorithms such as Stable Diffusion in the field of e-commerce creativity, image generation technology has made significant progress. Stable Diffusion is an open source diffusion model that can generate high-quality images based on text descriptions. Since its release in 2022, due to its powerful image generation capabilities and open ecosystem, a large number of generative models have emerged that are fine-tuned or fused based on the stable diffusion model.

[0003] However, although these generative models provide a wealth of options for creative design, their image generation quality varies and often generates non-good images that do not meet human preferences. At present, the industry mainly uses the following two methods to evaluate the effect of text images: 1. Use FID and CLIP scores for quantitative evaluation. FID measures the average distance between the generated image and the reference real image, but this indicator does not include real human preferences, and the fine-tuned model may not truly reflect its performance in actual use. In addition, FID performs poorly in distinguishing preferred images from non-preferred images and is not a reliable indicator. The CLIP score uses the CLIP model to measure the match between the generated image and the text, but a single indicator cannot fully reflect the quality and generalization of the image. 2. Use a fixed prompt set to score the similarity and realism of the generated images. This method relies on manual judgment, is full of subjectivity, and has many standards. It is usually scored by a general model and then judged subjectively by humans. This is not only time-consuming and labor-intensive, but also difficult to ensure consistency and objectivity. Summary of the invention

[0004] Based on this, it is necessary to provide a generated image evaluation method, device, computer equipment and storage medium that can perform multi-dimensional evaluation on diffusion model generated images in order to address the above technical problems.

[0005] In one aspect, a method for generating an image evaluation is provided, the method comprising:

[0006] Establish multi-dimensional evaluation metrics for generated images;

[0007] Acquire image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0008] According to the target evaluation index, target image configuration information and a generated image corresponding to the target image configuration information are obtained, and the target evaluation index is evaluated according to a preset rule to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility;

[0009] The evaluation results of the multi-dimensional evaluation indicators are obtained, and the diffusion model to be evaluated is evaluated.

[0010] In one embodiment, the image configuration information includes image type and description information; the step of obtaining the image configuration information and inputting it into a diffusion model to be evaluated, and outputting a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated, comprises:

[0011] Determining a target image type according to the diffusion model to be evaluated;

[0012] A plurality of groups of description information are acquired according to the target image type, and the plurality of groups of description information are respectively input into the diffusion model to be evaluated to obtain a plurality of generated images corresponding to each of the description information.

[0013] In one embodiment, the step of acquiring target image configuration information and a generated image corresponding to the target image configuration information according to a target evaluation indicator, and evaluating the target evaluation indicator according to a preset rule to obtain an evaluation result of the target evaluation indicator includes:

[0014] In response to the target evaluation indicator being one or more of image generation quality and preference, obtaining a plurality of first description information of a target image type and a plurality of first generated images corresponding to each of the first description information, wherein the image generation quality includes image aesthetics and / or consistency;

[0015] In response to the target evaluation indicator being image generation quality, inputting the first generated image into a pre-trained image aesthetic scoring model to obtain an image aesthetic score of the diffusion model to be evaluated; and / or,

[0016] The CLIP model is used to extract feature vectors of the first description information and the corresponding first generated image respectively, and the consistency score of the diffusion model to be evaluated is obtained by calculating the cosine similarity.

[0017] In one embodiment, the method further comprises:

[0018] In response to the target evaluation indicator being preference, inputting the first description information and the corresponding first generated image into a pre-trained HPSv2 model to obtain a first preference score of the diffusion model to be evaluated; and / or,

[0019] The first description information and the corresponding first generated image are input into a pre-trained ImageReward model to obtain a second preference score of the diffusion model to be evaluated.

[0020] In one embodiment, the step of acquiring target image configuration information and a generated image corresponding to the target image configuration information according to the target evaluation index, and evaluating the target evaluation index according to a preset rule to obtain an evaluation result of the target evaluation index further includes:

[0021] In response to the target evaluation indicator being style compatibility, obtaining a plurality of second description information of a target image type and a plurality of second generated images corresponding to each of the second description information, wherein the second description information respectively has a different style type;

[0022] Acquire a corresponding first style reference image according to the style type of the second description information;

[0023] A style loss between the second generated image and the corresponding first style reference image is calculated as a style compatibility score of the diffusion model to be evaluated.

[0024] In one embodiment, the step of acquiring target image configuration information and a generated image corresponding to the target image configuration information according to the target evaluation index, and evaluating the target evaluation index according to a preset rule to obtain an evaluation result of the target evaluation index further includes:

[0025] In response to the target evaluation indicator being LoRA compatibility, obtaining a plurality of LoRA models of the target image type and corresponding third description information, and a plurality of third generated images corresponding to each of the third description information, the LoRA models respectively having different style types;

[0026] Acquire a corresponding second style reference image according to the style type of the LoRA model;

[0027] A style loss between the third generated image and the corresponding second style reference image is calculated as a LoRA compatibility score of the diffusion model to be evaluated.

[0028] In one embodiment, obtaining the evaluation result of the multi-dimensional evaluation index and evaluating the diffusion model to be evaluated includes:

[0029] The scores of the image generation quality, preference, style compatibility and LoRA compatibility are obtained, and weighted summation and standardization are performed to obtain the evaluation result of the diffusion model to be evaluated.

[0030] In another aspect, a device for generating an image evaluation is provided, the device comprising:

[0031] Establish a module for establishing multi-dimensional evaluation indicators of image generation quality;

[0032] A generating module, configured to obtain image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0033] A first evaluation module is used to obtain target image configuration information and a corresponding generated image according to a target evaluation indicator, and evaluate the target evaluation indicator according to a preset rule to obtain an evaluation result of the target evaluation indicator, wherein the evaluation indicator includes image generation quality, preference, style compatibility and LoRA compatibility;

[0034] The second evaluation module is used to obtain the evaluation results of the multi-dimensional evaluation indicators and evaluate the diffusion model to be evaluated.

[0035] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0036] Establish multi-dimensional evaluation metrics for generated images;

[0037] Acquire image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0038] According to the target evaluation index, target image configuration information and a generated image corresponding to the target image configuration information are obtained, and the target evaluation index is evaluated according to a preset rule to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility;

[0039] The evaluation results of the multi-dimensional evaluation indicators are obtained, and the diffusion model to be evaluated is evaluated.

[0040] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0041] Establish multi-dimensional evaluation metrics for generated images;

[0042] Acquire image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0043] According to the target evaluation index, target image configuration information and a generated image corresponding to the target image configuration information are obtained, and the target evaluation index is evaluated according to a preset rule to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility;

[0044] The evaluation results of the multi-dimensional evaluation indicators are obtained, and the diffusion model to be evaluated is evaluated.

[0045] The above-mentioned generated image evaluation method, device, computer equipment and storage medium avoid the limitations of a single indicator by establishing a multi-dimensional evaluation indicator, and can more effectively screen out more suitable models in directional scenarios, laying the foundation for model fine-tuning, while avoiding the probability of generating image collapse under the black box, and through automated quantitative evaluation, it can save the resources required for manual scoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a process flow of generating an image evaluation method in one embodiment;

[0047] Figure 2 A structural block diagram of an image evaluation device is generated in one embodiment;

[0048] Figure 3 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Stable Diffusion: A diffusion model developed based on latent diffusion. The current mainstream StableDiffusion model versions are 1.5 / 2.x / sdxl.

[0051] Prompt: prompt word. This application evaluates the generated images of the open source model Stable Diffusion, so the prompts used are all in English. Stable Diffusion does not support Chinese. In the specific business process, the user inputs Chinese and uses chatgpt to output English prompts.

[0052] LoRA: Fine-tuning the large model can change the style and output effect of the Stable Diffusion large model.

[0053] CLIP: A graph-text retrieval method for computing image and text features.

[0054] In one embodiment, Figure 1 As shown, a method for generating image evaluation is provided, comprising the following steps:

[0055] Step S1, establishing a multi-dimensional evaluation index for the generated image.

[0056] In this embodiment, the diffusion model to be evaluated adopts a Stable Diffusion model, for example, dreamshape, chilloutmix, realistic vision, majicmix realistic, etc., which are more friendly to e-commerce scenarios, can be selected, and then the image quality generated by the model is evaluated through multi-dimensional evaluation indicators.

[0057] Specifically, the evaluation indicators for the Stable Diffusion model include four dimensions: 1. Image generation quality, including the image aesthetic score and consistency score of the generated image, which are used to evaluate the image generation quality and the consistency of following the prompt, which is the most basic ability of the model; 2. Preference, using the HPSv2 and ImageReward indicators. The HPSv2 indicator is used to evaluate whether the images generated by the model conform to human aesthetics and whether they are visually friendly in visual scenes. The ImageReward indicator is used to evaluate whether the images generated by the model will produce toxic or harmful content. These two indicators collectively reflect the alignment of the images generated by the model in the human dimension; 3. Style compatibility, that is, the stylization ability of the model itself; 4. LoRA compatibility, that is, the model's performance ability for the LoRA plug-in. LoRA is an important plug-in for the Stable Diffusion model. Through LoRA, compatibility with various scenes and styles can be achieved. For example, it can generate human platform pictures, gimbal pictures, etc., as well as different scenes, such as kitchens, living rooms, woods, etc., with rich content programs.

[0058] Step S2: acquiring image configuration information and inputting it into a diffusion model to be evaluated, and outputting a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated.

[0059] In one embodiment, the image configuration information includes an image type and description information; the image configuration information is obtained and input into a diffusion model to be evaluated, and a plurality of generated images corresponding to the image configuration information are output through the diffusion model to be evaluated, including: determining a target image type according to the diffusion model to be evaluated; obtaining multiple groups of description information according to the target image type, and inputting the multiple groups of description information into the diffusion model to be evaluated respectively, to obtain multiple generated images corresponding to each of the description information.

[0060] Among them, the image configuration information, including image type and description information, can be determined according to the user's requirements for generating images. The image type defines the theme or category of the image, such as natural scenery, pets, animation, etc., while the description information is a specific description of the image content, including the main elements expected by the user, style preferences, and image details. This description information is also called a prompt, which provides guidance for the diffusion model to be evaluated to follow when generating images.

[0061] Step S3, obtaining target image configuration information and a generated image corresponding to the target image configuration information according to the target evaluation index, and evaluating the target evaluation index according to preset rules to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility.

[0062] In one embodiment, the target image configuration information and the generated image corresponding to the target image configuration information are obtained according to the target evaluation indicator, and the target evaluation indicator is evaluated according to a preset rule to obtain an evaluation result of the target evaluation indicator, including: in response to the target evaluation indicator being one or more of image generation quality and preference, a plurality of first description information of the target image type and a plurality of first generated images corresponding to each of the first description information are obtained, and the image generation quality includes image aesthetics and / or consistency; in response to the target evaluation indicator being image generation quality, the first generated image is input into a pre-trained image aesthetic scoring model to obtain an image aesthetic score of the diffusion model to be evaluated; and / or, the feature vectors of the first description information and the corresponding first generated image are respectively extracted using a CLIP model, and the consistency score of the diffusion model to be evaluated is obtained by calculating the cosine similarity.

[0063] For example, for e-commerce scenarios, n (n ≥ 30) first description information (prompt) can be collected, and the collection sources include various platforms, among which only positive prompts are selected, and negative prompts are uniformly defaulted. The criteria for prompt selection include non-pornographic or soft pornographic, non-political, mostly based on physical objects, etc., and the prompt design often includes words such as close-up, HD resolution, modern design, and best quality. Then, the API mode of the StableDiffusion model webui platform is used, and the fastapi server is started on the cloud platform, and the local package is performed according to the post request to output the image. Each prompt generates m (m ≥ 30) pictures, and the first dimension evaluation generates a total of n*m images.

[0064] Furthermore, the evaluation indicators of image generation quality include image aesthetics and consistency. In this embodiment, the LAION-Aesthetics V2 model is used to evaluate image aesthetics, and the CLIP model is used to evaluate the consistency between the image and the description information.

[0065] Specifically, the aesthetic score of the image quality generated by the Stable Diffusion model is quantitatively evaluated by improved_aesthetic_predictor (LAION-Aesthetics V2). This is an image aesthetic scoring model proposed by LAION. The model is trained using most of the scoring data from 176,000 Simulacra Aesthetic Captions image scoring pairs, 15,000 LAION-Logos image scoring pairs, and 250,000 Aesthetic Visual Analysis (AVA) data. The generated images are input into the LAION-Aesthetics V2 model to obtain the image aesthetic score of each image. Since the Stable Dffusion model is also trained based on LAION-Aesthetics, the image aesthetic scoring indicator is relatively reliable, and the standard deviation of the image aesthetic score can be used to measure the model's good image rate.

[0066] Furthermore, the CLIP model is used to quantitatively evaluate the consistency between the content and description information of the image generated by the Stable Diffusion model, that is, whether the generated image can correctly reflect the content of the description information, that is, the accuracy of the generation.

[0067] Specifically, the CLIP model is used to output the feature vectors of the first description information and the corresponding first generated image respectively, and then the cosine similarity between the feature vectors of the two is calculated to obtain the CLIP score. The CLIP score is used to represent the matching degree and correlation between natural language and image pairs, so it can be used as a consistency score of the model. When the CLIP score is higher (close to 1), the correlation between the image-text pairs is higher. This score can also be used as an indicator to evaluate the compatibility of model image generation.

[0068] In one embodiment, in response to the target evaluation indicator being preference, the first description information and the corresponding first generated image are input into a pre-trained HPSv2 model to obtain a first preference score of the diffusion model to be evaluated; and / or, the first description information and the corresponding first generated image are input into a pre-trained ImageReward model to obtain a second preference score of the diffusion model to be evaluated.

[0069] The preference index may be evaluated by using the description information and generated pictures used in evaluating the image generation quality index. In this embodiment, the HPSv2 index and the ImageReward index are used.

[0070] Specifically, the Human Preference Score v2 (HPSv2) model is trained on the Human PreferenceDataset v2 (HPDv2) dataset, which is a large-scale human annotated dataset containing 798,000 human preference choices and 433,000 pairs of images. These images come from a wide range of sources, including multiple mainstream text-to-image generation models. The HPSv2 model is a preference prediction model trained on HPDv2. It can reflect human preferences for generated images. Therefore, this indicator is mainly used to evaluate the visual friendliness of the generated image itself for humans. It cannot produce images with distorted limbs, distorted faces, or deformed products.

[0071] Furthermore, the ImageReward metric can also be used to measure the preference of images generated by the Stable Diffusion model. ImageReward is a text-to-image human preference reward model built on the open source dataset ImageRewardDB. It is used to measure whether images generated under text guidance conform to human aesthetics. The dataset includes 10,000 candidate prompts, each with 4-9 generated images. The images are manually annotated and sorted in order of preference, and 137,000 comparison pairs are finally collected. This metric is mainly used to evaluate the model's text alignment capabilities and its ability to avoid toxic content output. In the images generated by Stable Diffusion, the model should be prevented from generating toxic content as much as possible.

[0072] In one embodiment, the target image configuration information and the generated image corresponding to the target image configuration information are obtained according to the target evaluation indicator, and the target evaluation indicator is evaluated according to a preset rule to obtain an evaluation result of the target evaluation indicator, and also includes: in response to the target evaluation indicator being style compatibility, a plurality of second description information of the target image type and a plurality of second generated images corresponding to each second description information are obtained, the second description information respectively having different style types; the corresponding first style reference image is obtained according to the style type of the second description information; and the style loss between the second generated image and the corresponding first style reference image is calculated as the style compatibility score of the diffusion model to be evaluated.

[0073] For example, for an e-commerce scene, k (k ≥ 10) styles of second description information (prompt) can be collected, where the styles are, for example, Photographic, Isometric, camera shot, Indoor natural style e-commerce background, minimalist style, etc. Each second description information generates m (m ≥ 30) images, and the third dimension evaluation generates a total of k*m images. At the same time, it is also necessary to collect relatively good stylized images generated by the second description information as the first style reference images. The number of images of each style is usually greater than or equal to 20.

[0074] Furthermore, the style compatibility of the image generated by the Stable Diffusion model is evaluated by calculating the style loss. Specifically, by extracting the multi-layer style features of the second description information and the first style reference image, and then calculating the gram matrix of the two for each layer of style features, and calculating the mean square error between the gram matrices, the value of the style loss is obtained. The lower this value is, the closer the generated image is to the style reference image in terms of overall style.

[0075] In one embodiment, the target image configuration information and the generated image corresponding to the target image configuration information are obtained according to the target evaluation indicator, and the target evaluation indicator is evaluated according to a preset rule to obtain an evaluation result of the target evaluation indicator, and also includes: in response to the target evaluation indicator being LoRA compatibility, a plurality of LoRA models of the target image type and corresponding third description information are obtained, and a plurality of third generated images corresponding to each of the third description information, the LoRA models respectively having different style types; a corresponding second style reference image is obtained according to the style type of the LoRA model; and the style loss between the third generated image and the corresponding second style reference image is calculated as the LoRA compatibility score of the diffusion model to be evaluated.

[0076] For example, for an image type of e-commerce scene, i (i ≥ 10) stylized LoRAs can be collected, such as Product Design v2.safetensors, WFProduct e-commerce scene_v1.1.safetensors, 3D e-commerce model V1_v1.0.safetensors, C4D geometry background geometry-bg_v2.5.safetensors, natural indoor e-commerce background_v1.0.safetensors, etc., and the latest prompt recommended by the LoRA is selected as the third description information. Each third description information generates m (m ≥ 30) pictures, and the fourth dimension evaluation generates a total of i*m images. At the same time, it is also necessary to collect relatively good stylized images generated under the current LoRA as the second style reference images. The number of images of each style is usually greater than or equal to 20.

[0077] Similarly, the compatibility of the image generation of the Stable Diffusion model with LoRA is evaluated by calculating the style loss. By extracting the multi-layer style features of the third description information and the second style reference image, and then calculating the gram matrix of the two for each layer of style features, and calculating the mean square error between the gram matrices, the value of the style loss is obtained. The lower this value is, the closer the generated image is to the style reference image in terms of overall style.

[0078] Step S4, obtaining the evaluation results of the multi-dimensional evaluation indicators, and evaluating the diffusion model to be evaluated.

[0079] In one embodiment, obtaining the evaluation results of the multi-dimensional evaluation indicators and evaluating the diffusion model to be evaluated includes: obtaining the scores of the image generation quality, preference, style compatibility and LoRA compatibility, performing weighted summation and normalization to obtain the evaluation results of the diffusion model to be evaluated.

[0080] Specifically, the original scores of each dimension are converted into values ​​within a unified range, as shown in Table 1. Users can adjust the type of different evaluation indicators according to specific circumstances. For example, when visual beauty is particularly emphasized, preference and style compatibility can be given higher weights.

[0081] Table 1 Evaluation results of diffusion model

[0082]

[0083] It is understandable that after obtaining the indicator scores of each dimension, it is not limited to the weighted summation method, but the comprehensive score of the diffusion model to be evaluated can also be calculated through methods such as geometric mean, hierarchical analysis method, and machine learning model.

[0084] In the above-mentioned generated image evaluation method, a process for scoring the image generation quality of the Stable Diffusion model is proposed. Jung established a multi-dimensional evaluation index, taking into account image generation quality, preference, stylization, and LoRA compatibility. It can efficiently evaluate existing or continuously updated Stable Diffusion models, and can also be used as an evaluation standard to evaluate newly trained large-scale Stable Diffusion-based Wenshengtu models, effectively improving the time-consuming and labor-intensive status of traditional image evaluation. In addition, style loss is introduced in the Stable Diffusion image generation model to calculate style compatibility and LoRA compatibility, which can better quantify these two key capabilities of the model.

[0085] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0086] In one embodiment, Figure 2 As shown, a device for generating image evaluation is provided, comprising: a building module, a generating module, a first evaluation module and a second evaluation module, wherein:

[0087] Establish a module for establishing multi-dimensional evaluation indicators of image generation quality;

[0088] A generating module, configured to obtain image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0089] A first evaluation module is used to obtain target image configuration information and a corresponding generated image according to a target evaluation indicator, and evaluate the target evaluation indicator according to a preset rule to obtain an evaluation result of the target evaluation indicator, wherein the evaluation indicator includes image generation quality, preference, style compatibility and LoRA compatibility;

[0090] The second evaluation module is used to obtain the evaluation results of the multi-dimensional evaluation indicators and evaluate the diffusion model to be evaluated.

[0091] In one embodiment, the generation module is further used to determine the target image type according to the diffusion model to be evaluated; obtain multiple groups of description information according to the target image type, and input the multiple groups of description information into the diffusion model to be evaluated respectively to obtain multiple generated images corresponding to each of the description information.

[0092] In one embodiment, the first evaluation module is further used to obtain multiple first description information of the target image type and multiple first generated images corresponding to each first description information in response to the target evaluation indicator being one or more of image generation quality and preference, and the image generation quality includes image aesthetics and / or consistency; in response to the target evaluation indicator being image generation quality, input the first generated image into a pre-trained image aesthetic scoring model to obtain an image aesthetic score of the diffusion model to be evaluated; and / or, use the CLIP model to respectively extract feature vectors of the first description information and the corresponding first generated image, and obtain the consistency score of the diffusion model to be evaluated by calculating cosine similarity.

[0093] In one embodiment, the first evaluation module is further used to, in response to the target evaluation indicator being preference, input the first description information and the corresponding first generated image into a pre-trained HPSv2 model to obtain a first preference score for the diffusion model to be evaluated; and / or, input the first description information and the corresponding first generated image into a pre-trained ImageReward model to obtain a second preference score for the diffusion model to be evaluated.

[0094] In one embodiment, the first evaluation module is further used to obtain multiple second description information of the target image type and multiple second generated images corresponding to each second description information in response to the target evaluation indicator being style compatibility, wherein the second description information respectively has different style types; obtain the corresponding first style reference image according to the style type of the second description information; and calculate the style loss between the second generated image and the corresponding first style reference image as the style compatibility score of the diffusion model to be evaluated.

[0095] In one embodiment, the first evaluation module is also used to obtain multiple LoRA models of the target image type and corresponding third description information, and multiple third generated images corresponding to each third description information in response to the target evaluation indicator being LoRA compatibility, and the LoRA models have different style types respectively; obtain the corresponding second style reference image according to the style type of the LoRA model; calculate the style loss between the third generated image and the corresponding second style reference image as the LoRA compatibility score of the diffusion model to be evaluated.

[0096] In one embodiment, the second evaluation module is further used to obtain the scores of the image generation quality, preference, style compatibility and LoRA compatibility, perform weighted summation and standardization processing to obtain the evaluation result of the diffusion model to be evaluated.

[0097] For the specific definition of the image evaluation device, please refer to the definition of the image evaluation method above, which will not be repeated here. Each module in the above-mentioned image evaluation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0098] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image configuration information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating image evaluation is implemented.

[0099] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0100] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0101] Establish multi-dimensional evaluation metrics for generated images;

[0102] Acquire image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0103] According to the target evaluation index, target image configuration information and a generated image corresponding to the target image configuration information are obtained, and the target evaluation index is evaluated according to a preset rule to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility;

[0104] The evaluation results of the multi-dimensional evaluation indicators are obtained, and the diffusion model to be evaluated is evaluated.

[0105] In one embodiment, when the processor executes the computer program, the following steps are further implemented: determining the target image type according to the diffusion model to be evaluated; obtaining multiple groups of description information according to the target image type, and inputting the multiple groups of description information into the diffusion model to be evaluated respectively to obtain multiple generated images corresponding to each of the description information.

[0106] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the target evaluation indicator being one or more of image generation quality and preference, multiple first description information of the target image type and multiple first generated images corresponding to each of the first description information are obtained, and the image generation quality includes image aesthetics and / or consistency; in response to the target evaluation indicator being image generation quality, the first generated image is input into a pre-trained image aesthetic scoring model to obtain an image aesthetic score of the diffusion model to be evaluated; and / or, feature vectors of the first description information and the corresponding first generated image are respectively extracted using the CLIP model, and the consistency score of the diffusion model to be evaluated is obtained by calculating the cosine similarity.

[0107] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the target evaluation indicator being preference, the first description information and the corresponding first generated image are input into the pre-trained HPSv2 model to obtain a first preference score of the diffusion model to be evaluated; and / or, the first description information and the corresponding first generated image are input into the pre-trained ImageReward model to obtain a second preference score of the diffusion model to be evaluated.

[0108] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the target evaluation indicator being style compatibility, multiple second description information of the target image type and multiple second generated images corresponding to each second description information are obtained, and the second description information respectively have different style types; according to the style type of the second description information, the corresponding first style reference image is obtained; and the style loss between the second generated image and the corresponding first style reference image is calculated as the style compatibility score of the diffusion model to be evaluated.

[0109] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the target evaluation indicator being LoRA compatibility, multiple LoRA models of the target image type and corresponding third description information, and multiple third generated images corresponding to each of the third description information, the LoRA models respectively having different style types; obtaining the corresponding second style reference image according to the style type of the LoRA model; calculating the style loss between the third generated image and the corresponding second style reference image as the LoRA compatibility score of the diffusion model to be evaluated.

[0110] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the scores of the image generation quality, preference, style compatibility and LoRA compatibility, performing weighted summation and normalization processing to obtain the evaluation result of the diffusion model to be evaluated.

[0111] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0112] Establish multi-dimensional evaluation metrics for generated images;

[0113] Acquire image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated;

[0114] According to the target evaluation index, target image configuration information and a generated image corresponding to the target image configuration information are obtained, and the target evaluation index is evaluated according to a preset rule to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility;

[0115] The evaluation results of the multi-dimensional evaluation indicators are obtained, and the diffusion model to be evaluated is evaluated.

[0116] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the target image type according to the diffusion model to be evaluated; obtaining multiple groups of description information according to the target image type, and inputting the multiple groups of description information into the diffusion model to be evaluated respectively to obtain multiple generated images corresponding to each of the description information.

[0117] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: in response to the target evaluation indicator being one or more of image generation quality and preference, multiple first description information of the target image type and multiple first generated images corresponding to each of the first description information are obtained, and the image generation quality includes image aesthetics and / or consistency; in response to the target evaluation indicator being image generation quality, the first generated image is input into a pre-trained image aesthetic scoring model to obtain an image aesthetic score of the diffusion model to be evaluated; and / or, feature vectors of the first description information and the corresponding first generated image are respectively extracted using a CLIP model, and a consistency score of the diffusion model to be evaluated is obtained by calculating cosine similarity.

[0118] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: in response to the target evaluation indicator being preference, the first description information and the corresponding first generated image are input into a pre-trained HPSv2 model to obtain a first preference score of the diffusion model to be evaluated; and / or, the first description information and the corresponding first generated image are input into a pre-trained ImageReward model to obtain a second preference score of the diffusion model to be evaluated.

[0119] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: in response to the target evaluation indicator being style compatibility, multiple second description information of the target image type and multiple second generated images corresponding to each second description information are obtained, and the second description information respectively have different style types; according to the style type of the second description information, the corresponding first style reference image is obtained; and the style loss between the second generated image and the corresponding first style reference image is calculated as the style compatibility score of the diffusion model to be evaluated.

[0120] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: in response to the target evaluation indicator being LoRA compatibility, multiple LoRA models of the target image type and corresponding third description information, and multiple third generated images corresponding to each of the third description information, the LoRA models respectively having different style types; obtaining a corresponding second style reference image according to the style type of the LoRA model; calculating the style loss between the third generated image and the corresponding second style reference image as the LoRA compatibility score of the diffusion model to be evaluated.

[0121] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the scores of the image generation quality, preference, style compatibility and LoRA compatibility, performing weighted summation and normalization processing to obtain the evaluation result of the diffusion model to be evaluated.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0123] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for generating an image evaluation, characterized in that: include: Establish multi-dimensional evaluation metrics for generated images; Acquire image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated; According to the target evaluation index, target image configuration information and a generated image corresponding to the target image configuration information are obtained, and the target evaluation index is evaluated according to a preset rule to obtain an evaluation result of the target evaluation index, wherein the evaluation index includes image generation quality, preference, style compatibility and LoRA compatibility; The evaluation results of the multi-dimensional evaluation indicators are obtained, and the diffusion model to be evaluated is evaluated.

2. The generated image evaluation method according to claim 1, characterized in that The image configuration information includes image type and description information; the step of obtaining the image configuration information and inputting it into the diffusion model to be evaluated, and outputting a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated, comprises: Determining a target image type according to the diffusion model to be evaluated; A plurality of groups of description information are acquired according to the target image type, and the plurality of groups of description information are respectively input into the diffusion model to be evaluated to obtain a plurality of generated images corresponding to each of the description information.

3. The generated image evaluation method according to claim 2, characterized in that: The step of acquiring target image configuration information and a generated image corresponding to the target image configuration information according to the target evaluation index, and evaluating the target evaluation index according to a preset rule to obtain an evaluation result of the target evaluation index includes: In response to the target evaluation indicator being one or more of image generation quality and preference, obtaining a plurality of first description information of a target image type and a plurality of first generated images corresponding to each of the first description information, wherein the image generation quality includes an image aesthetic score and / or a consistency score; In response to the target evaluation indicator being image generation quality, inputting the first generated image into a pre-trained image aesthetic scoring model to obtain an image aesthetic score of the diffusion model to be evaluated; and / or, The CLIP model is used to extract feature vectors of the first description information and the corresponding first generated image respectively, and the consistency score of the diffusion model to be evaluated is obtained by calculating the cosine similarity.

4. The generated image evaluation method according to claim 3, characterized in that: The method further comprises: In response to the target evaluation indicator being preference, inputting the first description information and the corresponding first generated image into a pre-trained HPSv2 model to obtain a first preference score of the diffusion model to be evaluated; and / or, The first description information and the corresponding first generated image are input into a pre-trained ImageReward model to obtain a second preference score of the diffusion model to be evaluated.

5. The generated image evaluation method according to claim 2, characterized in that: The step of acquiring target image configuration information and a generated image corresponding to the target image configuration information according to the target evaluation index, and evaluating the target evaluation index according to a preset rule to obtain an evaluation result of the target evaluation index, further includes: In response to the target evaluation indicator being style compatibility, obtaining a plurality of second description information of a target image type and a plurality of second generated images corresponding to each of the second description information, wherein the second description information respectively has a different style type; Acquire a corresponding first style reference image according to the style type of the second description information; A style loss between the second generated image and the corresponding first style reference image is calculated as a style compatibility score of the diffusion model to be evaluated.

6. The generated image evaluation method according to claim 2, characterized in that: The step of acquiring target image configuration information and a generated image corresponding to the target image configuration information according to the target evaluation index, and evaluating the target evaluation index according to a preset rule to obtain an evaluation result of the target evaluation index, further includes: In response to the target evaluation indicator being LoRA compatibility, obtaining a plurality of LoRA models of the target image type and corresponding third description information, and a plurality of third generated images corresponding to each of the third description information, the LoRA models respectively having different style types; Acquire a corresponding second style reference image according to the style type of the LoRA model; A style loss between the third generated image and the corresponding second style reference image is calculated as a LoRA compatibility score of the diffusion model to be evaluated.

7. The generated image evaluation method according to claim 1, characterized in that: The obtaining the evaluation result of the multi-dimensional evaluation index and evaluating the diffusion model to be evaluated includes: The scores of the image generation quality, preference, style compatibility and LoRA compatibility are obtained, and weighted summation and standardization are performed to obtain the evaluation result of the diffusion model to be evaluated.

8. A device for generating image evaluation, characterized in that: The device comprises: Establish a module for establishing multi-dimensional evaluation indicators of image generation quality; A generating module, configured to obtain image configuration information and input it into a diffusion model to be evaluated, and output a plurality of generated images corresponding to the image configuration information through the diffusion model to be evaluated; A first evaluation module is used to obtain target image configuration information and a corresponding generated image according to a target evaluation indicator, and evaluate the target evaluation indicator according to a preset rule to obtain an evaluation result of the target evaluation indicator, wherein the evaluation indicator includes image generation quality, preference, style compatibility and LoRA compatibility; The second evaluation module is used to obtain the evaluation results of the multi-dimensional evaluation indicators and evaluate the diffusion model to be evaluated.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images