Method and device for evaluating a text-to-image generation model, electronic equipment and storage medium

Abstract

The application relates to the technical field of image processing, in particular to a text-image generation model evaluation method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: searching a representative subset satisfying a preset representative condition from a text set; inputting the representative subset of the text set into a text-image generation model to be evaluated to obtain a generated picture set; and obtaining an evaluation result of the text-image generation model to be evaluated according to the generated picture set, wherein the evaluation result comprises an image quality index. Thus, the technical problem that in the related art, the time required for evaluating a large data set is relatively long, the evaluation result of a randomly sampled data set subset is relatively inaccurate, has large fluctuations and cannot meet the task requirements is solved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an evaluation method, apparatus, electronic device and storage medium for text-image generation models. Background Technology

[0002] In recent years, text and image generation models have achieved excellent generation performance. Evaluating the generation quality of text and image generation models is an important step in selecting existing models and developing new models. Therefore, the evaluation of text and image generation models is of great significance.

[0003] In related technologies, a large text dataset (e.g., tens of thousands of texts) is input into a text-to-image generation model to generate images and their quality is evaluated; or a subset (e.g., dozens of texts) is randomly sampled from a large text set to generate images for evaluation.

[0004] However, in related technologies, evaluating text and image generation models using large datasets takes a long time, generally up to 60 GPU (Graphics Processing Unit) hours (using RTX 3090), while the evaluation results of randomly sampled subsets of datasets are less accurate, fluctuate greatly, and cannot meet the task requirements, so improvements are urgently needed. Summary of the Invention

[0005] This application provides an evaluation method, apparatus, electronic device, and storage medium for text and image generation models, in order to solve the technical problems in related technologies, such as the long time required to evaluate text and image generation models using large datasets, and the inaccurate and volatile evaluation results of randomly sampled subsets of datasets, which cannot meet the requirements of the task.

[0006] The first aspect of this application provides an evaluation method for a text-image generation model, comprising the following steps: searching a representative subset from a text set that satisfies preset representativeness conditions; inputting the representative subset of the text set into the text-image generation model to be evaluated to obtain a generated image set; and obtaining an evaluation result of the text-image generation model to be evaluated based on the generated image set, wherein the evaluation result may include image quality indicators.

[0007] Optionally, in one embodiment of this application, the step of searching for a representative subset that meets a preset representativeness condition from the text set includes: randomly sampling text from the text set and combining it to form multiple candidate subsets; sorting the multiple candidate subsets according to the index of each candidate subset to obtain a sorted candidate subset; forming a new candidate text set according to the sorted candidate subsets, until a preset iteration stop condition is met to obtain the representative subset.

[0008] Optionally, in one embodiment of this application, before sorting the plurality of candidate subsets, the method further includes: obtaining an index for each candidate subset based on the correlation between the evaluation results of the preset text-image generation model set using each candidate subset and the evaluation results of the preset text-image generation model set using the text set.

[0009] Optionally, in one embodiment of this application, the step of constructing a new candidate text set based on the sorted candidate subset includes: selecting the multiple texts with the highest frequency of occurrence from the candidate subset that meet preset conditions, and generating the new candidate text set.

[0010] A second aspect of this application provides an evaluation apparatus for a text-image generation model, comprising: a search module for searching a representative subset from a text set that satisfies preset representativeness conditions; a generation module for inputting the representative subset of the text set into a text-image generation model to be evaluated to obtain a generated image set; and an evaluation module for obtaining an evaluation result of the text-image generation model to be evaluated based on the generated image set, wherein the evaluation result may include image quality indicators.

[0011] Optionally, in one embodiment of this application, the search module includes: a sampling unit, configured to randomly sample text from the text set and combine it to form multiple candidate subsets; a sorting unit, configured to sort the multiple candidate subsets according to the index of each candidate subset to obtain a sorted candidate subset; and an iteration unit, configured to form a new candidate text set based on the sorted candidate subsets until a preset iteration stop condition is met to obtain the representative subset.

[0012] Optionally, in one embodiment of this application, it further includes: an obtaining module, configured to obtain an index of each candidate subset based on the correlation between the evaluation results of the preset text-image generation model set using each candidate subset and the evaluation results of the preset text-image generation model set using the text set before sorting the plurality of candidate subsets.

[0013] Optionally, in one embodiment of this application, the iteration unit includes: a selection subunit, used to select the multiple texts with the highest frequency in a candidate subset that meet preset conditions, and generate the new candidate text set.

[0014] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the evaluation method for the text-image generation model as described in the above embodiments.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described evaluation method for the text-image generation model.

[0016] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the evaluation method for the text-image generation model as described above.

[0017] This application embodiment first searches for a representative subset that meets preset representativeness criteria from a text set, and then inputs this representative subset into the text-to-image generation model to be evaluated to obtain a set of generated images. The evaluation result of the text-to-image generation model is then obtained based on the generated image set, where the evaluation result includes image quality metrics. This application embodiment improves the accuracy, efficiency, and quality of text-to-image generation model evaluation, better meeting practical task requirements. Therefore, it solves the technical problems in related technologies, such as the long time required to evaluate text-to-image generation models using large datasets, and the inaccurate and volatile evaluation results of randomly sampled dataset subsets, which fail to meet task requirements.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0020] Figure 1 This is a flowchart of an evaluation method for a text-image generation model provided according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a method for searching a representative subset according to an embodiment of this application;

[0022] Figure 3 This is a block diagram of an evaluation apparatus for a text and image generation model provided according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0025] The following description, with reference to the accompanying drawings, describes an evaluation method, apparatus, electronic device, and storage medium for text-image generation models according to embodiments of this application. Addressing the technical problems mentioned in the background art, such as the long time required to evaluate text-image generation models using large datasets, and the inaccurate and fluctuating evaluation results of randomly sampled subsets of datasets failing to meet task requirements, this application provides an evaluation method for text-image generation models. In this method, a representative subset satisfying preset representativeness conditions is first searched from a text set. This representative subset is then input into the text-image generation model to be evaluated to obtain a set of generated images. The evaluation result of the text-image generation model is then obtained based on the generated image set, where the evaluation result includes image quality indicators. Embodiments of this application improve the accuracy of text-image generation model evaluation, enhance evaluation efficiency and quality, and better meet actual task requirements. Therefore, this solves the technical problems in related technologies, such as the long time required to evaluate text-image generation models using large datasets, and the inaccurate and fluctuating evaluation results of randomly sampled subsets of datasets failing to meet task requirements.

[0026] Specifically, Figure 1 This is a flowchart of an evaluation method for a text-image generation model provided according to an embodiment of this application.

[0027] like Figure 1 As shown, the evaluation method for this text-image generation model includes the following steps:

[0028] In step S101, a representative subset that meets the preset representativeness criteria is searched from the text set.

[0029] It is understood that the text set may be, but is not limited to, a large (e.g., complete, this application does not impose specific limitations) set (e.g., COCO, DiffusionDB dataset, this application does not impose specific limitations) or at least a portion thereof containing all relevant texts within a certain topic, keyword, or domain. The text form may be textual, such as books, articles, web pages, etc.; it may also be text pre-converted from non-textual data, wherein the non-textual data may be audio, such as audio, video, etc., or other forms, this application does not impose specific limitations. Furthermore, in the embodiments of this application, the text in the text set can be used as prompt words input into the text-image generation model to generate images, wherein any text may include characters, numbers, words, etc. of any length, this application does not impose specific limitations.

[0030] Furthermore, in the embodiments of this application, searching can be understood as the process of finding texts that meet preset representative conditions from a large set of texts (e.g., complete, without specific limitations in this application). The relevant search algorithms may include, but are not limited to, random iterative search algorithms, simulated annealing algorithms, genetic algorithms, etc., without specific limitations in this application.

[0031] Furthermore, the preset representativeness conditions can be set by those skilled in the art according to the actual situation, and this application does not impose specific restrictions. For example, the preset representativeness conditions may include a preset number of representative subsets, or the correlation or similarity between the evaluation results of the preset text-image generation model set using representative subsets and the evaluation results of the preset text-image generation model set using text sets, etc.

[0032] As one possible implementation, embodiments of this application can search for a representative subset that meets preset representativeness criteria from a text set, thereby providing a basis for evaluating the text generation model. The search method can be configured by those skilled in the art according to actual circumstances, and this application does not impose specific limitations.

[0033] Optionally, in one embodiment of this application, searching for a representative subset that meets a preset representativeness condition from a text set includes: randomly sampling text from the text set and combining it to form multiple candidate subsets; sorting the multiple candidate subsets according to the index of each candidate subset to obtain a sorted candidate subset; forming a new candidate text set based on the sorted candidate subsets, until a preset iteration stopping condition is met to obtain a representative subset. Before sorting the multiple candidate subsets, the index of each candidate subset can be obtained based on the correlation between the evaluation results of the preset text-image generation model set using each candidate subset and the evaluation results of the preset text-image generation model set using the text set.

[0034] It is understood that the metrics for each candidate subset may include, but are not limited to, the KD value (Kendall's Tau correlation coefficient), MAE (Mean Absolute Error), and MSE (Mean Squared Error). Furthermore, correlation may include, but is not limited to, ordinal correlation, semantic correlation, and structural correlation; this application does not impose specific limitations.

[0035] As one possible implementation, the flowchart for searching a representative subset that satisfies preset representativeness conditions from a text set in this application embodiment can be as follows: Figure 2 As shown, the specific steps can be as follows:

[0036] Step S1: Randomly sample text from the text set and combine it into multiple candidate subsets.

[0037] In other words, embodiments of this application can combine text randomly sampled from a large text set (e.g., the entire set, which is not specifically limited in this application) into multiple candidate subsets. Each candidate subset can have the same preset size (e.g., 10, 100, etc., which is not specifically limited in this application). Furthermore, in embodiments of this application, the preset size can be significantly smaller than the number of texts included in the text set (e.g., several thousand or tens of thousands, which is not specifically limited in this application).

[0038] Step S2: Sort according to one or more indicators for each candidate subset.

[0039] In other words, before ranking multiple candidate subsets, this application embodiment can obtain indicators (e.g., KD value, MAE, MSE, etc., without specific limitations) for each candidate subset based on the correlation (e.g., ranking correlation or similarity, which is not specifically limited in this application) between the evaluation results of the preset text-image generation model set using each candidate subset and the evaluation results of the preset text-image generation model set using the text set. Then, the candidate subsets are ranked according to the relevant indicators. The specific settings of the indicators can be made by those skilled in the art according to the actual situation, and this application does not impose specific limitations. Furthermore, the preset text-image generation model may include, but is not limited to, a diffusion model, which is not specifically limited in this application.

[0040] Furthermore, in some embodiments, the present application embodiments may assume that the preset text and image generation model set includes N m There are several models, each with a different architecture, parameters, etc., and this application does not impose specific restrictions. Furthermore, this set can be represented as M = {M...} i |i∈{1,...,N m}}, where M i For the i-th model, N m The integer is greater than 1. Further, in embodiments of this application, the original text set can be defined as P = {p i |i∈{1,...,N}}, where N is the number of texts in the text set, p i Let be the i-th text. Each text can contain characters, numbers, letters, punctuation marks, etc., of any length. The candidate subset can be represented as... Where, the index list I = {i1,...,i N′Furthermore, N′ represents the number of texts in the candidate subset. Further, in this embodiment, the text set P is input into each model in the text-image generation model set M to obtain a first generated image set. Based on the first generated image set, a first evaluation index is obtained for each model, where the first evaluation index can be expressed as... candidate subset Input each model in the text-image generation model set M to obtain a second set of generated images. Based on this second set of generated images, derive a second evaluation metric for each model. The second evaluation metric can be expressed as: Among them, candidate subset The indicators can be obtained based on the similarity or correlation between the first set of evaluation indicators and the second set of evaluation indicators, and this application does not impose any specific restrictions.

[0041] Specifically, in the embodiments of this application, the first or second evaluation metric may include, but is not limited to, evaluation metrics for a single image, such as CLIPScore, ImageReward, etc., which are not specifically limited in this application, or evaluation metrics for a set of images, such as FID (Frechet Inception Distance), which are not specifically limited in this application.

[0042] Furthermore, in this embodiment of the application, for the evaluation index of a single image, the values ​​of the evaluation index of each generated image in the first or second generated image set can be averaged to obtain the overall index of the first or second generated image set.

[0043] In some embodiments, the KD value can be used as an indicator for the candidate subset. The KD value can be calculated as follows: for any two models M in the text-image generation model set M... i and M j (where i≠j), if their first and second evaluation indicators are consistent in magnitude (i.e., x) i <x j And y i <y j , or x i >x j And y i >y j If they are consistent, then they are defined as a consistent pair; if their first and second evaluation indices are inconsistent (i.e., x... i <x j And y i >y j , or x i >x j And y i <y jIf they are not consistent pairs, then they are defined as inconsistent pairs.

[0044] Furthermore, in the embodiments of this application, N can be defined. c N is the number of consistent pairs among all model pairs. d Let n be the number of inconsistent pairs among all model pairs, and n1 be the number of model pairs that are equal in magnitude in the first evaluation metric set but unequal in magnitude in the second evaluation metric set (i.e., conforming to x). i =x j And y i ≠y j The number of model pairs, n², is the number of model pairs that are not equal in size in the first evaluation metric set but are equal in size in the second evaluation metric set (i.e., satisfying x). i ≠x j And y i =y j The number of model pairs. At this point, the candidate subset... The KD value can be expressed as Furthermore, it can be understood that the range of KD values ​​is generally [-1, 1], where KD = 1 can indicate the strongest positive correlation (e.g., the same order), KD = -1 can indicate the strongest negative correlation (e.g., the same order), and KD = 0 can indicate no correlation (i.e., completely independent).

[0045] Furthermore, embodiments of this application can sort the candidate subset based on the KD value, for example, from smallest to largest or from largest to smallest. The sorting method can be set by those skilled in the art according to actual circumstances, and this application does not impose specific limitations.

[0046] Step S3: Construct a new candidate text set based on the sorted candidate subsets.

[0047] In other words, the embodiments of this application can re-filter the sorted candidate subset to form a new candidate text set, and the method of forming the new candidate text set will be given below.

[0048] Step S4: Repeat the above steps until the preset iteration stopping condition is met, and finally obtain a representative subset. The preset iteration stopping condition can be set by those skilled in the art according to the actual situation, and this application does not impose specific restrictions.

[0049] In other words, the embodiments of this application can continuously repeat the above steps and iterate until the preset iteration stop condition is met. For example, the preset iteration stop condition may include one or more of the following: the number of iterations reaches a threshold, or the index of the candidate text set (e.g., KD value, which is not specifically limited in this application) reaches a threshold, or the size of the candidate text set reaches a preset set size (e.g., 10 or 20 texts, which is not specifically limited in this application), thereby obtaining a representative subset.

[0050] In actual implementation, the embodiments of this application can first randomly sample text from the complete text set and combine it to form multiple candidate subsets. Then, based on the indicators of each candidate subset, these candidate subsets are reordered to form a new candidate text set. This process is repeated until a preset iteration stop condition is met, and finally a representative subset is obtained.

[0051] Optionally, in one embodiment of this application, constructing a new candidate text set based on the sorted candidate subset includes: selecting the multiple texts with the highest frequency of occurrence from the candidate subset that meet the preset conditions, and generating a new candidate text set.

[0052] Here, the embodiments of this application provide a detailed explanation of the methods for constructing a new set of candidate texts.

[0053] It is understandable that the most frequently occurring texts can include, but are not limited to, the single most frequently occurring text, the top 15% of the most frequently occurring texts, the top 30% of the most frequently occurring texts, and the proportion of the most frequently occurring candidate text set (e.g., the proportion can be represented by k). s %, where k s (The value can be any value between 0 and 100) and can be set by those skilled in the art according to the actual situation. This application does not impose any specific restrictions.

[0054] Furthermore, this application embodiment selected the highest frequency as the indicator to meet the preset conditions. Alternatively, the highest KD value for a single specific text can also be selected as the indicator to meet the preset conditions. This application does not impose any specific limitations.

[0055] Additionally, preset conditions can be understood as conditions set when the indicator meets the highest conditions, such as the highest frequency or the highest KD value. The specific settings can be set by those skilled in the art according to the actual situation, and this application does not impose specific restrictions.

[0056] In actual implementation, embodiments of this application can sort candidate subsets (e.g., 1000, this application does not impose a specific limit) according to their relevance, such as KD value, and then select the frequency of occurrence as the selection criterion for text in the sorted candidate subsets. For example, the top 1% of candidate subsets with the highest relevance (e.g., 10, this application does not impose a specific limit) can be selected, the frequency of occurrence of all text in these candidate subsets can be counted, and a certain proportion (e.g., the top 15%, this application does not impose a specific limit) of text with the highest frequency of occurrence can be selected from these texts to generate a new candidate text set; alternatively, different numbers of texts can be selected from candidate subsets with different relevance, such as selecting all texts in the candidate subset with the highest relevance, a certain proportion (e.g., the top 50%, this application does not impose a specific limit) of text in the candidate subset with the second highest relevance, and so on, to generate a new candidate text set. The specific method can be set by those skilled in the art according to the actual situation, and this application does not impose a specific limit.

[0057] In step S102, a representative subset of the text set is input into the text-image generation model to be evaluated to obtain the generated image set.

[0058] It is understood that the representative subset obtained in the above steps can be used as input to the text-image generation model to be evaluated, and then the model can be evaluated based on the resulting set of generated images.

[0059] In some embodiments, the process of generating images based on input text by the text-image generation model of this application may include: (1) For each text, the image features corresponding to the text are obtained through a text-image feature model (e.g., CLIP model (Contrastive Language-Image Pre-training, a pre-trained model based on contrastive text-image pairs, which is not specifically limited in this application). (2) The obtained image features are subjected to mutual attention operation with intermediate results during the inference process of a visual generation model (e.g., diffusion model, which is not specifically limited in this application) to finally obtain the generated image.

[0060] Compared with using a complete set of text (approximately tens of thousands of texts), the embodiments of this application can use a representative subset obtained by searching (approximately dozens of texts) for evaluation. Image generation and metric averaging can be performed on only a smaller set, which improves the efficiency and quality of text-image generation model evaluation and is more in line with actual task requirements.

[0061] In step S103, the evaluation result of the text-image generation model to be evaluated is obtained based on the generated image set, wherein the evaluation result may include image quality indicators.

[0062] It is understood that image quality metrics may include, but are not limited to, evaluation metrics for a single image, such as CLIPScore, ImageReward, etc., which are not specifically limited in this application, or evaluation metrics for a set of images, such as FID, which are not specifically limited in this application.

[0063] Specifically, in the embodiments of this application, the CLIPScore evaluation process can be as follows: first, a generated image I and the corresponding text t are given, and then input into the CLIP model to obtain the features E in the text domain and the image domain, respectively. i With E t At this point, the CLIPScore metric can be calculated as: CLIPScore(I,t)=max(100*cos(E) i E t ),0).

[0064] In this embodiment of the application, the FID evaluation process can be as follows: first, a set of real images (i.e., images not generated by the model) {I} is given. r}, and generate image set {I g Then, for all images in the two image sets, the Inception spatial features {F} are extracted using the Inception model. r} and {F g Next, calculate the mean μ and covariance ∑ of the two feature sets respectively. Then, the formula for calculating the FID index can be expressed as: FID = |μ r -μ g | 2 +Tr(∑ r +∑ g -2∑r∑g12, where Tr· represents the trace.

[0065] This application embodiment can select appropriate metrics to evaluate the quality of images generated by the text-image generation model. Furthermore, for evaluation metrics targeting a single image, this application embodiment can average the evaluation metric values ​​of each generated image in the generated image set to obtain the overall metric for the generated image set. The metrics can be set by those skilled in the art according to actual conditions, such as image quality metrics, etc., and this application does not impose specific limitations.

[0066] For example, embodiments of this application may assume a text set of T and each text item of type t. i Furthermore, in this embodiment of the application, an image generation model is used to generate image I. i , among which, I i =f gen (t i ), where f gen(·) represents the text-image generation model, and the resulting set of generated images can be represented as {I}. Furthermore, in the embodiments of this application, the average image quality index of the generated image set {I} can be calculated to obtain the index of the entire generated image set.

[0067] The evaluation method for text-image generation models proposed in this application involves randomly sampling text from a large text set and combining it to form multiple candidate subsets. These candidate subsets are then sorted and selected according to indicators to form a new text set. This process is iterated until a preset iteration condition is met, resulting in a representative subset. This representative subset is then input into the text-image generation model to be evaluated to obtain a set of generated images. The evaluation result of the text-image generation model is then obtained based on the generated image set, where the evaluation result includes image quality indicators. This application improves the accuracy, efficiency, and quality of text-image generation model evaluation, better meeting practical task requirements. Therefore, it solves the technical problems in related technologies, such as the long evaluation time required when using large datasets, and the inaccurate and fluctuating evaluation results obtained using randomly sampled dataset subsets, which fail to meet task requirements.

[0068] Next, with reference to the accompanying drawings, an evaluation apparatus for a text and image generation model proposed according to an embodiment of this application is described.

[0069] Figure 3 This is a block diagram of an evaluation apparatus for a text and image generation model provided according to an embodiment of this application.

[0070] like Figure 3 As shown, the evaluation device 10 for the text and image generation model includes: a search module 100, a generation module 200, and an evaluation module 300.

[0071] Specifically, the search module 100 is used to search for a representative subset from the text set that meets preset representativeness criteria.

[0072] The generation module 200 is used to input a representative subset of the text set into the text-image generation model to be evaluated to obtain a set of generated images.

[0073] The evaluation module 300 is used to obtain the evaluation results of the text-image generation model to be evaluated based on the generated image set, wherein the evaluation results may include image quality indicators.

[0074] Optionally, in one embodiment of this application, the search module 100 includes a sampling unit, a sorting unit, and an iteration unit.

[0075] The sampling unit is used to randomly sample text from the text set and combine them to form multiple candidate subsets.

[0076] The sorting unit is used to sort multiple candidate subsets according to the index of each candidate subset, so as to obtain the sorted candidate subsets.

[0077] An iterative unit is used to construct a new candidate text set based on the sorted candidate subsets until a preset iteration stopping condition is met, thus obtaining a representative subset.

[0078] Optionally, in one embodiment of this application, it further includes: an acquisition module.

[0079] The acquisition module is used to obtain the index of each candidate subset based on the correlation between the evaluation results of the preset text-image generation model set using each candidate subset and the evaluation results of the preset text-image generation model set using the text set before sorting the multiple candidate subsets.

[0080] Optionally, in one embodiment of this application, the iteration unit includes: a selection subunit.

[0081] The selection sub-unit is used to select the most frequently occurring texts from the candidate subset that meet the preset conditions, and generate a new candidate text set.

[0082] It should be noted that the foregoing explanation of the evaluation method embodiment for the text-image generation model also applies to the evaluation device for the text-image generation model in this embodiment, and will not be repeated here.

[0083] The evaluation apparatus for text-image generation models proposed in this application can randomly sample and combine text from a large text set to form multiple candidate subsets. These candidate subsets are then sorted and selected according to indicators to form a new text set. This process is iterated until a preset iteration condition is met, resulting in a representative subset. This representative subset is then input into the text-image generation model to be evaluated to obtain a set of generated images. The evaluation result of the text-image generation model is then obtained based on the generated image set, where the evaluation result includes image quality indicators. This application improves the accuracy, efficiency, and quality of text-image generation model evaluation, better meeting practical task requirements. Therefore, it solves the technical problems in related technologies, such as the long evaluation time required when using large datasets, and the inaccurate and fluctuating evaluation results obtained using randomly sampled dataset subsets, which fail to meet task requirements.

[0084] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. The electronic device may include:

[0085] The memory 401 and the processor 402 are provided, wherein the memory 401 has a computer program stored on the memory 401 and can be run on the processor 402.

[0086] When the processor 402 executes the program, it implements the evaluation method for the text and image generation model provided in the above embodiments.

[0087] Furthermore, electronic devices also include:

[0088] Communication interface 403 is used for communication between memory 401 and processor 402.

[0089] The memory 401 is used to store computer programs that can run on the processor 402.

[0090] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0091] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0092] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0093] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0094] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described evaluation method for the text-image generation model.

[0095] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described evaluation method for the text-image generation model.

[0096] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0098] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0099] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0100] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0103] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An evaluation method for a text-image generation model, characterized in that, Includes the following steps: Search the text collection for a representative subset that meets the preset representativeness criteria; The representative subset of the text set is input into the text-image generation model to be evaluated to obtain the generated image set; as well as The evaluation result of the text-image generation model to be evaluated is obtained based on the generated image set, wherein the evaluation result includes image quality indicators; The step of searching for a representative subset from the text set that satisfies a preset representativeness condition includes: Texts are randomly sampled from the text set and combined to form multiple candidate subsets; The index of each candidate subset is obtained based on the correlation between the evaluation results of the preset text-image generation model set using each of the multiple candidate subsets and the evaluation results of the preset text-image generation model set using the text set. The candidate subsets are sorted according to the indicators of each candidate subset to obtain the sorted candidate subsets; A new candidate text set is formed based on the sorted candidate subsets until a preset iteration stopping condition is met, thus obtaining the representative subset; The metrics for obtaining each candidate subset include: The text set is input into each model in the preset text-image generation model set to obtain the first generated image set of the text set in the preset text-image generation model set, and the first evaluation index of each model is determined based on the first generated image set; The candidate subset is input into each model to obtain a second set of generated images of the candidate subset in the preset text and image generation model set, and a second evaluation index of each model is determined based on the second set of generated images. Based on the first evaluation index and the second evaluation index, the correlation between the candidate subsets and the text sets is determined; The index for each candidate subset is obtained based on the correlation.

2. The evaluation method for the text-image generation model according to claim 1, characterized in that, The step of constructing a new candidate text set based on the sorted candidate subset includes: Select the most frequently occurring texts from the candidate subset that meet the preset criteria, and generate the new candidate text set.

3. An evaluation device for a text-image generation model, characterized in that, The evaluation method for the text-image generation model as described in claim 1, wherein the apparatus comprises: The search module is used to search for a representative subset of a text collection that meets preset representativeness criteria. A generation module is configured to input the representative subset of the text set into the text-image generation model to be evaluated to obtain a generated image set; and An evaluation module is used to obtain an evaluation result of the text-image generation model to be evaluated based on the generated image set, wherein the evaluation result includes image quality indicators.

4. The evaluation device for the text-image generation model according to claim 3, characterized in that, The search module includes: A sampling unit is used to randomly sample text from the text set and combine them to form multiple candidate subsets; A sorting unit is used to sort the multiple candidate subsets according to the index of each candidate subset, so as to obtain a sorted candidate subset; An iterative unit is used to construct a new candidate text set based on the sorted candidate subset until a preset iteration stop condition is met, thereby obtaining the representative subset.

5. The evaluation apparatus for the text-image generation model according to claim 4, characterized in that, Also includes: The acquisition module is used to obtain an index for each candidate subset based on the correlation between the evaluation results of the preset text-image generation model set using each candidate subset and the evaluation results of the preset text-image generation model set using the text set, before sorting the plurality of candidate subsets.

6. The evaluation apparatus for the text-image generation model according to claim 4 or 5, characterized in that, The iterative unit includes: The selection subunit is used to select the multiple texts that appear most frequently in the candidate subset that meet the preset conditions, and generate the new candidate text set.

7. An electronic device, characterized in that, include: A memory and a processor, wherein the memory has a computer program stored on the memory and capable of running on the processor, the processor executing the program to implement the evaluation method of the text-image generation model as described in any one of claims 1-2.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the evaluation method for the text-image generation model as described in any one of claims 1-2.

Images