Method and device for analyzing text generated image model and electronic equipment

Abstract

The invention is suitable for the technical field of image processing, and provides an analysis method and device for a text-to-image model and electronic equipment, the method comprises the following steps: inputting a target evaluation text into a to-be-analyzed image generation model to obtain a target generation image output by the image generation model, the target evaluation text comprises a target attribute, and the target evaluation text comprises the target attribute; the target attribute is an attribute of an evaluation target in the target evaluation text, performing attribute identification on the target evaluation text to obtain a text target attribute, and performing attribute identification on the target generation image to obtain an image target attribute, and calculating an attribute score of the image generation model based on the text target attribute and the image target attribute, and performing quality analysis on the image generation model according to the attribute score. The problem that the text generation image model cannot be accurately analyzed can be solved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to an analysis method, device, electronic device and computer-readable storage medium for a text-generated image model. Background Art

[0002] With the rapid development of the text2img (text-to-image) model, the quality of text-to-image generation has reached a relatively high level, which can meet the basic needs of practical applications. For example, in the fields of public security criminal investigation, artistic creation, etc., generating facial images through text can play a greater role.

[0003] Analyzing the text-generated image model can better understand the model performance and thus evaluate the model quality. Existing analysis methods for text-generated image models mainly analyze the image generation capability of the text-generated image model based on the image feature distribution of the generated image (such as FID (Kernel Inception Distance, an indicator for evaluating the quality of images generated by the model), KID (Fréchet Inception Distance, an indicator for evaluating the quality of images generated by the model), etc.), but these methods cannot accurately reflect the text-generated image capability of the text-generated image model, resulting in inaccurate model quality analysis of the text-generated image model. Summary of the invention

[0004] The embodiments of the present application provide a method, device and electronic device for analyzing a text-generated image model, which can solve the problem of being unable to accurately analyze a text-generated image model.

[0005] In a first aspect, an embodiment of the present application provides an analysis method for a text-generated image model, comprising:

[0006] Inputting a target evaluation text into an image generation model to be analyzed to obtain a target generation image output by the image generation model, wherein the target evaluation text includes target attributes, and the target attributes are attributes of the evaluation target in the target evaluation text;

[0007] Performing attribute recognition on the target evaluation text to obtain text target attributes, and performing attribute recognition on the target generated image to obtain image target attributes;

[0008] Calculating an attribute score of the image generation model based on the text target attribute and the image target attribute;

[0009] The image generation model is subjected to quality analysis according to the attribute scores.

[0010] Optionally, performing attribute recognition on the target evaluation text to obtain text target attributes includes:

[0011] The target attribute text in the target evaluation text is extracted, and the text target attribute of the target evaluation text is detected according to the preset standard attribute text and the target attribute text, wherein the standard attribute text includes the matching text corresponding to the target attribute text.

[0012] Optionally, the calculating the attribute score of the image generation model based on the text target attribute and the image target attribute includes:

[0013] Attribute matching is performed on the text target attributes and the image target attributes, and an attribute score of the image generation model is calculated according to the attribute matching result.

[0014] Optionally, the performing attribute matching on the text target attribute and the image target attribute includes:

[0015] A binary attribute matching is performed on the text target attribute and the image target attribute, and the binary attribute matching is used to determine whether the text target attribute and the image target attribute are consistent.

[0016] Optionally, after performing quality analysis on the image generation model according to the attribute score, the method further includes:

[0017] The quality analysis result is used to perform model optimization training on the image generation model.

[0018] Optionally, the target generated image includes a generated target, and the generated target corresponds to the evaluation target. After inputting the target evaluation text into the image generation model to be analyzed and obtaining the target generated image output by the image generation model, the method further includes:

[0019] The number of generated targets in the target generated image is detected, and the stability of the image generation model is analyzed using the number of generated targets and the number of evaluated targets.

[0020] Optionally, it also includes:

[0021] Acquire a target evaluation image corresponding to the target evaluation text, wherein the target evaluation image is an image containing all target attributes in the target evaluation text;

[0022] Calculating a universal score of the image generation model based on the target evaluation image and the target generated image, wherein the universal score is used to measure the similarity between the target feature vector in the target evaluation image and the target feature vector in the target generated image;

[0023] A comprehensive quality analysis is performed on the image generation model based on the general score and the attribute score.

[0024] In a second aspect, an embodiment of the present application provides an analysis device for a text-generated image model, comprising:

[0025] An image generation module, used for inputting a target evaluation text into an image generation model to be analyzed, and obtaining a target generation image output by the image generation model, wherein the target evaluation text includes target attributes, and the target attributes are attributes of the evaluation target in the target evaluation text;

[0026] An attribute recognition module is used to perform attribute recognition on the target evaluation text to obtain text target attributes, and to perform attribute recognition on the target generated image to obtain image target attributes;

[0027] An attribute score calculation module, used to calculate the attribute score of the image generation model based on the text target attribute and the image target attribute;

[0028] A quality analysis module is used to perform quality analysis on the image generation model according to the attribute score.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for analyzing the text-generated image model described in the first aspect when executing the computer program.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the analysis method of the text-generated image model described in the first aspect are implemented.

[0031] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the analysis method of the text-generated image model described in any one of the first aspects above.

[0032] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0033] In the embodiment of the present application, by inputting the target evaluation text into the image generation model to be analyzed, the target generation image output by the image generation model is obtained. Since the target evaluation text includes the target attributes of the evaluation target, the attribute score calculated based on the text target attribute and the image target attribute can better analyze the image generation model. Specifically, since the target generation image is generated based on the target evaluation text, and the target evaluation text includes the target attributes of the evaluation target, the text target attribute can be obtained by attribute recognition of the target evaluation text, and the image target attribute can be obtained by attribute recognition of the target generation image, and the attribute score corresponding to the target attribute can be quickly calculated based on the above text target attribute and image target attribute; and, because the text target attribute is equivalent to the description information of the evaluation target in the target evaluation text, and the image target attribute is equivalent to the description information of the evaluation target in the target generation image, the attribute score calculated by the text target attribute and the image target attribute can reflect the degree of conversion of the description information in the text to the description information in the image, compared with the existing image distribution analysis indicators (FID, KID, etc.), and then reflect the ability of the model to generate images through text, so the quality analysis of the text generation image model can be better performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 It is a flowchart of an analysis method of a text-generated image model provided in an embodiment of the present application;

[0036] Figure 2 It is a structural schematic diagram of an analysis device for a text-generated image model provided in an embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0039] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0040] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0041] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0042] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0043] The text2img (text to image) task generally contains two data: input text data and output image data, which correspond to each other. The text-to-image model generates image data containing the same descriptive information through the descriptive information in the text data. With the development of artificial intelligence, the quality of text-generated images has reached a relatively high level, and has a certain degree of controllability, meeting the basic needs of practical applications, such as generating face images from text. However, the evaluation indicators of generated images only include indicators such as FID and KID that analyze the distribution of image data. For example, the FID indicator evaluates the generated image by calculating the distance between the feature vectors of the real image and the generated image. The smaller the FID value, the higher the similarity. However, these generated image evaluation indicators do not reflect the degree of conversion from the input text data to the output image data. For example, when generating face images, it is impossible to reflect the model's ability to convert text data to image data from attributes such as gender and age, so it is impossible to accurately analyze the generation ability of the text-to-image model.

[0044] In order to improve the analysis effect of the text-generated image model, an embodiment of the present application provides an analysis method for the text-generated image model, which performs quality analysis on the image generation model based on attribute scores calculated based on text target attributes of the target evaluation text and image target attributes of the target generated image.

[0045] Figure 1 A flow chart of an analysis method of a text-generated image model provided in an embodiment of the present application is shown, and is described in detail as follows:

[0046] S1. Input the target evaluation text into the image generation model to be analyzed to obtain the target generation image output by the image generation model. The target evaluation text includes target attributes, which are the attributes of the evaluation target in the target evaluation text.

[0047] In an embodiment of the present application, the target evaluation text refers to a descriptive text including one or more evaluation targets. The target evaluation text usually records multiple target attributes of the evaluation targets, wherein the evaluation targets correspond to the target attributes. For example, when the evaluation target is a face, the corresponding target attribute is a face attribute; and when the evaluation target is a landscape, the corresponding target attribute is a landscape attribute.

[0048] In the embodiment of the present application, the image generation model to be analyzed refers to a model that can generate an image according to the target attribute (or description information) recorded in the text, such as a text2img (text generation image) model. When the image generation model to be analyzed is a text2img model, the target generation image of the present application embodiment refers to the image generated by the text2img (text generation image) model according to the description information in the target evaluation text, for example, the image generation model to be analyzed can be a text generation image (text2img) model such as a generative adversarial network (Generative Adversarial Nets, GAN) model, a diffusion model (Diffusion Model), etc. At the same time, since the target generation image output by the image generation model is generated according to the target evaluation text, therefore corresponding to the evaluation target in the target evaluation text, the target generation image also includes a generation target, such as an evaluation target is a human face, and the corresponding target generation image generates a target that is also a human face.

[0049] In some optional embodiments, taking the generation of a face image according to an evaluation text as an example, at this time, the target evaluation text is a face evaluation text, and the evaluation target is a face, that is, the face evaluation text is a description text containing face attributes. Assuming that the content of the face evaluation text is "This is a young man, his skin color is fair, his face shape is a square face, his hair is not too long or too short, his eyebrows are straight eyebrows, his eyes are squinting, his eyelids are single, his nose is wide, his mouth is thin lips, his ears are pointed, and he does not wear glasses", according to the above content, it can be seen that the face evaluation text includes face attributes such as gender, hairstyle, face shape, and age, and the image generation model to be analyzed generates a corresponding face generation image according to each face attribute in the above face evaluation text.

[0050] S2. Performing attribute recognition on the target evaluation text to obtain text target attributes, and performing attribute recognition on the target generated image to obtain image target attributes.

[0051] In the embodiment of the present application, performing attribute recognition on the target evaluation text refers to identifying the attribute description information about the evaluation target from the target evaluation text, that is, the text target attribute; performing attribute recognition on the target generated image refers to identifying the attribute description information about the generated target from the target evaluation image, that is, the image target attribute. The text target attributes and image target attributes obtained by attribute recognition can better reflect the information of the target evaluation text and the target generated image.

[0052] In an optional embodiment of the present application, since the target evaluation text and the target generated image are text data and image data, respectively, different methods can be used to perform attribute recognition on the target evaluation text and the target generated image, for example, performing attribute recognition on the target evaluation text through regular expressions, performing attribute recognition on the target generated image through a model trained by deep learning, etc.

[0053] S3. Calculate the attribute score of the image generation model based on the text target attributes and the image target attributes.

[0054] In the embodiments of the present application, since the target generated image is an image obtained by the image generation model based on the target evaluation text, the text target attributes and the image target attributes should also correspond to each other. However, due to the limitations of the generation capability of the image generation model, there is an incomplete correspondence between the text target attributes and the image target attributes, such as the quantity is not completely the same. The above-mentioned attribute score is a score used to reflect the degree of alignment (including quantity, similarity, etc.) between the text target attributes of the target evaluation text and the image target attributes of the target generated image.

[0055] S4. Perform quality analysis on the above image generation model based on the above attribute scores.

[0056] In some embodiments, taking text-generated facial images as an example, multiple facial evaluation texts contain similar descriptions of "male gender", and the same image generation model is used to obtain multiple facial generation images based on different facial evaluation texts. The number of images in which the face is "male" in the multiple facial generation images is counted, and the attribute score of the facial attribute of "male gender" can be calculated based on the counted number, thereby enabling quality analysis of the model generation capability based on the attribute generation dimension to improve the accuracy of the model analysis.

[0057] In the embodiment of the present application, the above-mentioned attribute scoring can be used to evaluate the alignment degree of the text target attribute of the target evaluation text and the image target attribute of the target generated image, and then the attribute analysis of the generation of different target attributes in the target evaluation text can be performed, which can improve the analysis accuracy of the text generation image capability of the text generation image model. At the same time, the image generation model is analyzed by attribute scoring, without limiting the specific type of the text generation image model. Any model that needs to be analyzed for the text generation image capability can be used, which greatly improves the versatility and extensibility of the text generation image quality analysis.

[0058] In the embodiment of the present application, by inputting the target evaluation text into the image generation model to be analyzed, the target generation image output by the image generation model is obtained. Since the target evaluation text includes the target attributes of the evaluation target, the attribute score calculated based on the text target attribute and the image target attribute can better analyze the image generation model. Specifically, since the target generation image is generated based on the target evaluation text, and the target evaluation text includes the target attributes of the evaluation target, the text target attribute can be obtained by attribute recognition of the target evaluation text, and the image target attribute can be obtained by attribute recognition of the target generation image, and the attribute score corresponding to the target attribute can be quickly calculated based on the above text target attribute and image target attribute; and, because the text target attribute is equivalent to the description information of the evaluation target in the target evaluation text, and the image target attribute is equivalent to the description information of the evaluation target in the target generation image, the attribute score calculated by the text target attribute and the image target attribute can reflect the degree of conversion of the description information in the text to the description information in the image, compared with the existing image distribution analysis indicators (FID, KID, etc.), and then reflect the ability of the model to generate images through text, so the quality analysis of the text generation image model can be better performed.

[0059] It should be noted that due to the wide application scenarios of text-generated facial images, such as public security criminal investigation, artistic creation and other fields, the analysis of the text-generated image model is used for explanation in the embodiments of the present application, that is, the evaluation target is a human face. When the evaluation target is a human face, the above-mentioned target evaluation text is a face evaluation text, the above-mentioned target generated image is a face generated image, the above-mentioned generation target is a human face, and the above-mentioned target attributes are human face attributes.

[0060] In one embodiment of the present application, the above-mentioned attribute recognition is performed on the above-mentioned target evaluation text to obtain the text target attribute, including:

[0061] The target attribute text in the target evaluation text is extracted, and the text target attribute of the target evaluation text is detected according to the preset standard attribute text and the target attribute text, wherein the standard attribute text includes the matching text corresponding to the target attribute text.

[0062] In an optional embodiment of the present application, the target evaluation text is taken as an example of a face evaluation text, and the face attribute text in the face evaluation text is extracted, wherein the face attribute text refers to a description text containing face attributes, for example, "male youth", "black medium hair", "not wearing glasses", etc. Due to the flexibility of text description, even for the same face attribute, the description method is different, such as "youth" can be described as "young people", so a preset standard attribute text is required for attribute matching. The standard attribute text is a pre-set matching text used as a comparison benchmark. The matching text is used to match the face attribute text in the face evaluation text. For example, the standard attribute text corresponding to the face attribute of age includes: "young people", "old people", etc.

[0063] In some embodiments, regular expressions and word2vec models (bag of words models) can be used to perform attribute recognition on face evaluation text to obtain text face attributes, wherein regular expressions are used to extract face attribute text in face evaluation text according to various face attributes and associated information in the text, and word2vec models are used to expand the face attribute text, and obtain the face attributes of the face evaluation text by matching the expanded face attribute text with the preset standard attribute text. By combining regular expressions and word2vec models, face attributes can be quickly detected from face evaluation text.

[0064] In an optional embodiment of the present application, it is assumed that the face evaluation text is "This is a young male with fair skin, a square face, medium-length straight black hair, straight eyebrows, squinting eyes, single eyelids, a wide nose, thin lips, pointed ears, and no glasses." Taking facial attributes including age, gender, wearing glasses and hairstyle as an example, facial attributes such as male, straight black hair, and glasses are extracted through regular expressions, and related information such as hair [neither long nor short], [no] glasses is matched, so as to determine that the facial attribute text in the face evaluation text includes: "no glasses", "male", "youth" and "medium black hair"; at the same time, the word2vec model will perform vocabulary expansion on the facial attribute text, and then calculate the similarity between the facial attribute text after the vocabulary expansion and the preset standard attribute text, and determine the attribute corresponding to the standard attribute text with the highest similarity as the text facial attribute of the face evaluation text.

[0065] In an optional embodiment of the present application, the text vector of each face attribute text can be obtained through the word2vec model, including: emb , A emb , where W emb Represents the text vector corresponding to the face attribute text, A embRepresents the text vector corresponding to the standard attribute text. The similarity between the face attribute text and the preset standard attribute text is calculated using the following formula:

[0066] similary_scoFe=cos(norm(W em□ ), norm(A emb ))

[0067] Among them, norm represents the normalization operation.

[0068] For example, the facial attribute text is "youth", and the preset standard attribute texts are "young people" and "old people". Through calculation, the similarity between "youth" and "young people" is the highest, so the attribute "young people" corresponding to the standard attribute text is determined as the text facial attribute of the face evaluation text.

[0069] In another optional embodiment of the present application, the above-mentioned attribute recognition is performed on the above-mentioned target evaluation text to obtain the text target attribute, including:

[0070] The text attribute detection algorithm is used to detect the text target attributes of the above target evaluation text.

[0071] In some embodiments, the above-mentioned text attribute detection algorithm is used to quickly identify the text target attributes of the evaluation target from the target evaluation text, and the above-mentioned text attribute detection algorithm may include a text attribute detection algorithm based on deep learning (such as based on Transformer, based on Attention, etc.). For example, the long short-term memory network (Long Short-Term Memory, LSTM) is used to identify the text face attributes of the face from the face evaluation text.

[0072] In an optional embodiment of the present application, the above-mentioned attribute recognition is performed on the above-mentioned target generated image to obtain the image target attribute, including:

[0073] The image target attributes of the above target generated image are detected using an image attribute detection algorithm.

[0074] In some embodiments, the above-mentioned image attribute detection algorithm is used to quickly identify the text target attributes of the generated target from the target generated image. The above-mentioned image attribute detection algorithm can be an image attribute detection model trained by an open source training image set, etc. For attribute detection of different types of images, it can be trained by different types of training image sets, including: a face attribute detection model trained by a face training image set, a landscape attribute detection model trained by a landscape training image set, etc. For example, the image face attributes of a face are identified from the face evaluation text by the face attribute detection model.

[0075] In an optional embodiment of the present application, when analyzing the text-generated face image model, the above-mentioned image attribute detection algorithm can be a face attribute detection model, which is used to quickly identify the image face attributes in the face image. For example, Deepface, Fastface and other models are used to quickly identify the image face attributes in the face image.

[0076] In the embodiment of the present application, the attribute score of the image generation model is calculated based on the text target attribute and the image target attribute, including:

[0077] Attribute matching is performed on the text target attributes and the image target attributes, and the attribute score of the image generation model is calculated according to the attribute matching result.

[0078] In some embodiments, the text-generated face image model is analyzed for explanation, the above-mentioned text attribute detection algorithm is used to extract the text face attributes corresponding to the face evaluation text through the description text in the face evaluation text, and the image attribute detection algorithm is used to detect the image face attributes in the face generation image; attribute matching refers to the alignment degree matching of different face attributes in the face evaluation text and the face generation image, and the attribute score of the image generation model is calculated according to the attribute matching result. For example, if 10 face evaluation texts contain descriptions similar to "gender male", the corresponding 10 face generation images are obtained by using the model, and the number of face generation images detected as "male" in the 10 face generation images is 9. Through attribute matching, it is determined that for the face attribute of "gender male", the attribute matching results of 9 face evaluation texts and face generation images are successful, and the attribute matching result of 1 face evaluation text and face generation image is unsuccessful. Therefore, through this attribute scoring, the degree of conversion of the text target attributes of the target evaluation text and the image target attributes of the target generated image can be quantified, so as to better analyze the quality of the text-generated face model.

[0079] In an optional embodiment of the present application, the attribute matching of the text target attribute and the image target attribute includes:

[0080] A binary attribute matching is performed on the text target attribute and the image target attribute. The binary attribute matching is used to determine whether the text target attribute and the image target attribute are consistent.

[0081] In some embodiments, binary attribute matching refers to indicating whether the textual face attributes of the face evaluation text and the image face attributes of the face generated image are consistent by means of 0 or 1, and calculating the attribute score based on the attribute matching result, wherein, if the textual face attributes of the face evaluation text and the image face attributes of the face generated image are the same, for example, the textual face attributes of the face evaluation text are "male" and the image face attributes of the face generated image are also "male", assuming that "1" is used to indicate that the textual face attributes and the image face attributes are consistent, then after determining that the binary attribute matching is successful, it is recorded as 1; if the textual face attributes of the face evaluation text and the image face attributes of the face generated image are not the same, for example, the textual face attributes are "male" and the image face attributes are "female", then it is determined that the binary attribute matching is unsuccessful and recorded as 0.

[0082] In one embodiment of the present application, for the above binary attribute matching, the attribute score can be calculated using the following formula:

[0083]

[0084] Among them, score 属性 Score the attribute of a target attribute. For example, 100 face evaluation texts contain the face attribute "male", and the text-generated face model is used to obtain the corresponding 100 face generated images, of which 95 face generated images are detected as having the face attribute of "male". When performing binary attribute matching, 95 matches are successful. The attribute score of the face attribute calculated according to the above formula is 95 / 100=0.95.

[0085] In some embodiments, the quality analysis results of different facial attributes can be determined by setting an attribute threshold. For example, the attribute threshold is set to 0.9. When the attribute score is greater than or equal to 0.9, it indicates that the text-generated face model has a good generation effect on the attribute. When the attribute score is less than 0.9, it indicates that the text-generated face model has a poor generation effect on the attribute.

[0086] In another optional embodiment of the present application, for a text-generated image model of multiple scenes, that is, the input text of the text-generated image model includes descriptions of scenery, characters, objects, etc. at the same time, and the output image corresponding to the text-generated image model also includes scenery, characters, objects, etc. at the same time, when analyzing the text-generated image model, the target evaluation text will include evaluation texts corresponding to different scenes. For example, if model A is used to generate a face image including a background, then the evaluation target includes a face and a background. When analyzing the model, the target evaluation text including description information such as background and face is input into the image generation model to be analyzed to obtain a target generated image; at the same time, when performing attribute recognition on the target evaluation text and the target generated image, it is necessary to perform corresponding attribute recognition on different evaluation targets, that is, perform attribute recognition on the background and face in the target evaluation text and the target generated image respectively, obtain text background attributes and image background attributes, text face attributes and image face attributes, and calculate the attribute scores corresponding to the background and face respectively, and use the attribute scores corresponding to the background and face to perform quality analysis on model A.

[0087] In some embodiments, for the quality analysis of the text-generated image model in multiple scenes, since the target evaluation text and the target generated image include multiple evaluation targets, when performing attribute recognition, multiple methods or multiple models can be combined to perform attribute recognition. For example, when performing attribute recognition on the target generated image, the face attribute detection model and the background attribute detection model can be used simultaneously to perform attribute recognition to obtain the image face attributes and the image background attributes. At the same time, when performing quality analysis, different weights can be set for the attribute scores of the background and the attribute scores of the face to improve the flexibility of the quality analysis of the text-generated image model in multiple scenes.

[0088] In another optional embodiment of the present application, after performing attribute analysis on the image generation model according to the attribute score, the following step is further included:

[0089] The quality analysis results are used to perform model optimization training on the above image generation model.

[0090] In some embodiments, the above-mentioned quality analysis results reflect the generation ability of the current image generation model for a certain target attribute when generating an image. For example, the lower the attribute score of a certain facial attribute, the poorer the generation ability of the model for the attribute when generating an image. Therefore, the image generation model can be optimized and trained according to the quality analysis results to continuously improve the image generation performance of the image generation model, including optimizing the training by adjusting the training data or adjusting the model parameters. For example, if the attribute score of facial attribute A is lower than 0.6, the model can be optimized and trained by adjusting the training data (such as increasing the number of training data containing the facial attribute A, or increasing the proportion of training data containing the facial attribute A, etc.) or model parameters (such as upsampling the training data containing the facial attribute A (for amplifying the features corresponding to the facial attribute A), and adjusting the loss weight corresponding to the training data of the facial attribute, etc.) during the iterative training of the model.

[0091] In another optional embodiment of the present application, the target generation image includes a generation target, and the generation target corresponds to the evaluation target. After the target evaluation text is input into the image generation model to be analyzed and the target generation image output by the image generation model is obtained, the following further includes:

[0092] The number of generated targets in the target generated image is detected, and the stability of the image generation model is analyzed using the generated number and the evaluation number of the evaluation targets.

[0093] In some embodiments, since the generated target in the target generated image corresponds to the evaluation target in the target evaluation text (e.g., if the evaluation target is a human face, then the generated target is also a human face), the generated number of the generated target and the evaluated number of the evaluation target should also correspond, and the relationship between the generated number and the evaluated number can reflect the stability of the generation ability of the image generation model. For example, the generated number of the generated target in the target generated image can be detected by a target number detection algorithm.

[0094] In an optional embodiment of the present application, the above-mentioned target number detection algorithm may be a face number detection algorithm, and the stability analysis of the image generation model is performed by the number of faces detected by the face number detection algorithm. It can be calculated by the following formula:

[0095]

[0096] Among them, score 稳定性Represents the stability score. For example, if all 100 face evaluation texts describe one face, that is, the evaluation number of the evaluation target is 1, then the more face generation images that contain only one face in the face generation images output by the text generation face model, the stronger the stability of the text generation face model. When the text generation face model is used to obtain the corresponding 100 (that is, the total number of face generation images) face generation images, there are 90 images with a generation number of 1 (that is, the number of face generation images with a generation number equal to the evaluation number), and there are 10 images with a face number of 0 or greater than or equal to 1, then At this time, we can also perform stability analysis based on the preset stability threshold. For example, if the stability threshold is 0.85, if the stability score is greater than or equal to 0.85, it means that the image generation model is relatively stable. If it is less than 0.85, it means that the image generation model is relatively unstable. The stability of the face image generated by the text generation face model can be accurately analyzed by the actual number of generated face images and the number of evaluation targets.

[0097] In another optional embodiment of the present application, it also includes:

[0098] Obtaining a target evaluation image corresponding to the target evaluation text, wherein the target evaluation image is an image containing all target attributes in the target evaluation text;

[0099] Calculating a universal score of the image generation model based on the target evaluation image and the target generated image, wherein the universal score is used to measure the similarity between the target feature vector in the target evaluation image and the target feature vector in the target generated image;

[0100] A comprehensive quality analysis of the above image generation model is performed based on the above general score and the above attribute score.

[0101] In some embodiments, since the attribute score reflects the ability of the image generation model to generate attributes of different dimensions, in order to further improve the accuracy of the quality analysis of the image generation model, the image generation model can be comprehensively analyzed by combining the target evaluation image and the general score. The above-mentioned target evaluation image corresponds to the target evaluation text and contains all the target attributes of the evaluation target in the target evaluation text, that is, the target attributes in the target evaluation text can correspond one-to-one with the target attributes in the target evaluation image. For example, for the face evaluation image corresponding to the face evaluation text, the face evaluation image contains all the face attributes in the above-mentioned face evaluation text. The above-mentioned general score may include at least one of the FID evaluation index and the KID evaluation index. For example, when calculating the FID evaluation index, the pre-trained Inception model is used to extract the feature vectors of the face evaluation image and the face generation image, and then these feature vectors are used to calculate the Fréchet distance between the face evaluation image and the face generation image as the FID score. Since the Fréchet distance is used to characterize the difference in mean and variance between the distributions of two images, the smaller the distance, the closer the distribution between the target evaluation image and the face generation image, and the better the quality, the KID evaluation index is similar and will not be repeated.

[0102] In an optional embodiment of the present application, the general score can reflect the similarity between image distributions as a whole, and the attribute score can reflect the generation degree of different target attributes from a fine-grained attribute level. Therefore, by combining the general score and the attribute score, the quality analysis of text-generated images can be more accurately performed.

[0103] In an optional embodiment, taking the general scoring including both the FID evaluation index and the KID evaluation index as an example, the quality analysis of the image generation model is performed using the following formula:

[0104]

[0105] Among them, score 总得分 Indicates the total score of quality analysis, score FID Indicates the FID score of the FID evaluation indicator, score KID The KID score represents the KID evaluation index. Since the smaller the FID score and KID score, the better, and the larger the attribute score, the better, the FID, KID score and (1-attribute score) are combined as the total quality analysis score. The smaller the total quality analysis score, the higher the quality of the text-generated image, that is, the stronger the text-generated image capability of the text-generated image model.

[0106] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0107] Corresponding to the analysis method of the text-generated image model described in the above embodiment, Figure 2 A schematic diagram of the structure of an analysis device for a text-generated image model provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0108] Reference Figure 2 The device may include an analysis device 21 for a text-generated image model. According to the implemented functions, the analysis device 21 for a text-generated image model may include an image generation module 211, an attribute recognition module 212, an attribute score calculation module 213 and a quality analysis module 214.

[0109] Reference Figure 2 , the analysis device 21 of the text-generated image model comprises:

[0110] The image generation module 211 is used to input the target evaluation text into the image generation model to be analyzed, and obtain the target generation image output by the image generation model, wherein the target evaluation text includes target attributes, and the target attributes are the attributes of the evaluation target in the target evaluation text;

[0111] The attribute recognition module 212 is used to perform attribute recognition on the target evaluation text to obtain text target attributes, and to perform attribute recognition on the target generated image to obtain image target attributes;

[0112] The attribute score calculation module 213 is used to calculate the attribute score of the image generation model based on the text target attribute and the image target attribute;

[0113] The quality analysis module 214 is used to perform quality analysis on the image generation model according to the attribute score.

[0114] In some embodiments, in the analysis device 21 of the text-to-image model, the target evaluation text includes face evaluation text, the target generation image includes face generation image, the evaluation target includes face, and the target attributes include face attributes.

[0115] In some embodiments, the attribute recognition module 212 performs attribute recognition on the target evaluation text to obtain text target attributes through the following steps, including:

[0116] The target attribute text in the target evaluation text is extracted, and the text target attribute of the target evaluation text is detected according to the preset standard attribute text and the target attribute text, wherein the standard attribute text includes the matching text corresponding to the target attribute text.

[0117] In some other embodiments, the attribute recognition module 212 performs attribute recognition on the target evaluation text to obtain text target attributes through the following steps, including:

[0118] The text target attributes of the target evaluation text are detected using a text attribute detection algorithm.

[0119] In some other embodiments, the attribute recognition module 212 performs attribute recognition on the target generated image to obtain the image target attribute through the following steps, including:

[0120] The image target attributes of the target generated image are detected using an image attribute detection algorithm.

[0121] In some embodiments, the attribute score calculation module 213 calculates the attribute score of the image generation model based on the text target attribute and the image target attribute through the following steps, including:

[0122] Attribute matching is performed on the text target attributes and the image target attributes, and an attribute score of the image generation model is calculated according to the attribute matching result.

[0123] In some embodiments, the attribute score calculation module 213 performs attribute matching on the text target attribute and the image target attribute through the following steps, including:

[0124] A binary attribute matching is performed on the text target attribute and the image target attribute, and the binary attribute matching is used to determine whether the text target attribute and the image target attribute are consistent.

[0125] In another embodiment, the analysis device 21 of the text-generated image model further includes an optimization training module, and the optimization training module is used to, after performing quality analysis on the image generation model according to the attribute score, include:

[0126] The quality analysis result is used to perform model optimization training on the image generation model.

[0127] In another embodiment, the target generated image includes a generated target, and the generated target corresponds to the evaluation target. The analysis device 21 of the text generated image model also includes a stability analysis module. The stability analysis module is used for inputting the target evaluation text into the image generation model to be analyzed to obtain the target generated image output by the image generation model, including:

[0128] The generation number of the generated target in the target generated image is detected, and the stability analysis of the image generation model is performed using the generation number and the evaluation number of the evaluation target.

[0129] In another embodiment, the text-to-image model analysis device 21 further includes a comprehensive analysis module, and the comprehensive analysis module includes:

[0130] Acquire a target evaluation image corresponding to the target evaluation text, wherein the target evaluation image is an image containing all target attributes in the target evaluation text;

[0131] Calculating a universal score of the image generation model based on the target evaluation image and the target generated image, wherein the universal score is used to measure the similarity between the target feature vector in the target evaluation image and the target feature vector in the target generated image;

[0132] A comprehensive quality analysis is performed on the image generation model based on the general score and the attribute score.

[0133] It should be noted that the information interaction, execution process, etc. between the devices / units are based on the same concept as the method embodiments of the present application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0134] Figure 3 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment includes: at least one processor 30 ( Figure 3 The at least one processor 30 includes a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the method embodiments are implemented.

[0135] The electronic device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of the electronic device 3 and does not constitute a limitation of the electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include an input sending device, a network access device, a bus, etc.

[0136] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0137] In some embodiments, the memory 31 may be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 31 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 3. Further, the memory 31 may also include both an internal storage unit of the electronic device 3 and an external storage device. The memory 31 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 31 may also be used to temporarily store data that has been sent or is to be sent.

[0138] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the functional units and modules is used as an example. In practical applications, the function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0139] An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the method embodiments when executing the computer program.

[0140] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the various method embodiments can be implemented.

[0141] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the various method embodiments when executing the computer program product.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0143] In the embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0145] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0146] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for analyzing a text-generated image model, characterized in that: include: Inputting a target evaluation text into an image generation model to be analyzed to obtain a target generation image output by the image generation model, wherein the target evaluation text includes target attributes, and the target attributes are attributes of the evaluation target in the target evaluation text; Performing attribute recognition on the target evaluation text to obtain text target attributes, and performing attribute recognition on the target generated image to obtain image target attributes; Calculating an attribute score of the image generation model based on the text target attribute and the image target attribute; The image generation model is subjected to quality analysis according to the attribute scores.

2. The text-generated image model analysis method according to claim 1, characterized in that: The step of performing attribute recognition on the target evaluation text to obtain text target attributes includes: The target attribute text in the target evaluation text is extracted, and the text target attribute of the target evaluation text is detected according to the preset standard attribute text and the target attribute text, wherein the standard attribute text includes the matching text corresponding to the target attribute text.

3. The text-generated image model analysis method according to claim 1, characterized in that: The calculating the attribute score of the image generation model based on the text target attribute and the image target attribute includes: Attribute matching is performed on the text target attributes and the image target attributes, and an attribute score of the image generation model is calculated according to the attribute matching result.

4. The text-generated image model analysis method according to claim 3, characterized in that: The attribute matching of the text target attribute and the image target attribute includes: A binary attribute matching is performed on the text target attribute and the image target attribute, and the binary attribute matching is used to determine whether the text target attribute and the image target attribute are consistent.

5. The text-generated image model analysis method according to any one of claims 1 to 4, characterized in that: After performing quality analysis on the image generation model according to the attribute score, the method further includes: The quality analysis result is used to perform model optimization training on the image generation model.

6. The text-generated image model analysis method according to any one of claims 1 to 4, characterized in that: The target generation image includes a generation target, and the generation target corresponds to the evaluation target. After inputting the target evaluation text into the image generation model to be analyzed and obtaining the target generation image output by the image generation model, the method further includes: The number of generated targets in the target generated image is detected, and the stability of the image generation model is analyzed using the number of generated targets and the number of evaluated targets.

7. The text-generated image model analysis method according to claim 1, characterized in that: Also includes: Acquire a target evaluation image corresponding to the target evaluation text, wherein the target evaluation image is an image containing all target attributes in the target evaluation text; Calculating a universal score of the image generation model based on the target evaluation image and the target generated image, wherein the universal score is used to measure the similarity between the target feature vector in the target evaluation image and the target feature vector in the target generated image; A comprehensive quality analysis is performed on the image generation model based on the general score and the attribute score.

8. An analysis device for a text-generated image model, characterized in that: include: An image generation module, used for inputting a target evaluation text into an image generation model to be analyzed, and obtaining a target generation image output by the image generation model, wherein the target evaluation text includes target attributes, and the target attributes are attributes of the evaluation target in the target evaluation text; An attribute recognition module is used to perform attribute recognition on the target evaluation text to obtain text target attributes, and to perform attribute recognition on the target generated image to obtain image target attributes; An attribute score calculation module, used to calculate the attribute score of the image generation model based on the text target attribute and the image target attribute; A quality analysis module is used to perform quality analysis on the image generation model according to the attribute score.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images