Image quality evaluation method, device, equipment, storage medium and program product

Abstract

The present invention discloses an image quality evaluation method, device, equipment, storage medium and program product, which relates to the field of artificial intelligence technology, including: after obtaining a description text and a target image generated based on the description text, obtaining a target keyword according to the description text, the target keyword at least includes a keyword extracted from the description text; performing a first evaluation of the content of the target image at least based on the target keyword to obtain a first evaluation result, the first evaluation result characterizing whether the content in the target image matches the target keyword. The present application improves the reliability of the image quality evaluation result, and thus can improve the accuracy of measuring the ability of the text-based image model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image quality assessment method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of artificial intelligence technology, text-generated graph technology has gradually become a research hotspot. Text-generated graph, that is, the technology of generating corresponding images through natural language description text (referred to as description text), has been increasingly used in the fields of art creation, advertising design, and auxiliary education.

[0003] The text graph technology is usually implemented based on the text graph model, that is, the description text is input into the text graph model, and the text graph model processes the received description text to generate the corresponding image (also called picture). In order to measure the ability of the text graph model to generate images, it is necessary to conduct a quality evaluation on the pictures generated by the text graph model based on the description text, and then determine the ability of the text graph model based on the quality evaluation results of the pictures. Among them, evaluating the content of the picture is an important aspect of the quality evaluation of the picture. For each picture, the way to evaluate the content is to give a general evaluation question and provide multiple evaluation result options accordingly, for example,

[0004] Does this picture seem consistent with your description?

[0005] a) Completely inconsistent

[0006] b) There is a significant difference

[0007] c) There are several small differences

[0008] d) There is a slight difference

[0009] e) Exactly matches the description

[0010] Relevant personnel select one evaluation result as the evaluation result of the content of the picture.

[0011] The above-mentioned evaluation method for the content of the image is a completely general judgment, without specific objective information as a reference. The reliability of the evaluation results of the image content is low, resulting in a low accuracy in measuring the ability of the Wensheng graph model. Summary of the invention

[0012] In view of the above problems, the present application provides an image quality evaluation method, device, equipment, storage medium and program product to improve the reliability of image evaluation. The specific scheme is as follows:

[0013] The first aspect of the present application provides an image quality evaluation method, comprising:

[0014] Obtaining a description text and a target image generated based on the description text;

[0015] Obtaining target keywords according to the description text, wherein the target keywords at least include keywords extracted from the description text;

[0016] A first evaluation is performed on the target image in terms of content based at least on the target keyword to obtain a first evaluation result, wherein the first evaluation result indicates whether the content in the target image matches the target keyword.

[0017] In a possible implementation, the target keyword is a keyword of a preset part of speech extracted from the description text; accordingly, performing a first evaluation of the content of the target image based on the target keyword includes:

[0018] If the target image contains content corresponding to the target keyword, determining that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, determining that the first evaluation result indicates that the content in the target image does not match the target keyword;

[0019] or,

[0020] If there is content corresponding to the target keyword in the target image and the content corresponding to the target keyword is correct, it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

[0021] In a possible implementation, obtaining the target keyword according to the description text includes: extracting a keyword of a preset part of speech from the description text, and obtaining a target noun based on the description text and the target image, wherein the target image includes content corresponding to the target noun, and the description text does not include the target noun; accordingly, performing a first evaluation of the content of the target image based on the target keyword includes:

[0022] If the target image contains content corresponding to the target keyword, and the content corresponding to the target noun is correct, determining that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, determining that the first evaluation result indicates that the content in the target image does not match the target keyword;

[0023] or,

[0024] If there is content corresponding to the target keyword in the target image and the content corresponding to the target keyword is correct, it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

[0025] In a possible implementation, the process of determining whether the target image contains content corresponding to the keyword of the preset part of speech includes:

[0026] Input the keywords of the preset part of speech and the target image into a pre-trained first evaluation model to obtain a first evaluation result output by the first evaluation model, wherein the first evaluation result indicates whether there is content corresponding to the keywords of the preset part of speech in the target image; the first evaluation model is obtained by training using keywords of the preset part of speech extracted from a description text sample and an image generated from the description text sample as training samples, and using whether there is content corresponding to the keywords of the preset part of speech extracted from the description text sample in the image generated from the description text sample as a sample label;

[0027] or,

[0028] The keywords of the preset part of speech and the target image are displayed through a human-computer interaction interface; and a second evaluation result input is received through the human-computer interaction interface, wherein the second evaluation result indicates whether there is content corresponding to the keywords of the preset part of speech in the target image.

[0029] In a possible implementation, the process of determining whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct includes:

[0030] The target keyword and the target image are input into a pre-trained second evaluation model to obtain a third evaluation result output by the second evaluation model, wherein the third evaluation result indicates whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct; the second evaluation model is trained by taking the target keyword obtained based on the description text sample and the image generated by the description text sample as training samples, and taking whether there is content corresponding to the target keyword obtained based on the description text sample in the image generated by the description text sample and whether the content corresponding to the target keyword is correct as sample labels;

[0031] or,

[0032] The target keyword and the target image are displayed through a human-computer interaction interface; a fourth evaluation result is received as input through the human-computer interaction interface, and the fourth evaluation result represents whether there is content corresponding to the target keyword in the target image, and whether the content corresponding to the target keyword is correct.

[0033] In a possible implementation, before performing a first evaluation of the target image in terms of content based at least on the target keyword, the method further includes:

[0034] Extracting ethnic information from the description text;

[0035] If ethnic information is extracted, the first evaluation of the target image in terms of content based at least on the target keyword includes:

[0036] Based on the target keyword and the ethnicity information, a first evaluation is performed on the target image in terms of content, and the first evaluation result indicates whether the content in the target image matches the target keyword and the ethnicity information.

[0037] In a possible implementation, the method further includes:

[0038] Based on the target image, a second visual evaluation is performed on the target image to obtain a second evaluation result, wherein the second evaluation result represents the aesthetics of the target image; different aesthetics correspond to different visual descriptions of the target image from multiple visual dimensions;

[0039] A target evaluation result of the target image is determined based on the first evaluation result and the second evaluation result.

[0040] In a possible implementation, performing a second visual evaluation on the target image based on the target image includes:

[0041] The target image and a plurality of aesthetic evaluation description options are displayed through a human-computer interaction interface, each aesthetic evaluation description option visually describes the target image from a plurality of visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics; the target aesthetic evaluation description option selected by the user is determined through the human-computer interaction interface; and the second evaluation result is determined based on the aesthetics corresponding to the target aesthetic evaluation description option;

[0042] or,

[0043] The target image is input into a pre-trained third evaluation model to obtain a fifth evaluation result output by the third evaluation model, wherein the fifth evaluation result is a target aesthetic evaluation description option among multiple aesthetic evaluation description options, each aesthetic evaluation description option visually describes the target image from multiple visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics; the second evaluation result is determined based on the aesthetics corresponding to the target aesthetic evaluation description option; the third evaluation model is trained using images generated by descriptive text samples as training samples and one of the multiple aesthetic evaluation description options as sample labels.

[0044] In a possible implementation, determining a target evaluation result of the target image based on the first evaluation result and the second evaluation result includes:

[0045] If the first evaluation result indicates that the content in the target image matches the target keyword, and the aesthetics of the target image indicated by the second evaluation result satisfies a preset condition, the target evaluation result of the target image is determined to be quality-passed; otherwise, the target evaluation result of the target image is determined to be quality-failed.

[0046] In a possible implementation, performing a second visual evaluation on the target image based on the target image includes:

[0047] If the first evaluation result indicates that the content in the target image matches the target keyword, performing a second visual evaluation on the target image based on the target image;

[0048] If the first evaluation result indicates that the content in the target image does not match the target keyword, the step of performing a second visual evaluation on the target image based on the target image is not performed.

[0049] A second aspect of the present application provides an image quality assessment device, comprising:

[0050] A first acquisition module, used to obtain a description text and a target image generated based on the description text;

[0051] A second obtaining module, configured to obtain target keywords according to the description text, wherein the target keywords at least include keywords extracted from the description text;

[0052] The first evaluation module is used to perform a first evaluation on the content of the target image based at least on the target keyword to obtain a first evaluation result, wherein the first evaluation result indicates whether the content in the target image matches the target keyword.

[0053] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the image quality assessment method of the first aspect or any implementation of the first aspect.

[0054] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0055] The memory is used to store computer programs;

[0056] The processor is used to execute the computer program so that the electronic device can implement the image quality assessment method of the first aspect or any implementation manner of the first aspect.

[0057] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the image quality assessment method of the first aspect or any implementation of the first aspect.

[0058] By means of the above technical scheme, the present application discloses an image quality assessment method, device, equipment, storage medium and program product. After obtaining a description text and a target image generated based on the description text, a target keyword is obtained according to the description text, and the target keyword at least includes a keyword extracted from the description text; at least a first evaluation of the content of the target image is performed based on the target keyword to obtain a first evaluation result, and the first evaluation result represents whether the content in the target image matches the target keyword. When the present application performs a quality assessment on the content of an image, at least objective factors such as the keywords in the description text and whether the content of the target image matches the keywords are considered, so that the reliability of the image quality assessment result is improved, and thus the accuracy of measuring the capabilities of the text-based image model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0060] Figure 1 A flowchart of an implementation of the image quality evaluation method provided in this application;

[0061] Figure 2 Example of four images generated based on the description text "comic-style cat" provided in this application;

[0062] Figure 3 Example images of four images generated based on the description text "a soft comic-style sofa, a big golden retriever and a puppy are lying on it" provided for this application;

[0063] Figure 4 Example images of four images generated based on the description text "a full and bright red comic-style strawberry" provided in this application;

[0064] Figure 5 Example images of four images generated based on the description text "Cartoon of the surging Yellow River, lush reeds and wetlands on both sides" provided for this application;

[0065] Figure 6 Example images of four images generated based on the description text "A bouquet of flowers on the dining table, cartoon style" provided for this application;

[0066] Figure 7 Example images of four images generated based on the description text "a handsome boy is riding a horse, behind the horse is a carriage, ink painting style" provided for this application;

[0067] Figure 8 Example images of four images generated based on the description text "a kitten lies on the roof, basking in the sun and licking its fur" provided in this application;

[0068] Fig. 9 Example images of four images generated based on the description text "2 teenagers" provided in this application;

[0069] Fig.10 This is an example of four images generated based on the description text "999 roses" provided by this application;

[0070] Fig.11 Example images of four images generated based on the description text "a handsome man, comic style" provided in this application;

[0071] Fig.12 Example images of four images generated for this application based on the description text "a cartoon of a black-skinned girl with naturally glowing skin. She stands on the beach with long curly black hair";

[0072] Fig.13 Example images of four images generated based on the description text "delicious comic-style cake" provided in this application;

[0073] Fig.14 Example images of four images generated based on the description text "AI robot cartoon" provided in this application;

[0074] Fig.15Example images of four images generated based on the description text "oil painting of dog" provided in this application;

[0075] Fig.16 Another implementation flow chart of the image quality evaluation method provided in this application;

[0076] Fig.17 A structural schematic diagram of the image quality evaluation device provided in this application;

[0077] Fig.18 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0078] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0079] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0080] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0081] The solution provided in the present application can be applied to electronic devices, which can be of various types, such as terminal-side devices, intermediate devices, service-side devices, cloud-side devices, etc.

[0082] As mentioned above, when the existing technology evaluates the content quality of images generated by the text graph model, the evaluation questions given are relatively general and completely rely on the user's subjective and general judgment. The quality evaluation results obtained in this way fluctuate greatly, the reliability of the evaluation results is low, and the reference value is small.

[0083] In order to improve the reliability of image quality evaluation, this application scheme is proposed.

[0084] See also Figure 1 , Figure 1A flowchart for implementing the image quality evaluation method provided in the embodiment of the present application may include:

[0085] Step S101: Obtain description text and a target image generated based on the description text.

[0086] The description text can be input into the Wensheng graph model to obtain the image generated by the Wensheng graph model. The description text describes the content that needs to be specifically drawn by the Wensheng graph model. The drawing content can include but is not limited to the following: people, scenery, animals, urban buildings, food, plants, etc. For example, the description text can be "draw a cute cat", "draw a landscape", etc.

[0087] The text-generated graph model can generate one or more images based on the description text. In the case where multiple images are generated based on the description text, the target image is any one of the multiple images. For each image generated by the text-generated graph model, a quality assessment is performed based on the solution of this application.

[0088] Step S102: obtaining target keywords according to the description text, where the target keywords at least include keywords extracted from the description text.

[0089] In the present application, the target keywords may only include keywords extracted from the description text, or the target keywords include: keywords extracted from the description text and keywords recognized from the target image.

[0090] Optionally, the target keyword may include multiple keywords. The multiple keywords may be keywords of preset parts of speech. The preset parts of speech may include only one part of speech, or may include multiple different parts of speech. As an example, the predicted part of speech may include but is not limited to at least one of the following: noun, adjective, preposition, verb, numeral, etc.

[0091] Step S103: performing a first evaluation on the target image in terms of content based at least on the target keyword to obtain a first evaluation result; the first evaluation result indicates whether the content in the target image matches the target keyword.

[0092] In the case where the target keyword includes multiple keywords, the first evaluation result represents whether the content in the target image matches each keyword.

[0093] The first evaluation can be a manual evaluation or an automatic evaluation by the system.

[0094] The image quality evaluation method provided in the present application, when evaluating the quality of the content of an image, at least considers objective factors such as the keywords in the description text and whether the content of the target image matches the keywords, and evaluates the image content by keywords, thereby refining the evaluation dimensions of the content and reducing subjective differences, thereby improving the reliability of the image quality evaluation results, and further improving the accuracy of measuring the capabilities of the text-based image model.

[0095] In an optional embodiment, the target keyword is a keyword of a preset part of speech extracted from the description text, such as noun, adjective, preposition, etc.

[0096] Accordingly, one implementation method of performing a first evaluation of the content of the target image based on the target keyword may be:

[0097] If the target keyword is reflected in the target image (i.e., there is content corresponding to the target keyword in the target image), it is determined that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result indicates that the content in the target image does not match the target keyword.

[0098] Optionally, one implementation method of determining whether content corresponding to the target keyword exists in the target image may be:

[0099] The target keyword and the target image are input into a pre-trained first evaluation model to obtain a first evaluation result output by the first evaluation model. The first evaluation result indicates whether there is content corresponding to the target keyword in the target image.

[0100] The first evaluation model is obtained by training with keywords of preset parts of speech extracted from the description text sample and images generated by the description text sample as training samples, and with whether the images generated by the description text sample contain content corresponding to the keywords of preset parts of speech extracted from the description text sample as sample labels. Specifically, the first evaluation model can be obtained by training with the first training set, and each training sample in the first training set includes: keywords of preset parts of speech extracted from the description text sample and images generated by the description text sample; the labels of the training samples are pre-annotated target information, and the target information represents whether the keywords of preset parts of speech in the training sample are reflected in the images in the training sample (that is, whether the images in the training sample contain content corresponding to the keywords of preset parts of speech in the training sample). When training the first evaluation model, the training sample is input into the first evaluation model to obtain the evaluation result output by the first evaluation model, and the parameters of the first evaluation model are updated with the goal that the evaluation result output by the first evaluation model is close to the label of the training sample. That is, the first evaluation model is updated based on the evaluation results output by the first evaluation model and the labels of the training samples, so that the evaluation results obtained by the updated first evaluation model based on the training samples are closer to the labels of the training samples than the evaluation results obtained by the first evaluation model before the update based on the training samples.

[0101] Optionally, another implementation method of determining whether there is content corresponding to the target keyword in the target image may be:

[0102] The target keyword and the target image are displayed through the human-computer interaction interface; and a second evaluation result input is received through the human-computer interaction interface, where the second evaluation result indicates whether there is content corresponding to the target keyword in the target image.

[0103] Optionally, for each keyword in the target keywords, a group of options is displayed, which includes two options: "reflected" and "not reflected". For each keyword, the user determines whether the keyword is reflected in the target image (i.e., determines whether there is content corresponding to the keyword in the target image) by observing the target image. If it is reflected, the user selects "reflected"; otherwise, the user selects "not reflected". After all options are selected for each keyword in the target keywords, the evaluation result (i.e., the second evaluation result) is submitted.

[0104] Although this scheme is also manually selected, the present application is selected by the user based on whether the keywords in the description text are reflected in the image, which refines the evaluation direction, and each evaluation direction is specific and objective. Therefore, compared with the existing general evaluation method, the reliability of the obtained evaluation results is still improved.

[0105] For example, the description text is "Cartoon-style cat", and the noun is: cat. Figure 2 As shown, it is an example of four images generated based on the description text "comic-style cat" provided by this application. In this example, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. Obviously, there is a cat in the target image, so the target image matches the noun. Since there are no adjectives and prepositions in the description text of this example, the first evaluation result indicates that the target keyword is reflected in the target image.

[0106] For example, the description text is "a soft cartoon-style sofa with a big golden retriever and a puppy lying on it", in which the nouns are: sofa, golden retriever, puppy. Figure 3 As shown, this is an example of four images generated based on the description text "soft cartoon-style sofa, a big golden retriever and a puppy are lying" provided by this application. In this example, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. Obviously, the target image has a golden retriever and a puppy, but no sofa. Therefore, the target image does not match the noun. Since there are no adjectives and prepositions in the description text of this example, the first evaluation result indicates that not every keyword is reflected in the target image.

[0107] In a text, when a word is used to express the nature, attribute or state of a person or thing, the word is an adjective. For example, "big" is generally used to describe a person or thing as being large, such as drawing a big apple. Here, "big" is used to describe the state of the apple, so "big" is an adjective. In the generated image, you can find the specific person or thing corresponding to the adjective, and find the corresponding value, attribute or state. For example, an image generated based on the description text "a big apple" draws an apple that takes up 60% of the picture. Here you can see that the apple is large. It is considered that the generated image matches the adjective "big".

[0108] For example, the description text is "a full and bright red cartoon-style strawberry", in which the noun is: strawberry; the adjective is: full and bright red. Figure 4 As shown, it is an example image of four images generated based on the description text "a plump and bright red comic-style strawberry" provided by the present application. In the example image, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. Obviously, there are strawberries in the target image, so the target image matches the noun. The adjective "plump and bright red" reflects the shape of the strawberry. "Plump and bright red" can be broken down into that the strawberry is round and about to burst, and its color is red and bright. The information that can be extracted from it is that the strawberry is round and red. From Figure 4It can be seen that the properties of the strawberries in the target image are red and round objects. It can be seen that the target image matches the adjectives. Since there are no prepositions in the description text of this example, the first judgment result indicates that the target keywords are reflected in the target image.

[0109] For example, the description text is "A comic of the surging Yellow River, with lush reeds and wetlands on both sides". Among them, the nouns are: Yellow River, reeds, wetlands; the adjectives are: surging, lush. As Figure 5 shown, it is an example diagram of four images generated based on the description text "A comic of the surging Yellow River, with lush reeds and wetlands on both sides" provided by this application. In this example diagram, thumbnails of the four images and a preview of one of the images (i.e., the target image) are shown. Obviously, there are the Yellow River, reeds, and wetlands in the target image. Therefore, the target image matches the nouns. The adjective "surging" reflects the state of the Yellow River. From Figure 5 it can be seen that the Yellow River painted in the target image is flowing slowly without any waves, which does not match "surging", that is, the target image does not match the adjectives. Since there are no prepositions in the description text of this example, the first judgment result indicates that not every keyword is reflected in the target image.

[0110] In a text, a word used to indicate the direction or position of the existence of a specific thing or person is a preposition. For example, there is a cat on the table. In this text, the specific things include the table and the cat. "On" indicates the specific direction and position of the cat, which is consistent with the definition of a preposition. That is, in this text, the preposition is "on".

[0111] For example, the description text is "There is a bunch of flowers on the dining table, in a comic style". Among them, the nouns are: dining table, flowers; the preposition is: on. As Figure 6 shown, it is an example diagram of four images generated based on the description text "There is a bunch of flowers on the dining table, in a comic style" provided by this application. In this example diagram, thumbnails of the four images and a preview of one of the images (i.e., the target image) are shown. Obviously, there are a dining table and flowers in the target image. Therefore, the target image matches the nouns. For the preposition "on" that shows the direction and position of things, from Figure 6 the target image shown, it can be seen that the specific things are the table and the flowers, and the flowers are placed on the table. That is, the target image matches the preposition. Since there are no adjectives in the description text of this example, the first judgment result indicates that the target keywords are reflected in the target image.

[0112] If in the generated image, the flowers are on the ground, then the target image does not match the preposition. Correspondingly, the first judgment result indicates that not every keyword is reflected in the target image.

[0113] In the above embodiment, the image is judged in terms of content relevance, that is, the focus is on whether the keywords of each part of speech are reflected in the target image, and whether the keywords are correctly reflected in the target image is not paid attention to. However, in actual applications, there may be drawing errors in the generated image, such as drawing deformity or other errors.

[0114] In an optional embodiment, in order to further improve the reliability of image quality judgment, the judgment of content correctness can be introduced. Based on this, another implementation method of the first evaluation of the content of the target image based on the target keyword provided by the present application can be:

[0115] If the target keyword is correctly reflected in the target image (i.e., there is content corresponding to the target keyword in the target image, and the content corresponding to the target keyword is correct), it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

[0116] For adjectives and media, if the adjectives and prepositions are reflected in the target image, they must be correct. However, for nouns, they may be incorrect. For example, when the specific thing in the text is a person, a person is a head, two arms, and two legs. In the image, the specific thing is a head, two arms, and three legs. The comparison is inconsistent, and it can be considered that the generated image does not correctly reflect the noun. Alternatively, in the case of inconsistent comparison, we can further pay attention to the proportion of the inconsistent part in the image. If the proportion is less than the threshold, it is considered that the generated image correctly reflects the noun, otherwise it is considered that the generated image does not correctly reflect the noun.

[0117] In this example, the keywords of the preset parts of speech may include numerals and verbs in addition to the aforementioned nouns, adjectives and prepositions.

[0118] Optionally, one implementation method of determining whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword in the target image is correctly reflected may be:

[0119] The target keyword and the target image are input into a pre-trained second judgment model to obtain a third judgment result output by the second judgment model. The third judgment result indicates whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct.

[0120] The second evaluation model is trained by using the target keywords obtained based on the description text sample and the image generated by the description text sample as training samples, and whether the content corresponding to the target keyword obtained based on the description text sample exists in the image generated by the description text sample, and whether the content corresponding to the target keyword is correct as the sample label. Specifically, the second evaluation model can be obtained by training with the second training set, and each training sample in the second training set includes: keywords of preset parts of speech extracted from the description text sample, and images generated by the description text sample; the labels of the training samples are pre-annotated target information, and the target information represents whether the keywords of preset parts of speech in the training sample are correctly reflected in the images in the training samples (that is, whether the images in the training samples contain content corresponding to the keywords of preset parts of speech in the training samples, and whether the content corresponding to the keywords of preset parts of speech in the training samples in the images in the training samples is correct). When training the second evaluation model, the training sample is input into the second evaluation model to obtain the evaluation result output by the second evaluation model, and the parameters of the second evaluation model are updated with the evaluation result output by the second evaluation model approaching the label of the training sample as the goal. That is, the second judgment model is updated based on the judgment results output by the second judgment model and the labels of the training samples, so that the judgment results obtained by the updated second judgment model based on the training samples are closer to the labels of the training samples than the judgment results obtained by the second judgment model before the update based on the training samples.

[0121] Optionally, another implementation method of determining whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword in the target image is correct may be:

[0122] The target keyword and the target image are displayed through the human-computer interaction interface; the fourth evaluation result is received through the human-computer interaction interface, and the fourth evaluation result represents whether there is content corresponding to the target keyword in the target image, and whether the content corresponding to the target keyword in the target image is correct.

[0123] Optionally, for each keyword in the target keywords, a group of options is displayed, and the group of options includes two options: "correctly reflected" and "incorrectly reflected". For each keyword, the user determines whether the keyword is correctly reflected in the target image by observing the target image (that is, whether there is content corresponding to the keyword in the target image, and whether the content corresponding to the keyword in the target image is correct). If it is correctly reflected, the user selects "correctly reflected", otherwise, the user selects "incorrectly reflected". After all keywords in the keywords with preset parts of speech are selected, the evaluation result (i.e., the fourth evaluation result) is submitted. For each keyword, correct reflection means that there is content corresponding to the keyword in the target image, and the content corresponding to the keyword in the target image is correct. Otherwise, it is incorrectly reflected.

[0124] Although this scheme is also manually selected, the present application is selected by the user based on whether the keywords in the description text are correctly reflected in the image, which refines the evaluation direction, and each evaluation direction is specific and objective. Therefore, compared with the existing general evaluation method, the reliability of the obtained evaluation results is still improved.

[0125] For example, the description text is "A handsome boy is riding a horse, and behind the horse is a carriage, in ink painting style", where the nouns are: boy, horse, carriage. Figure 7 As shown in FIG. 1 , an example image of four images generated based on the description text “a handsome boy is riding a horse, behind the horse is a carriage, ink painting style” provided by the present application is shown. In the example image, thumbnails of the four images and a preview of one of the images (i.e., the target image) are shown. The horse should have four legs, and Figure 7 The target image has 7 legs, and the error area is manually marked in the image (such as the rectangular box area in the figure), so that the image quality evaluation device can determine the proportion of the error area to the entire image according to the size of the error area and the size of the entire image. In this example, the proportion of the error area to the entire image is greater than 20% (that is, the threshold is 20%). It can be determined that not every part of speech keyword is correctly reflected in the target image. Therefore, the first evaluation result indicates that the content in the target image does not match the target keyword.

[0126] Furthermore, the error related information "object: horse; error description: deformity; prompt (ie description text)" may be input to determine the related information of the first evaluation result.

[0127] When there are specified verbs in the description text (such as running, jumping, climbing up and down, etc.), they need to be drawn in the generated image.

[0128] Among them, if the object (noun) that should perform the action in the image is not drawn to perform the action, the image does not match the verb, otherwise, the image matches the verb.

[0129] Alternatively, if the object (noun) that should perform the action in the image is not drawn to perform the action, and the proportion of the object that should perform the action in the entire screen exceeds a threshold (for example, 10%), then the image does not match the verb, otherwise, the image matches the verb. The user can mark the object that should perform the action, and then the image quality assessment device determines the proportion of the object that should perform the action in the entire screen based on the size of the marked area and the size of the entire image. Other calculation methods that need to calculate the proportion of a local area in the image to the entire screen can refer to this method, and this application will not repeat them one by one.

[0130] For example, the description text is "The kitten lies on the roof, basking in the sun and licking its fur", where the nouns are: kitten, roof; and the verb is: licking fur. Figure 8 As shown, it is an example of four images generated based on the description text "kitten lying on the roof, basking in the sun and licking fur" provided by this application. In this example, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. In the target image, there are roofs and kittens, and both are correct; but there is no verb (licking fur). The kitten is the subject of the action, accounting for more than 10%, so the target image does not match the verb. It can be determined that not every keyword of the part of speech is correctly reflected in the target image. Therefore, the first evaluation result indicates that the content in the target image does not match the target keyword.

[0131] For numerals, only numerals within a preset number (for example, 5) are judged for specific values, while for numerals exceeding the preset number, no specific value judgment is performed, as long as they are greater than the preset number.

[0132] For example, the description text is "2 teenagers", where the noun is: teenager; the numeral is: 2. Fig. 9 As shown, it is an example of four images generated based on the description text "2 teenagers" provided by this application. In this example, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed, wherein, in the target image, exactly two teenagers are displayed, so the target image matches both the noun and the numeral. It can be determined that the keywords of each part of speech are correctly reflected in the target image, so the first evaluation result indicates that the content in the target image matches the target keyword.

[0133] For example, the description text is "999 roses", where the noun is: roses and the numeral is: 999. Fig.10As shown, it is an example of four images generated based on the description text "999 roses" provided by the present application. In the example, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. Among them, the numeral is greater than 5, so the requirement for the numeral in the target image is that it is greater than 5. Obviously, roses are displayed in the target image, and the number of roses is greater than 5. Therefore, the target image matches both the noun and the numeral. It can be determined that the keywords of each part of speech are correctly reflected in the target image. Therefore, the first evaluation result indicates that the content in the target image matches the target keyword.

[0134] In the aforementioned embodiment, the target keywords only include keywords extracted from the description text. In an optional embodiment, in addition to the keywords extracted from the description text, the target keywords may also include nouns that are reflected in the target image but not included in the description text (for ease of description and distinction, they are recorded as target nouns).

[0135] Optionally, the target image can be converted into target text through an image-to-text model, and then nouns in the target text can be extracted, and the nouns in the target text can be compared with the nouns in the description text to determine the nouns reflected in the target image but not included in the description text as target nouns.

[0136] Based on this, another implementation of the first evaluation of the content of the target image based on at least the target keyword provided by the present application may be:

[0137] If the target keyword (including the keyword of the preset part of speech and the target noun) is reflected in the target image (i.e., there is content corresponding to the target keyword in the target image), and the target noun is correctly reflected in the target image (i.e., there is content corresponding to the target noun in the target image, and the content corresponding to the target noun in the target image is correct), it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword. In this example, only whether the content corresponding to the target noun in the target image is correct is concerned, and whether the content corresponding to the keyword of the preset part of speech is correct is not concerned.

[0138] The implementation method of determining whether the target keyword is reflected in the target image can refer to the aforementioned embodiment, which will not be described in detail here.

[0139] The target noun that is reflected in the target image but not included in the description text is correctly reflected in the target image, including: the content corresponding to the target noun exists in the target image and the content is correct, and the proportion of the content corresponding to the target noun in the target image to the entire image is less than a threshold (for example, 10%); or, the content corresponding to the noun exists in the target image and the content is erroneous, but the proportion of the erroneous content to the entire image is less than a threshold (for example, 3%).

[0140] Another implementation of the first evaluation of the content of the target image based on the target keyword provided by the present application may be:

[0141] If the target keyword (including keywords of preset parts of speech and target nouns) is reflected in the target image (i.e., there is content corresponding to the target keyword in the target image), and the target keyword is correctly reflected in the target image (i.e., the content corresponding to the target keyword in the target image is correct), it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

[0142] The implementation method of determining whether the target keyword is correctly reflected in the target image can refer to the aforementioned embodiment, which will not be described in detail here.

[0143] The target noun that is reflected in the target image but not included in the description text is correctly reflected in the target image, including: the content corresponding to the target noun exists in the target image and the content is correct, and the proportion of the content corresponding to the target noun in the target image to the entire image is less than a threshold (for example, 10%); or, the content corresponding to the noun exists in the target image and the content is erroneous, but the proportion of the erroneous content to the entire image is less than a threshold (for example, 3%).

[0144] As mentioned above, the content in the description text may involve characters, and characters may involve ethnic information. In order to further improve the reliability of image quality evaluation, the present application may also consider ethnic information when evaluating the quality of images.

[0145] Based on this, the image quality evaluation method provided by this application may also include:

[0146] Prior to the first evaluation of the target image in terms of content based at least on the target keywords, it also includes:

[0147] The ethnic information is extracted from the description text. Of course, if the description text involves a person, the ethnic information will be extracted. If the description text does not involve a person, the ethnic information will not be extracted.

[0148] Correspondingly, if ethnic information is extracted, one implementation of the first evaluation of the content of the target image at least based on the target keyword may be:

[0149] Based on the target keywords and ethnic information, a first evaluation is performed on the target image in terms of content, and the first evaluation result indicates whether the content in the target image matches the target keywords and ethnic information.

[0150] A content evaluation of the target image may be performed based on the target keywords, and another content evaluation of the target image may be performed based on the ethnic information. These two evaluations constitute a first evaluation.

[0151] The process of evaluating the content of the target image based on the target keyword can be found in the above-mentioned embodiment, which will not be described in detail here.

[0152] The process of evaluating the target image based on ethnic information may include:

[0153] Optionally, if the description text involves a person but does not clearly state the race information, or there is no race-related information, the race information is determined to be preset race information (eg, Asian).

[0154] If the description text involves a person and specifies race information, or there is a description of race-related information, the race is determined to be the race information described in the description text, or the race information determined based on the description of race-related information.

[0155] As an example, a pre-trained ethnic information recognition model can be input into the target image to obtain an ethnic information recognition result output by the ethnic information recognition model. If the ethnic information represented by the ethnic information recognition result is the same as the specified ethnic information in the description text, or the description of the ethnic related information represented by the ethnic information recognition result is the same as or semantically the same as the description of the ethnic related information in the description text, it is determined that the content in the target image matches the ethnic information; otherwise, it is determined that the content in the target image does not match the ethnic information.

[0156] As an example, the extracted racial information or racial related information and the target image can be displayed through the human-computer interaction interface; the sixth evaluation result is received as input through the human-computer interaction interface, and the sixth evaluation result represents whether the racial information of the person in the target image matches the extracted racial information or racial related information.

[0157] For example, the description text is "a handsome man, comic style", where the noun is: man; the adjective is: handsome; and there is no ethnic information. Fig.11As shown, it is an example of four images generated based on the description text "a handsome man, comic style" provided by this application. In this example, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. The person in the target image is a handsome man who is Asian. Therefore, the target image matches all nouns, adjectives, and ethnic information. Therefore, the first evaluation result indicates that the content in the target image matches all target keywords and ethnic information.

[0158] For example, the description text is "a cartoon of a black-skinned girl with naturally shiny skin. She stands on the beach with long curly black hair", where the nouns are: girl, beach; and the ethnic-related information is: black-skinned girl. Fig.12 As shown, it is an example image of four images generated based on the description text "a cartoon of a black-skinned girl with naturally shiny skin. She stands on the beach with long curly black hair" provided by this application. In this example image, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed. The character in the target image is a girl with black skin, naturally shiny skin, and long curly black hair. Therefore, the target image matches both the noun and the ethnic information. Therefore, the first evaluation result indicates that the content in the target image matches both the target keyword and the ethnic information.

[0159] The above content introduces the implementation method of content quality evaluation of images generated by the text-based graph model. In some scenarios, it may be necessary to only evaluate the quality of the images. In other scenarios, it may be necessary to evaluate the quality of the images generated by the text-based graph model in terms of both content and visual aspects.

[0160] Currently, when performing visual quality evaluation on images generated by the Wensheng graph model, the following questions are output for users to choose from:

[0161] In your opinion, does this picture meet your aesthetic standards?

[0162] a) Ugly, unacceptable, and totally devoid of any aesthetic value

[0163] b) Ugly, somewhat repulsive, poor composition, poor presentation of the main content of the picture

[0164] c) Ordinary, acceptable, with mediocre composition and main content

[0165] d) Excellent, can be recommended, the picture is beautiful, and the content of the picture is aesthetically pleasing

[0166] e) Exquisite, beautiful composition, beautiful and exquisite picture content

[0167] Obviously, the current image quality evaluation method also makes general judgments without specific objective information as a reference. The reliability of the evaluation results of the image content is low, resulting in low accuracy in measuring the capabilities of the Wensheng graph model.

[0168] In order to further improve the accuracy of image quality evaluation, Figure 1 Based on the illustrated embodiment, the image quality evaluation method provided by the present application may further include:

[0169] Step S104: Based on the target image, a visual evaluation is performed on the target image to obtain a second evaluation result; the second evaluation result represents the aesthetics of the target image. Different aesthetics correspond to different visual descriptions of the target image from multiple visual dimensions.

[0170] The target image may be evaluated in multiple visual dimensions to obtain a second evaluation result. The multiple visual dimensions here may include but are not limited to: color, composition, proportion, shape, overall style, and white edge / signature / watermark, etc. Accordingly, the second evaluation result represents the degree of conformity of the target image with the user's aesthetic expectations in multiple visual dimensions, and different degrees of conformity represent different aesthetics of the target image.

[0171] The second evaluation can be a manual evaluation or an automatic system evaluation.

[0172] Step S105: Determine a target evaluation result of the target image based on the first evaluation result and the second evaluation result.

[0173] The present application combines the quality evaluation results of the content (ie, the first evaluation results) and the quality evaluation results of the vision (ie, the second evaluation results) to obtain the final quality evaluation results of the target image (ie, the target evaluation results).

[0174] In an optional embodiment, one implementation of the second visual evaluation of the target image based on the target image may be:

[0175] The target image and multiple aesthetic evaluation description options are displayed through a human-computer interaction interface. Each aesthetic evaluation description option describes the target image visually from multiple visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics.

[0176] The target aesthetic evaluation description option selected by the user is determined through the human-computer interaction interface.

[0177] The second evaluation result is determined based on the aesthetics corresponding to the target aesthetics evaluation description option.

[0178] As shown in Table 1, it is an example of multiple aesthetic evaluation description options and their corresponding aesthetics provided in an embodiment of the present application.

[0179] Table 1

[0180]

[0181]

[0182] In the example shown in Table 1, each aesthetic evaluation description option describes the target image visually from multiple visual dimensions such as color, composition, proportion, shape, and overall style. The aesthetics is the score corresponding to the aesthetic evaluation description option.

[0183] In an optional embodiment, another implementation of the second visual evaluation of the target image based on the target image may be:

[0184] The target image is input into the pre-trained third evaluation model to obtain the fifth evaluation result output by the third evaluation model, and the fifth evaluation result is the target aesthetic evaluation description option among multiple aesthetic evaluation description options (as shown in Table 1), each aesthetic evaluation description option visually describes the target image from multiple visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics. The third evaluation model is trained using images generated by description text samples as training samples and one of the multiple aesthetic evaluation description options as sample labels.

[0185] The second evaluation result is determined based on the aesthetics corresponding to the target aesthetics evaluation description option, as shown in Table 1.

[0186] For example, the description text is "delicious cartoon-style cake", and the noun is: cake. Fig.13 As shown, this is an example image of four images generated based on the description text "Delicious comic-style cake" provided by the present application. In the example image, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed, wherein the character in the target image is a cake, and the image is bright in color, with coordinated colors and a centered and uncluttered composition. The strawberries and candles in the cake are in a coordinated proportion, and the shape of the cake is consistent with real life. The style is the common comic style, and there are no white edges, signatures, or watermarks on the screen. However, the overall details are poor, and the strawberry similarity is low, which meets the description of "the color, composition, proportion, shape, and overall style are basically in line with expectations, with only one item not meeting expectations." Therefore, its aesthetics is 3 points, and its aesthetics passes the quality evaluation.

[0187] For example, the description text is "AI robot cartoon", and the noun is: robot. Fig.14As shown, this is an example image of four images generated based on the description text "AI Robot Comics" provided by this application. In the example image, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed, wherein the target image shows multiple robots. This image has bright colors, coordinated colors, messy composition, many messy robots, blurred lines, and unclear images. The style is the common cartoon style, and there are no white edges, signatures, or watermarks on the screen. It meets the description of "the color, composition, proportion, shape, and overall style are basically in line with expectations, with only two items not meeting expectations." Therefore, its aesthetics is 2 points, and its aesthetics failed the quality evaluation.

[0188] For example, the description text is "oil painting of dog", and the noun is: dog. Fig.15 As shown, it is an example image of four images generated based on the description text "oil painting of dog" provided by this application. In the example image, thumbnails of the four images and a preview of one of the images (i.e., the target image) are displayed, wherein the target image shows three dogs. There is an obvious signature in the lower right corner of this image, so its aesthetics is 2 points, and its aesthetics failed the quality evaluation.

[0189] In an optional embodiment, an implementation method of determining the target evaluation result of the target image based on the first evaluation result and the second evaluation result may be:

[0190] If the first evaluation result indicates that the content in the target image matches the target keyword, and the aesthetics of the target image indicated by the second evaluation result satisfies the preset conditions, the target evaluation result of the target image is determined to be quality passed; otherwise, the target evaluation result of the target image is determined to be quality failed.

[0191] That is, as long as the first evaluation result indicates that the content in the target image does not match the target keyword, and / or the aesthetics of the target image indicated by the second evaluation result does not meet the preset conditions, the target evaluation result of the target image is determined to be quality failure.

[0192] Optionally, in order to reduce the amount of calculation, after obtaining the first evaluation result, first determine whether the first evaluation result indicates that the content in the target image matches the target keyword. If so, then perform a second visual evaluation on the target image. Otherwise, directly determine that the target evaluation result of the target image is not of good quality, without performing a second visual evaluation on the target image. That is, one implementation method of the present application for performing a second visual evaluation on the target image based on the target image may be:

[0193] If the first evaluation result indicates that the content in the target image matches the target keyword, then based on the target image, performing a second visual evaluation on the target image;

[0194] If the first evaluation result indicates that the content in the target image does not match the target keyword, the step of performing a second visual evaluation on the target image based on the target image is not performed.

[0195] like Fig.16 As shown, another implementation flow chart of the image quality evaluation method provided in the embodiment of the present application may include:

[0196] The prompt (i.e., description text) input by the user is given to the interface of the text-generated graph model. After receiving the prompt, the text-generated graph model performs calculations, generates a picture, and outputs it.

[0197] First, we evaluate the content relevance based on the prompt and the pictures generated by the text graph model. The evaluation content is shown in Table 2.

[0198] Table 2

[0199]

[0200] If the evaluation results corresponding to each part of speech shown in Table 2 are: all keywords of each part of speech are matched, then the content correctness is evaluated based on the prompt and the picture generated by the text graph model. The evaluation content is shown in Table 3.

[0201] Table 3

[0202]

[0203] If the evaluation results corresponding to each part of speech shown in Table 3 are: all keywords of each part of speech are matched, then a visual evaluation is performed based on the pictures generated by the text graph model, and the evaluation content is shown in Table 1.

[0204] If the aesthetics of the image is determined to be passed based on Table 1, the quality evaluation result of the image is that the quality is passed.

[0205] If the evaluation results corresponding to the parts of speech shown in Table 2 are: not matching at least one keyword, then the quality evaluation result of the image is determined to be unqualified, and there is no need to evaluate the content correctness based on Table 3.

[0206] Similarly, if the evaluation results corresponding to the parts of speech shown in Table 3 are: not matching at least one keyword, then the quality evaluation result of the image is determined to be unqualified, and there is no need to perform visual evaluation based on Table 1.

[0207] In addition, for the convenience of statistics, after obtaining the target evaluation results of each image, the image can be marked with the target evaluation results. Subsequently, the ratio of the number of images that pass the quality test to the total number of images can be counted, and the capability of the Wensheng graph model can be determined based on the ratio.

[0208] Corresponding to the method embodiment, the present application also provides an image quality evaluation device. A structural schematic diagram of the image quality evaluation device provided in the embodiment of the present application is as follows: Fig.17 As shown, it may include:

[0209] A first obtaining module 1701, a second obtaining module 1702, a first evaluation module 1703, a second evaluation module 1704 and a determination module 1705;

[0210] The first obtaining module 1701 is used to obtain a description text and a target image generated based on the description text;

[0211] The second obtaining module 1702 is used to obtain target keywords according to the description text, and the target keywords at least include keywords extracted from the description text;

[0212] The first evaluation module 1703 is used to perform a first evaluation on the target image in terms of content based at least on the target keyword to obtain a first evaluation result, where the first evaluation result indicates whether the content in the target image matches the target keyword.

[0213] The image quality assessment device provided in the embodiment of the present application, when evaluating the quality of the content of the image, at least considers objective factors such as the keywords in the description text and whether the content of the target image matches the keywords, and evaluates the image content according to the keywords, thereby refining the evaluation dimensions of the content and reducing the subjective gap, thereby improving the reliability of the image quality assessment results, and further improving the accuracy of the measurement of the capabilities of the text-based image model.

[0214] In an optional embodiment, the target keyword is a keyword of a preset part of speech extracted from the description text; accordingly, when the first evaluation module 1703 performs a first evaluation of the content of the target image based on the target keyword, it is used to:

[0215] If the target image contains content corresponding to the target keyword, determining that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, determining that the first evaluation result indicates that the content in the target image does not match the target keyword;

[0216] or,

[0217] If there is content corresponding to the target keyword in the target image and the content corresponding to the target keyword is correct, it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

[0218] In an optional embodiment, the second obtaining module 1702 is used to: extract keywords of preset parts of speech from the description text, and obtain a target noun based on the description text and the target image, wherein the target image contains content corresponding to the target noun, and the description text does not contain the target noun; correspondingly, when the first evaluation module 1703 performs a first evaluation of the target image in terms of content based on the target keywords, it is used to:

[0219] If the target image contains content corresponding to the target keyword, and the content corresponding to the target noun is correct, determining that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, determining that the first evaluation result indicates that the content in the target image does not match the target keyword;

[0220] or,

[0221] If there is content corresponding to the target keyword in the target image and the content corresponding to the target keyword is correct, it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

[0222] In an optional embodiment, when the first evaluation module 1703 determines whether there is content corresponding to the keyword of the preset part of speech in the target image, it is used to:

[0223] Input the keywords of the preset part of speech and the target image into a pre-trained first evaluation model to obtain a first evaluation result output by the first evaluation model, wherein the first evaluation result indicates whether there is content corresponding to the keywords of the preset part of speech in the target image; the first evaluation model is obtained by training using keywords of the preset part of speech extracted from a description text sample and an image generated from the description text sample as training samples, and using whether there is content corresponding to the keywords of the preset part of speech extracted from the description text sample in the image generated from the description text sample as a sample label;

[0224] or,

[0225] The keywords of the preset part of speech and the target image are displayed through a human-computer interaction interface; and a second evaluation result input is received through the human-computer interaction interface, wherein the second evaluation result indicates whether there is content corresponding to the keywords of the preset part of speech in the target image.

[0226] In an optional embodiment, when the first evaluation module 1703 determines whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct, it is used to:

[0227] The target keyword and the target image are input into a pre-trained second evaluation model to obtain a third evaluation result output by the second evaluation model, wherein the third evaluation result indicates whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct; the second evaluation model is trained by taking the target keyword obtained based on the description text sample and the image generated by the description text sample as training samples, and taking whether there is content corresponding to the target keyword obtained based on the description text sample in the image generated by the description text sample and whether the content corresponding to the target keyword is correct as sample labels;

[0228] or,

[0229] The target keyword and the target image are displayed through a human-computer interaction interface; a fourth evaluation result is received as input through the human-computer interaction interface, and the fourth evaluation result represents whether there is content corresponding to the target keyword in the target image, and whether the content corresponding to the target keyword is correct.

[0230] In an optional embodiment, before performing a first evaluation of the target image in terms of content based at least on the target keyword, the first evaluation module 1703 is further configured to:

[0231] Extracting ethnic information from the description text;

[0232] If ethnic information is extracted, the first evaluation module 1703 performs a first evaluation of the content of the target image based at least on the target keyword, for:

[0233] Based on the target keyword and the ethnicity information, a first evaluation is performed on the target image in terms of content, and the first evaluation result indicates whether the content in the target image matches the target keyword and the ethnicity information.

[0234] In an optional embodiment, the device further comprises:

[0235] The second evaluation module 1704 is used to perform a visual evaluation on the target image based on the target image to obtain a second evaluation result, where the second evaluation result represents the aesthetics of the target image;

[0236] The determination module 1705 is used to determine the target evaluation result of the target image based on the first evaluation result and the second evaluation result.

[0237] In an optional embodiment, when the second evaluation module 1704 performs a second visual evaluation on the target image based on the target image, it is used to:

[0238] The target image and a plurality of aesthetic evaluation description options are displayed through a human-computer interaction interface, each aesthetic evaluation description option visually describes the target image from a plurality of visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics; the target aesthetic evaluation description option selected by the user is determined through the human-computer interaction interface; and the second evaluation result is determined based on the aesthetics corresponding to the target aesthetic evaluation description option;

[0239] or,

[0240] The target image is input into a pre-trained third evaluation model to obtain a fifth evaluation result output by the third evaluation model, wherein the fifth evaluation result is a target aesthetic evaluation description option among multiple aesthetic evaluation description options, each aesthetic evaluation description option visually describes the target image from multiple visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics; the second evaluation result is determined based on the aesthetics corresponding to the target aesthetic evaluation description option; the third evaluation model is trained using images generated by descriptive text samples as training samples and one of the multiple aesthetic evaluation description options as sample labels.

[0241] In an optional embodiment, when the determination module 1705 determines the target evaluation result of the target image based on the first evaluation result and the second evaluation result, it is configured to:

[0242] If the first evaluation result indicates that the content in the target image matches the target keyword, and the aesthetics of the target image indicated by the second evaluation result satisfies a preset condition, the target evaluation result of the target image is determined to be quality-passed; otherwise, the target evaluation result of the target image is determined to be quality-failed.

[0243] In an optional embodiment, when the second evaluation module 1704 performs a second visual evaluation on the target image based on the target image, it is used to:

[0244] If the first evaluation result indicates that the content in the target image matches the target keyword, performing a second visual evaluation on the target image based on the target image;

[0245] If the first evaluation result indicates that the content in the target image does not match the target keyword, the step of performing a second visual evaluation on the target image based on the target image is not performed.

[0246] The present application also provides an electronic device in an embodiment. Fig.18 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include but is not limited to fixed terminals such as mobile phones, notebook computers, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc., and may also be a server or a server cluster, etc. Fig.18 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0247] like Fig.18 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1802 or a program loaded from a storage device 1808 to a random access memory (RAM) 1803. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1803. The processing device 1801, the ROM 1802, and the RAM 1803 are connected to each other via a bus 1804. An input / output (I / O) interface 1805 is also connected to the bus 1804.

[0248] Typically, the following devices may be connected to the I / O interface 1805: an input device 1806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1808 including, for example, a memory card, a hard disk, etc.; and a communication device 1809. The communication device 1809 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.18 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0249] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any image quality assessment method provided in the embodiment of the present application.

[0250] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any image quality assessment method provided in the embodiment of the present application.

[0251] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0252] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0253] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0254] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for evaluating image quality, characterized in that: include: Obtaining a description text and a target image generated based on the description text; Obtaining target keywords according to the description text, wherein the target keywords at least include keywords extracted from the description text; A first evaluation is performed on the target image in terms of content based at least on the target keyword to obtain a first evaluation result, wherein the first evaluation result indicates whether the content in the target image matches the target keyword.

2. The method according to claim 1, characterized in that The target keyword is a keyword of a preset part of speech extracted from the description text; accordingly, a first evaluation of the content of the target image is performed based on the target keyword, including: If the target image contains content corresponding to the target keyword, determining that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, determining that the first evaluation result indicates that the content in the target image does not match the target keyword; or, If there is content corresponding to the target keyword in the target image and the content corresponding to the target keyword is correct, it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

3. The method according to claim 1, characterized in that The step of obtaining the target keyword according to the description text includes: extracting a keyword of a preset part of speech from the description text, and obtaining a target noun based on the description text and the target image, wherein the target image includes content corresponding to the target noun, and the description text does not include the target noun; accordingly, performing a first evaluation of the content of the target image based on the target keyword includes: If the target image contains content corresponding to the target keyword, and the content corresponding to the target noun is correct, determining that the first evaluation result indicates that the content in the target image matches the target keyword; otherwise, determining that the first evaluation result indicates that the content in the target image does not match the target keyword; or, If there is content corresponding to the target keyword in the target image and the content corresponding to the target keyword is correct, it is determined that the first evaluation result represents that the content in the target image matches the target keyword; otherwise, it is determined that the first evaluation result represents that the content in the target image does not match the target keyword.

4. The method according to claim 3, characterized in that The process of determining whether there is content corresponding to the keyword of the preset part of speech in the target image includes: Input the keywords of the preset part of speech and the target image into a pre-trained first evaluation model to obtain a first evaluation result output by the first evaluation model, wherein the first evaluation result indicates whether there is content corresponding to the keywords of the preset part of speech in the target image; the first evaluation model is obtained by training using keywords of the preset part of speech extracted from a description text sample and an image generated from the description text sample as training samples, and using whether there is content corresponding to the keywords of the preset part of speech extracted from the description text sample in the image generated from the description text sample as a sample label; or, The keywords of the preset part of speech and the target image are displayed through a human-computer interaction interface; and a second evaluation result input is received through the human-computer interaction interface, wherein the second evaluation result indicates whether there is content corresponding to the keywords of the preset part of speech in the target image.

5. The method according to claim 3, characterized in that: The process of determining whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct includes: The target keyword and the target image are input into a pre-trained second evaluation model to obtain a third evaluation result output by the second evaluation model, wherein the third evaluation result indicates whether there is content corresponding to the target keyword in the target image and whether the content corresponding to the target keyword is correct; the second evaluation model is trained by taking the target keyword obtained based on the description text sample and the image generated by the description text sample as training samples, and taking whether there is content corresponding to the target keyword obtained based on the description text sample in the image generated by the description text sample and whether the content corresponding to the target keyword is correct as sample labels; or, The target keyword and the target image are displayed through a human-computer interaction interface; a fourth evaluation result is received as input through the human-computer interaction interface, and the fourth evaluation result represents whether there is content corresponding to the target keyword in the target image, and whether the content corresponding to the target keyword is correct.

6. The method according to claim 1, characterized in that Before performing a first evaluation of the target image in terms of content based at least on the target keyword, the method further includes: Extracting ethnic information from the description text; If ethnic information is extracted, the first evaluation of the target image in terms of content based at least on the target keyword includes: Based on the target keyword and the ethnicity information, a first evaluation is performed on the target image in terms of content, and the first evaluation result indicates whether the content in the target image matches the target keyword and the ethnicity information.

7. The method according to claim 1, characterized in that Also includes: Based on the target image, performing a second visual evaluation on the target image to obtain a second evaluation result, wherein the second evaluation result represents the aesthetics of the target image; Different aesthetics correspond to different visual descriptions of the target image from multiple visual dimensions; A target evaluation result of the target image is determined based on the first evaluation result and the second evaluation result.

8. The method according to claim 7, characterized in that The step of performing a second visual evaluation on the target image based on the target image comprises: The target image and a plurality of aesthetic evaluation description options are displayed through a human-computer interaction interface, each aesthetic evaluation description option visually describes the target image from a plurality of visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics; the target aesthetic evaluation description option selected by the user is determined through the human-computer interaction interface; and the second evaluation result is determined based on the aesthetics corresponding to the target aesthetic evaluation description option; or, The target image is input into a pre-trained third evaluation model to obtain a fifth evaluation result output by the third evaluation model, wherein the fifth evaluation result is a target aesthetic evaluation description option among multiple aesthetic evaluation description options, each aesthetic evaluation description option visually describes the target image from multiple visual dimensions, and different aesthetic evaluation description options correspond to different degrees of aesthetics; the second evaluation result is determined based on the aesthetics corresponding to the target aesthetic evaluation description option; the third evaluation model is trained using images generated by descriptive text samples as training samples and one of the multiple aesthetic evaluation description options as sample labels.

9. The method according to claim 7, characterized in that: The determining the target evaluation result of the target image based on the first evaluation result and the second evaluation result includes: If the first evaluation result indicates that the content in the target image matches the target keyword, and the aesthetics of the target image indicated by the second evaluation result satisfies a preset condition, the target evaluation result of the target image is determined to be quality-passed; otherwise, the target evaluation result of the target image is determined to be quality-failed.

10. The method according to claim 9, characterized in that The step of performing a second visual evaluation on the target image based on the target image comprises: If the first evaluation result indicates that the content in the target image matches the target keyword, performing a second visual evaluation on the target image based on the target image; If the first evaluation result indicates that the content in the target image does not match the target keyword, the step of performing a second visual evaluation on the target image based on the target image is not performed.

11. An image quality evaluation device, characterized in that: include: A first acquisition module, used to obtain a description text and a target image generated based on the description text; A second obtaining module, configured to obtain target keywords according to the description text, wherein the target keywords at least include keywords extracted from the description text; The first evaluation module is used to perform a first evaluation on the content of the target image based at least on the target keyword to obtain a first evaluation result, wherein the first evaluation result indicates whether the content in the target image matches the target keyword.

12. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the image quality assessment method as described in any one of claims 1 to 10.

13. A computer program product, characterized in that The method comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the image quality assessment method as claimed in any one of claims 1 to 10.

14. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the image quality assessment method as described in any one of claims 1 to 10.

Images