A method, system, electronic device, storage medium and program for evaluating image generation

Abstract

The present invention discloses a method, system, electronic device, storage medium and program for evaluating a generated image, wherein the method comprises: obtaining an original image, a generated image and style specification information as evaluation data, and transmitting the evaluation data to a model detection module and a visual question-answering module respectively, wherein the model detection module and the visual question-answering module have at least one identical detection point; for the identical detection point, determining a first type of evaluation index of the generated image through the model detection module based on the evaluation data, and generating a second type of evaluation index of the generated image through the visual question-answering module; determining a picture evaluation result of the generated image based on the first type of evaluation index and the second type of evaluation index according to a preset evaluation rule. The embodiment of the present invention can realize the automatic evaluation of the generated image, solve the problem of poor accuracy of the evaluation result, improve the objectivity of the image generation evaluation process, improve the accuracy of the image generation evaluation result, and improve the image quality of the generated image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, electronic device, storage medium and program for evaluating image quality. Background Art

[0002] With the development of artificial intelligence technology, multimodal large models have gradually become a research hotspot in the current field of artificial intelligence. Multimodal large models can process a variety of media data, including but not limited to text, images, audio and video, and realize intelligent information processing by learning the connection between different modalities. As an application of multimodal large models, image-generated image technology will be widely used in smart cars, smart homes and other fields. However, the quality of image-generated images is often unsatisfactory. There are often defects in the picture quality and picture content, which leads to poor user experience of the image-generated image function. To address this problem, it is necessary to introduce the evaluation of image generation results to achieve feedback on multimodal large models, thereby improving the processing effect of image-generated images. However, the existing evaluation methods for image generation results mainly rely on manual observation, which is often subject to the subjective impression of the evaluators and lacks the objectivity and consistency of the image generation result evaluation. In addition, the entire evaluation process cannot be automated, resulting in low feedback efficiency of the evaluation results. Summary of the invention

[0003] The present invention provides a method, system, electronic device, storage medium and program for evaluating image generation results, so as to realize automatic evaluation of image generation results, solve the problem of poor accuracy of image generation evaluation results caused by inconsistent manual evaluation standards, improve the objectivity of image generation evaluation process based on the automatic evaluation process, improve the accuracy of image generation evaluation results, thereby assisting in improving the image generation processing effect, and improving the image quality of generated images.

[0004] According to one aspect of the present invention, a method for evaluating a graph is provided, wherein the method comprises:

[0005] Acquire the original image, the generated image, and the style specification information as evaluation data, and transmit the evaluation data to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point;

[0006] For the same detection point, determining a first type of evaluation index of the generated graph through the model detection module based on the evaluation data, and generating a second type of evaluation index of the generated graph through the visual question answering module;

[0007] According to preset evaluation rules, an image evaluation result of the generated image is determined based on the first type of evaluation indicators and the second type of evaluation indicators.

[0008] According to another aspect of the present invention, a system for evaluating graph generation is provided, wherein the system comprises:

[0009] A data input module, used for the data input module, for obtaining the original image, the generated image and the style specification information as evaluation data, and transmitting the evaluation data input to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point;

[0010] An indicator detection module, used for determining, for the same detection point, a first type of evaluation indicator of the generated graph through the model detection module based on the evaluation data, and generating a second type of evaluation indicator of the generated graph for the detection point through the visual question answering module;

[0011] An evaluation result module is used to determine the picture evaluation result based on the first type of evaluation indicators and the second type of evaluation indicators according to preset evaluation rules.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the graphic evaluation method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for evaluating the image generation graph described in any embodiment of the present invention when executed.

[0017] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the method for evaluating the image raw graph described in any embodiment of the present invention is implemented.

[0018] The technical solution of the embodiment of the present invention takes the original image, the generated image and the style specification information as evaluation data, and inputs the evaluation data into a model detection module and a visual question-answering module having at least one same detection point, generates a first type of evaluation indicator and a second type of evaluation indicator for the above-mentioned generated image for each same detection point in the model detection module and the visual question-answering module, and generates an image evaluation result of the corresponding generated image based on the first type of evaluation indicator and the second type of evaluation indicator according to preset evaluation rules, thereby realizing automatic evaluation of the generated image, unifying the evaluation criteria of the generated image through preset evaluation rules, solving the problem of strong subjectivity of manual evaluation criteria, improving the accuracy of image quality assessment, and facilitating feedback on the process of processing the generated image, thereby assisting in improving the image quality of the generated image.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a method for evaluating a graph according to the first embodiment of the present invention;

[0022] Figure 2 is a flowchart of another graph generation and graph evaluation method provided according to the second embodiment of the present invention;

[0023] Figure 3 is a flowchart of another graph generation and graph evaluation method provided according to Embodiment 3 of the present invention;

[0024] Figure 4 is a flowchart of another graph generation and graph evaluation method provided according to Embodiment 4 of the present invention;

[0025] Figure 5 This is an example diagram of a method for generating and evaluating a graph according to Embodiment 5 of the present invention;

[0026] Figure 6 This is an example diagram of a graph generation and graph evaluation architecture provided according to Embodiment 5 of the present invention;

[0027] Figure 7 is a structural schematic diagram of a graph generation and evaluation system provided according to Embodiment 6 of the present invention;

[0028] Figure 8 It is a structural schematic diagram of an electronic device for implementing the image generation and image evaluation method of an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 This is a flow chart of a method for evaluating a raw image according to the first embodiment of the present invention. This embodiment is applicable to the case of automatically evaluating images generated by image generation technology. The method can be executed by a raw image evaluation system. The raw image evaluation system can be implemented in the form of hardware and / or software. The raw image evaluation system is generally configured in a terminal, a server or a server cluster. Figure 1 As shown, the method includes:

[0033] Step 110: obtain the original image, the generated image and the style specification information as evaluation data, and transmit the evaluation data to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point.

[0034] The original image may be an original image used for image generation, and the original image may be acquired in real time through a camera, or actively uploaded by a user. Exemplarily, a user may start a camera on the driver's seat of a vehicle to acquire an image inside the vehicle as an original image, and the style specification information may be used to describe the art style information of the image to be generated, such as Japanese anime style, ink painting style, optimization style, etc. The style specification information may include text or voice, etc. Further, in some embodiments, the style specification information may include indication information of a pre-configured style type, and the style specification information may identify the pre-configured style type through characters such as numbers, letters, or special symbols, that is, different style guidance information may correspond to different pre-configured style types.

[0035] In an embodiment of the present invention, the model detection module and the visual question-answering module may have at least one indicator extraction module for the same detection point. The same detection point may mean that the same or related indicators can be detected in the model detection module and the visual question-answering module. The model detection module may be a module for processing and identifying the detection point based on a pre-trained image processing model. The model detection module may include at least a model for processing at least one of the original image, the generated image, and the style specification information. The visual question-answering module is a model that can perform quality detection on the generated image in the form of questions and answers. It can be understood that the quality may include aspects such as the image display quality, the degree of similarity with the original image content, the degree of similarity with the original image style, and the aesthetics. The visual question-answering model may be implemented based on a multimodal model. The multimodal model may receive the generated image, the original image, and the style specification information, and analyze the generated image generation quality in the form of questions and answers. For example, the user may input the question "Are the contents of the generated image consistent with those of the original image?" to the visual model. The multimodal model may generate answers such as "consistent", "inconsistent" or "the portrait of the generated image is inconsistent", "the generated image loses a certain content", etc. based on the received generated image, the original image, and the style specification information.

[0036] Specifically, the original image, the generated image and the style specification information can be collected. The original image, the generated image and the style specification information can be collected during the image generation process or obtained by user input. It can be understood that each group of original images, generated images and style specification information can be used as a group of evaluation data, and the obtained original image, generated and style specification information can be multiple groups, that is, multiple groups of evaluation data can be obtained. The one or more groups of evaluation data obtained above can be input into the model detection module and the visual question and answer module respectively. The model detection module may include multiple detection points for detecting the evaluation data, and the visual question and answer module may also include the detection points for detecting the above evaluation data. There is at least one detection point in the model detection module and the visual question and answer module as a corresponding relationship, and the corresponding relationship may include the detection points being the same or associated. It can be understood that the detection points can be detected in different forms in the model detection module and the visual question and answer module. For example, in the model detection module, different deep learning models can be used to detect the detection points, and in the visual question and answer module, the detection points can be detected in the form of preset questions.

[0037] Step 120: For the same detection point, determine the first type of evaluation indicators of the generated graph through the model detection module based on the evaluation data, and generate the second type of evaluation indicators of the generated graph through the visual question answering module.

[0038] In an embodiment of the present invention, the model detection module can analyze the evaluation data for each set detection point to generate indicators for evaluating different detection points of the corresponding generated graph. These indicators can be used as the first type of evaluation indicators, that is, the model detection module can process the evaluation data for each detection point to generate indicators for evaluating the generated graph as the first type of evaluation indicators, and the second type of evaluation indicators can be indicators generated by evaluating the generated graph for the detection point in the visual question and answer module. The indicators generated by the visual question and answer module can be used as the second type of evaluation indicators. It can be understood that since the model detection module and the visual question and answer module have at least one corresponding detection point, there is a corresponding relationship between at least one of the generated first type of evaluation indicators and the second type of evaluation indicators. The corresponding relationship can be determined by the detection point of the generated indicator, that is, there are two detection points with corresponding relationships in the model detection module and the visual question and answer module, and the indicators generated for these two detections can also have a corresponding relationship.

[0039] Step 130: According to the preset evaluation rules, based on the first type of evaluation indicators and the second type of evaluation indicators, determine the image evaluation result of the generated image.

[0040] The preset evaluation rules may be preset rules for statistically analyzing the first type of evaluation indicators and the second type of evaluation indicators. The statistical results corresponding to the preset evaluation rules may be used to measure the generation quality of the generated image. The preset evaluation rules may evaluate the generation quality of the generated image in different dimensions, and the dimensions may include but are not limited to evaluating the similarity of the generated image with the original image, the correctness of the style of the generated image with the original image, and the aesthetics of the generated image. The picture evaluation result may include the indicator statistical results generated by statistically analyzing the first type of evaluation indicators and the second type of evaluation indicators based on the preset evaluation rules. The indicator statistical results may reflect the quality effect of the generated image. The picture evaluation result may also include the generated image evaluation information corresponding to the above-mentioned indicator statistical results. The generated image evaluation information may include the evaluation of the quality of the generated image. For example, the generated image evaluation information may include whether the generated image effect is good or not.

[0041] In an embodiment of the present invention, preset evaluation rules can be called, and statistical analysis can be performed on the first type of evaluation indicators and the second type of evaluation indicators according to the preset evaluation rules to determine the indicator statistical results, and generate corresponding generation graph evaluation information according to the indicator statistical results. The above-mentioned indicator statistical results and the generation graph evaluation information can be combined as the generation graph evaluation results. It can be understood that in some embodiments, the generation graph evaluation results can also be visualized to assist users in obtaining the advantages or disadvantages of the generation graph.

[0042] In an embodiment of the present invention, an original image, a generated image and style specification information are obtained as evaluation data, and the above evaluation data is transmitted to a model detection module and a visual question-answering module, wherein the model detection module and the visual question-answering module include at least one identical detection point, and the model detection module and the visual question-answering module are respectively called to process the generated image, and a first type of evaluation indicator and a second type of evaluation indicator corresponding to the above detection point are generated, and the first type of evaluation indicator and the second type of evaluation indicator are analyzed according to preset evaluation rules to determine the image evaluation result of the generated image, thereby realizing automatic evaluation of the generated image, and evaluating the generated image based on standardized preset evaluation rules can reduce the influence of existing manual subjective evaluation on the evaluation result of the generated image, and can improve the accuracy of image quality assessment, and facilitate feedback on the processing process of the generated image, thereby assisting in improving the image quality of the generated image.

[0043] Embodiment 2

[0044] Figure 2 is a flowchart of another graph generation graph evaluation method provided according to the second embodiment of the present invention. The embodiment of the present invention is a specific embodiment of the above invention embodiment, and describes the generation process of the evaluation result of the generated graph. Figure 2 The method provided in the embodiment of the present invention specifically comprises the following steps:

[0045] Step 210: obtain the original image, the generated image and the style specification information as evaluation data, and transmit the evaluation data to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point.

[0046] Step 220: For the same detection point, determine the first type of evaluation indicators of the generated graph through the model detection module based on the evaluation data, and generate the second type of evaluation indicators of the generated graph through the visual question answering module.

[0047] Step 230: Map the first type of evaluation indicators and the second type of evaluation indicators to at least one preset evaluation dimension according to corresponding indicator types.

[0048] Among them, the preset evaluation dimensions can generate evaluation dimensions of different attributes of the image. The preset evaluation dimensions may include dimensions of similarity to the original image, dimensions of style correctness, and dimensions of aesthetics, etc. It can be understood that the above-mentioned preset evaluation dimensions are only for example and not for limitation. Other attribute dimensions of the generated image are also within the scope of the preset evaluation dimensions. For example, dimensions such as clarity, brightness, storage size, etc. of the generated image may also fall within the scope of the preset evaluation dimensions. The indicator type can reflect that the first type of evaluation indicator or the second type of evaluation indicator belongs to the evaluation indicator of the evaluation attribute dimension of the generated graph. The indicator type can have a mapping relationship with the evaluation attribute dimension of the generated graph. The mapping relationship can be pre-configured based on experience or by an artificial intelligence algorithm. The indicator type can be determined based on the detection point corresponding to the first evaluation indicator or the second type of evaluation indicator. The indicator type can be divided into race indicator, expression indicator, hair condition indicator, face shape indicator, mouth shape indicator, beard condition indicator, glasses condition indicator, number of people indicator, gender indicator, appearance indicator, head posture indicator, limb posture indicator, redundant humanoid object indicator, human body deformity indicator, overall style indicator, local style indicator, aesthetic score indicator, picture white edge condition indicator, picture signature condition indicator and picture watermark condition indicator according to the corresponding detection point.

[0049] In an embodiment of the present invention, for each evaluation indicator of the first category of evaluation indicators and the second category of evaluation indicators obtained, the indicator type can be determined according to the detection point corresponding to the evaluation indicator, and according to the mapping relationship between the indicator type and the preset evaluation dimension, each evaluation indicator of the first category of evaluation indicators and the second category of evaluation indicators is respectively mapped to a different preset evaluation dimension, so that the first category of evaluation indicators and the second category of evaluation indicators can be statistically analyzed with respect to different preset evaluation dimensions to improve the refinement of the evaluation of the generated image and further improve the evaluation quality of the generated image.

[0050] Step 240: According to the preset evaluation rules, the image evaluation information of the mapped first-category evaluation indicators and the second-category evaluation indicators are respectively determined under different preset evaluation dimensions.

[0051] Among them, the image evaluation information can be information for evaluating the generated image under different preset evaluation dimensions. The image evaluation information can include the statistical results of the first type of evaluation indicators and the second type of evaluation indicators under the preset evaluation dimensions and the analysis and description information of the statistical results. The image evaluation information can also include detailed indicator types and indicator values.

[0052] In an embodiment of the present invention, the first category of evaluation indicators and the second category of evaluation indicators can be mapped to different preset evaluation dimensions according to their respective indicator types. For different preset evaluation dimensions, the first category of evaluation indicators and the second category of evaluation indicators mapped under the corresponding preset evaluation dimensions can be statistically analyzed according to preset evaluation rules to generate image evaluation information. It can be understood that each image evaluation information can correspond to a different preset evaluation dimension, that is, each preset evaluation dimension can generate one or a group of image evaluation information according to the preset evaluation rules. It can be understood that the preset evaluation rules can be an evaluation rule set, which can include multiple evaluation rules, and each evaluation rule can perform statistical analysis on the mapped first category of evaluation indicators and the second category of evaluation indicators under different preset evaluation dimensions.

[0053] Step 250: Merge the evaluation information of each picture as a picture evaluation result.

[0054] In an embodiment of the present invention, all acquired image evaluation information may be aggregated and merged to generate an image evaluation result, and the merging process may include listing the image evaluation information according to different preset evaluation dimensions, or aggregating different image evaluation information into one image evaluation result. Exemplarily, different image evaluation information may be input into a multimodal large model, and a summary description of all image evaluation information may be generated by the multimodal large model as an image evaluation result, or the image evaluation information may be filled into corresponding positions of a preset report according to its corresponding preset evaluation dimensions, and the preset report filled with image evaluation information of different preset evaluation dimensions may be used as an image evaluation result.

[0055] In an embodiment of the present invention, an original image, a generated image and style specification information are used as evaluation data, and the evaluation data are respectively input into a model detection module and a visual question-answering module. The model detection module and the visual question-answering module specifically have at least one corresponding detection point. For the detection point, the evaluation data is processed into a first type of evaluation indicator in the model detection module and the evaluation data is processed into a second type of evaluation indicator in the visual question-answering module. The first type of evaluation indicator and the second type of evaluation indicator are respectively mapped to at least one preset evaluation dimension according to their respective corresponding indicator types. According to preset evaluation rules, picture evaluation information corresponding to the mapped first type of evaluation indicator and the second type of evaluation indicator is determined under different preset evaluation dimensions. The picture evaluation information is merged into a picture evaluation result, thereby realizing automatic evaluation of generated pictures, which can improve the objectivity of the evaluation rules of the generated pictures. By evaluating the generated pictures through different evaluation dimensions, the refinement of the evaluation of the picture generation effect can be improved, and the evaluation accuracy of the generated pictures can be further improved. Feedback update of the picture generation picture processing process can be assisted, thereby helping to improve the image quality of the generated pictures.

[0056] Further, based on the above-mentioned embodiment of the invention, the preset evaluation dimension includes at least one of the following: a picture content similarity dimension, a picture style correctness dimension, and a picture aesthetics dimension, and the mapping relationship between the indicator type and the preset evaluation dimension includes at least one of the following:

[0057] The image content similarity dimension corresponds to at least one of the following indicators: race indicator, expression indicator, hair condition indicator, face shape indicator, mouth shape indicator, beard condition indicator, glasses condition indicator, number of people indicator, gender indicator, appearance indicator, head posture indicator, limb posture indicator, extra humanoid object indicator, and human deformity indicator in the indicator type;

[0058] The image style correctness dimension corresponds to at least one of the overall style index and the local style index in the index type;

[0059] The image aesthetics dimension corresponds to at least one of the aesthetics score index, the image white edge condition index, the image signature condition index, and the image watermark condition index in the indicator type.

[0060] In an embodiment of the present invention, the picture content similarity dimension may be a dimension for evaluating the similarity between the generated image and the original image from the perspective of the picture content. The picture content similarity dimension may be obtained by comparing the content of the generated image with the content of the original image. The picture content similarity dimension may include, but is not limited to, evaluating whether the poses of the characters in the generated image and the original image are the same, whether the objects included in the generated image and the original image are the same, whether the background colors of the generated image and the original image are the same, etc. The picture style correctness dimension may be an evaluation dimension for evaluating whether the style of the generated image is consistent with the style of the original image. The picture style correctness dimension may be evaluated by evaluating whether the local style or the overall style of the generated image is consistent with the original title. The picture aesthetics dimension may be a dimension for evaluating the aesthetic effect of the generated image. The picture aesthetics dimension may be evaluated by evaluating the aesthetics of the generated image, whether the image has a white edge, or whether the image has a watermark, etc. Each preset evaluation dimension may have a mapping relationship with different indicator types, that is, each preset evaluation dimension may have one or more indicator types mapped, and the indicators of this indicator type may be used to evaluate the generated image under the mapped preset evaluation dimension.

[0061] In an exemplary implementation, the mapping relationship between the picture content similarity dimension, the picture style correctness dimension, and the picture aesthetics dimension in the preset evaluation dimensions and their corresponding indicator types can be shown in the following table:

[0062] Table 1 Example of mapping relationship between preset evaluation dimensions and indicator types

[0063]

[0064]

[0065] Embodiment 3

[0066] Figure 3 is a flowchart of another graph generation graph evaluation method provided according to the third embodiment of the present invention. The embodiment of the present invention is a specific embodiment of the above invention embodiment, and describes the generation process of the first type of evaluation indicators. Figure 3 The method provided in the embodiment of the present invention specifically comprises the following steps:

[0067] Step 310: obtain the original image, the generated image and the style specification information as evaluation data, and transmit the evaluation data to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point, and the model detection module includes at least an image detection unit and a label detection unit, and the image detection unit includes image detection models corresponding to each detection point respectively.

[0068] In an embodiment of the present invention, the model detection module may include at least an image detection unit and a label detection unit, wherein the image detection unit includes at least one or more image detection models corresponding to the detection points, and at least one of the detection points is the same as or associated with the detection point of the visual question answering model. The image detection model may be a deep learning model for processing images, and the image detection model may extract indicator parameters of different indicator types in the original image and / or the generated image for the corresponding detection points, and the label detection unit may generate indicators of the corresponding indicator types by comparing and analyzing the indicator parameters generated by the image detection unit processing, and the image detection model may be a picture analysis model generated by training a picture training set for different detection points, and the image detection model includes but is not limited to a graph convolutional network model, a graph residual network model, and the like.

[0069] Specifically, the acquired original image, generated image and style specification information can be used as evaluation data, and the evaluation data can be input into the model detection module and the visual question-answering module respectively. It can be understood that since the model detection module includes at least one image detection unit, the evaluation data input into the model detection module can be copied, and each copy of the evaluation data can be input into an image detection unit respectively, so that the image detection unit can generate an indicator parameter corresponding to at least one detection point based on the evaluation data. The indicator parameter can be parameter information generated by the image detection unit through identification and analysis of the generated image or the original image in the evaluation data, so that the label detection unit can conduct comparative analysis through the above indicator parameters.

[0070] Step 320: Generate image analysis results of the detection points based on the evaluation data through each image detection model, wherein the image analysis results include at least one of a first image analysis result generated based on the original image, a second image analysis result generated based on the generated image, and a third image analysis result generated based on the generated image and style specified information.

[0071] Among them, the image detection model can be a pre-trained picture processing model, and the image detection model can identify one or more detection points in the picture. For example, the image detection model can include a trained face recognition model, through which multiple detection points such as race, skin color, beard, number of people, etc. can be processed and analyzed, thereby generating indicator parameters of corresponding indicator types. For another example, the image detection model can include a generated human posture recognition model, and the human posture recognition model can only analyze and process one detection point of human posture, and can only generate indicator parameters of the indicator type of human posture indicator. The input of the image detection model can include a generated image, an original image, a generated image, and style specified information, etc., respectively. The result generated by the image detection model processing the generated image can be used as the first image analysis result, the result generated by the image detection model processing the original image can be used as the second image analysis result, and the result generated by the image detection model processing the generated image and the style specified information can be used as the third image analysis result. It can be understood that according to the different types of image detection models configured in the module detection module, the image analysis result generated by the image detection model based on the evaluation data can include at least one of the above-mentioned first image analysis result, second image analysis result, and third image analysis result.

[0072] In an embodiment of the present invention, the model detection module may include multiple image detection models, each image detection model may be targeted at different detection points, an image detection model may receive input evaluation data, the image detection model processes a set of evaluation data for its corresponding detection point, the image detection model may use the result generated by processing the original image in the evaluation data as the first image analysis result, may use the result generated by the image detection model according to the generated graph in the evaluation data as the second image analysis result, and the image detection model may also generate a third image analysis result according to the generated graph and style specification information in the evaluation data. It is understandable that the first image analysis result, the second image analysis result, and the third image analysis result of a set of evaluation data for one or more detection points may be generated by one image detection model. In another exemplary embodiment, the model detection module includes multiple image detection models, and each image detection model can be configured to detect one or more detection points. One image detection model can generate a first image analysis result for a detection point A according to the original image in the evaluation data, while another image detection model can process the generated image in the evaluation data into a second image analysis result for the above-mentioned detection point A. There can also be an image detection model that processes the generated image and style specification information in the evaluation data into a third image analysis result for the above-mentioned detection point A. It can be understood that a set of first image analysis results, second image analysis results and third image analysis results for one or more detection points of the evaluation data can be obtained by processing multiple image detection models respectively.

[0073] Step 330: Process the image analysis result into a first type of evaluation indicator through a label detection unit.

[0074] Specifically, the label detection unit can be a unit that processes the image analysis results. The label detection unit can compare and analyze the first image analysis results, the second image analysis results and the third image analysis results corresponding to different detection points in the image analysis results, and perform statistics, so as to use the statistical analysis of the image analysis results as the first type of evaluation indicators. The label detection unit can include, for different detection points, comparing the corresponding first image analysis results with the second image analysis results and summarizing the third image analysis results.

[0075] Step 340: Obtain question and answer information including each detection point configured by the visual question and answer module.

[0076] The question and answer information may be question information used to analyze the evaluation data in the visual question and answer module. The question and answer information may be set according to the detection points. Each question in the question and answer information may correspond to one or more detection points. For example, the question and answer information may include whether the races in the generated graph are consistent. The question and answer information may be a question about the detection point of the race of the generated graph. The question and answer information may be input by the user in real time or may be pre-set as a configuration file.

[0077] In an embodiment of the present invention, the visual question and answer module can be configured with question and answer information, which can be input by the user in real time or obtained by reading a configuration file. The question and answer information configured for the visual question and answer module can be extracted. The question and answer information can include at least one question, and each question can be used to detect at least one detection point in the visual question and answer module.

[0078] Step 350: input the question and answer information, the original image, the generated image, and the style specification information into the visual question and answer module, so that the visual question and answer module generates an answer result corresponding to the question and answer information based on the semantics of the original image, the generated image, and the style specification information, and uses the answer result as the second type of evaluation indicator.

[0079] The semantics may be the meaning contained in the style-specified information, or the meaning of the concept represented by the object in the real world corresponding to the characters corresponding to the style-specified information.

[0080] In an embodiment of the present invention, question-and-answer information and evaluation data can be input into a visual question-and-answer module, and the visual question-and-answer module can analyze and identify the semantics of the original image, generated image, and style-specified information in the evaluation data. The visual question-and-answer module can generate corresponding question-and-answer results according to the analysis results of the semantics of the original image, generated image, and style-specified information. The question-and-answer result can be an answer corresponding to the question-and-answer information, and the answer can be determined by the visual question-and-answer module based on the semantics of the original image, generated image, and style guidance information. Exemplarily, the question-and-answer information is whether the race of the generated image is consistent with that of the original image. The visual question-and-answer model can generate an answer to the question-and-answer information by identifying the features of the generated image and the original image: consistent or inconsistent, and the above answer can be used as the answer result corresponding to the question-and-answer information. The answer results generated by the visual question-and-answer module for the question-and-answer information can be used as the second type of evaluation indicators.

[0081] Step 360: According to the preset evaluation rules, based on the first type of evaluation indicators and the second type of evaluation indicators, determine the image evaluation result of the generated image.

[0082] In an embodiment of the present invention, the original image, the generated image and the style specified information are used as evaluation data, and the evaluation data is input into a model detection module and a visual question-answering module. Each image detection model in the model detection module generates an image analysis result of a corresponding detection point based on the evaluation data. The label detection unit of the model detection module processes the image analysis result as a first type of evaluation indicator, obtains question-answering information of the visual question-answering module, and the question-answering information is configured for different detection points. The evaluation data and the question-answering information are processed in the visual question-answering module to obtain an answer result generated by the visual question-answering module for the question-answering information, and the answer result is used as a second type of evaluation indicator. The image evaluation results corresponding to the first and second evaluation indicators are generated according to the preset evaluation rules, thereby realizing the automatic evaluation of the generated images. The generated images are recognized by the image detection model and semantic recognition is added to the generated images based on the visual question answering model, which can improve the diversity of the detection dimensions of the generated images and the accuracy of the evaluation of the image generation effect. The generated results are evaluated by standardized preset evaluation rules to solve the problem of excessive subjectivity in the existing evaluation process, improve the evaluation effect of the generated images, and facilitate feedback on the processing process of the generated images, thereby assisting in improving the image quality of the generated images.

[0083] Further, based on the above-mentioned embodiment of the invention, the image detection model includes at least one of a face analysis model, a body analysis model, a picture style recognition model and a picture aesthetics scoring model, and the image analysis results of the detection points generated by each of the image detection models based on the evaluation data include at least one of the following:

[0084] The race, expression, hair condition, face shape, mouth shape, beard condition, glasses condition, number of people, gender and appearance of the original image and the generated image are respectively extracted from the face analysis model as the first image analysis result and the second image analysis result;

[0085] The head posture, limb posture, redundant human-shaped objects and human body deformity in the original image and the generated image are respectively extracted as the first image analysis result and the second image analysis result through the human body analysis model;

[0086] Generate the overall style and the local style corresponding to the original image and the generated image respectively as the first image analysis result and the second image analysis result through the image style recognition model;

[0087] The overall style and local style generated by the image style recognition model based on the generated graph and the style specified information are used as the third image analysis result;

[0088] The image aesthetics scoring model is used to extract and generate the aesthetics score of the corresponding detection point in the image, the image white edge condition, the image signature condition and the image watermark condition as the second image analysis result.

[0089] In an embodiment of the present invention, the image detection model in the model detection module may include at least a face analysis model, a human body analysis model, a picture style recognition model and a picture aesthetic scoring model. The face analysis model may be trained to analyze and recognize faces in pictures. The face analysis model may include but is not limited to DeepFace, FaceNet, VGGFace, SphereFace, ArcFace, etc. The detection points corresponding to the face analysis model may include race, expression, hair condition, face shape, mouth shape, beard condition, glasses condition, number of people, gender and appearance, etc. The human body analysis model may be trained to be a model for analyzing and identifying human bodies in pictures. The human body analysis model especially analyzes and identifies the head and torso of human bodies in car pictures. The human body analysis model may include but is not limited to YOLO, OpenCV, SMPLify, etc. The detection points corresponding to the human body analysis model may include head posture, limb posture, redundant human-shaped objects and human body deformities, etc. A picture style recognition model can be trained to recognize picture style and a picture aesthetic scoring model can be trained to recognize the aesthetic score of a generated picture. The picture style recognition model can include but is not limited to a graph convolutional network, a graph residual network, etc. The detection points of the picture style recognition model can include overall style, local style, etc.; and the picture aesthetic scoring model can adopt an expert system, a convolutional network, etc. The detection points of the picture aesthetic scoring model at least include the aesthetics of the picture.

[0090] Specifically, the original image and the generated image in the evaluation data can be processed in the face analysis model, and the race, expression, hair condition, face shape, mouth shape, beard condition, glasses condition, number of people, gender and appearance of the original image can be extracted as the first image analysis result, and the race, expression, hair condition, face shape, mouth shape, beard condition, glasses condition, number of people, gender and appearance of the generated image can be extracted as the first image analysis result as the second image analysis result; in the human body analysis model, the head posture, limb posture, superfluous human-shaped objects and human body deformities can be extracted from the original image as the first image analysis result, and the head posture, limb posture, superfluous human-shaped objects and human body deformities generated by processing the generated image in the human body analysis model can be used as the second image analysis result; the picture style recognition model can extract the overall style and local style according to the generated image and the style specification information, and the extracted overall style and local style can be used as the third image analysis result; the picture aesthetics scoring model analyzes the generated image to determine the aesthetics score, the white edge of the image, the signature of the image and the watermark of the image as the second image analysis result.

[0091] Further, based on the above-mentioned embodiment of the invention, the image analysis result is processed into a first type of evaluation index by the label detection unit, including at least one of the following:

[0092] Generate a first comparison result of the first image analysis result and the second image analysis result in the label detection unit, and use the first comparison result as a first type of evaluation indicator;

[0093] Generating, in the label detection unit, a second comparison result between the second image analysis result and the first preset threshold, and using the second comparison result as a first type of evaluation indicator;

[0094] A third comparison result between the third image analysis result and the second preset threshold is generated in the label detection unit, and the third comparison result is used as the first type of evaluation indicator.

[0095] In an embodiment of the present invention, the label detection unit may be configured with one or more processing rules for performing statistical analysis on the image analysis results. The processing rules may include comparing the first image analysis result with the second image analysis result to determine whether the first image analysis result is the same as the second image analysis result. The comparison result of the first image analysis result with the second image analysis result may be recorded as a first comparison result, and the first comparison result may be used as a first type of evaluation indicator generated by the label detection unit; the processing rules configured by the label detection unit may also include comparing the second image analysis result with a first preset threshold, and the comparison result may be used as a first type of evaluation indicator. The first preset threshold may be a pre-configured critical value, which is used to determine whether the generated image meets the preset generation effect; the label detection unit may also be configured with a second preset threshold for comparison with the third image analysis result. The label detection unit may compare the third image analysis result with the second preset threshold through the configured processing rules, and use the determined comparison result as the first type of evaluation indicator.

[0096] Based on the above-mentioned embodiment of the invention, the question and answer information includes at least one of the following:

[0097] Whether the ethnic categories of the two pictures are consistent; whether the task expressions of the two pictures are consistent; whether the hair lengths of the characters in the two pictures are consistent; whether the mouth shapes of the characters in the two pictures are consistent; whether the beard status of the characters in the two pictures is consistent; whether the glasses wearing status of the characters in the two pictures is consistent; whether the number of people in the two pictures is the same; whether the genders of the characters in the two pictures are consistent; whether the appearance of the generated picture is higher than the specified score; whether the head postures of the characters in the two pictures are consistent; whether the body movements of the characters in the two pictures are consistent; whether there are extra human-shaped objects in the generated picture; whether there are human body deformities in the generated picture; whether the style of the generated picture is consistent with the specified generation style; whether the local style of the generated picture is consistent with the specified generation style; whether the aesthetics of the generated picture meets the standards; whether the generated picture has white edges; whether the generated picture has watermarks; whether the generated picture has signatures.

[0098] In an embodiment of the present invention, question and answer information may be configured in the form of a configuration file, in which one or more questions may be configured. The questions are used to detect evaluation data for detection points in a visual question and answer model, and the questions may include at least one of the following: whether the ethnic categories of the two pictures are consistent; whether the expressions of the characters in the two pictures are consistent; whether the hair lengths of the characters in the two pictures are consistent; whether the mouth shapes of the characters in the two pictures are consistent; whether the beard states of the characters in the two pictures are consistent; whether the glasses wearing states of the characters in the two pictures are consistent; whether the number of people in the two pictures is the same; whether the genders of the characters in the two pictures are consistent; whether the appearance of the generated picture is higher than a specified score; whether the head postures of the characters in the two pictures are consistent; whether the body movements of the characters in the two pictures are consistent; whether there are extra human-shaped objects in the generated picture; whether there are human body deformities in the generated picture; whether the style of the generated picture is consistent with the specified generation style; whether the local style of the generated picture is consistent with the specified generation style; whether the aesthetics of the generated picture meets the standards; whether the generated picture has white edges; whether the generated picture has watermarks; whether the generated picture has signatures. It will be understood that the form of the above questions is for illustrative purposes only and is not intended to be limiting. Other questions regarding racial category, facial expression, hair length, mouth shape, beard status, glasses wearing status, number of people, gender, appearance, head posture, body movements, extra human-shaped objects, human deformities, overall style, local style, aesthetics, white edges on images, image signatures, image watermarks and other detection points may all be used as question and answer information.

[0099] Embodiment 4

[0100] Figure 4 is a flowchart of another method for generating and evaluating images according to Embodiment 4 of the present invention. The embodiment of the present invention is a specific embodiment of the above-mentioned invention. The process of generating image evaluation information is described in detail in the embodiment of the present invention. Figure 4 The method provided in the embodiment of the present invention specifically includes the following steps:

[0101] Step 410: obtain the original image, the generated image and the style specification information as evaluation data, and transmit the evaluation data to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point.

[0102] Step 420: For the same detection point, determine the first type of evaluation indicators of the generated graph through the model detection module based on the evaluation data, and generate the second type of evaluation indicators of the generated graph through the visual question answering module.

[0103] Step 430: Map the first type of evaluation indicators and the second type of evaluation indicators to at least one preset evaluation dimension according to the corresponding indicator types.

[0104] Step 440: extract the indicator evaluation statistical rules, statistical result classification and evaluation information template corresponding to the statistical result classification configured for each preset evaluation dimension in the preset evaluation rules.

[0105] Among them, the preset evaluation rules can be rules configured for different preset evaluation dimensions, and the rules may include indicator evaluation statistical rules, statistical result grading, and evaluation information templates, etc. The indicator evaluation statistical rules can be used for analytical statistical methods when evaluating the first type of evaluation indicators and the second type of evaluation indicators. The indicator evaluation statistical rules may include a method of screening abnormal indicators, screening normal indicators, screening indicators exceeding a threshold, or screening specified indicators through the first type of evaluation indicators and the second type of evaluation indicators. Taking the screening of abnormal indicators as an example, the indicator evaluation statistical rules may include determining abnormal indicators based on the first type of evaluation indicators when the first type of evaluation indicators and the second type of evaluation indicators are inconsistent; determining abnormal indicators based on the second type of evaluation indicators when the first type of evaluation indicators and the second type of evaluation indicators are inconsistent; determining abnormal indicators based on the weighted operation results of the first type of evaluation indicators and the second type of evaluation indicators; when the first type of evaluation indicators and the second type of evaluation indicators are the same, determining abnormal indicators based on one of the first type of evaluation indicators and the second type of evaluation indicators, etc. The statistical result grading may be a result grading generated after statistics are performed on the first type of evaluation indicators and the second type of evaluation indicators. The statistical result grading may include multiple result gradings, and each result grading may correspond to a critical value of a statistical result. The statistical result classification can correspond to the evaluation information template. Different result classifications within the statistical result classification can respectively have one or more corresponding evaluation information templates. The evaluation information template may include that the XX evaluation dimension of the generated graph failed the inspection, the XX indicator of the XX evaluation dimension failed the inspection, etc. The XX in the preset information template can represent a reserved position. The XX can be used to fill in a specific parameter. The specific value of the parameter can be determined based on the statistical result or preset evaluation dimension corresponding to the evaluation information template.

[0106] In an embodiment of the present invention, a corresponding set of rules can be configured for different preset evaluation dimensions within the preset evaluation rules, and the rules can at least include indicator evaluation statistical rules, statistical result grading, and evaluation information templates corresponding to the statistical result grading, etc. Before evaluating and statistically analyzing the first type of evaluation indicators and the second type of evaluation indicators, the indicator evaluation statistical rules, statistical result grading, and evaluation information templates corresponding to the statistical result grading for each preset evaluation dimension can be extracted from the preset evaluation rules. The rules for each preset evaluation dimension in the preset evaluation rules can exist in the form of a configuration file or a configuration table.

[0107] Step 450: For each preset evaluation dimension, the mapped first-category evaluation indicator and the second-category evaluation indicator are evaluated and counted according to the corresponding indicator evaluation and statistical rules to obtain statistical results.

[0108] Among them, the statistical results can be the results generated after evaluation and statistics are performed according to the indicator evaluation statistical rules. The statistical results can correspond to the indicator evaluation statistical rules. When the indicator evaluation statistical rules include rules for counting abnormal indicators, the statistical results can include the number of abnormal indicators. When the indicator evaluation statistical rules include rules for counting normal indicators, the statistical results can include the number of normal indicators, etc.

[0109] In an embodiment of the present invention, the first category of evaluation indicators and the second category of evaluation indicators mapped to each preset evaluation dimension can be obtained, and the first category of evaluation indicators and the second category of evaluation indicators can be analyzed and counted according to the indicator evaluation statistical rules of the preset evaluation dimension, and the generated analysis and statistical results can be used as the statistical results of the preset evaluation dimension.

[0110] Step 460: for each preset evaluation dimension, determine the statistical result classification that matches the statistical result, and generate the picture evaluation information according to the evaluation information template of the statistical result classification.

[0111] In an embodiment of the present invention, the statistical results of each preset evaluation dimension can be matched with the configured statistical result grade to determine the statistical result grade to which the statistical results belong. The matching method may include comparing the statistical result with the value range corresponding to the statistical result grade. If the statistical result is within the value range corresponding to the statistical result grade, it is determined that the statistical result matches the statistical result grade. An evaluation information template corresponding to the matched statistical result grade can be obtained, and picture evaluation information can be generated according to the evaluation information template and the statistical result. For example, the statistical result can be filled into the determined evaluation information template to obtain the picture evaluation information. In some embodiments, the preset evaluation dimension corresponding to the statistical result can also be filled into the evaluation information module to generate the picture evaluation information.

[0112] Step 470: Merge the evaluation information of each picture as a picture evaluation result.

[0113] In an embodiment of the present invention, by acquiring an original image, a generated image and style specification information as evaluation data, the evaluation data is input into a model detection module and a visual question-answering module respectively, the model detection module and the visual question-answering model have at least one same detection point, the evaluation data is processed in the model detection module for the same detection point to determine a first type of evaluation indicator for the generated image, and the evaluation data is processed in the visual question-answering module to determine a second type of evaluation indicator for the generated image, the first type of evaluation indicator and the second type of evaluation indicator are mapped to at least one preset evaluation dimension according to the indicator type, the indicator evaluation statistical rules, the statistical result classification and the evaluation information template corresponding to the statistical result classification for each preset evaluation dimension in the preset evaluation rules are extracted, and for each The preset evaluation dimensions determine the statistical results corresponding to the first and second evaluation indicators after mapping according to the indicator evaluation statistical rules, determine the statistical result classification according to the statistical result values, obtain the evaluation information template corresponding to the statistical result classification, and use the evaluation information template to generate image evaluation information under the preset evaluation dimensions, and merge the image evaluation information of all preset evaluation dimensions as the image evaluation results, thereby realizing the automatic evaluation of generated images, realizing the standardization of generated image evaluation, reducing the influence of manual subjective evaluation on the evaluation results of generated images, improving the accuracy of image generation effect evaluation, improving the effect of generated image evaluation, facilitating feedback on the image generation process, and assisting in improving the image quality of generated images.

[0114] Further, based on the above-mentioned embodiment of the invention, the indicator evaluation statistical rules at least include abnormal quantity statistical rules and one-vote veto item statistical rules. For each preset evaluation dimension, the mapped first category evaluation indicators and second category evaluation indicators are evaluated and counted according to the corresponding indicator evaluation statistical rules to obtain statistical results, including:

[0115] In each preset evaluation dimension, the mapped first-category evaluation indicators and second-category evaluation indicators are counted according to the abnormal judgment conditions of the abnormal number statistical rules to obtain the number of abnormal indicators of the corresponding preset evaluation dimension; in each preset evaluation dimension, the mapped first-category evaluation indicators and second-category evaluation indicators are matched with the one-vote veto item judgment conditions in the one-vote veto statistical rules to obtain the one-vote veto item statistical results of the corresponding preset evaluation dimension; for each preset evaluation dimension, the corresponding number of abnormal indicators and one-vote veto item statistical results are used as statistical results.

[0116] Among them, the abnormal judgment condition may be a condition for determining abnormal indicators in the first category of evaluation indicators and the second category of evaluation indicators, and the abnormal judgment condition may include but is not limited to determining abnormal indicators based on the first category of evaluation indicators when the first category of evaluation indicators and the second category of evaluation indicators are inconsistent; determining abnormal indicators based on the second category of evaluation indicators when the first category of evaluation indicators and the second category of evaluation indicators are inconsistent; determining abnormal indicators based on the weighted calculation results of the first category of evaluation indicators and the second category of evaluation indicators; determining abnormal indicators based on one of the first category of evaluation indicators and the second category of evaluation indicators when the first category of evaluation indicators and the second category of evaluation indicators are the same, etc. The number of abnormal indicators may be the total number of abnormal indicators obtained through abnormal indicator statistics, and the number of abnormal indicators may correspond to a preset evaluation dimension. The veto item statistical rules can screen whether there are rules for the first and second category evaluation indicators that belong to the veto indicators. The veto items can include abnormal indicator items configured under a preset evaluation dimension. Once the abnormal indicator item appears, the evaluation of the generated graph under the preset evaluation dimension will fail. The veto item statistical results can include the number of items that meet the veto items or whether they meet the veto items, etc.

[0117] In an embodiment of the present invention, within each preset evaluation dimension, abnormal indicator statistics and veto item statistics can be performed for the first category of evaluation indicators and the second category of evaluation indicators mapped to the preset evaluation dimension. By judging the first category of evaluation indicators and the second category of evaluation indicators for statistical analysis, the number of abnormal indicators is determined as the number of abnormal indicators of the preset evaluation dimension. By judging whether there is a veto item in the first category of evaluation indicators and the second category of evaluation indicators, the veto item statistical result of the preset evaluation dimension is obtained. The veto item statistical result and the number of abnormal indicators can be used as the statistical result of the preset evaluation dimension.

[0118] In other embodiments of the invention, for each preset evaluation dimension, determining the statistical result classification that matches the statistical result, and generating the picture evaluation information according to the evaluation information template of the statistical result classification, includes:

[0119] For each preset evaluation dimension, extract the abnormal indicator grading threshold and the one-vote veto item statistical threshold for statistical result grading, match the number of abnormal indicators in the statistical result and the one-vote veto item statistical result with the abnormal indicator number grading threshold and the one-vote veto item statistical threshold for statistical result grading, respectively, and obtain the target statistical result grading that matches the statistical results of each preset evaluation dimension;

[0120] For each preset evaluation dimension, determine the target evaluation information template corresponding to the statistical result classification in each evaluation information template;

[0121] The statistical results corresponding to the preset evaluation dimensions are filled into the target evaluation information template corresponding to the preset evaluation dimensions to obtain the image evaluation information of the preset evaluation dimensions.

[0122] Among them, the abnormal indicator quantity classification threshold can be the critical threshold for dividing the statistical result classification, and the abnormal indicator data classification threshold can be the value range of different statistical result classifications. It can be understood that each statistical result classification can include at least one abnormal indicator data classification threshold. For example, the abnormal indicator data classification threshold can include the lowest threshold and / or the highest threshold of the statistical result classification. The one-vote veto item statistical threshold can be the critical threshold for judging whether the statistical result meets the one-vote veto. The one-vote veto item statistical threshold can include 1, or a higher value.

[0123] In an embodiment of the present invention, within each preset evaluation dimension, the abnormal number grading threshold and the one-vote veto item statistical threshold configured for different statistical result grading within the preset evaluation dimension can be extracted, and the number of abnormal indicators and the one-vote veto item statistical results can be extracted in the statistical result. The number of abnormal indicators and the one-vote veto item statistical results of the statistical result can be compared with the abnormal number grading threshold and the one-vote veto item statistical threshold of each statistical result grading, respectively, to determine the abnormal number grading threshold satisfied by the number of abnormal indicators and the one-vote veto item statistical threshold satisfied by the one-vote veto statistical result. The statistical result grade corresponding to the satisfied abnormal number grading threshold and the one-vote veto item statistical threshold can be determined as the target statistical result grade matching the statistical result. The evaluation information template of the target statistical result grade can be used as the target evaluation information template of the statistical result. The statistical result and all or part of the information of the preset evaluation dimension of the statistical result can be filled into the target evaluation information template, thereby constituting image evaluation information.

[0124] Further, based on the above-mentioned embodiment of the invention, the first type of evaluation indicators and the second type of evaluation indicators mapped are evaluated and counted according to the corresponding indicator evaluation and statistical rules, including at least one of the following:

[0125] When the first category of evaluation indicators and the second category of evaluation indicators are inconsistent, evaluation statistics are performed based on the first category of evaluation indicators;

[0126] When the first type of evaluation indicators and the second type of evaluation indicators are inconsistent, the evaluation statistics are performed based on the second type of evaluation indicators;

[0127] Perform evaluation statistics based on the weighted calculation results of the first and second evaluation indicators;

[0128] When the first type of evaluation indicators and the second type of evaluation indicators are the same, evaluation statistics are performed based on one of the first type of evaluation indicators and the second type of evaluation indicators.

[0129] In an embodiment of the present invention, there are multiple ways to determine evaluation statistics within different preset evaluation dimensions, which may specifically include: if the first category of evaluation indicators and the second category of evaluation indicators within the preset evaluation dimension are inconsistent, the inconsistency may be that the indicator values ​​corresponding to one or more of the detection points corresponding to the first category of evaluation indicators and the second category of evaluation indicators are inconsistent, and evaluation statistics are performed using the first category of evaluation indicators; or, if the first category of evaluation indicators and the second category of evaluation indicators mapped by the preset evaluation dimension are inconsistent, evaluation statistics may be performed using the second category of evaluation indicators; or, evaluation may be performed using the weighted calculation results of the first category of evaluation indicators and the second category of evaluation indicators. Statistics, that is, the first category of evaluation indicators and the second category of evaluation indicators can have different weights. The first category of evaluation indicators and the second category of evaluation indicators can be weighted according to their respective corresponding weights, and then the weighted operation results generated by the operation are used as parameters for evaluation statistics. The weighted operation method may include but does not include summation, average, variance, standard deviation, etc.; or, the first category of evaluation indicators and the second category of evaluation indicators mapped under the preset evaluation dimension are the same, and one of the first category of evaluation indicators and the second category of evaluation indicators can be randomly selected for evaluation statistics, or, the first category of evaluation indicators or the second category of evaluation indicators can be fixedly selected for evaluation statistics.

[0130] Further, on the basis of the above-mentioned embodiments of the invention, for each preset evaluation dimension, the abnormal indicator quantity classification threshold and the one-vote veto item statistical threshold for statistical result classification are extracted, and the abnormal indicator quantity and the one-vote veto item statistical results in the statistical results are matched with the abnormal indicator quantity classification threshold and the one-vote veto item statistical threshold in the statistical result classification, respectively, to obtain the target statistical result classification matching the statistical results of each preset evaluation dimension, including:

[0131] If the statistical result of the one-vote veto item is greater than or equal to the one-vote veto item statistical threshold of the first result classification within the statistical result classification, then the target statistical result classification of the statistical result is determined to be the first result classification, and the target evaluation information template of the first result classification is that the generated graph effect does not meet the requirements of the preset evaluation dimension;

[0132] If the statistical result of the one-vote veto item is less than the one-vote veto item statistical threshold of the first result classification within the statistical result classification, and the number of abnormal indicators is greater than or equal to the abnormal indicator number classification threshold of the first result classification within the statistical result classification, then the target statistical result classification of the statistical result is determined to be the first result classification, and the target evaluation information template of the first result classification is that the generated graph effect does not meet the requirements of the preset evaluation dimension;

[0133] If the statistical result of the veto item is smaller than the statistical threshold of the veto item of the first result classification within the statistical result classification, and the number of abnormal indicators is smaller than the grading threshold of the number of abnormal indicators corresponding to the first result classification of the first result classification within the statistical result classification, then the target statistical result classification of the statistical result is determined to be the second result classification, and the target evaluation information of the second result classification is that the generated graph effect meets the requirements of the preset evaluation dimension.

[0134] In an embodiment of the present invention, the statistical result of the veto item can be first compared with the statistical threshold of the veto item of the first result classification in the statistical result classification. If the statistical result of the veto item is greater than or equal to the statistical threshold of the veto item of the first result classification, the target statistical result classification of the statistical result of the preset evaluation dimension is the first result classification, and the target evaluation information template of the first result classification is that the generated graph effect does not meet the requirements of the preset evaluation dimension; otherwise, the number of abnormal indicators is compared with the abnormal indicator number classification threshold of the first result classification to determine whether the number of abnormal indicators is less than or equal to the abnormal indicator number classification threshold. If so, the statistical result of the preset evaluation dimension does not belong to the first result classification, but to the second result classification, and the target evaluation information template of the preset evaluation dimension is that the generated graph effect meets the requirements of the preset evaluation dimension; if not, that is, the number of abnormal indicators is greater than the abnormal indicator number classification threshold of the first result classification, then the target evaluation information template of the preset evaluation dimension can be that the generated graph effect does not meet the requirements of the preset evaluation dimension.

[0135] Furthermore, on the basis of the above-mentioned embodiments of the invention, it also includes: for each preset evaluation dimension, the number of abnormal indicators and the statistical results of veto items in the statistical results, and the matching results of the number of abnormal indicators and the statistical results of veto items with the abnormal indicator number grading threshold and the veto item statistical threshold are respectively visualized.

[0136] In an embodiment of the present invention, the number of abnormal indicators and the statistical results of veto items, the first comparison situation of the number of abnormal indicators and the grading threshold of the number of abnormal indicators, and the second comparison situation of the statistical results of veto items and the veto statistical threshold can be obtained for each preset evaluation dimension, and the above-mentioned abnormal indicators, the number of abnormal indicators, the first comparison situation and the second comparison situation can be visualized. It can be understood that the visualization process can display the above-mentioned abnormal indicators, the number of abnormal indicators, the first comparison situation and the second comparison situation in separate columns according to the preset evaluation dimensions, so as to facilitate users to obtain the evaluation situation of the generated graph under different preset evaluation dimensions.

[0137] Embodiment 5

[0138] Figure 5is an example diagram of a method for evaluating image generation provided by embodiment 5 of the present invention. In this embodiment of the present invention, a user can input an original image and a specified generation style. The generation style can be a pre-configured style type, which can include but is not limited to a fairy tale style, a spaceship style, a yacht style, a Star Wars style, and a princess carriage style, etc. A generated image with a corresponding style can be generated according to the original image and the specified generation style through a pre-configured image generation system. See Figure 5 The above original image, generated style and generated image can be used as input data, and the input data can be input into the automated evaluation service. The automated evaluation service can perform detailed index detection on the three dimensions of similarity, style correctness and aesthetics. The above three evaluation dimensions can be refined into vertical indicators about face, body, image style and image aesthetics. The above vertical indicators can be detected in the automated evaluation service. The above vertical indicators can specifically include at least one of the following:

[0139] Regarding faces: whether the races of the people in the two pictures are consistent; whether the face shapes of the people in the two pictures are consistent; whether the mouth shapes of the people in the two pictures are consistent; whether the expressions of the people in the two pictures are consistent; whether the hair lengths of the people in the two pictures are consistent; whether the beards of the people in the two pictures are consistent; whether the glasses of the people in the two pictures are consistent; whether the number of people in the two pictures is the same; whether the genders of the people in the two pictures are consistent; whether the appearance of the generated picture is higher than XX points, etc.;

[0140] Regarding the human body: whether the head postures of the characters in the two pictures are consistent; whether the body movements of the characters in the two pictures are consistent; whether there are extra human-shaped objects in the generated picture; whether there are human body deformities in the generated picture, etc.;

[0141] Regarding the image style: whether the generated image style is consistent with the specified generated style; whether the local style of the generated image is consistent with the specified generated style, etc.;

[0142] Regarding the aesthetics of the image: whether the generated image meets the aesthetic standards; whether the generated image has white edges; whether the generated image has watermarks; whether the generated image has signatures, etc.

[0143] In an embodiment of the present invention, the automated evaluation service can be implemented based on an AI algorithm module and a visual question answering model, wherein see Figure 6The AI ​​algorithm module may include at least an image model detection system and a label verification system. The image model detection system may include at least a face analysis model, a body analysis model, a picture style recognition model and a picture aesthetics scoring model. The label verification system may analyze different label indicators generated by the image model detection to generate evaluation results for corresponding detection points. The visual question answering model asks visual questions based on the input data through a large visual model, obtains answers to all detection points in the form of questions and answers, and uses the obtained answers as evaluation results.

[0144] Specifically, the detection results generated by the above-mentioned automatic evaluation service are mapped to the three evaluation dimensions of similarity, style correctness and aesthetics, and a detailed evaluation report is output through statistical analysis of the detection results of different dimensions to provide quality feedback for the image generation system. In an exemplary embodiment, the mapping method of the detection results to similarity, style correctness and aesthetics can be shown in the following Table 2:

[0145] Table 2 Mapping relationship between detection results and evaluation dimensions

[0146]

[0147]

[0148] In an embodiment of the present invention, through an automated evaluation process, combined with multi-dimensional evaluations such as face analysis, body analysis, image style analysis, and image aesthetics analysis, the three evaluation dimensions of similarity, style correctness, and aesthetics are evaluated together with the help of an AI algorithm module and a large visual model. This can improve the reliability of the generated image evaluation, effectively liberate human resources, and improve the efficiency of the generated image evaluation. Through visualized evaluation results, the readability and comprehensibility of the evaluation report are improved, which helps to improve the generation quality of the image generation system.

[0149] Embodiment 6

[0150] Figure 7 is a structural diagram of a graph generation and evaluation system provided according to Embodiment 6 of the present invention. Figure 7 As shown, the system includes:

[0151] The data input module 610 is used to obtain the original image, the generated image and the style specification information as the evaluation data, and transmit the evaluation data input to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point.

[0152] The indicator detection module 620 is used to determine the first type of evaluation indicators of the generated graph based on the evaluation data through the model detection module for the same detection point, and to generate the second type of evaluation indicators of the generated graph for the detection point through the visual question answering module.

[0153] The evaluation result module 630 is used to determine the picture evaluation result based on the first type of evaluation indicators and the second type of evaluation indicators according to the preset evaluation rules.

[0154] In an embodiment of the present invention, the original image, the generated image and the style specification information are used as evaluation data through a data input module, and the evaluation data are respectively input into a model detection module and a visual question and answer module having at least one same detection point. The indicator detection module generates a first type of evaluation indicator and a second type of evaluation indicator for the above-mentioned generated image for each detection point in the model detection module and the visual question and answer module. The evaluation result module generates an image evaluation result of the corresponding generated image based on the first type of evaluation indicator and the second type of evaluation indicator according to preset evaluation rules, thereby realizing automatic evaluation of the generated image. The evaluation standard of the generated image is unified through preset evaluation rules, which can solve the problem of strong subjectivity of the manual evaluation standard, improve the accuracy of image quality assessment, and facilitate feedback on the process of image processing, thereby assisting in improving the image quality of the generated image.

[0155] Based on the above-mentioned embodiment of the invention, the evaluation result module 630 includes:

[0156] The mapping processing unit is used to map the first type of evaluation indicators and the second type of evaluation indicators to at least one preset evaluation dimension according to the corresponding indicator types.

[0157] The dimension evaluation unit is used to determine the image evaluation information of the mapped first-category evaluation indicators and the second-category evaluation indicators respectively under different preset evaluation dimensions according to preset evaluation rules.

[0158] Combined with the merging unit, it is used to merge the evaluation information of each picture as the picture evaluation result.

[0159] Based on the above-mentioned embodiment of the invention, the preset evaluation dimension includes at least one of the following: picture content similarity dimension, picture style correctness dimension and picture aesthetics dimension, and the mapping relationship between the indicator type and the preset evaluation dimension includes at least one of the following:

[0160] The image content similarity dimension corresponds to at least one of the following indicators: race indicator, expression indicator, hair condition indicator, face shape indicator, mouth shape indicator, beard condition indicator, glasses condition indicator, number of people indicator, gender indicator, appearance indicator, head posture indicator, limb posture indicator, extra humanoid object indicator, and human deformity indicator in the indicator type;

[0161] The image style correctness dimension corresponds to at least one of the overall style index and the local style index in the index type;

[0162] The image aesthetics dimension corresponds to at least one of the aesthetics score index, the image white edge condition index, the image signature condition index, and the image watermark condition index in the indicator type.

[0163] Based on the above-mentioned embodiment of the invention, the indicator detection module 620 includes:

[0164] A model processing unit is used to generate image analysis results of detection points based on evaluation data through each image detection model, wherein the image analysis results at least include at least one of a first image analysis result generated based on the original image, a second image analysis result generated based on the generated image, and a third image analysis result generated based on the generated image and style specified information.

[0165] The label verification unit is used to process the image analysis result in the label detection unit into the first type of evaluation index.

[0166] On the basis of the above-mentioned embodiment of the invention, the image detection model includes at least one of a face analysis model, a body analysis model, a picture style recognition model and a picture aesthetics scoring model. In each image detection model, an image analysis result corresponding to at least one detection point is generated based on the evaluation data, including at least one of the following:

[0167] The race, expression, hair condition, face shape, mouth shape, beard condition, glasses condition, number of people, gender and appearance of the original image and the generated image are respectively extracted from the face analysis model as the first image analysis result and the second image analysis result;

[0168] The head posture, limb posture, redundant human-shaped objects and human body deformity in the original image and the generated image are respectively extracted as the first image analysis result and the second image analysis result through the human body analysis model;

[0169] Generate the overall style and the local style corresponding to the original image and the generated image respectively as the first image analysis result and the second image analysis result through the image style recognition model;

[0170] The overall style and local style generated by the image style recognition model based on the generated graph and the style specified information are used as the third image analysis result;

[0171] The image aesthetics scoring model is used to extract and generate the aesthetics score of the corresponding detection point in the image, the image white edge condition, the image signature condition and the image watermark condition as the second image analysis result.

[0172] Based on the above-mentioned embodiments of the invention, the label verification unit is specifically used for at least one of the following:

[0173] Generate a first comparison result of the first image analysis result and the second image analysis result by a label detection unit, and use the first comparison result as a first type of evaluation indicator;

[0174] Generate a second comparison result between the second image analysis result and the first preset threshold value by the label detection unit, and use the second comparison result as the first type of evaluation indicator;

[0175] A third comparison result between the third image analysis result and the second preset threshold is generated by the label detection unit, and the third comparison result is used as the first type of evaluation indicator.

[0176] Based on the above-mentioned embodiment of the invention, the indicator detection module 620 further includes:

[0177] The question and answer information unit is used to obtain the question and answer information of each detection point configured by the visual question and answer module.

[0178] The question-answering processing unit is used to input question-answering information, original images, generated images, and style-specified information into the visual question-answering module, so that the visual question-answering module generates an answer result corresponding to the question-answering information based on the semantics of the original images, generated images, and style-specified information, and uses the answer result as the second type of evaluation indicator.

[0179] Based on the above-mentioned embodiment of the invention, the question and answer information includes at least one of the following:

[0180] Whether the ethnic categories of the two pictures are consistent; whether the expressions of the characters in the two pictures are consistent; whether the hair lengths of the characters in the two pictures are consistent; whether the mouth shapes of the characters in the two pictures are consistent; whether the beard status of the characters in the two pictures is consistent; whether the glasses wearing status of the characters in the two pictures is consistent; whether the number of people in the two pictures is the same; whether the genders of the characters in the two pictures are consistent; whether the appearance of the generated picture is higher than the specified score; whether the head postures of the characters in the two pictures are consistent; whether the body movements of the characters in the two pictures are consistent; whether there are extra human-shaped objects in the generated picture; whether there are human body deformities in the generated picture; whether the style of the generated picture is consistent with the specified generation style; whether the local style of the generated picture is consistent with the specified generation style; whether the aesthetics of the generated picture meets the standards; whether the generated picture has white edges; whether the generated picture has watermarks; whether the generated picture has signatures.

[0181] Based on the above-mentioned embodiment of the invention, the dimension evaluation unit includes:

[0182] The rule extraction unit is used to extract the indicator evaluation statistical rules, statistical result classification and evaluation information template corresponding to the statistical result classification configured for each preset evaluation dimension in the preset evaluation rules.

[0183] The evaluation and statistical unit is used to evaluate and count the mapped first-category evaluation indicators and second-category evaluation indicators according to the corresponding indicator evaluation and statistical rules for each preset evaluation dimension to obtain statistical results.

[0184] The picture evaluation unit is used to determine the statistical result classification that matches the statistical result for each preset evaluation dimension, and generate picture evaluation information according to the evaluation information template of the statistical result classification.

[0185] On the basis of the above-mentioned embodiments of the invention, the evaluation and statistical unit is specifically used for: within each preset evaluation dimension, performing abnormal indicator statistics on the mapped first-category evaluation indicators and second-category evaluation indicators according to the abnormal judgment conditions of the abnormal number statistical rules, and obtaining the number of abnormal indicators corresponding to the preset evaluation dimension; within each preset evaluation dimension, matching the mapped first-category evaluation indicators and second-category evaluation indicators with the one-vote veto item judgment conditions in the one-vote veto statistical rules, and obtaining the one-vote veto item statistical results of the corresponding preset evaluation dimension; for each preset evaluation dimension, taking the corresponding abnormal indicator quantity and one-vote veto item statistical results as statistical results.

[0186] On the basis of the above-mentioned embodiments of the invention, the picture evaluation unit is specifically used for: for each preset evaluation dimension, extracting the abnormal indicator quantity grading threshold and the one-vote veto item statistical threshold for the statistical result grading, matching the abnormal indicator quantity and the one-vote veto item statistical results in the statistical result with the abnormal indicator quantity grading threshold and the one-vote veto item statistical threshold in the statistical result grading respectively, to obtain the target statistical result grading that matches the statistical results of each preset evaluation dimension; for each preset evaluation dimension, determining the target evaluation information template corresponding to the statistical result grading in each evaluation information template; filling the statistical results corresponding to the preset evaluation dimension into the target evaluation information template corresponding to the preset evaluation dimension, to obtain the picture evaluation information of the preset evaluation dimension.

[0187] On the basis of the above-mentioned embodiment of the invention, the evaluation and statistics unit performs evaluation and statistics on the mapped first-category evaluation indicators and second-category evaluation indicators according to the corresponding indicator evaluation and statistics rules, including at least one of the following:

[0188] When the first category of evaluation indicators and the second category of evaluation indicators are inconsistent, evaluation statistics are performed based on the first category of evaluation indicators;

[0189] When the first type of evaluation indicators and the second type of evaluation indicators are inconsistent, the evaluation statistics are performed based on the second type of evaluation indicators;

[0190] Perform evaluation statistics based on the weighted calculation results of the first and second evaluation indicators;

[0191] When the first type of evaluation indicators and the second type of evaluation indicators are the same, evaluation statistics are performed based on one of the first type of evaluation indicators and the second type of evaluation indicators.

[0192] On the basis of the above-mentioned embodiment of the invention, the image evaluation unit extracts the abnormal indicator quantity classification threshold and the one-vote veto item statistical threshold for each preset evaluation dimension, and matches the abnormal indicator quantity and the one-vote veto item statistical result in the statistical result with the abnormal indicator quantity classification threshold and the one-vote veto item statistical threshold in the statistical result classification, respectively, to obtain the target statistical result classification matching the statistical results of each preset evaluation dimension, including:

[0193] If the statistical result of the one-vote veto item is greater than or equal to the one-vote veto item statistical threshold of the first result classification within the statistical result classification, then the target statistical result classification of the statistical result is determined to be the first result classification, and the target evaluation information template of the first result classification is that the generated graph effect does not meet the requirements of the preset evaluation dimension;

[0194] If the statistical result of the one-vote veto item is less than the one-vote veto item statistical threshold of the first result classification within the statistical result classification, and the number of abnormal indicators is greater than or equal to the abnormal indicator number classification threshold of the first result classification within the statistical result classification, then the target statistical result classification of the statistical result is determined to be the first result classification, and the target evaluation information template of the first result classification is that the generated graph effect does not meet the requirements of the preset evaluation dimension;

[0195] If the statistical result of the one-vote veto item is less than the one-vote veto item statistical threshold of the first result classification within the statistical result classification, and the number of abnormal indicators is less than the abnormal indicator number classification threshold corresponding to the first result classification of the first result classification within the statistical result classification, then the target statistical result classification of the statistical result is determined to be the second result classification, and the target evaluation information of the second result classification is that the generated graph effect meets the requirements of the preset evaluation dimension. Based on the above-mentioned embodiment of the invention, the dimension evaluation unit also includes:

[0196] The information display sub-unit is used to visualize the number of abnormal indicators and the statistical results of veto items in the statistical results for each preset evaluation dimension, and the matching results of the number of abnormal indicators and the statistical results of veto items with the classification threshold of the number of abnormal indicators and the statistical threshold of veto items.

[0197] The image generation and evaluation system provided in the embodiment of the present invention can execute the image generation and evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0198] Embodiment 7

[0199] Figure 8It is a structural schematic diagram of an electronic device that implements the graph evaluation method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0200] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0201] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0202] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the image generation and evaluation method.

[0203] In some embodiments, the image raw image evaluation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image raw image evaluation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image raw image evaluation method in any other appropriate manner (e.g., by means of firmware).

[0204] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0205] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0206] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0207] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0208] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0209] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0210] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0211] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating a graph, characterized in that: The method comprises: Acquire the original image, the generated image, and the style specification information as evaluation data, and transmit the evaluation data to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point; For the same detection point, determining a first type of evaluation index of the generated graph through the model detection module based on the evaluation data, and generating a second type of evaluation index of the generated graph through the visual question answering module; According to preset evaluation rules, an image evaluation result of the generated image is determined based on the first type of evaluation indicators and the second type of evaluation indicators.

2. The method according to claim 1, characterized in that: The step of determining the image evaluation result of the generated image based on the first type of evaluation indicators and the second type of evaluation indicators according to the preset evaluation rules includes: Mapping the first type of evaluation indicators and the second type of evaluation indicators to at least one preset evaluation dimension according to corresponding indicator types; According to the preset evaluation rules, under different preset evaluation dimensions, respectively determine the image evaluation information of the first type of evaluation indicators and the second type of evaluation indicators after mapping; The picture evaluation information is combined as the picture evaluation result.

3. The method according to claim 1, characterized in that: The model detection module at least includes an image detection unit and a label detection unit, the image detection unit includes an image detection model for processing at least one of the detection points, and for the same detection point, determining the first type of evaluation index of the generated graph through the model detection module based on the evaluation data includes: Generate, by each of the image detection models, an image analysis result of the detection point based on the evaluation data, wherein the image analysis result includes at least one of a first image analysis result generated based on the original image, a second image analysis result generated based on the generated image, and a third image analysis result generated based on the generated image and the style specified information; The image analysis result is processed into the first type of evaluation indicator by the label detection unit.

4. The method according to claim 3, characterized in that: The processing of the image analysis result into the first type of evaluation indicator by the label detection unit includes at least one of the following: Generate a first comparison result between the first image analysis result and the second image analysis result by the label detection unit, and use the first comparison result as the first type of evaluation indicator; Generate, by the label detection unit, a second comparison result between the second image analysis result and the first preset threshold, and use the second comparison result as the first type of evaluation indicator; A third comparison result between the third image analysis result and a second preset threshold is generated by the label detection unit, and the third comparison result is used as the first type of evaluation indicator.

5. The method according to claim 1, characterized in that: The second type of evaluation index of the generated graph is generated by the visual question answering module based on the evaluation data for the detection point, including: Acquire the question and answer information including each of the detection points configured by the visual question and answer module; The question and answer information, the original image, the generated image and the style specification information are input into the visual question and answer module, so that the visual question and answer module generates an answer result corresponding to the question and answer information based on the semantics of the original image, the generated image and the style specification information, and uses the answer result as the second type of evaluation indicator.

6. The method according to claim 2, characterized in that: The step of determining the image evaluation information of the first type of evaluation indicators and the second type of evaluation indicators after mapping under different preset evaluation dimensions according to the preset evaluation rules includes: Extracting the indicator evaluation statistical rules, statistical result classification and evaluation information template corresponding to the statistical result classification configured for each of the preset evaluation dimensions in the preset evaluation rules; For each of the preset evaluation dimensions, the mapped first category evaluation indicators and the second category evaluation indicators are evaluated and counted according to the corresponding indicator evaluation statistical rules to obtain statistical results; For each of the preset evaluation dimensions, the statistical result classification that the statistical result matches is determined, and the picture evaluation information is generated according to the evaluation information template of the statistical result classification.

7. The method according to claim 6, characterized in that: The indicator evaluation statistical rules at least include abnormal quantity statistical rules and one-vote veto item statistical rules. For each of the preset evaluation dimensions, the first category of evaluation indicators and the second category of evaluation indicators mapped are evaluated and counted according to the corresponding indicator evaluation statistical rules to obtain statistical results, including: In each of the preset evaluation dimensions, abnormal indicator statistics are performed on the mapped first-category evaluation indicators and the second-category evaluation indicators according to the abnormality judgment conditions of the abnormality quantity statistical rule to obtain the number of abnormal indicators corresponding to the preset evaluation dimension; In each of the preset evaluation dimensions, the mapped first-category evaluation indicators and the second-category evaluation indicators are matched with the one-vote veto item judgment conditions in the one-vote veto statistical rules to obtain the one-vote veto item statistical results corresponding to the preset evaluation dimensions; For each of the preset evaluation dimensions, the corresponding number of abnormal indicators and the statistical result of the veto items are taken as the statistical result.

8. The method according to claim 6, characterized in that: The step of determining the statistical result classification that matches the statistical result for each of the preset evaluation dimensions, and generating the picture evaluation information according to the evaluation information template of the statistical result classification includes: For each of the preset evaluation dimensions, extract the abnormal indicator quantity grading threshold and the one-vote veto item statistical threshold of the statistical result grading, match the abnormal indicator quantity and the one-vote veto item statistical results in the statistical result with the abnormal indicator quantity grading threshold and the one-vote veto item statistical threshold in the statistical result grading respectively, and obtain the target statistical result grading matched by the statistical results of each of the preset evaluation dimensions; For each of the preset evaluation dimensions, determining a target evaluation information template corresponding to the statistical result classification in each of the evaluation information templates; The statistical result corresponding to the preset evaluation dimension is filled into the target evaluation information template corresponding to the preset evaluation dimension to obtain the picture evaluation information of the preset evaluation dimension.

9. A graph evaluation system, characterized in that: The system comprises: A data input module, used for obtaining the original image, the generated image and the style specification information as evaluation data, and transmitting the evaluation data input to the model detection module and the visual question answering module respectively, wherein the model detection module and the visual question answering module have at least one common detection point; An indicator detection module, used for determining, for the same detection point, a first type of evaluation indicator of the generated graph through the model detection module based on the evaluation data, and generating a second type of evaluation indicator of the generated graph for the detection point through the visual question answering module; An evaluation result module is used to determine the picture evaluation result based on the first type of evaluation indicators and the second type of evaluation indicators according to preset evaluation rules.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the graphic evaluation method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the graphic evaluation method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method for evaluating graph raw images according to any one of claims 1 to 8.

Images