Multi-dimensional automatic evaluation method for image processing and storage medium

Abstract

The application discloses a kind of multi-dimension automatic evaluation methods, systems and storage medium for image processing, including obtaining the original image corresponding to the image editing task to be evaluated, edited image and editing instruction;Establish double-level evaluation architecture including basic model evaluation layer and vertical domain workflow evaluation layer, based on visual language model, the structured semantic evaluation of original image, edited image and editing instruction, based on semantic score index, and combined with at least one auxiliary evaluation index, the edited image is evaluated in multidimension, according to the fusion calculation of preset fusion strategy evaluation index, generates the comprehensive evaluation result corresponding to image editing model or vertical domain workflow.The application can improve the comprehensiveness and accuracy of image editing evaluation.

Description

Technical Field

[0001] This application relates to the field of intelligent image editing technology, specifically to a multi-dimensional automated evaluation method, system, and storage medium for image processing. Background Technology

[0002] With the development of generative artificial intelligence technology, image editing technology based on large-scale pre-trained models has been gradually applied to scenarios such as digital content generation, commercial visual design, portrait photography, advertising production, and virtual photography. Large-scale image editing models can modify people, backgrounds, clothing, poses, lighting, or specific details in original images based on user-inputted text commands. As model versions and image editing workflows continuously iterate, how to objectively, automatically, and repeatedly evaluate the actual effects of different image editing models and different business workflows has become a crucial technical issue in model optimization and business implementation.

[0003] Existing methods for evaluating image editing effects mainly include pixel-level image quality assessment, global semantic similarity assessment, subjective human evaluation, and single-dimensional automatic scoring methods. Traditional pixel-level metrics such as PSNR and SSIM are primarily used to evaluate low-level pixel or structural differences between images. However, image editing tasks often allow for significant changes to the target region; therefore, pixel-level similarity is insufficient to accurately reflect whether the editing result conforms to user instructions, nor can it assess whether non-target regions have been reasonably preserved. On the other hand, while global semantic similarity metrics such as CLIP Score can reflect the high-level semantic consistency between the original and edited images to some extent, they are mainly based on comparisons of overall feature representations and have limited ability to identify fine-grained issues such as changes in local attributes, discrepancies in person identification, incorrect relationships, changes in quantity, and excessive modification of non-target regions.

[0004] In practice, manual visual evaluation remains a common method. This involves professionals observing each edit and providing feedback based on factors such as subject consistency, background appropriateness, image naturalness, and the absence of obvious AI-generated traces. However, this method relies on the subjective experience of the evaluators, making it difficult to maintain consistent evaluation standards among different personnel. The repeatability of evaluation results is poor, and it incurs high manpower and time costs when portrait editing models require frequent, multi-version iterations. While some existing automated evaluation solutions can score based on single dimensions such as aesthetic quality, text alignment, or image similarity, a single indicator cannot simultaneously reflect multiple aspects of image editing tasks, including the degree of instruction execution, preservation of original content, image quality, subject identity consistency, and human visual preferences. Therefore, it is difficult to obtain evaluation results that comprehensively characterize the image editing effect.

[0005] Furthermore, existing evaluation systems typically focus on uniformly testing the base model itself, while image editing in real-world commercial applications often involves multiple models, processing nodes, or post-processing steps forming a complete workflow. The evaluation results of the base model on general datasets do not directly reflect its real-world performance in specific business scenarios. For example, in portrait photography or virtual photography scenarios, it is necessary not only to determine whether editing instructions are executed, but also to simultaneously assess whether the subject's identity and clothing are preserved, whether the subject and background blend naturally, whether lighting and shadow conform to spatial relationships, and whether significant structural distortion occurs. The lack of an effective mapping between existing general evaluation metrics and specific business constraints makes it difficult to conduct targeted evaluations of end-to-end image editing workflows.

[0006] Therefore, existing technologies have at least the following shortcomings: First, the evaluation granularity of traditional pixel-level or global similarity indicators is limited, making it difficult to identify local semantic errors and abnormal changes in non-target regions; second, single-dimensional scoring is difficult to reflect the overall quality of image editing results; third, manual evaluation is highly subjective, costly, and difficult to reproduce on a large scale; and fourth, there is a lack of hierarchical and scenario-based evaluation mechanisms between general evaluation of basic models and specific business workflows. Summary of the Invention

[0007] In view of the above problems, this application provides a multi-dimensional automated evaluation method for image processing to solve the technical problems of insufficient refinement, objectivity and reproducibility in the above image editing evaluation.

[0008] To achieve the above objectives, the inventors provide a multi-dimensional automated evaluation method for image processing, comprising the following steps:

[0009] A two-tier evaluation architecture is established, comprising a basic model evaluation layer and a vertical workflow evaluation layer. The basic model evaluation layer is used to establish corresponding first evaluation rules and one or more first evaluation indicators for the general editing capabilities of image editing models, and selects one image editing model as the base of the vertical workflow based on the evaluation results. The vertical workflow evaluation layer is used to establish corresponding second evaluation rules and one or more second evaluation indicators for specific application scenarios.

[0010] Acquire the original image, editing instructions, and edited image corresponding to the image editing task to be evaluated. The edited image includes the image obtained by the image editing model and / or the image obtained by the vertical workflow.

[0011] Based on a visual language model, a structured semantic evaluation is performed on the original image, editing instructions, and the edited image to obtain a semantic scoring index for characterizing the image editing effect.

[0012] Based on the evaluation rules and evaluation indicators determined by the two-level evaluation architecture, combined with the semantic scoring indicators and at least one auxiliary evaluation indicator, the edited image is evaluated in multiple dimensions to obtain multiple corresponding evaluation results. The auxiliary evaluation indicators include one or more of the following: image similarity indicators, facial feature similarity indicators, aesthetic evaluation indicators, and business evaluation indicators set for specific application scenarios.

[0013] The multiple evaluation results are fused and calculated according to a preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model and / or the vertical workflow.

[0014] Furthermore, based on the visual language model, a structured semantic evaluation is performed on the original image, editing instructions, and edited image to obtain a semantic scoring index for characterizing the image editing effect, including:

[0015] The original image, the edited image, and the corresponding editing instructions are input into the visual language model;

[0016] A corresponding visual question-answering task is generated based on preset semantic scoring and evaluation rules, wherein the semantic scoring and evaluation rules include at least task objective constraints, editing area constraints, and evaluation criterion constraints.

[0017] The visual language model is used to reason about the visual question-answering task to obtain a semantic scoring index that characterizes the degree of matching between the image editing result and the editing target.

[0018] Furthermore, the semantic scoring metric includes at least two evaluation dimensions, wherein the evaluation dimensions include:

[0019] Instruction compliance metrics are used to evaluate whether the editing results meet the requirements of the editing instructions.

[0020] And an editing conservatism metric for evaluating whether non-target regions retain the original image features;

[0021] The semantic score result is determined based on the lowest score among the multiple evaluation dimensions.

[0022] Furthermore, the phrase "based on a visual language model, performing structured semantic evaluation on the original image, editing instructions, and edited image to obtain a semantic scoring index for characterizing the image editing effect" includes:

[0023] The visual language model is used to perform structured semantic evaluation on the original image, editing instructions, and the image processed by the image editing model to obtain a semantic scoring index that characterizes the image editing effect of the image editing model.

[0024] And / or based on a visual language model, perform structured semantic evaluation on the original image, editing instructions, and images processed by the vertical workflow to obtain a semantic scoring index that characterizes the effect of the vertical workflow on image editing.

[0025] Furthermore, the phrase "based on the evaluation rules and evaluation indicators determined by the two-level evaluation architecture, combined with the semantic scoring indicator and at least one auxiliary evaluation indicator, to perform multi-dimensional evaluation of the edited image and obtain corresponding multiple evaluation results" includes:

[0026] Based on the first evaluation rule and first evaluation index determined by the two-level evaluation architecture, combined with the semantic scoring index of the image editing effect of the image editing model and at least one auxiliary evaluation index, the image processed by the image editing model is evaluated in multiple dimensions to obtain multiple corresponding evaluation results.

[0027] And / or based on the second evaluation rules and second evaluation indicators determined by the two-level evaluation architecture, combined with the semantic scoring indicators of the image editing effect of the vertical workflow and at least one auxiliary evaluation indicator, the images processed by the vertical workflow are evaluated in multiple dimensions to obtain multiple corresponding evaluation results.

[0028] Furthermore, the step of "fusion calculation of the multiple evaluation results according to a preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model or vertical workflow" includes:

[0029] Based on a preset fusion strategy, multiple evaluation results of the image obtained from the image editing model are fused and calculated to generate a comprehensive evaluation result corresponding to the image editing model.

[0030] And / or perform fusion calculations on multiple evaluation results of the images obtained from the vertical workflow processing according to a preset fusion strategy to generate a comprehensive evaluation result corresponding to the vertical workflow.

[0031] Furthermore, the image similarity index is obtained by extracting the visual feature vectors of the original image and the edited image respectively, and calculating the degree of similarity between the visual feature vectors;

[0032] The visual feature vectors are obtained through a pre-trained visual coding model, and the similarity includes evaluation results calculated based on vector distance, cosine similarity, or other feature space distances.

[0033] The facial feature similarity index includes:

[0034] Face region detection is performed on the original image and the edited image;

[0035] Obtain standardized face region images based on the detection results;

[0036] Extract the facial feature vector corresponding to the standardized facial region image;

[0037] Calculate the similarity between the two facial feature vectors to evaluate the consistency of the person's identity features before and after editing;

[0038] The aesthetic evaluation index is obtained by inputting the image to be evaluated, the corresponding text description information, and the preset evaluation dimensions into a multimodal evaluation model; wherein, the evaluation dimensions include at least one of the following: overall image style, color performance, lighting effect, clarity, detail quality, and artistic quality of the image.

[0039] Furthermore, the vertical workflow evaluation layer performs task-decoupled reconstruction of the first evaluation index according to the specific application scenario to obtain a second evaluation index suitable for the target business scenario.

[0040] In the context of character image editing, the second evaluation metric includes: character consistency metric, background consistency metric, image quality metric, and lighting consistency metric; the character consistency metric is used to evaluate the retention of one or more of the following: character identity, clothing, action posture, props, and human body structure.

[0041] Furthermore, the background consistency index is evaluated using a multi-level judgment method;

[0042] The multi-level determination method includes: a first state indicating that the background is kept reasonable or can be enhanced; a second state indicating that there are slight color or light and shadow changes but the degree of influence cannot be determined; and a third state indicating that there is structural distortion.

[0043] The background consistency evaluation result is determined based on the first state, the second state, and the third state.

[0044] To address the aforementioned technical problems, this application also provides another technical solution:

[0045] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the multi-dimensional automated evaluation method for image processing described in any of the above technical solutions.

[0046] Unlike existing technologies, the above-mentioned technical solution is a multi-dimensional automated evaluation method for image processing. By constructing a two-level evaluation architecture including a basic model evaluation layer and a vertical workflow evaluation layer, it acquires the original image, the edited image, and the editing instructions. Based on a visual language model, it converts the image editing quality into a structured semantic evaluation task to obtain the corresponding semantic scoring index. It then combines at least one auxiliary evaluation index to evaluate the image editing results in multiple dimensions. Finally, it fuses and calculates multiple evaluation results according to a preset fusion strategy to obtain a comprehensive evaluation result of the image editing model or the image editing workflow. Compared to existing methods that rely on pixel-level metrics, single global similarity metrics, or manual visual evaluation, this invention improves the ability to identify instruction execution, local semantic deviations, and abnormal changes in non-target regions through a visual language model. It reduces evaluation bias caused by single metrics through multi-dimensional metric collaboration and fusion calculation, enabling evaluation results to simultaneously reflect different aspects of image editing, such as consistency, quality, identity preservation, and aesthetics. Furthermore, the two-tiered evaluation architecture allows the same evaluation system to be used for general capability evaluation of the basic model, and the evaluation rules and metrics can be reconstructed according to specific business scenarios. This improves the applicability, scalability, and scenario matching capability of the evaluation method. Automated processing reduces the subjectivity and labor costs of manual evaluation, improving the repeatability of evaluation results and the efficiency of model iteration.

[0047] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description

[0048] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of the present invention and other related contents, and should not be considered as limitations on this application.

[0049] In the accompanying drawings of the instruction manual:

[0050] Figure 1 A flowchart illustrating the multi-dimensional automated evaluation method for image processing described in a specific implementation;

[0051] Figure 2 This is a schematic diagram of a computer-readable storage medium according to a specific embodiment.

[0052] The reference numerals used in the above figures are explained as follows:

[0053] 200 types of computer-readable storage media; Detailed Implementation

[0054] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0055] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0056] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0057] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0058] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0059] Without further limitations, the use of terms such as “comprising,” “including,” “having,” or other similar open-ended expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0060] In this application, expressions such as "greater than", "less than", and "exceeding" are understood to exclude the stated number; expressions such as "above", "below", and "within" are understood to include the stated number. Furthermore, in the description of the embodiments of this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times", unless otherwise explicitly specified.

[0061] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0062] The processor described in the embodiments of this application can be implemented by hardware, firmware, software, or a combination thereof. It can be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. It also includes other physical, biological, or chemical structures that can implement the same or equivalent functions as the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of this application, or any combination of the steps mentioned therein.

[0063] The computer program involved in the embodiments can be stored in a computer device readable storage medium, which includes, but is not limited to, disks, magnetic tapes, magnetic cards, floppy disks, flash memory, optical disks, optical cards, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical, or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, proteins, and other units with information storage capabilities. In specific embodiments, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiments can be centrally stored in a single medium, or distributed and stored in multiple media. The memory containing the computer device readable storage medium can be non-volatile memory or random access memory. These computer device readable storage media can be built into the device, or can be connected to the device involved in the embodiments as an external device or part of an external device. In some embodiments, the memory having a computer device readable storage medium is deployed locally; in other embodiments, the memory may be deployed remotely from the processor, for example, as a network-attached memory accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), or a suitable combination thereof, as long as computer device access to the memory is enabled. Furthermore, the computer program involved in the embodiments may be stored in plaintext / ciphertext form, or it may be designed as training data, integrated and recombined through model training and implicitly stored in the parameter states of a deep neural network or other machine learning model.

[0064] In existing technologies, the evaluation of image editing models typically relies on a single metric. However, image editing tasks differ from traditional image generation tasks. Their core requirements include not only the visual quality of the generated image but also whether user editing instructions are accurately executed, whether key content of the original image is preserved, whether unexpected changes are produced, and whether specific business process requirements are met. Therefore, a single metric cannot comprehensively evaluate the actual capabilities of an image editing model.

[0065] Please see Figure 1To address the aforementioned issues, this embodiment provides a multi-dimensional automated evaluation method for image processing. This method is used for automated, structured, and multi-dimensional evaluation of edited images based on generative artificial intelligence models. This embodiment is particularly suitable for scenarios such as portrait image editing, commercial visual generation, and digital content creation. The method automates evaluation based on a "two-tier evaluation architecture" and "multi-dimensional indicator fusion," combining general model capability evaluation with specific business workflow evaluation, and utilizing a visual language model to achieve structured semantic analysis, thereby obtaining more comprehensive image editing effect evaluation results.

[0066] This multi-dimensional automated evaluation method for image processing includes the following steps:

[0067] S101. Establish a two-tier evaluation architecture including a basic model evaluation layer and a vertical workflow evaluation layer. The basic model evaluation layer is used to establish corresponding first evaluation rules and one or more first evaluation indicators for the general editing capabilities of image editing models, and select one image editing model as the base of the vertical workflow based on the evaluation results. The vertical workflow evaluation layer is used to establish corresponding second evaluation rules and one or more second evaluation indicators for specific application scenarios.

[0068] S102. Obtain the original image, editing instructions, and edited image corresponding to the image editing task to be evaluated. The edited image includes the image obtained by the image editing model and / or the image obtained by the vertical workflow.

[0069] S103. Based on the visual language model, perform structured semantic evaluation on the original image, editing instructions and the edited image to obtain a semantic scoring index for characterizing the image editing effect;

[0070] S104. Based on the evaluation rules and evaluation indicators determined by the two-level evaluation architecture, and combined with the semantic scoring indicators and at least one auxiliary evaluation indicator, the edited image is evaluated in multiple dimensions to obtain multiple corresponding evaluation results. The auxiliary evaluation indicators include one or more of the following: image similarity indicators, facial feature similarity indicators, aesthetic evaluation indicators, and business evaluation indicators set for specific application scenarios.

[0071] S105. Perform fusion calculation on the multiple evaluation results according to the preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model and / or the vertical workflow.

[0072] In step S101, a two-tiered evaluation architecture is established, comprising a basic model evaluation layer and a vertical workflow evaluation layer, to perform tiered evaluations of model capabilities and business workflows. The basic model evaluation layer is used to establish corresponding first evaluation rules and one or more first evaluation indicators for the general editing capabilities of the image editing model, thereby evaluating the general capabilities of the image editing model itself and selecting one image editing model as the base for the vertical workflow based on the evaluation results. These include: instruction understanding capability; image content preservation capability; editing capability; and visual quality generation capability, etc. The vertical workflow evaluation layer is used to establish corresponding second evaluation rules and one or more second evaluation indicators for specific business scenarios. For example, in a photo studio portrait business scenario, in addition to evaluating whether the model completes the editing instructions, it is also necessary to further evaluate: whether the person's identity is preserved; whether the clothing meets the requirements; whether the background is natural; whether the lighting and shadow relationships are reasonable; and whether it meets commercial delivery standards.

[0073] In this embodiment, the basic model evaluation layer adopts a unified evaluation index to enable horizontal comparison between different models; the vertical workflow evaluation layer adjusts the evaluation dimensions according to business needs so that the evaluation results can reflect the real application effect.

[0074] Compared to existing evaluation methods that typically only assess basic model capabilities and fail to cover actual business processes, this embodiment establishes a two-tiered structure, separating "model capability evaluation" from "business task completion quality evaluation." This makes the evaluation system both universal and adaptable to different scenarios. Therefore, it avoids the problem of existing models performing well on public datasets but performing poorly in real-world business scenarios.

[0075] In step S102, the input data corresponding to the image editing task to be evaluated is first acquired. The input data includes: the original image, the edited image, and the editing instructions input by the user. The edited image includes the image processed by the image editing model and / or the image processed by the vertical workflow. The editing instructions describe the target content to be modified in the original image, such as: modifying clothing; modifying the background environment; adjusting the pose; adding or deleting image elements. For example, in a portrait generation scenario, the input may include: the original image uploaded by the user, the portrait image generated by the model, and editing instructions input by the user such as "adjust the portrait to a summer beach portrait style." By simultaneously acquiring the original image, the edited image, and the editing instructions, the subsequent evaluation process not only focuses on the image itself but also considers the user's needs to determine whether the edited image meets expectations.

[0076] In step S103, a structured semantic evaluation is performed based on a visual language model to obtain a semantic scoring index. Specifically, after obtaining the original image, the edited image, and the editing instructions, this embodiment further utilizes a visual language model for structured semantic evaluation. This structured semantic evaluation based on the visual language model to obtain the semantic scoring index specifically includes: performing a structured semantic evaluation on the original image, the editing instructions, and the image processed by the image editing model based on the visual language model to obtain a semantic scoring index characterizing the image editing effect of the image editing model; and / or performing a structured semantic evaluation on the original image, the editing instructions, and the image processed by the vertical workflow based on the visual language model to obtain a semantic scoring index characterizing the image editing effect of the vertical workflow.

[0077] Specifically, the original image, the edited image, and the editing instructions are input into the visual language model. Based on the preset semantic scoring evaluation rules, the image editing evaluation task is transformed into a visual question answering task. The semantic scoring evaluation rules include at least the task objective constraints, the editing area constraints, and the evaluation criteria constraints.

[0078] For example, given the editing instruction: "Change the character's clothes to a red formal dress while maintaining the character's identity," the evaluation questions are: "Does the edited clothing color meet the instruction requirements?"; "Does the character's identity remain consistent after editing?"; "Are there any significant changes in other areas besides the target area?". The visual language model performs joint reasoning based on the image content and text instructions, and outputs corresponding evaluation results. Evaluation results may include: whether the editing requirements are met, and a score indicating the degree of satisfaction.

[0079] According to the preset evaluation rules, the output of the visual language model is converted into structured semantic scoring indicators. For example: output instruction compliance: 0-100 points; content retention: 0-100 points; semantic consistency: 0-100 points.

[0080] This step transforms the traditional image quality assessment problem into a "visual language understanding problem." Compared to existing methods that only calculate image feature distances, this method utilizes a visual language model to understand: the editing target, image context, and local semantic relationships. Therefore, it can more accurately identify image quality and avoid erroneous results where generated images appear similar but do not actually complete the instructions.

[0081] In step S104, the multi-dimensional evaluation of the edited image includes: based on the first evaluation rule and first evaluation index determined by the two-level evaluation architecture, combined with the semantic scoring index of the image editing effect of the image editing model and at least one auxiliary evaluation index, to perform a multi-dimensional evaluation of the image processed by the image editing model and obtain multiple corresponding evaluation results; and / or based on the second evaluation rule and second evaluation index determined by the two-level evaluation architecture, combined with the semantic scoring index of the image editing effect of the vertical workflow and at least one auxiliary evaluation index, to perform a multi-dimensional evaluation of the image processed by the vertical workflow and obtain multiple corresponding evaluation results. In step S104, the multi-dimensional evaluation results are obtained by combining auxiliary evaluation indicators and then fused to avoid bias caused by single-index evaluation. Specifically, after obtaining the semantic scoring index, at least one auxiliary evaluation index is further combined to perform a multi-dimensional evaluation of the editing results. Among them, the auxiliary evaluation indicators include, but are not limited to: image similarity index, facial feature similarity index, aesthetic evaluation index, and evaluation indicators set for specific business scenarios.

[0082] Specifically: For image similarity evaluation, visual feature vectors of the original and edited images are extracted, and their similarity in the feature space is calculated to evaluate the preservation of the unedited region or the overall visual structure. For facial feature evaluation, facial regions are detected in the images before and after editing, and facial identity feature vectors are extracted to determine whether the person's identity remains consistent. For aesthetic evaluation, a multimodal evaluation model is used to evaluate the overall visual effect of the image, including composition, color, lighting, sharpness, and artistic expression. Finally, the multiple evaluation indicators obtained from the above different dimensions are input into the fusion calculation module, and a comprehensive evaluation result is obtained according to the preset fusion strategy.

[0083] In step 105, the fusion strategy can employ weighted fusion, rule-based fusion, multi-index constraint fusion, or aggregation based on constraints between evaluation dimensions. The process of fusing and calculating the multiple evaluation results according to the preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model and / or the vertical workflow includes: fusing and calculating the multiple evaluation results of the image processed by the image editing model according to the preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model; and / or fusing and calculating the multiple evaluation results of the image processed by the vertical workflow according to the preset fusion strategy to generate a comprehensive evaluation result corresponding to the vertical workflow.

[0084] In the above embodiments, the visual language model is used to transform the evaluation of image editing effects into a structured evaluation task based on image content understanding and text semantic understanding. In an optional embodiment, the visual language model can be a multimodal large model with visual understanding and natural language reasoning capabilities. The multimodal large model can include, but is not limited to, a visual language model based on the Transformer architecture. Specifically, evaluation question templates can be pre-established according to different types of image editing tasks. The original image, the edited image, and the editing instructions are input into the visual language model, so that the model not only judges the degree of matching between the editing result and the target instructions, but also judges whether abnormal changes occur in non-target areas during the editing process in combination with the original image content. For example, for character dressing-up editing tasks, a question can be constructed to evaluate the preservation of character identity: "Does the edited character still maintain consistency with the original character?"; for background replacement tasks, a question can be constructed to evaluate the rationality of scene changes: "Does the edited background conform to the target description and blend naturally with the character?"; for style transfer tasks, a question can be constructed to evaluate style conformity: "Does the style of the edited image conform to the specified style requirements?" The above method can transform the originally difficult-to-quantify manual evaluation criteria into a structured evaluation problem that can be understood by a visual language model, and further output calculable scoring indicators.

[0085] In the above embodiments, to avoid the inaccuracy of a single evaluation dimension in reflecting the image editing effect, multiple semantic evaluation dimensions can be set for different evaluation goals, and the final semantic score can be obtained based on the constraint relationship between the multiple dimensions. In some embodiments, the semantic evaluation dimensions may include instruction compliance and editing conservatism. Instruction compliance is used to evaluate whether the editing result completes the user-specified modification goal, and editing conservatism is used to evaluate whether non-target areas retain the original image features. For example, if the model correctly modifies a person's clothing but changes the person's identity features at the same time, although it satisfies some editing instructions, the overall editing effect is not ideal; if the model maintains a high degree of consistency with the original image but fails to complete the specified modification task, the editing quality cannot be considered high either. Therefore, in some embodiments, the lowest score among multiple evaluation dimensions can be used as the final score for the corresponding evaluation item, thereby avoiding a high score in a single dimension from masking other key issues and improving the consistency between the evaluation results and the real user experience.

[0086] In the above embodiments, the auxiliary evaluation metrics are used to further evaluate the edited image from different perspectives, and their specific types can be adjusted according to the evaluation object and application scenario. In some embodiments, the auxiliary evaluation metrics may include image similarity evaluation metrics. Specifically, a pre-trained visual feature extraction model can be used to extract visual feature representations of the original image and the edited image respectively, and the similarity between the two feature representations can be calculated to evaluate the degree of preservation of the overall structure, content relationships, or non-edited areas of the images before and after editing. The visual feature extraction model can be a visual model based on image-text joint training, such as the CLIP model, or other models capable of extracting semantic features of images.

[0087] It should be noted that this invention is not limited to using a single image similarity calculation method. In other embodiments, pixel feature comparison, local region feature comparison, depth visual feature distance calculation, or other forms of image consistency evaluation methods may be used according to actual application needs.

[0088] In the above embodiments, when the evaluation object involves the task of editing human images, a human consistency evaluation index can be further set to determine whether the human identity features remain consistent before and after editing. Specifically, face region detection can be performed on the original image and the edited image first to obtain corresponding face region images. Then, feature extraction is performed on the face regions to obtain face feature vectors that can represent human identity information. The similarity between the face feature vectors before and after editing is further calculated, and the consistency of human identity is determined based on the similarity results.

[0089] In some implementations, face detection can be achieved using deep learning-based face detection algorithms, while face feature extraction can be implemented using face recognition networks, such as face feature extraction models based on ArcFace. This approach avoids the problem of inaccurate identification of individuals due to changes in clothing, background, or pose in traditional image similarity evaluation, making the evaluation results more aligned with the actual needs of scenarios such as portrait photography and digital human generation.

[0090] In other embodiments, aesthetic evaluation metrics can be set for image editing tasks that require evaluation of visual aesthetic effects. Specifically, the image to be evaluated can be input into a multimodal aesthetic evaluation model, and the overall visual effect of the image can be evaluated by combining the image content and corresponding text description information. The aesthetic evaluation metrics may include dimensions such as composition rationality, color harmony, naturalness of light and shadow, detail representation ability, and overall visual appeal.

[0091] Compared with traditional methods that rely solely on human aesthetic evaluation, this embodiment uses a model to automatically extract visual features from images and combine them with semantic information for evaluation. This reduces subjective differences in the human evaluation process and improves consistency and efficiency in large-scale model testing.

[0092] In the above implementation, to address the issue of differing evaluation requirements between the basic model and specific business scenarios, the vertical workflow evaluation layer can further reconstruct the evaluation metrics based on specific application scenarios. For example, in the portrait generation scenario, general evaluation metrics can be further refined into metrics such as subject consistency, background consistency, clothing consistency, action consistency, and lighting consistency; in the product image generation scenario, further attention can be paid to metrics such as maintaining the main subject of the product, material realism, and scene integration effect.

[0093] The consistency evaluation of individuals can be further broken down into consistency of identity, clothing, movement, and human anatomy. Specifically, visual models can be used to determine whether the identity of the person in the edited image is maintained, whether the clothing changes meet the editing requirements, whether the person's posture is natural, and whether there are any abnormal deformations in the human anatomy.

[0094] Background consistency evaluation can employ a multi-state evaluation approach, such as categorizing evaluation results into reasonable, pending confirmation, and unreasonable states. A reasonable state indicates that the edited background meets the target requirements and blends naturally with the subject; a pending confirmation state indicates some changes but it's unclear whether they affect the overall effect; and an unreasonable state indicates obvious structural errors, semantic mismatches, or visual conflicts. This multi-state evaluation method avoids the problem of insufficient information expression in traditional binary classification evaluation and improves the accuracy of evaluations in complex scenarios.

[0095] After obtaining the above multi-dimensional evaluation results, the final comprehensive evaluation result can be generated through the fusion calculation module. In some embodiments, the fusion calculation can adopt a weighted fusion method, setting corresponding weights according to the importance of different evaluation results; in other embodiments, a rule-based fusion method can also be adopted, comprehensively judging multiple results according to preset evaluation logic; in other embodiments, a machine learning model can also be used to fuse multiple evaluation results to obtain a comprehensive score that is more in line with human preferences.

[0096] The above technical solutions enable a shift from single image quality evaluation to "semantic understanding - multi-dimensional index analysis - comprehensive evaluation". On the one hand, the use of visual language models improves the ability to understand the relationship between editing instructions and image content, enabling the comprehensive evaluation results to reflect the degree of completion of real tasks. On the other hand, by integrating image similarity, character consistency, aesthetic quality, and business scenario indicators, the evaluation system can cover different application needs, improving the comprehensiveness, objectivity, and scalability of image processing evaluation.

[0097] In the above implementation, in order to obtain an evaluation result that comprehensively reflects the image editing effect, it is necessary to fuse and calculate evaluation indicators from different sources and dimensions. Since different evaluation indicators focus on different objectives, for example, the semantic score output by the visual language model mainly reflects the degree of completion of the editing task, the image similarity index mainly reflects the degree of content preservation, the facial feature similarity index mainly reflects the degree of identity preservation, and the aesthetic evaluation index mainly reflects the quality of visual performance, therefore, a single indicator cannot be simply used as the final evaluation basis.

[0098] In some embodiments, multiple evaluation results can first be categorized according to the evaluation objective to form different evaluation dimensions. For example, evaluation results used to determine whether the editing conforms to the user's instructions can be categorized into the editing consistency dimension, evaluation results used to determine the visual quality of the image can be categorized into the image quality dimension, evaluation results used to determine the preservation of people or subjects can be categorized into the content preservation dimension, and evaluation results used to determine the user's visual preferences can be categorized into the aesthetic evaluation dimension.

[0099] Furthermore, the evaluation results of each dimension are fused and calculated based on their relative importance. In other embodiments, to avoid overestimating other key deficiencies by giving an excessively high score to one evaluation dimension, a constrained fusion strategy can be adopted. For example, in the editorial consistency evaluation, the degree of instruction compliance and the degree of editorial conservatism are treated as two independent evaluation results, and a weighted fusion strategy is used to calculate the final score of each dimension. If either evaluation result has a low score, it indicates a significant quality defect in the editing result; by adjusting the weights, a higher weight can be assigned to the weakest evaluation result, amplifying the constraining effect of low-scoring items on the overall result, thus obtaining a consistency score that reflects the overall editing quality.

[0100] By employing the aforementioned fusion approach, the bias inherent in traditional evaluation methods due to single-result biases can be avoided. For example, when using only image similarity evaluation, the model might achieve a high score by reducing image variations, but in reality, it might not have completed the user-specified editing task; similarly, when using only semantic matching evaluation, the model might complete the text description, but this could lead to abnormal changes in the person's identity, structure, or background. Therefore, by leveraging the mutual constraints among multi-dimensional evaluation metrics, the actual capabilities of the image editing model can be more accurately reflected.

[0101] In other implementations, the fusion calculation process can be dynamically adjusted according to different business scenarios. For example, in portrait generation scenarios, the weights of portrait consistency and aesthetic quality indicators can be increased; in product image generation scenarios, the weights of subject preservation and material authenticity indicators can be increased; and in artistic creation scenarios, the weights of style matching and visual performance indicators can be increased. Through scenario-based weight adjustments, the same evaluation framework can be applied to different types of image editing tasks.

[0102] In the above implementation, the vertical workflow evaluation layer is used to address the problem that the evaluation results of the basic model cannot directly reflect the actual business application effect. Since image editing tasks in actual applications are usually not completed by a single model, but consist of multiple processing stages such as image understanding, content editing, detail enhancement, and style adjustment, it is necessary to establish an evaluation system for the complete workflow.

[0103] In some embodiments, the vertical workflow evaluation layer can establish corresponding evaluation rules based on the target business process. That is, it can establish corresponding second evaluation rules and one or more second evaluation indicators for specific application scenarios, and convert general evaluation indicators into fine-grained indicators that meet business needs. For example, in the portrait generation process, a portrait consistency evaluation indicator can be established for the subject, a background consistency evaluation indicator can be established for scene replacement, and an image quality evaluation indicator can be established for the final image presentation effect.

[0104] The character consistency evaluation can be further analyzed through multiple sub-indicators, including character identity consistency, clothing consistency, posture consistency, and the rationality of human body structure. Specifically, visual language models can be used to determine whether the character in the edited image still corresponds to the original character; object detection and attribute recognition models can be used to determine whether the clothing meets the editing requirements; posture estimation models can be used to determine whether the character's movements are natural; and human body structure analysis can be used to determine whether there are problems such as limb abnormalities or proportional distortions.

[0105] Background consistency evaluation can be achieved by comparing the target background description with the actual background content in the edited image. For example, when a user requests that a person be placed in a seaside scene, the evaluation system can determine whether the edited result contains a seaside environment that meets the requirements, and further determine whether the spatial relationship, lighting relationship, and color relationship between the person and the background are harmonious.

[0106] Through the above methods, the vertical workflow evaluation layer can transform the originally abstract concept of "whether the image effect is good or not" into multiple calculable and comparable business evaluation indicators, making the evaluation results closer to actual application needs.

[0107] like Figure 2As shown, another embodiment provides a computer-readable storage medium 200. The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the multi-dimensional automated evaluation method for image processing described in any of the above embodiments.

[0108] The computer-readable storage medium can be volatile memory or non-volatile memory, or a combination of both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage.

[0109] The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synclink dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The computer-readable storage media described in the embodiments of the present invention are intended to include these and any other suitable types of memory.

[0110] In some embodiments, the processor may be implemented by software, hardware, firmware, or a combination thereof, and may be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor, thereby enabling the processor to execute some or all of the steps or any combination of the steps in the multi-dimensional automated evaluation method for image processing described in the various embodiments of this application.

[0111] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.

Claims

1. A multi-dimensional automated evaluation method for image processing, characterized in that, Includes the following steps: A two-tier evaluation architecture is established, comprising a basic model evaluation layer and a vertical workflow evaluation layer. The basic model evaluation layer is used to establish corresponding first evaluation rules and one or more first evaluation indicators for the general editing capabilities of image editing models, and selects one image editing model as the base of the vertical workflow based on the evaluation results. The vertical workflow evaluation layer is used to establish corresponding second evaluation rules and one or more second evaluation indicators for specific application scenarios. Acquire the original image, editing instructions, and edited image corresponding to the image editing task to be evaluated. The edited image includes the image obtained by the image editing model and / or the image obtained by the vertical workflow. Based on a visual language model, a structured semantic evaluation is performed on the original image, editing instructions, and the edited image to obtain a semantic scoring index for characterizing the image editing effect. Based on the evaluation rules and evaluation indicators determined by the two-level evaluation architecture, combined with the semantic scoring indicators and at least one auxiliary evaluation indicator, the edited image is evaluated in multiple dimensions to obtain multiple corresponding evaluation results. The auxiliary evaluation indicators include one or more of the following: image similarity indicators, facial feature similarity indicators, aesthetic evaluation indicators, and business evaluation indicators set for specific application scenarios. The multiple evaluation results are fused and calculated according to a preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model and / or the vertical workflow.

2. The multi-dimensional automated evaluation method for image processing according to claim 1, characterized in that, The structured semantic evaluation of the original image, editing instructions, and edited image based on the visual language model yields a semantic scoring index for characterizing the image editing effect, including: The original image, the edited image, and the corresponding editing instructions are input into the visual language model; The corresponding visual question-answering task is generated according to the preset semantic scoring and evaluation rules, wherein the semantic scoring and evaluation rules include at least the task objective constraints, the editing area constraints, and the evaluation criteria constraints. The visual language model is used to reason about the visual question-answering task to obtain a semantic scoring index that characterizes the degree of matching between the image editing result and the editing target.

3. The multi-dimensional automated evaluation method for image processing according to claim 2, characterized in that, The semantic scoring metric includes at least two evaluation dimensions, wherein the evaluation dimensions include: Instruction compliance metrics are used to evaluate whether the editing results meet the requirements of the editing instructions. And an editing conservatism metric for evaluating whether non-target regions retain the original image features; The semantic score result is determined based on the lowest score among the multiple evaluation dimensions.

4. The multi-dimensional automated evaluation method for image processing according to claim 1, characterized in that, The phrase "based on a visual language model, performing structured semantic evaluation on the original image, editing instructions, and edited image to obtain a semantic scoring index for characterizing the image editing effect" includes: Based on the visual language model, a structured semantic evaluation is performed on the original image, editing instructions, and the image processed by the image editing model to obtain a semantic scoring index that characterizes the image editing effect of the image editing model. And / or based on a visual language model, perform structured semantic evaluation on the original image, editing instructions, and images processed by the vertical workflow to obtain a semantic scoring index that characterizes the effect of the vertical workflow on image editing.

5. The multi-dimensional automated evaluation method for image processing according to claim 4, characterized in that, The phrase "based on the evaluation rules and evaluation indicators determined by the two-level evaluation architecture, combined with the semantic scoring indicator and at least one auxiliary evaluation indicator, to perform multi-dimensional evaluation of the edited image and obtain multiple corresponding evaluation results" includes: Based on the first evaluation rule and first evaluation index determined by the two-level evaluation architecture, combined with the semantic scoring index of the image editing effect of the image editing model and at least one auxiliary evaluation index, the image processed by the image editing model is evaluated in multiple dimensions to obtain multiple corresponding evaluation results. And / or based on the second evaluation rules and second evaluation indicators determined by the two-level evaluation architecture, combined with the semantic scoring indicators of the image editing effect of the vertical workflow and at least one auxiliary evaluation indicator, the images processed by the vertical workflow are evaluated in multiple dimensions to obtain multiple corresponding evaluation results.

6. The multi-dimensional automated evaluation method for image processing according to claim 5, characterized in that, The phrase "fusing and calculating the multiple evaluation results according to a preset fusion strategy to generate a comprehensive evaluation result corresponding to the image editing model and / or the vertical workflow" includes: Based on a preset fusion strategy, multiple evaluation results of the image obtained from the image editing model are fused and calculated to generate a comprehensive evaluation result corresponding to the image editing model. And / or perform fusion calculations on multiple evaluation results of the images obtained from the vertical workflow processing according to a preset fusion strategy to generate a comprehensive evaluation result corresponding to the vertical workflow.

7. The multi-dimensional automated evaluation method for image processing according to claim 1, characterized in that, The image similarity index is obtained by extracting the visual feature vectors of the original image and the edited image respectively, and calculating the degree of similarity between the visual feature vectors; The visual feature vectors are obtained through a pre-trained visual coding model, and the similarity includes evaluation results calculated based on vector distance, cosine similarity, or other feature space distances. The facial feature similarity index includes: Face region detection is performed on the original image and the edited image; Obtain standardized face region images based on the detection results; Extract the facial feature vector corresponding to the standardized facial region image; Calculate the similarity between the two facial feature vectors to evaluate the consistency of the person's identity features before and after editing; The aesthetic evaluation index is obtained by inputting the image to be evaluated, the corresponding text description information, and the preset evaluation dimensions into a multimodal evaluation model; wherein, the evaluation dimensions include at least one of the following: overall image style, color performance, lighting effect, clarity, detail quality, and artistic quality of the image.

8. The multi-dimensional automated evaluation method for image processing according to claim 1, characterized in that, The vertical workflow evaluation layer performs task-decoupled reconstruction of the first evaluation index according to the specific application scenario to obtain a second evaluation index that is suitable for the target business scenario. In the context of character image editing, the second evaluation metric includes: character consistency metric, background consistency metric, image quality metric, and lighting consistency metric; the character consistency metric is used to evaluate the retention of one or more of the following: character identity, clothing, action posture, props, and human body structure.

9. The multi-dimensional automated evaluation method for image processing according to claim 8, characterized in that, The background consistency index is evaluated using a multi-level judgment method; The multi-level determination method includes: a first state indicating that the background is kept reasonable or can be enhanced; a second state indicating that there are slight color or light and shadow changes but the degree of influence cannot be determined; and a third state indicating that there is structural distortion. The background consistency evaluation result is determined based on the first state, the second state, and the third state.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-dimensional automated evaluation method for image processing as described in any one of claims 1-8.

Images