Automatic test method and device for image editing task and electronic equipment

Abstract

The invention discloses an automatic test method and device for an image editing task and electronic equipment, and relates to the technical field of computer vision, and the method comprises the steps: obtaining an input image and a target region mask, and enabling the target region mask to be located in the input image; generating a target editing instruction set based on the input image and the target area mask; editing the input image and the target area mask according to at least one editing instruction in the target editing instruction set to obtain an edited image; and performing dual quality evaluation on the editing area and the non-editing area to obtain a test result. Through dual quality evaluation, the problem that quality evaluation is incomplete due to the fact that a traditional evaluation method neglects regional differences is solved, the whole processes of preparation, execution and verification of the image editing test are connected into an automatic assembly line in series, manual intervention and waiting in all links are eliminated, and the test efficiency is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to automated testing methods, apparatus and electronic devices for image editing tasks. Background Technology

[0002] With the rapid development of generative artificial intelligence, significant progress has been made in image generation technology based on natural language instructions. Currently, the research focus is gradually shifting from "ability to generate" to "controllability and editability," enabling fine-grained local editing by combining natural language instructions and region masks without redrawing the entire image.

[0003] However, current image generation technologies still suffer from a series of instabilities and distortions, such as artifacts at the boundaries of the editing area and background, leakage of content outside the mask, color / material / geometric deviations, conflicts between global style and local consistency, and insufficient semantic alignment of instructions. Corresponding testing and evaluation also face challenges: a lack of unified and reproducible benchmarks and annotation standards, difficulty in breaking down evaluation dimensions, insufficient correlation between objective indicators and subjective scores, high labor costs, limited coverage and scale of evaluation, and insufficient repeatability and cross-model comparability of results.

[0004] Therefore, improving the efficiency and comprehensiveness of automated testing for image editing tasks is a problem that urgently needs to be solved. Summary of the Invention

[0005] The main objective of this application is to provide an automated testing method, apparatus, and electronic device for image editing tasks, aiming to solve the technical problem of how to improve the efficiency and comprehensiveness of automated testing for image editing tasks.

[0006] To achieve the above objectives, this application proposes an automated testing method for image editing tasks, the method comprising: Obtain an input image and a target region mask, wherein the target region mask is located in the input image; Based on the input image and the target region mask, a set of target editing instructions is generated; The input image and the target region mask are edited according to at least one editing instruction from the target editing instruction set to obtain the edited image; Based on the edited image, a dual quality assessment is performed on the edited and unedited areas to obtain the test results.

[0007] In one embodiment, the step of acquiring an input image and a target region mask, wherein the target region mask is located in the input image, includes: Perform target detection on the input image to obtain at least one target object and its corresponding location information; A binary mask and a blur mask are generated based on the target object and its corresponding location information, wherein the blur mask is used to preserve the clarity of the target area; The target region mask is determined based on the binary mask and the blurred mask.

[0008] In one embodiment, the step of generating a set of target editing instructions based on the input image and the target region mask includes: The input image and the target region mask are input into a large language model, and a set of candidate editing instructions is generated by combining them with a preset task template; The candidate editing instruction set is filtered based on semantic similarity to generate a target editing instruction set, which includes representative instructions and diverse instructions.

[0009] In one embodiment, the step of filtering the candidate editing instruction set based on semantic similarity to generate a target editing instruction set, the target editing instruction set including representative instructions and diverse instructions, includes: The candidate editing instruction set is clustered based on the target object type to obtain clustered editing instructions; Based on the clustered editing instructions, the editing instructions with the most keywords in each cluster are selected as representative instructions, and an initial set and a candidate set are generated. The initial set includes multiple representative instructions, and the candidate set includes multiple candidate instructions. The candidate instructions are the remaining editing instructions in each cluster. Based on each candidate instruction in the candidate set, the semantic similarity between the current candidate instruction and each representative instruction is calculated according to the semantic embedding vector, and the similarity set of the current candidate instruction is obtained. Based on the candidate set, the maximum value in the similarity set of the current candidate instruction is selected to obtain multiple target candidate values; The minimum value is selected from the plurality of target candidate values ​​to obtain the target candidate instruction; The target candidate instruction is removed from the candidate set and added to the initial set; The process involves returning to the step of calculating the semantic similarity between the current candidate instruction and each representative instruction based on the semantic embedding vector, and obtaining the similarity set of the current candidate instruction. When the number of editing instructions in the initial set reaches a preset number, the target editing instruction set is obtained.

[0010] In one embodiment, the step of editing the input image and the target region mask according to at least one editing instruction from the target editing instruction set to obtain an edited image includes: Based on the preset size of the editing model, the input image and the target region mask are resized to obtain the adjusted image; The adjusted image is edited based on at least one editing instruction from the target editing instruction set to generate an initial edited image; The initial edited image is restored to its original size to obtain the edited image.

[0011] In one embodiment, the step of performing dual quality assessment on the edited and unedited regions based on the edited image to obtain test results includes: The quality of the edited image is evaluated by verifying the consistency between the edited area and the current editing instruction, and a first verification result is obtained. The second verification result was obtained by verifying the stability of the non-editable region; The test results are determined based on the first verification result and the second verification result.

[0012] In one embodiment, the step of performing a quality assessment on the edited image and obtaining a first verification result by verifying the consistency between the edited region and the editing instructions includes: Based on the current editing command, the expected attributes are parsed to generate a set of structured verification questions; The visual question-answering model is used to answer the set of verification questions on the edited image to obtain the answer results; The answer is matched with the expected attribute to obtain the first verification result.

[0013] In one embodiment, the step of obtaining a second verification result by performing stability verification on the non-editable region includes: Based on the instance segmentation mask, at least one non-edited region is determined from the edited image; The non-editable region features are obtained by extracting features from the non-editable region using a pre-trained visual encoder. Calculate the cosine similarity between the features of the non-edited region and the corresponding region features in the input image; The cosine similarity is converted into a rate of change, and a preset stability threshold is compared with the rate of change to obtain a comparison result. Based on the comparison results, it is determined whether there is any out-of-bounds modification in the non-editable area, thus obtaining the second verification result.

[0014] Furthermore, to achieve the above objectives, this application also proposes an automated testing device for image editing tasks, the automated testing device for image editing tasks comprising: An image acquisition module is used to acquire an input image and a target region mask, wherein the target region mask is located in the input image; The instruction generation module is used to generate a set of target editing instructions based on the input image and the target region mask; The image editing module is used to edit the input image and the target region mask according to at least one editing instruction from the target editing instruction set to obtain the edited image; The quality evaluation module is used to perform dual quality evaluation on the edited and non-edited areas based on the edited image to obtain test results.

[0015] In addition, to achieve the above objectives, this application also proposes an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of an automated testing method for the image editing task as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: The system acquires the input image and a target region mask, with the target region mask located within the input image. By acquiring both the input image and the target region mask, it solves the problem of inaccurate region specification in traditional image editing, achieving precise positioning of the editing target and laying the foundation for subsequent automated processing.

[0017] Based on the input image and the target region mask, a set of target editing instructions is generated. This method solves the problems of incompleteness, low efficiency, and poor consistency in manually designed editing instructions, achieving intelligent, comprehensive, and automated instruction generation, thus improving editing efficiency.

[0018] The input image and the target region mask are edited according to at least one editing instruction from the target editing instruction set to obtain the edited image. By editing the image according to the editing instructions, the problems of time-consuming and error-prone manual operation are solved, enabling fast and accurate image modification and ensuring the smoothness of the editing process.

[0019] Based on the edited image, a dual quality assessment is performed on both the edited and non-edited regions to obtain test results. This dual quality assessment addresses the problem of incomplete overall quality evaluation caused by traditional methods neglecting regional differences, enabling collaborative monitoring of both edited and non-edited regions and improving the reliability and consistency of editing results. By automating the entire process of image editing testing—preparation, execution, and verification—a pipeline is created, achieving highly efficient, scalable, and high-precision testing of image editing models without significant manual intervention. It is applicable to consistency comparisons and regression evaluations across models and datasets, eliminating manual intervention and waiting in each stage and significantly improving testing efficiency. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the first embodiment of the automated testing method for image editing tasks in this application; Figure 2 This is a flowchart illustrating the automated generation of test cases in this application; Figure 3 This is a flowchart illustrating the quality assessment process for the editing results in this application; Figure 4 This is a visual representation of the automated test case generation results in this application; Figure 5 This is a visual representation of the quality assessment results of the editing results in this application; Figure 6 This is a schematic diagram of the module structure of the automated testing device for image editing tasks in this application.

[0023] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0025] This application provides an automated testing method for image editing tasks, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the automated testing method for image editing tasks in this application.

[0026] In this embodiment, the automated testing method for image editing tasks includes steps S10 to S40: Step S10: Obtain the input image and the target region mask, wherein the target region mask is located in the input image.

[0027] It should be noted that the input image can be understood as the original image data to be edited, serving as the starting point for the entire testing process. The target region mask is a binary image located within the input image, used to precisely identify the specific region that needs to be edited. The mask values ​​are used to distinguish the target region from the background, ensuring the targeted nature of the editing operation.

[0028] Step S20: Generate a set of target editing instructions based on the input image and the target region mask.

[0029] It should be noted that the target editing instruction set is a set of editing instructions automatically generated based on the input image and the target region mask. These instructions define specific editing operations for the target region, such as color adjustment and filter application, forming an executable command sequence. It can be understood that the target editing instruction set includes multiple pre-processed editing instructions. Pre-processing may include operations such as deduplication and cleansing.

[0030] Step S30: Edit the input image and the target region mask according to at least one editing instruction in the target editing instruction set to obtain the edited image.

[0031] It should be noted that the input image and the target region mask can be edited based on one editing instruction from the target editing instruction set, or multiple editing instructions from the target editing instruction set can be used to edit the input image and the target region mask.

[0032] Step S40: Based on the edited image, perform dual quality assessment on the edited and non-edited areas to obtain the test results.

[0033] It should be noted that the edited image is the output image after processing by the target editing instruction set. The target region has been modified according to the editing instructions, while the non-target region remains unchanged. The edited region is the part of the image corresponding to the target region mask, i.e., the area directly modified by the editing operation. The non-edited region is the part of the image other than the edited region, which remains unchanged during the editing process and retains its original state. Dual quality assessment can be understood as a quality evaluation method that independently evaluates the edited and non-edited regions to check the editing effect and the integrity of the non-edited regions, ensuring overall quality. The test results are the quantitative or qualitative output obtained after dual quality assessment, used to evaluate the success or failure and quality level of the editing operation.

[0034] In this embodiment, by acquiring the input image and the target region mask, the problem of inaccurate region specification in traditional image editing is solved, achieving precise positioning of the editing target and laying the foundation for subsequent automated processing. By generating a set of target editing instructions based on the input image and the target region mask, the problems of incomplete, inefficient, and inconsistent manual design of editing instructions are solved, achieving intelligent, comprehensive, and automated instruction generation, thus improving editing efficiency. Editing the image according to the editing instructions solves the problems of time-consuming and error-prone manual operation, achieving fast and accurate image modification and ensuring the smoothness of the editing process. Finally, through dual quality evaluation, the problem of incomplete overall quality assessment caused by ignoring regional differences in traditional evaluation methods is solved, achieving collaborative monitoring of the edited and non-edited regions, improving the reliability and consistency of the editing results. By connecting the entire process of image editing test preparation, execution, and verification into an automated pipeline, high-efficiency, scalable, and high-precision testing of the image editing model is achieved without extensive manual intervention. It is applicable to consistency comparison and regression evaluation across models and datasets, eliminating manual intervention and waiting in each stage, and achieving a significant improvement in testing efficiency.

[0035] In the second embodiment, step S10 includes: Step S101: Perform target detection on the input image to obtain at least one target object and its corresponding location information.

[0036] It should be noted that the target object refers to the specific information or object that needs to be identified in the input image. By performing target detection on the input image, specific objects in the image can be identified and their location information, such as bounding boxes, can be determined.

[0037] For example, taking the MSCOCO dataset as an example, the MSCOCO dataset is collected from real-world scenes and covers 80 common object categories. 5000 images from this dataset are selected as the validation set as the baseline data; for each input image... Annotation is performed; for example, annotations include instance segmentation polygons or RLEs, class IDs, bounding boxes, etc. An image can be selected. The target object is obtained through object detection. .

[0038] Step S102: Generate a binary mask and a blur mask based on the target object and its corresponding location information, wherein the blur mask is used to preserve the clarity of the target area.

[0039] It should be noted that a binary mask contains only two pixel values, such as 0 and 1, and is used to strictly divide an image into target and background regions. For example, the target region corresponds to a pixel value of 1, and the background region corresponds to a pixel value of 0. A blur mask, on the other hand, is a mask in which the pixel value changes continuously between 0 and 1. It is used to represent the transition zone at the edge of the target region. Its function is to make the transition between the target region and the background natural and avoid harsh boundaries.

[0040] Step S103: Determine the target region mask based on the binary mask and the blurred mask.

[0041] It should be noted that the target region mask represents a final mask jointly determined by a binary mask and a blur mask. It preserves the precise shape of the core target region and includes information on the smooth transition of the edges.

[0042] For example, based on the target object Generate the corresponding black and white binary mask. Based on this, the area outside the mask is blurred to obtain a blurred image where only the target area is clear. .

[0043] In this embodiment, target detection technology is used to automatically identify target objects and their locations, solving the problems of low efficiency and inaccuracy in manual target selection. This automates target area localization and lays a high-efficiency foundation for subsequent processing. By generating binary and blurred masks, the problem of harsh and unnatural edges in the editing area caused by a single hard boundary mask is solved. Preprocessing to smooth the transition of the target area edges improves the visual quality of the editing effect. By combining two types of masks to determine the target area mask, the problem of inaccurate masks that may be caused by relying on a single information source is solved. This achieves optimized integration of mask information, thereby ensuring the accuracy and efficiency of subsequent editing command generation and execution.

[0044] In the third embodiment, step S20 includes: Step S201: Input the input image and target region mask into the large language model, and generate a set of candidate editing instructions by combining them with a preset task template.

[0045] It should be noted that, based on the capabilities of the Large Language Model (LLM) in natural language understanding and generation, editing instructions are automatically constructed by combining image context information. The preset task template can include multimodal information such as task type and constraints, image target objects, and contextual descriptions, thereby generating diverse and semantically clear editing instruction text. The image target objects can be obtained through detection or annotation, and the contextual descriptions can include image captions. The candidate editing instruction set refers to a collection of image editing instructions initially generated by the Large Language Model; its content is extensive and contains redundancy.

[0046] For example, by setting relevant constraints based on different editing instruction types, the target object... Its constraints are merged into a text description. and the image to They are all input into the LLM to generate different types of editing instructions. For example, templates and rules can be designed for 10 preset types of editing instructions. A hybrid strategy of "template constraints + large language model" can be used to automatically generate diverse and executable editing instructions. Common fields are defined for all types, including... , and optional , This provides a reference for subsequent error detection. To avoid structural duplication and improve coverage, under each task category, the same image is analyzed. object Ten semantically unique candidate instructions are generated as a candidate set for subsequent screening and experimentation. For example, the types of editing instructions can be found in Table 1.

[0047] Table 1 Step S202: Based on semantic similarity, the candidate editing instruction set is filtered to generate the target editing instruction set, which includes representative instructions and diverse instructions.

[0048] It should be noted that by introducing linguistic diversity constraints and task construction strategies, multiple semantically distinct and executable commands are generated for each image, thus forming a target set of editing commands. The candidate set of editing commands is filtered by measuring the semantic similarity between two text segments to obtain the target editing commands. Representative commands refer to those that cover the most common or important editing needs. Diverse commands refer to those that reflect different editing directions and avoid homogenization of test cases.

[0049] For example, for each type of editing instruction, you can first select all instructions. Filter representative instructions Then, from the remaining candidates, the instruction least similar to or furthest from the selected set is chosen, thus balancing instruction quality and diversity. The final set of target editing instructions will be used for subsequent image editing. The above process can be referenced. Figure 2 .

[0050] In this embodiment, the input image and target region mask are input into a large language model and combined with a preset task template to generate a candidate editing instruction set. This solves the problems of inefficiency and limited creativity inherent in traditional methods that rely entirely on manual design of test instructions. It enables large-scale, automated, and rapid generation of initial test instructions, greatly improving the speed of test case construction. Furthermore, the candidate instruction set is filtered based on semantic similarity to generate a target editing instruction set containing representative and diverse instructions. This solves the problems of instruction redundancy, repetition, or focus dispersion that may occur when directly using the original output of the large language model. It achieves automated optimization and condensation of the initial instruction set, ensuring that the test instruction set fully covers the core scenario while avoiding unnecessary repeated testing. This improves the efficiency of instruction generation while also ensuring the efficiency of subsequent test execution.

[0051] In one implementation, step S202 includes: clustering the candidate editing instruction set based on the target object type to obtain clustered editing instructions; selecting the editing instruction with the most keywords in each cluster as a representative instruction based on the clustered editing instructions, and generating an initial set and a candidate set, wherein the initial set includes multiple representative instructions and the candidate set includes multiple candidate instructions, and the candidate instructions are the remaining editing instructions in each cluster; calculating the semantic similarity between the current candidate instruction and each representative instruction based on the semantic embedding vector for each candidate instruction in the candidate set to obtain a similarity set of the current candidate instruction; selecting the maximum value in the similarity set of the current candidate instruction based on the candidate set to obtain multiple target candidate values; selecting the minimum value from the multiple target candidate values ​​to obtain a target candidate instruction; deleting the target candidate instruction from the candidate set and adding it to the initial set; returning to the step of calculating the semantic similarity between the current candidate instruction and each representative instruction based on the semantic embedding vector for each candidate instruction in the candidate set to obtain a similarity set of the current candidate instruction; when the number of editing instructions in the initial set reaches a preset number, a target editing instruction set is obtained.

[0052] It should be noted that candidate instructions can be grouped according to the target object type, so that instructions of the same type are grouped together in the same cluster. After clustering, each cluster contains a set of semantically related instructions. The representative instruction is the instruction with the most keywords selected from each cluster, representing the core editing intent of that cluster. The initial set contains representative instructions selected from each cluster, serving as the basis for the target instruction set. The remaining instructions in each cluster, excluding the representative instructions, are defined as candidate instructions and temporarily stored in the candidate set. The semantic embedding vector is a representation that maps text instructions to numerical vectors, used to quantify semantic information. Semantic similarity is the degree of semantic similarity between two instructions calculated based on the embedding vector. The similarity set is the set of values ​​obtained after calculating the semantic similarity between each candidate instruction and all representative instructions in the initial set. The target candidate value can be understood as the maximum value taken from its similarity set for each candidate instruction, representing the maximum similarity between that instruction and the current representative instruction set. The target candidate instruction is the instruction selected from the candidate set with the smallest target candidate value, i.e., the least similar to the existing representative instructions. The preset quantity represents a pre-set threshold for the size of the target instruction set. The target editing instruction set is the final optimized instruction set, which includes representative and diverse instructions.

[0053] For example, under each editing instruction category, first select all candidate instruction sets. according to Perform clustering, This corresponds to MSCOCO category 80. For each cluster of instructions, after removing stop words, the instruction with the highest score based on keyword count is selected as... Ensure that one entry is retained for each category, covering 80 categories. After obtaining the categories... Then, the remaining instructions are used as a candidate set for diversity selection. Specifically, the embedding vector of each instruction is calculated to measure semantic similarity, and two instructions... , The similarity is taken in cosine form. Distance is defined as According to various categories The union of the initial set is the set of the union of the initial set. A Max-Min greedy algorithm can be used to iteratively select instructions from the candidate set that are least similar to the selected set and add them one at a time. The retention rule is as follows: when the number of instructions in a certain category... When the value is ≤5, retain 50%, that is, round up; when If the value is greater than 5, 10% will be retained, meaning at least one instruction will be kept. The final set of merged instructions will be the target editing instruction set, which will then serve as input for subsequent image editing.

[0054] In this implementation, the candidate editing instruction set is clustered based on the target object type, solving the problem of low processing efficiency caused by the initial disorganized instruction set. This achieves automated preliminary classification of instructions, laying an efficient foundation for subsequent screening. By selecting the instruction with the most keywords in each cluster as representative instructions and generating an initial set and candidate set, the time-consuming and subjective problem of manually selecting representative instructions is solved, enabling rapid identification of core instructions and improving the starting efficiency of instruction set construction. Semantic similarity calculation between candidate instructions and representative instructions based on semantic embedding vectors overcomes the bottleneck of traditional methods being unable to quantify semantic differences, providing data support for efficient screening. By selecting the maximum similarity value of each candidate instruction as the target candidate value, and further selecting the instruction corresponding to the minimum target candidate value as the target candidate instruction, automated optimization of diverse instructions is achieved, avoiding homogenization of test cases. By iteratively adding target candidate instructions to the initial set until a preset number is reached, the problem of manual intervention in instruction set size control is solved, achieving automated optimization of instruction set size and ensuring the efficiency and completeness of the testing process.

[0055] In the fourth embodiment, step S30 includes: Step S301: Adjust the size of the input image and the target region mask according to the preset size of the editing model to obtain the adjusted image.

[0056] It should be noted that the editing model refers to a pre-trained artificial intelligence model used to perform image editing tasks; in this embodiment, it refers to the image editing model under test. For example, the input image and the target region mask can be scaled proportionally according to the preset size of the editing model to obtain a scaled image and mask. The scaled image and mask are then centered on a canvas of a specified size, and blank areas are filled to obtain an adjusted image. Simultaneously, scaling parameter information and fill position information are generated, which can be used for image restoration.

[0057] Specifically, taking Stable Diffusion's inpainting as an example, Stable Diffusion is an image editing model, and inpainting is a local image editing function within Stable Diffusion. Input image It is an RGB three-channel color image with dimensions (3, w, h). Let the longer side... short side : When L < 512, scale up proportionally; when Scaling down or maintaining the same size as before Get the scaled size Paste the scaled result in the center of a 512×512 canvas and fill in the margins. . Scaled Paste to Position, fill blank areas with white; apply the scaled black and white mask. Paste the image to the same position, fill the blank areas with black, and obtain an image and mask aligned to (512, 512) with the model input. After performing inpainting, perform reverse cropping to restore the original image. (Horizontal diagram, with the longer side being the width), then from Start of the process Otherwise (including) (Vertical / Square image), from the top left corner Cut Therefore, the output size is... The result maintains the same aspect ratio as the original image. As can be seen from the above-described proportional scaling, directional edge patching, and reverse cropping processes, this embodiment has no special requirements for the original image size and possesses universality and portability.

[0058] Step S302: Edit the image based on at least one editing instruction from the target editing instruction set to generate an initial edited image.

[0059] It should be noted that the initial edited image can be understood as the intermediate result image obtained after applying editing commands to the resized image.

[0060] Step S303: Restore the initial edited image to its original size to obtain the edited image.

[0061] It should be noted that the original size refers to the initial size of the input image, and the edited image can be understood as the final output image that has been edited and restored to the original size.

[0062] In this embodiment, the input image and target region mask are resized according to the preset size of the editing model. This solves the problem of processing failure or inefficiency caused by the mismatch between the original image size and the model input requirements, achieving automated adaptation of image preprocessing and improving the startup efficiency of the editing workflow. By editing the adjusted image based on the target editing instruction set, efficient execution of editing operations at standardized sizes is achieved, reducing computational overhead. Finally, by restoring the initial edited image to its original size, automated size restoration of the editing result is achieved, ensuring the consistency of output efficiency and thus improving the overall pipeline efficiency of image editing testing.

[0063] In the fifth embodiment, step S40 includes: Step S401: Perform a quality assessment on the edited image. By verifying the consistency between the edited area and the current editing instruction, a first verification result is obtained.

[0064] It should be noted that the first verification result is the result obtained through consistency verification, indicating whether the edited area meets the instruction requirements. Step S401 includes: parsing the expected attributes based on the current editing instruction to generate a structured verification question set; using a visual question answering model to answer the verification question set on the edited image to obtain the answer result; and matching the answer result with the expected attributes to obtain the first verification result.

[0065] Here, the expected attributes are the attributes that the image should possess, such as color and texture, parsed from the editing instructions. The structured verification question set represents a set of structured questions generated based on the expected attributes, used for systematically verifying the image. The visual question answering model can be understood as an artificial intelligence model that can automatically answer questions based on image content. The answer result represents the visual question answering model's response to the verification question set. For example, the steps of generating the structured verification question set based on parsing the expected attributes from the current editing instructions may include: jointly inputting the editing instructions and the original image description into a large language model; driving the large language model to automatically generate judgment-type questions and single-choice questions, covering at least one dimension of existence, quantity, attribute, orientation, and relative relationship.

[0066] For example, the area of ​​the mask region in the image can be calculated, and filtering can be performed based on a preset area threshold. Specifically, let the set of mask pixels be denoted as... The total area of ​​the image is Mask ratio Set area threshold .when Samples are considered valid and proceed to further processing if they are not; otherwise, they are skipped. Based on the type of editing instruction (e.g., adding / deleting / replacing targets, color / shape / material changes, etc.), corresponding templates and rules are established. The editing instructions and the original image description (prompt) are used as joint input to drive a Large Language Model (LLM) to automatically generate a structured validation question set, covering both judgment and multiple-choice questions. Judgment questions include yes / no options, while multiple-choice questions can be set to multiple selections. For example, judgment questions can be set to Yes / No: such as "Does the target exist?", "Is it located in a specified position / relative relationship?", "Do the attributes match?"; multiple-choice questions can include: "Quantity / Color / Shape / Orientation / Posture", etc.

[0067] Step S402: By performing stability verification on the non-editable region, a second verification result is obtained.

[0068] It should be noted that the second verification result is the result obtained through stability verification, indicating whether the non-edited region is stable. Step S402 includes: determining at least one non-edited region from the edited image based on an instance segmentation mask; extracting features from the non-edited region using a pre-trained visual encoder to obtain non-edited region features; calculating the cosine similarity between the non-edited region features and the corresponding region features in the input image; converting the cosine similarity into a rate of change, and comparing a preset stability threshold with the rate of change to obtain a comparison result; determining whether the non-edited region has been modified beyond its bounds based on the comparison result to obtain the second verification result.

[0069] It should be noted that the instance segmentation mask is used to accurately identify each object instance in the image, thereby distinguishing non-editable regions. Out-of-bounds modification refers to the situation where a non-editable region is accidentally modified. The rate of change is compared with a preset stability threshold; if the rate of change exceeds the preset stability threshold, the non-editable region is determined to have undergone out-of-bounds modification.

[0070] For example, for each generated image, we can first analyze the non-edited regions in its corresponding original image, i.e. Does the region undergo unexpected changes? The rate of change can be defined. The stability threshold is set as a percentage. If there exists any make If a significant change occurs outside the mask area, i.e., an out-of-bounds modification, the sample is directly marked as a failure. For samples with an area below the threshold... It does not participate in the judgment to suppress noise and boundary effects.

[0071] Step S403: Determine the test result based on the first verification result and the second verification result.

[0072] It should be noted that the test result is a comprehensive conclusion. The judgment rule is that the final test result is "qualified" only when both the first and second verification results are "qualified"; otherwise, if any one of the verification results is "unqualified", the final test result is "unqualified".

[0073] For example, similarity can be calculated in the out-of-mask region based on a fixed encoder, such as CLIP (Contrastive Language–Image Pre-training), and the invariance of the non-edited region can be determined using a rate-of-change threshold. Visual Question Answering (VQA) models or detection-segmentation models can then be used to refine the output image. The above answers are matched item by item with the expected facts to determine the consistency of the instructions in the editing area. The evaluation process is referenced. Figure 3The test is considered "successful" only if both conditions are met simultaneously, meaning the test result is "qualified". Both generated test cases and test results can be visualized; see reference [link / reference]. Figure 4 and Figure 5 , Figure 4 This represents the visual results of the automated test case generation. Figure 5 This is a visualization of the quality assessment results of the editing process.

[0074] In this embodiment, a visual question-answering model is used to automatically verify the consistency between the edited region and the current editing instruction, solving the problem of time-consuming and error-prone manual inspection of editing effects. This achieves rapid automation of edited region quality assessment and improves testing efficiency. By employing a technique that uses instance segmentation masks to locate non-edited regions and extracts features using a pre-trained visual encoder to calculate cosine similarity for stability verification, the inefficiency of manually comparing changes in non-edited regions is solved, achieving objective and efficient processing of non-edited region stability verification. Finally, the test result is determined by automatically combining the first and second verification results, achieving closed-loop automation of the testing process and significantly improving overall testing efficiency.

[0075] Based on the same inventive concept, such as Figure 6 As shown, this application also provides an automated testing apparatus for image editing tasks, which includes: Image acquisition module 10 is used to acquire an input image and a target region mask, wherein the target region mask is located in the input image; Instruction generation module 20 is used to generate a set of target editing instructions based on the input image and the target region mask; Image editing module 30 is used to edit the input image and the target region mask according to at least one editing instruction in the target editing instruction set to obtain an edited image; The quality assessment module 40 is used to perform dual quality assessments on the edited and non-edited areas based on the edited image to obtain test results.

[0076] It should be noted that the automated testing device for image editing tasks provided in this application embodiment and the automated testing method for image editing tasks provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned automated testing method for image editing tasks, and the repeated parts will not be described again.

[0077] In some embodiments, an electronic device provided in this application includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement an automated testing method for the above-described image editing task.

[0078] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0079] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0080] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An automated testing method for image editing tasks, characterized in that, The method includes: Obtain an input image and a target region mask, wherein the target region mask is located in the input image; Based on the input image and the target region mask, a set of target editing instructions is generated; The input image and the target region mask are edited according to at least one editing instruction from the target editing instruction set to obtain the edited image; Based on the edited image, a dual quality assessment is performed on the edited and unedited areas to obtain the test results.

2. The method as described in claim 1, characterized in that, The step of acquiring the input image and the target region mask, wherein the target region mask is located in the input image, includes: Perform target detection on the input image to obtain at least one target object and its corresponding location information; A binary mask and a blur mask are generated based on the target object and its corresponding location information, wherein the blur mask is used to preserve the clarity of the target area; The target region mask is determined based on the binary mask and the blurred mask.

3. The method as described in claim 1, characterized in that, The step of generating a set of target editing instructions based on the input image and the target region mask includes: The input image and the target region mask are input into a large language model, and a set of candidate editing instructions is generated by combining them with a preset task template; The candidate editing instruction set is filtered based on semantic similarity to generate a target editing instruction set, which includes representative instructions and diverse instructions.

4. The method as described in claim 3, characterized in that, The step of filtering the candidate editing instruction set based on semantic similarity to generate a target editing instruction set, wherein the target editing instruction set includes representative instructions and diverse instructions, includes: The candidate editing instruction set is clustered based on the target object type to obtain clustered editing instructions; Based on the clustered editing instructions, the editing instructions with the most keywords in each cluster are selected as representative instructions, and an initial set and a candidate set are generated. The initial set includes multiple representative instructions, and the candidate set includes multiple candidate instructions. The candidate instructions are the remaining editing instructions in each cluster. Based on each candidate instruction in the candidate set, the semantic similarity between the current candidate instruction and each representative instruction is calculated according to the semantic embedding vector, and the similarity set of the current candidate instruction is obtained. Based on the candidate set, the maximum value in the similarity set of the current candidate instruction is selected to obtain multiple target candidate values; The minimum value is selected from the plurality of target candidate values ​​to obtain the target candidate instruction; The target candidate instruction is removed from the candidate set and added to the initial set; The process involves returning to the step of calculating the semantic similarity between the current candidate instruction and each representative instruction based on the semantic embedding vector, and obtaining the similarity set of the current candidate instruction. When the number of editing instructions in the initial set reaches a preset number, the target editing instruction set is obtained.

5. The method as described in claim 1, characterized in that, The step of editing the input image and the target region mask according to at least one editing instruction from the target editing instruction set to obtain the edited image includes: Based on the preset size of the editing model, the input image and the target region mask are resized to obtain the adjusted image; The adjusted image is edited based on at least one editing instruction from the target editing instruction set to generate an initial edited image; The initial edited image is restored to its original size to obtain the edited image.

6. The method as described in claim 1, characterized in that, The step of performing dual quality assessment on the edited and unedited regions based on the edited image to obtain test results includes: The quality of the edited image is evaluated by verifying the consistency between the edited area and the current editing instruction, and a first verification result is obtained. The second verification result was obtained by verifying the stability of the non-editable region; The test results are determined based on the first verification result and the second verification result.

7. The method as described in claim 6, characterized in that, The step of evaluating the quality of the edited image and obtaining a first verification result by verifying the consistency between the edited area and the editing instructions includes: Based on the current editing command, the expected attributes are parsed to generate a set of structured verification questions; The visual question-answering model is used to answer the set of verification questions on the edited image to obtain the answer results; The answer is matched with the expected attribute to obtain the first verification result.

8. The method as described in claim 6, characterized in that, The step of obtaining a second verification result by verifying the stability of the non-editable region includes: Based on the instance segmentation mask, at least one non-edited region is determined from the edited image; The non-editable region features are obtained by extracting features from the non-editable region using a pre-trained visual encoder. Calculate the cosine similarity between the features of the non-edited region and the corresponding region features in the input image; The cosine similarity is converted into a rate of change, and a preset stability threshold is compared with the rate of change to obtain a comparison result. Based on the comparison results, it is determined whether there is any out-of-bounds modification in the non-editable area, thus obtaining the second verification result.

9. An automated testing device for image editing tasks, characterized in that, The automated testing device for the image editing task includes: An image acquisition module is used to acquire an input image and a target region mask, wherein the target region mask is located in the input image; The instruction generation module is used to generate a set of target editing instructions based on the input image and the target region mask; The image editing module is used to edit the input image and the target region mask according to at least one editing instruction from the target editing instruction set to obtain the edited image; The quality evaluation module is used to perform dual quality evaluation on the edited and non-edited areas based on the edited image to obtain test results.

10. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of an automated testing method for an image editing task as described in any one of claims 1 to 8.

Images