Image evaluation method, model training method, device, medium and equipment

Abstract

The present application provides an image evaluation method, model training method, apparatus, medium, and device, the method comprising: in response to an input original image and an extended image, performing image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to a preset object type, wherein: performing image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine a foreground object evaluation result of the extended image; calling a pre-trained feature evaluation model to perform feature difference evaluation on the original image and the extended image to determine a feature evaluation result of the extended image; and determining a comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result. Based on the solution of the present application, the efficiency of evaluating extended images can be improved.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and more specifically, to an image evaluation method, model training method, apparatus, medium, and device in the field of computer technology. Background Art

[0002] With the rapid development of AIGC (Artificial Intelligence Generated Content) technology, its application scope is becoming increasingly broad. Users can easily use AIGC to generate diverse content to meet personalized and diverse needs. In particular, in the e-commerce sector, merchants are now able to use AIGC to create creative and attractive extended images based on the original product images. These extended images often incorporate rich background elements to enhance the product's display on e-commerce platforms.

[0003] However, before these extended images can be used for display on e-commerce platforms, an essential step is to evaluate their applicability to ensure that the quality of the extended images meets certain requirements. This evaluation process currently relies mainly on manual operations, which is inefficient. Summary of the Invention

[0004] This specification provides an image evaluation method, model training method, device, medium and equipment, which can improve the efficiency of extended image evaluation.

[0005] In a first aspect, an image evaluation method is provided, the method comprising:

[0006] In response to an input original image and an extended image, performing image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image;

[0007] performing image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine a foreground object evaluation result of the extended image;

[0008] Calling a pre-trained feature evaluation model to evaluate the feature difference between the original image and the extended image, and determining the feature evaluation result of the extended image;

[0009] Based on the foreground object evaluation result and the feature evaluation result, a comprehensive evaluation result of the extended image is determined.

[0010] In combination with the first aspect, in some possible implementations, there is at least one first foreground object and at least one second foreground object; image comparison is performed based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine the foreground object evaluation result of the extended image, including: for each first foreground object in at least one first foreground object and each second foreground object in at least one second foreground object, respectively determining the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object; based on the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object, determining at least one segmented image group, wherein any one segmented image group in the at least one segmented image group includes a corresponding segmented image of a first foreground object and a segmented image of a second foreground object; respectively obtaining parameters of each segmented image group under a preset content comparison index; respectively obtaining parameters of each segmented image group under a preset layout analysis index; and determining the foreground object evaluation result of the extended image based on the parameters of each segmented image group under the preset content comparison index and the parameters under the preset layout analysis index.

[0011] In combination with the first aspect and the above-mentioned implementation manner, in some possible implementation manners, respectively obtaining parameters of each segmented image group under a preset content comparison index includes: traversing at least one segmented image group; for any one target segmented image group in the at least one traversed segmented image group, if the preset content comparison index includes a peak signal-to-noise ratio index, obtaining a peak signal-to-noise ratio value of a segmented image of a target first foreground object in the target segmented image group, and a peak signal-to-noise ratio value of a segmented image of a target second foreground object in the target segmented image group; determining parameters of the target segmented image group under the peak signal-to-noise ratio index based on the peak signal-to-noise ratio value of the segmented image of the target first foreground object and the peak signal-to-noise ratio value of the segmented image of the target second foreground object; if the preset content comparison index includes a multi-scale structural similarity index, obtaining the target segmented image group. The method comprises the steps of: determining a multi-scale structural similarity index value of a segmented image of a target first foreground object and a multi-scale structural similarity index value of a segmented image of a target second foreground object; comparing the multi-scale structural similarity index value of the segmented image of the target first foreground object and the multi-scale structural similarity index value of the segmented image of the target second foreground object, and determining parameters of the target segmented image group under the multi-scale structural similarity index indicator; if the preset content comparison indicator includes a foreground object area overlap rate indicator, obtaining the area of the target first foreground object in the segmented image of the target first foreground object and the area of the target second foreground object in the segmented image of the target second foreground object; and determining parameters of the target segmented image group under the foreground object area overlap rate indicator according to the comparison of the area of the target first foreground object and the area of the target second foreground object.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the parameters of each segmented image group under the preset layout analysis index are obtained respectively, including: traversing at least one segmented image group; for any target segmented image group in the at least one traversed segmented image group, if the preset layout analysis index includes a display position index, obtaining the center pixel point of the target second foreground object in the segmented image of the target second foreground object in the target segmented image group; analyzing according to the center pixel point of the target second foreground object and the preset center pixel reference range, and determining the parameters of the target segmented image group under the display position index; if the preset layout analysis index includes an area ratio index, obtaining the area ratio of the target second foreground object in the segmented image of the target second foreground object; analyzing according to the area ratio of the target second foreground object and the preset area ratio reference range, and determining the parameters of the target segmented image group under the area ratio index.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, image segmentation is performed on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and image segmentation is performed on the extended image to obtain a segmented image of a second foreground object that conforms to a preset object type, including: obtaining segmentation prompt information corresponding to the preset object type; inputting the segmentation prompt information and the original image into a pre-trained image segmentation model to obtain a segmented image of the first foreground object output by the image segmentation model; inputting the segmentation prompt information and the extended image into the image segmentation model to obtain a segmented image of the second foreground object output by the image segmentation model.

[0014] In a second aspect, a model training method is provided, the method comprising:

[0015] Creating an initial feature evaluation model; obtaining an image pair sample set, wherein the image pair sample set includes multiple image pair samples with feature evaluation result annotation information, and any target image pair sample in the multiple image pair samples includes an original image sample and an extended image sample, and the extended image sample is generated by background extension based on the foreground object in the original image sample; iteratively training the initial feature evaluation model based on the image pair sample set; if a preset convergence condition is met, determining the initial feature evaluation model as a pre-trained feature evaluation model.

[0016] In combination with the second aspect, in some possible implementations, the initial feature evaluation model includes an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer; the image feature extraction layer is used to receive the input original image sample and the extended image sample, and obtain the image feature matrix of the original image sample and the image feature matrix of the extended image sample, respectively; the feature difference analysis layer is used to determine the feature difference information between the original image sample and the extended image sample based on the image feature matrix of the original image sample and the image feature matrix of the extended image sample; the feature evaluation layer is used to determine the feature evaluation result of the extended image sample based on the feature difference information between the original image sample and the extended image sample.

[0017] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the feature difference analysis layer is also used to extract the global feature vector of the original image sample from the image feature matrix of the original image sample, extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample, and determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample.

[0018] In a third aspect, an image evaluation device is provided, the device comprising:

[0019] a segmentation unit configured to, in response to an input original image and an extended image, perform image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and to perform image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image;

[0020] a first evaluation unit, configured to perform image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object, and determine an evaluation result of the foreground object of the extended image;

[0021] The second evaluation unit is used to call a pre-trained feature evaluation model to evaluate the feature difference between the original image and the extended image, and determine the feature evaluation result of the extended image;

[0022] The determination unit is used to determine the comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result.

[0023] In a fourth aspect, a model training device is provided, comprising:

[0024] A creation unit for creating an initial feature evaluation model;

[0025] an acquisition unit, configured to acquire an image pair sample set, wherein the image pair sample set includes a plurality of image pair samples annotated with feature evaluation result information, wherein any target image pair sample among the plurality of image pair samples includes an original image sample and an extended image sample, wherein the extended image sample is generated by performing background extension based on a foreground object in the original image sample;

[0026] A training unit, for iteratively training an initial feature evaluation model based on an image pair sample set;

[0027] The determining unit is configured to determine the initial feature evaluation model as a pre-trained feature evaluation model if a preset convergence condition is met.

[0028] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0029] In a sixth aspect, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0030] In a seventh aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0031] In the embodiments of the present specification: First, through image segmentation, the segmented image of the first foreground object and the segmented image of the second foreground object that meet the preset object type can be accurately extracted from the original image and the extended image respectively, and the segmented image of the first foreground object and the segmented image of the second foreground object are used to perform image comparison to determine the foreground object evaluation result. Secondly, the feature difference evaluation of the original image and the extended image is performed using a pre-trained feature evaluation model, which can objectively analyze the difference between the two at the feature level. This difference generally comes from the background diffusion in the process of generating the extended image, and further obtains a feature evaluation result that can reflect the image background difference. Using the foreground object evaluation result and the feature evaluation result, the comprehensive evaluation result of the extended image can be determined. The comprehensive evaluation result effectively reflects the quality of the extended image, and at the same time does not require manual intervention in the evaluation process, effectively improving the efficiency of the extended image evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of an image evaluation scenario provided by an embodiment of this specification;

[0033] Figure 2 This is a flow chart of an image evaluation method provided in an embodiment of this specification;

[0034] Figure 3 is an example schematic diagram of an original image and an expanded image provided in an embodiment of this specification;

[0035] Figure 4 This is a flow chart of determining a foreground object evaluation result provided by an embodiment of this specification;

[0036] Figure 5 This is an example schematic diagram of segmenting an original image and an expanded image provided in an embodiment of this specification;

[0037] Figure 6 This is a flow chart of obtaining parameters of a segmented image group under a preset content comparison index provided by an embodiment of this specification;

[0038] Figure 7 This is a flow chart of obtaining parameters of a segmented image group under a preset layout analysis index provided by an embodiment of this specification;

[0039] Figure 8 This is an example schematic diagram of the position and proportion of a second foreground object provided in an embodiment of this specification;

[0040] Figure 9 This is an example schematic diagram of the position and proportion of a second foreground object provided in an embodiment of this specification;

[0041] Figure 10 This is a flow chart of a model training method provided in an embodiment of this specification;

[0042] Figure 11 This is an example diagram of a training feature analysis model provided in an embodiment of this specification;

[0043] Figure 12 This is a schematic diagram illustrating an example of an image evaluation effect provided in an embodiment of this specification;

[0044] Figure 13 This is a schematic diagram illustrating an example of an image evaluation effect provided in an embodiment of this specification;

[0045] Figure 14 This is a schematic diagram of the structure of an image evaluation device provided in an embodiment of this specification;

[0046] Figure 15 This is a schematic diagram of the structure of a model training device provided in an embodiment of this specification;

[0047] Figure 16This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0048] The following will clearly and thoroughly describe the technical solutions in this specification with reference to the accompanying drawings. In the description of the embodiments of this specification, unless otherwise specified, " / " means or. For example, A / C can mean A or C. "And / or" in the text is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or C can mean: A exists alone, A and C exist at the same time, and C exists alone. In addition, in the description of the embodiments of this specification, "multiple" means two or more than two.

[0049] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0050] With the rapid development of AIGC technology, its application scope is becoming increasingly broad. Users can now easily utilize advanced AIGC technology to generate diverse, high-quality content through algorithmic models and deep learning frameworks to meet personalized and diverse needs. In particular, in the e-commerce sector, merchants can leverage AIGC technology to extend the background of products in their original images through image generation algorithms and creative optimization strategies, creating creative and highly eye-catching extended images. These extended images not only retain the core features of the products but also incorporate rich background elements and visual effects, aiming to significantly improve the display of products on e-commerce platforms and enhance users' willingness to purchase.

[0051] However, before these expanded images with background extensions can be displayed on e-commerce platforms, an essential step is evaluation. This evaluation process aims to ensure that the expanded images meet certain requirements. However, this evaluation process currently relies primarily on manual labor, which not only requires a significant amount of manpower and time, but also suffers from inefficiencies.

[0052] In response to the above-mentioned problems in the related art, the main solutions provided by the embodiments of this specification are as follows: First, through image segmentation, the segmented image of the first foreground object and the segmented image of the second foreground object that meet the preset object type can be accurately extracted from the original image and the extended image respectively, and the foreground object evaluation result can be determined by performing image comparison using the segmented image of the first foreground object and the segmented image of the second foreground object. Secondly, the feature difference evaluation of the original image and the extended image is performed using a pre-trained feature evaluation model, which can objectively analyze the difference between the two at the feature level. This difference generally comes from the background diffusion in the process of generating the extended image, and further obtains a feature evaluation result that can reflect the image background difference. Using the foreground object evaluation result and the feature evaluation result, the comprehensive evaluation result of the extended image can be determined. The comprehensive evaluation result effectively reflects the quality of the extended image, and at the same time does not require manual intervention in the evaluation process, effectively improving the efficiency of the extended image evaluation.

[0053] See Figure 1 , Figure 1 This is a schematic diagram of a scene of image evaluation provided by an embodiment of this specification. Based on the solution provided by an embodiment of this specification, the user can generate an extended image based on the original image by operating on the user side. Then, the user side transmits the original image and the extended image to the server side, and the server side receives the input original image and extended image. In response to the input original image and extended image, the server side performs image segmentation on the original image to obtain a segmented image of a first foreground object that meets the preset object type, and performs image segmentation on the extended image to obtain a segmented image of a second foreground object that meets the preset object type; performs image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine the foreground object evaluation result of the extended image; calls a pre-trained feature evaluation model to perform feature difference evaluation on the original image and the extended image to determine the feature evaluation result of the extended image; and determines the comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result. If the comprehensive evaluation result of the extended image indicates that the evaluation is passed, the server side returns the first evaluation result information to the client side, wherein the first evaluation result information is at least used to prompt the user that the extended image has passed the evaluation. If the comprehensive evaluation result of the extended image indicates that the evaluation fails, the server returns second evaluation result information to the client, wherein the second evaluation result information is at least used to prompt the user that the extended image fails the evaluation.

[0054] based on Figure 1 The following is a schematic diagram of the scene Figure 2 - Figure 13 , the image evaluation method provided in the embodiments of this specification is introduced in detail.

[0055] See Figure 2, is a flow chart of an image evaluation method provided in the embodiment of this specification. Figure 2 As shown, the method of the embodiment of this specification may include the following steps S102 to S108.

[0056] S102, in response to the input original image and the extended image, performing image segmentation on the original image to obtain a segmented image of a first foreground object that meets the preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that meets the preset object type, wherein the extended image is generated by background extension based on the first foreground object in the original image.

[0057] Specifically, in response to the input original image and the extended image, the extended image needs to be evaluated in terms of foreground objects and background features respectively to determine whether the extended image passes the evaluation.

[0058] Regarding foreground objects, the original image must first be segmented to obtain a segmented image of a first foreground object that matches a preset object type, and the extended image must also be segmented to obtain a segmented image of a second foreground object that matches a preset object type. Image segmentation is a technique that divides an image into multiple regions or objects so that each region or object can be processed or analyzed separately. In this step, particular attention is paid to foreground objects that match a preset object type. The preset object type can be any predefined object that needs to be identified and extracted in the original and extended images, such as goods, people, animals, vehicles, and other entities with specific characteristics or meaning.

[0059] For the original image, a segmented image of a first foreground object that matches a preset object type is accurately extracted using a preset image segmentation model or image segmentation algorithm. Similarly, for the expanded image, a segmented image of a second foreground object that matches a preset object type can also be accurately extracted using a preset image segmentation model or image segmentation algorithm.

[0060] It should be noted that the segmented image of the first foreground object and the segmented image of the second foreground object may be resized to keep them consistent in size, so as to facilitate subsequent comparison and analysis.

[0061] It's important to note that foreground objects and background are the two basic elements that make up an image. The foreground object is the portion of the image that's of primary interest or focus. The background, on the other hand, is the portion of the image other than the foreground object, providing the image's surroundings and context.

[0062] Furthermore, the extended image is generated by extending the background of the first foreground object in the original image. In one possible implementation, the original image is processed using AIGC technology, specifically by extending the background of the first foreground object in the original image, thereby generating a corresponding extended image. It is understood that the background of the extended image is different from that of the original image because the background of the extended image is newly generated after AIGC processing, while the background of the original image is the unprocessed original background. In terms of technical principles, AIGC technology utilizes deep learning and computer vision algorithms to identify and segment the first foreground object in the original image. Through precise image segmentation technology, the foreground object can be separated from the original background while preserving its shape, size, color, and other characteristics. Subsequently, AIGC technology generates a new background and fuses it with the segmented foreground object to generate the extended image. It is understood that there is a one-to-one correspondence between the first foreground object in the original image and the second foreground object in the extended image.

[0063] To visually demonstrate the relationship between the original image and the expanded image, see Figure 3 , Figure 3 is a schematic diagram illustrating an example of an original image and an extended image provided in an embodiment of this specification. Original image 001 includes first foreground object 0011 and first foreground object 0012. Extended image 002 is obtained by performing background extension based on first foreground object 0011 and first foreground object 0012 in original image 001. In extended image 002, second foreground object 0021 corresponds to first foreground object 0011, and second foreground object 0022 corresponds to first foreground object 0012. Furthermore, the backgrounds of original image 001 and extended image 002 differ.

[0064] S104 : performing image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine a foreground object evaluation result of the extended image.

[0065] Specifically, because the extended image is generated based on the original image, the second foreground object in the extended image corresponds one-to-one to the first foreground object in the original image. During the generation of the extended image, the second foreground object may deviate from the corresponding first foreground object in terms of content and layout. For example, the second foreground object may have redundant content generated at the edge compared to the corresponding first foreground object; another example is that the second foreground object may be distorted in some pixels compared to the corresponding first foreground object; another example is that the second foreground object may be offset in position compared to the corresponding first foreground object; another example is that the second foreground object may have a different area compared to the corresponding first foreground object.

[0066] Therefore, the segmented image of the first foreground object and the segmented image of the second foreground object can be compared in terms of content and layout to determine the foreground object evaluation result of the extended image. It can be understood that the foreground object evaluation result of the extended image reflects the quality performance of the extended image in terms of foreground objects.

[0067] In one possible implementation, the foreground object evaluation result of the extended image is expressed in the form of a numerical value, and the size of the numerical value reflects the quality of the extended image in terms of the foreground object; in one possible implementation, the foreground object evaluation result of the extended image is expressed in the form of a state, and the state reflects the quality of the extended image in terms of the foreground object.

[0068] S106 , calling a pre-trained feature evaluation model to evaluate feature differences between the original image and the extended image, and determining a feature evaluation result of the extended image.

[0069] Specifically, during the generation of the expanded image, due to background diffusion, the background of the expanded image will differ from that of the original image. These differences may manifest in multiple aspects, such as color distribution, texture pattern, brightness contrast, etc. To accurately evaluate these background differences, it is necessary to introduce a feature evaluation model that can deeply extract and analyze image features.

[0070] The feature evaluation model is a pre-trained model tool based on machine learning technology. It can learn the feature processing capabilities of images during the training process, analyze the feature difference information between the original image and the extended image, and establish a mapping relationship between the feature difference information and the feature evaluation results.

[0071] Specifically, the feature evaluation model receives the original image and the expanded image as input, extracts image feature information of the original image and the expanded image, and represents this image feature information in an image feature matrix or other form. The feature evaluation model then determines feature difference information between the original image and the expanded image based on the image feature information of the original image and the expanded image. The feature evaluation model then determines a feature evaluation result for the expanded image based on the feature difference information between the original image and the expanded image.

[0072] In one possible implementation, the original and expanded images can undergo feature saliency processing before being input into the feature evaluation model. Feature saliency processing aims to enhance key features in the image, enabling the feature evaluation model to more accurately capture and analyze the features of the original and expanded images. Feature saliency processing can be implemented in a variety of ways, including using image enhancement techniques such as contrast enhancement and sharpening to highlight image edges and details, making key features more prominent.

[0073] In one possible implementation, the feature evaluation result of the extended image is expressed in the form of a numerical value, and the size of the numerical value reflects the quality of the extended image in terms of background features; in one possible implementation, the feature evaluation result of the extended image is expressed in the form of a state, and the state reflects the quality of the extended image in terms of background features.

[0074] S108 , determining a comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result.

[0075] Specifically, based on the foreground object evaluation result and the feature evaluation result, the overall quality of the extended image is comprehensively evaluated, and a comprehensive evaluation result can be determined, which comprehensively reflects the quality of the extended image in terms of foreground object scheme and background features.

[0076] It should be noted that the foreground object evaluation results and feature evaluation results may be expressed in numerical form to facilitate comparison and analysis. The foreground object evaluation results and feature evaluation results may be expressed in the form of status, which supports logical judgment to obtain comprehensive evaluation results.

[0077] The following are examples of how numerical values are represented:

[0078] In one example, assume that the foreground object evaluation result of the extended image is 0.9 (expressed numerically, ranging from 0 to 1, with 1 indicating complete foreground object consistency), and the feature evaluation result of the extended image is 0.8 (also expressed numerically, ranging from 0 to 1, with 1 indicating complete feature consistency). If the threshold is set to 0.85, and the foreground object evaluation result and the feature evaluation result are considered equally important, then the comprehensive evaluation result can be calculated as (0.9 + 0.8) / 2 = 0.85. Since the comprehensive evaluation result is equal to the set threshold, it can be considered that the quality of the extended image meets the requirements and the evaluation passes. Conversely, if the comprehensive evaluation result is less than the set threshold, then the quality of the extended image is considered to be unsatisfactory and the evaluation fails.

[0079] In one example, the foreground object evaluation result of the expanded image is 0.7, and the feature evaluation result of the expanded image is 0.6. If the threshold is still set at 0.85, and the foreground object evaluation result is considered more important than the feature evaluation result, the weight of the foreground object evaluation result can be set to 0.6, and the weight of the feature evaluation result can be set to 0.4. The calculated comprehensive evaluation result is 0.6 * 0.7 + 0.4 * 0.6 = 0.54. Since the comprehensive evaluation result is less than the set threshold, it can be considered that the quality of the expanded image does not meet the requirements and the evaluation fails.

[0080] In one example, if the fidelity of foreground objects is high, the weight of the foreground object evaluation result might be set to 0.9, and the weight of the feature evaluation result might be set to 0.1. Assume that the foreground object evaluation result of the extended image is 0.95 (numerically expressed), and the feature evaluation result is 0.5 (numerically expressed). The calculated comprehensive evaluation result is 0.9 * 0.95 + 0.1 * 0.5 = 0.895. If the threshold is set to 0.9, then because the comprehensive evaluation result is lower than the set threshold, it can be considered that the extended image needs improvement in terms of foreground object fidelity, even though its foreground object evaluation result is already quite high. In this case, the evaluation can be determined to have failed.

[0081] The following are examples of how status may be represented:

[0082] In one example, suppose the foreground object evaluation result of the extended image is "passed" (indicating that the foreground object is completely consistent or meets the preset standard), and the feature evaluation result is "passed" (indicating that the features are completely consistent or meet the preset standard). If the comprehensive evaluation rule is set as "if both the foreground object evaluation result and the feature evaluation result pass, the comprehensive evaluation result passes," then the comprehensive evaluation result of the extended image can be determined to be "passed."

[0083] In one example, the foreground object evaluation result of an extended image is "passed," but the feature evaluation result is "failed." If the comprehensive evaluation rule is set as "if both the foreground object evaluation result and the feature evaluation result pass, the comprehensive evaluation result passes; otherwise, fails," then the comprehensive evaluation result of the extended image can be determined to be "failed."

[0084] In one example, if the fidelity requirements for foreground objects are high, a more stringent comprehensive evaluation rule may be set. Suppose the foreground object evaluation result of the extended image is "passed," but the feature evaluation result is "failed." Even if the foreground object evaluation result is excellent, due to the failure of the feature evaluation result, the comprehensive evaluation result of the extended image can still be determined to be "failed" according to the set comprehensive evaluation rule: "If both the foreground object evaluation result and the feature evaluation result pass, the comprehensive evaluation result passes; otherwise, it fails."

[0085] In this embodiment, first, through image segmentation, the segmented image of the first foreground object and the segmented image of the second foreground object that meet the preset object type can be accurately extracted from the original image and the extended image respectively. The segmented image of the first foreground object and the segmented image of the second foreground object are used to perform image comparison to determine the foreground object evaluation result. Secondly, the feature difference evaluation of the original image and the extended image is performed using a pre-trained feature evaluation model, which can objectively analyze the differences between the two at the feature level. This difference generally comes from the background diffusion in the process of generating the extended image, and further obtains a feature evaluation result that can reflect the image background difference. Using the foreground object evaluation result and the feature evaluation result, the comprehensive evaluation result of the extended image can be determined. The comprehensive evaluation result effectively reflects the quality of the extended image. At the same time, no manual intervention in the evaluation process is required, which effectively improves the efficiency of the extended image evaluation.

[0086] See Figure 4 , provides a flow chart of determining the foreground object evaluation result for an embodiment of the present application. Wherein, each of the first foreground object and the second foreground object is at least one. Figure 4 As shown, the method of the embodiment of the present application may include the following steps S202 to S210, and steps S202 to S210 may be used as Figure 2 The refinement of step S104 in the illustrated embodiment.

[0087] S202, for each first foreground object in the at least one first foreground object and each second foreground object in the at least one second foreground object, respectively determining pixel similarities between a segmented image of each first foreground object and a segmented image of each second foreground object;

[0088] S204, determining at least one segmented image group based on pixel similarities between the segmented images of each first foreground object and the segmented images of each second foreground object, wherein any one of the at least one segmented image group includes a corresponding segmented image of the first foreground object and a segmented image of the second foreground object;

[0089] S206, respectively obtaining parameters of each segmented image group under a preset content comparison index;

[0090] S208, respectively obtaining parameters of each segmented image group under a preset layout analysis index;

[0091] S210 , determining a foreground object evaluation result of the expanded image based on parameters of each segmented image group under a preset content comparison index and parameters under a preset layout analysis index.

[0092] Specifically, the original image may contain one or more first foreground objects, while the number of second foreground objects in the extended image is greater than or equal to the number of first foreground objects in the original image. There are two reasons why the number of first foreground objects in the original image and second foreground objects in the extended image may not correspond:

[0093] On the one hand, the extended image is generated by performing background expansion based on the first foreground object in the original image. During the background expansion process, objects that are not present in the original image may appear in the background of the extended image. For example, there is only one first foreground object in the original image, specifically a desk lamp. By performing background expansion based on the desk lamp in the original image, an extended image can be obtained. The content in the extended image shows a desk lamp placed on a table, and the table is the content that is not present in the original image. It is understandable that the table and desk lamp in the extended image are both potential second foreground objects, depending on the definition of the preset object type.

[0094] On the other hand, the image segmentation of the original image and the expanded image is guided by the preset object type. Based on the above example, assuming that the first and second foreground objects are limited by the preset object type "electrical appliances," then the first foreground object in the original image is only a table lamp, and the second foreground object in the expanded image is also only a table lamp; assuming that the first and second foreground objects are limited by the preset object type "commodity," then the first foreground object in the original image is only a table lamp, while the second foreground object in the expanded image is a table lamp and a table. In other words, the segmented images of the second foreground object in the expanded image will be more than the segmented images of the first foreground object in the original image, and the redundant segmented images of the second foreground objects in the expanded image will not be used in subsequent image comparison (for example, the segmented image of "table" in the above example will not be used in subsequent image comparison).

[0095] For the reasons described above, before performing image comparison, it is necessary to determine a one-to-one correspondence between the first foreground objects in the original image and the second foreground objects in the expanded image. Specifically, for each first foreground object in the at least one first foreground object and each second foreground object in the at least one second foreground object, the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object is determined.

[0096] Specifically, pixel difference calculation methods in image processing technology, such as mean square error (MSE) or structural similarity (SSIM), can be used to quantify the similarity between the two segmented images. Through such a comparison, the pixel similarity between the segmented image of any first foreground object and the segmented image of any second foreground object can be obtained, and the pixel similarity is expressed in the form of a numerical value.

[0097] Next, at least one segmented image group needs to be determined. A segmented image group is composed of a corresponding segmented image of a first foreground object and a segmented image of a second foreground object. When determining the segmented image groups, the image groups can be sorted based on pixel similarity. The segmented image of the first foreground object and the segmented image of the second foreground object with the highest similarity are selected to form a segmented image group. This continues in this order until all first foreground objects and second foreground objects are assigned to corresponding segmented image groups.

[0098] Subsequently, parameters of each segmented image group under a preset content comparison index are obtained. The parameters under the preset content comparison index reflect the quality of the segmented image of the second foreground object at the content level, and indirectly reflect the quality of the expanded image at the content level.

[0099] Next, parameters of each segmented image group under a preset layout analysis index are obtained. The parameters under the preset layout analysis index reflect the quality of the segmented image of the second foreground object at the layout level, and indirectly reflect the quality of the expanded image at the layout level.

[0100] Finally, the foreground object evaluation result of the extended image is determined based on the parameters of each segmented image group under the preset content comparison index and the parameters under the preset layout analysis index.

[0101] In one possible implementation, corresponding weights can be set for the parameters under the preset content comparison index, and corresponding weights can be set for the parameters under the preset layout analysis index, respectively. The parameters under the preset content comparison index and the parameters under the preset layout analysis index are weighted and summed to obtain a comprehensive numerical value as the evaluation result of the foreground object of the extended image.

[0102] In one possible implementation, a first parameter threshold and a second parameter threshold may be obtained, respectively. If the parameter under the preset content comparison index is greater than or equal to the first parameter threshold, then the extended image is determined to have passed the evaluation at the content level of the foreground object; if the parameter under the preset content comparison index is less than the first parameter threshold, then the extended image is determined to have failed the evaluation at the content level of the foreground object. If the parameter under the preset layout analysis index is greater than or equal to the second parameter threshold, then the extended image is determined to have passed the evaluation at the layout level of the foreground object; if the parameter under the preset layout analysis index is less than the second parameter threshold, then the extended image is determined to have failed the evaluation at the layout level of the foreground object. If the extended image passes both the content level and the layout level of the foreground object, then the foreground object evaluation result of the extended image is determined to be in the "evaluation passed" state; if the extended image fails either the content level or the layout level of the foreground object, then the foreground object evaluation result of the extended image is determined to be in the "evaluation failed" state.

[0103] To understand the image segmentation process in this example, please refer to Figure 5 , Figure 5 This is an example schematic diagram of segmenting an original image and an expanded image provided in an embodiment of this specification.

[0104] Original image 001 includes first foreground object 0011 and first foreground object 0012. Background extension is performed based on first foreground object 0011 and first foreground object 0012 in original image 001 to obtain extended image 002. In extended image 002, second foreground object 0021 corresponds to first foreground object 0011, and second foreground object 0022 corresponds to first foreground object 0012. After image segmentation, segmented image 0031 of first foreground object 0011, segmented image 0032 of first foreground object 0012, segmented image 0033 of second foreground object 0021, and segmented image 0034 of second foreground object 0022 are obtained. After analysis and processing, we can obtain pixel similarities between segmented image 0031 of first foreground object 0011 and segmented image 0033 of second foreground object 0021, pixel similarities between segmented image 0031 of first foreground object 0011 and segmented image 0034 of second foreground object 0022, pixel similarities between segmented image 0032 of first foreground object 0012 and segmented image 0033 of second foreground object 0021, and pixel similarities between segmented image 0032 of first foreground object 0012 and segmented image 0034 of second foreground object 0022. Then, based on the comparison of these pixel similarities, segmented image 0031 of first foreground object 0011 and segmented image 0033 of second foreground object 0021 can be grouped into the same segmented image group, and segmented image 0032 of first foreground object 0012 and segmented image 0034 of second foreground object 0022 can be grouped into the same segmented image group.

[0105] In this embodiment, first, by comparing the pixel similarity of foreground objects in the original image and the expanded image, the corresponding first and second foreground objects are accurately determined, thereby forming a segmented image group. Second, using preset content comparison indicators and preset layout analysis indicators, the expanded image is comprehensively evaluated at both the content and layout levels. This dual evaluation mechanism can more comprehensively reflect the quality of the expanded image, thereby improving the accuracy and reliability of the evaluation. Finally, by combining the evaluation results at both the content and layout levels, a more comprehensive and accurate foreground object evaluation result can be obtained, providing effective guidance for the expanded image evaluation process.

[0106] See Figure 6 , provides a flow chart of obtaining parameters of a segmented image group under a preset content comparison index for an embodiment of the present application, such as Figure 6 As shown, the method of the embodiment of the present application may include the following steps S302 to S308, and steps S302 to S308 may be used as Figure 4 The refinement of step S206 in the illustrated embodiment.

[0107] S302, traversing at least one segmented image group;

[0108] S304: For any target segmented image group in the at least one traversed segmented image group, if the preset content comparison index includes a peak signal-to-noise ratio index, obtaining a peak signal-to-noise ratio value of a segmented image of a target first foreground object in the target segmented image group and a peak signal-to-noise ratio value of a segmented image of a target second foreground object in the target segmented image group; comparing the peak signal-to-noise ratio value of the segmented image of the target first foreground object with the peak signal-to-noise ratio value of the segmented image of the target second foreground object to determine parameters of the target segmented image group under the peak signal-to-noise ratio index;

[0109] S306: If the preset content comparison index includes a multi-scale structural similarity index, obtaining a multi-scale structural similarity index value of the segmented image of the first target foreground object and a multi-scale structural similarity index value of the segmented image of the second target foreground object; comparing the multi-scale structural similarity index value of the segmented image of the first target foreground object with the multi-scale structural similarity index value of the segmented image of the second target foreground object to determine parameters of the target segmented image group under the multi-scale structural similarity index;

[0110] S308, if the preset content comparison index includes a foreground object area overlap rate index, then obtain the area of the target first foreground object in the segmented image of the target first foreground object, and the area of the target second foreground object in the segmented image of the target second foreground object; compare the area of the target first foreground object and the area of the target second foreground object to determine the parameters of the target segmented image group under the foreground object area overlap rate index.

[0111] Specifically, it is first necessary to traverse all the segmented image groups to be analyzed.

[0112] For each target segmented image group traversed, if the preset content comparison metric includes the Peak Signal-to-Noise Ratio (PSNR) metric, the following operations need to be performed:

[0113] Obtaining Peak Signal-to-Noise Ratio (PSNR): Calculate the PSNR of the segmented image of the first foreground object and the PSNR of the segmented image of the second foreground object in the target segmented image group. PSNR is a measure of image distortion or quality loss; higher PSNR values indicate better image quality.

[0114] Comparing Peak Signal-to-Noise Ratio (PSNR): Compare the PSNR of the segmented image of the first foreground object with the PSNR of the segmented image of the second foreground object. The purpose of this comparison is to assess the similarity or difference in image quality between the two segmented images.

[0115] Determine parameters: Based on the comparison results, determine the parameters of the target segmented image group under the peak signal-to-noise ratio indicator. This parameter can be the difference, ratio, or other statistical quantity between the two peak signal-to-noise ratio values, which is used to quantify the difference in image quality between the two segmented images.

[0116] For each target segmented image group traversed, if the preset content comparison metric includes the Multi-Scale Structural Similarity Index (MSSSIM), the following operations need to be performed:

[0117] Obtaining Multiscale Structural Similarity Index Values: Calculate the multiscale structural similarity index values for the segmented image of the first foreground object and the segmented image of the second foreground object. The multiscale structural similarity index measures the structural similarity between two images; higher values indicate more similar images.

[0118] Comparing Multi-Scale Structural Similarity Index Values: Compare the multi-scale structural similarity index values of the segmented image of the first foreground object with the multi-scale structural similarity index values of the segmented image of the second foreground object. The purpose of this comparison is to assess the structural similarity or difference between the two segmented images.

[0119] Determine parameters: Based on the comparison results, determine the parameters of the target segmented image group under the multi-scale structural similarity index indicator. This parameter can be the difference, ratio, or other statistical quantity of the two multi-scale structural similarity index values, which is used to quantify the structural differences between the two segmented images.

[0120] For each target segmentation image group traversed, if the preset content comparison index includes the foreground object area overlap ratio index, the following operations need to be performed:

[0121] Obtaining foreground object areas: Calculate the area of the first foreground object in the segmented image of the first foreground object, and the area of the second foreground object in the segmented image of the second foreground object. The foreground object areas are obtained by counting the number of pixels of the foreground objects in the segmented image.

[0122] Compare foreground object areas: Compare the area of the first foreground object to the area of the second foreground object. The purpose of this comparison is to evaluate the size similarity or difference of the foreground objects in the two segmented images.

[0123] Determine parameters: Based on the comparison results, determine the parameters of the target segmented image group under the foreground object area overlap ratio metric. This parameter can be the ratio of the intersection over union (IoU) of the two foreground object areas, which quantifies the degree of overlap of the foreground object areas in the two segmented images.

[0124] In this embodiment, all segmented image groups to be analyzed are first traversed to ensure that each segmented image group can be accurately evaluated. Secondly, for each target segmented image group, the peak signal-to-noise ratio, multi-scale structural similarity index, and foreground object area overlap ratio are compared and analyzed based on different preset content comparison indicators. This diversified comparison can more comprehensively reflect the similarities or differences between the two segmented images at the content level. Finally, based on the comparison results, the parameters of the target segmented image group under each preset content comparison indicator are determined. These parameters can quantify the differences between the two segmented images at the content level, providing an effective reference for evaluating the extended object.

[0125] See Figure 7 , provides a flow chart of obtaining parameters of a segmented image group under a preset layout analysis index for an embodiment of the present application, such as Figure 7 As shown, the method of the embodiment of the present application may include the following steps S402-S406.

[0126] S402, traversing at least one segmented image group;

[0127] S404, for any target segmented image group in the at least one traversed segmented image group, if the preset layout analysis index includes a display position index, obtaining a center pixel point of the target second foreground object in the segmented image of the target segmented image group; performing analysis based on the center pixel point of the target second foreground object and a preset center pixel reference range to determine parameters of the target segmented image group under the display position index;

[0128] S406, if the preset layout analysis index includes an area ratio index, then obtain the area ratio of the target second foreground object in the segmented image of the target second foreground object; analyze according to the area ratio of the target second foreground object and the preset area ratio reference range to determine the parameters of the target segmented image group under the area ratio index.

[0129] Specifically, it is first necessary to traverse all the segmented image groups to be analyzed.

[0130] For each target segmented image group traversed, if the preset layout analysis indicators include display position indicators, the following operations need to be performed:

[0131] Get Center Pixel: Calculate the center pixel of the target second foreground object in the segmented image of the target second foreground object in the target segmented image group. The center pixel is the pixel that represents the location of the target second foreground object in the segmented image. This can be obtained by calculating the average position of all pixels of the target second foreground object in the segmented image.

[0132] Analyze display position: This function analyzes the center pixel of the target secondary foreground object and the preset center pixel reference range. The preset center pixel reference range is a region defined within the segmented image that determines whether the foreground object is in the expected position. By comparing the center pixel of the target secondary foreground object to see if it falls within the preset center pixel reference range, you can assess whether the foreground object's display position matches your expectations.

[0133] Determine parameters: Based on the analysis results, determine the parameters of the target segmented image group under the display position indicator. This parameter can be a Boolean value (indicating whether the foreground object is in the expected position) or a distance metric between the center pixel and a preset reference range of center pixels (indicating the degree to which the foreground object deviates from the expected position).

[0134] To understand this embodiment, please refer to Figure 8 as well as Figure 9 , Figure 8 is an example diagram of the position and proportion of a second foreground object provided in an embodiment of this specification. Figure 9 This is an example diagram of the position and proportion of a second foreground object provided in an embodiment of this specification. Figure 8 In the example, the preset center pixel reference range defines an area 005 in the segmented image 004. The center pixel 007 of the target second foreground object 006 is outside the area 005. In this case, it can be considered that the target second foreground object 006 has deviated from the expected position. The parameter of the target segmented image group under the display position indicator may be determined as a Boolean value of 0 (indicating that the target second foreground object 006 has deviated from the expected position). Figure 9 In the example, the preset center pixel reference range defines an area 005 in the segmented image 004, and the center pixel 007 of the target second foreground object 006 is located in the area 005. Therefore, it can be considered that the target second foreground object 006 meets the expected position, and the parameter of the target segmented image group under the display position indicator may be determined as a Boolean value 1 (indicating that the target second foreground object 006 meets the expected position).

[0135] For each target segmentation image group traversed, if the preset layout analysis indicator includes the area ratio indicator, the following operations need to be performed:

[0136] Get Area Fraction: Calculates the area fraction of the target second foreground object in its segmented image. The area fraction is the proportion of the area occupied by the target second foreground object in the segmented image. This can be calculated by calculating the ratio of the number of pixels of the target second foreground object to the total number of pixels in the segmented image.

[0137] Analyze Area Ratio: Analyzes the area ratio of the target secondary foreground object and the preset area ratio reference range. The preset area ratio reference range is a defined area ratio range within the image that is used to determine whether the area ratio of the target secondary foreground object meets expectations. By comparing whether the area ratio of the target secondary foreground object falls within the preset area ratio reference range, you can assess whether the size of the target secondary foreground object meets expectations.

[0138] Determine parameters: Based on the analysis results, determine the parameters for the target segmented image group under the area ratio metric. This parameter can be a Boolean value (indicating whether the foreground object's area ratio meets expectations) or a measure of the difference between the area ratio and a preset reference range (indicating the degree to which the foreground object's area ratio deviates from expectations).

[0139] In this embodiment, first, all the segmented image groups to be analyzed are traversed to ensure that each segmented image group can be comprehensively evaluated. Secondly, for each target segmented image group, the display position index and the area ratio index are analyzed respectively according to the different preset layout analysis indicators. In the analysis of the display position index, by calculating the center pixel point of the target second foreground object and comparing it with the preset center pixel reference range, it is possible to accurately judge whether the display position of the foreground object meets expectations. In the analysis of the area ratio index, by calculating the area ratio of the target second foreground object and comparing it with the preset area ratio reference range, it is possible to accurately evaluate whether the area size of the foreground object meets expectations. Based on the above precise and targeted analysis method, more accurate and valuable layout analysis information is provided for the evaluation process of the extended image.

[0140] In one embodiment, Figure 2 In the embodiment shown, step S102 is further refined and may further include the following steps:

[0141] Get the segmentation prompt information corresponding to the preset object type;

[0142] Inputting the segmentation prompt information and the original image into a pre-trained image segmentation model to obtain a segmented image of the first foreground object output by the image segmentation model;

[0143] The segmentation prompt information and the expanded image are input into the image segmentation model to obtain a segmented image of the second foreground object output by the image segmentation model.

[0144] Specifically, the image segmentation model involved in this embodiment is a model that can receive text input as image segmentation guidance, such as SAM (Segment Anything Model).

[0145] First, we need to obtain segmentation hints corresponding to the preset object types. Segmentation hints are guiding data that provide information on how to identify and segment specific types of objects in an image. This information may include the object's name, category, shape, color, texture, and the object's likely location in the image.

[0146] Once the segmentation hints are obtained, they can be fed into a pre-trained image segmentation model along with the original image and the expanded image. For example, SAM is a deep learning-based model that can learn and understand complex features in an image, enabling accurate segmentation of foreground objects. When the segmentation hints and the original image are fed into SAM, SAM processes the original image based on the feature guidance provided by the segmentation hints and outputs a segmented image of the first foreground object. This segmented image may be a binary image, where the foreground object is labeled with a specific pixel value (e.g., 1) and the background with another pixel value (e.g., 0). Alternatively, in some cases, the segmented image may be an image in a format other than binary, such as a grayscale or color image. Similarly, when the segmentation hints and the expanded image are fed into SAM, SAM processes the expanded image based on the feature guidance provided by the segmentation hints and outputs a segmented image of the second foreground object.

[0147] Through the above steps, we can obtain two segmentation images of the foreground object: one is the segmentation image of the original image, and the other is the segmentation image of the expanded image. These two segmentation images represent the recognition and segmentation results of the image segmentation model under different image inputs.

[0148] In this embodiment, the segmentation hint information provides clear guidance to the image segmentation model, helping it more accurately identify and segment foreground objects in the image. Secondly, by inputting the segmentation hint information into the image segmentation model along with the original image and the expanded image, segmented images of two different foreground objects are obtained, improving segmentation accuracy and robustness.

[0149] See Figure 10 , which is a flow chart of a model training method provided in the embodiment of this specification. Figure 10 As shown, the method of the embodiment of this specification may include the following steps S502 to S508.

[0150] S502: Create an initial feature evaluation model.

[0151] Specifically, as the starting point of the model training process, the key lies in building a basic framework capable of learning and understanding image features. The initial feature evaluation model created in this embodiment is based on deep learning technologies, such as convolutional neural networks (CNNs), which can automatically extract and evaluate key features in images. When creating the initial feature evaluation model, parameters such as the structure, number of layers, and number of nodes can be configured for the initial feature evaluation model. The settings of these parameters will directly affect the complexity and learning ability of the initial feature evaluation model.

[0152] S504, obtaining an image pair sample set, wherein the image pair sample set includes multiple image pair samples with feature evaluation result annotation information, and any target image pair sample in the multiple image pair samples includes an original image sample and an extended image sample, and the extended image sample is generated by background extension based on the foreground object in the original image sample.

[0153] Specifically, in order to support the training of the initial feature evaluation model, it is necessary to obtain an image pair sample set. The image pair sample set contains multiple image pair samples with feature evaluation result annotation information, where each image pair sample consists of an original image sample and an extended image sample. The original image sample is an unprocessed natural image, while the extended image sample is generated by background extension based on the foreground object in the original image. This background extension can be achieved through AIGC technology, such as replacing or modifying the background content while keeping the foreground object unchanged. For the definition of feature evaluation result annotation information in the image pair sample set, please refer to Figure 2 The explanation of the feature evaluation results in the illustrated embodiment is different in that the feature evaluation result annotation information exists in the form of annotation information, which will serve as the goal of model training to guide the model to learn how to evaluate images.

[0154] S506, iteratively training the initial feature evaluation model based on the image pair sample set.

[0155] Specifically, in order to ensure that the initial feature evaluation model has sufficient generalization capabilities, it is necessary to iteratively train the initial feature evaluation model based on the image pair sample set. In each iteration, the initial feature evaluation model traverses the image pair sample set, extracts features and predicts feature evaluation results for each image pair sample, and then calculates the loss function value based on the difference between the predicted feature evaluation results and the feature evaluation result annotation information. The loss function value reflects the prediction error of the initial feature evaluation model under the current parameter settings. By adjusting the model parameters through optimization algorithms (such as gradient descent), the loss function value can be gradually reduced, thereby improving the prediction accuracy of the initial feature evaluation model. The iterative training process requires traversing the image pair sample set multiple times until the performance of the initial feature evaluation model reaches a stable level.

[0156] S508: If the preset convergence condition is met, the initial feature evaluation model is determined as the pre-trained feature evaluation model.

[0157] Specifically, the convergence conditions involved in this embodiment are pre-set, and the convergence conditions refer to the criteria used to determine whether the training is completed during the model training process, and may include the range of change of the loss function value, the upper limit of the number of iterations, etc. When the change in the loss function value of the initial feature evaluation model is very small in several consecutive iterations or reaches the upper limit of the number of iterations, it can be considered that the initial feature evaluation model has converged to a stable performance level. At this time, the initial feature evaluation model can be determined as a pre-trained feature evaluation model for subsequent image feature evaluation tasks. If the initial feature evaluation model does not meet the convergence conditions, it is necessary to continue iterative training until convergence is reached.

[0158] In this embodiment, first, the initial feature evaluation model constructed based on deep learning technology has powerful image feature extraction capabilities, laying a solid foundation for subsequent model training. Second, by obtaining an image pair sample set containing multiple image pair samples and annotating the feature evaluation results for each image pair sample, rich and high-quality training data is provided for model training. This enables the model to learn more accurate and comprehensive image feature evaluation knowledge during training. Finally, the initial feature evaluation model is optimized through iterative training until the preset convergence conditions are met, ensuring that the resulting feature evaluation model has stable performance and high prediction accuracy.

[0159] In one embodiment, Figure 10 The initial feature evaluation model in the illustrated embodiment includes an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer;

[0160] The image feature extraction layer is used to receive the input original image samples and the extended image samples, and obtain the image feature matrix of the original image samples and the image feature matrix of the extended image samples respectively;

[0161] A feature difference analysis layer, configured to determine feature difference information between the original image sample and the extended image sample based on an image feature matrix of the original image sample and an image feature matrix of the extended image sample;

[0162] The feature evaluation layer is used to determine the feature evaluation result of the extended image sample based on the feature difference information between the original image sample and the extended image sample.

[0163] Specifically, the initial feature evaluation model is configured as a deep learning model with a multi-layered structure, including an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer. The following is a detailed explanation of the specific implementation of these three layers:

[0164] Image Feature Extraction Layer: Serving as the input to the initial feature evaluation model, the image feature extraction layer receives the original and expanded image samples and performs deep feature extraction on them, obtaining image feature matrices for each. To achieve this, the image feature extraction layer is built on the image encoder portion of the CLIP-ViT model. The CLIP-ViT model is a vision-language pre-trained model whose image encoder automatically extracts rich feature information from images, such as edges, textures, and shapes, and represents this information in the form of a feature matrix. This allows both original and expanded image samples to obtain their respective feature representations through the image feature extraction layer.

[0165] Feature Difference Analysis Layer: The task of the Feature Difference Analysis Layer is to determine the feature difference information between the original image sample and the extended image sample based on the image feature matrix of the original image sample and the image feature matrix of the extended image sample output by the image feature extraction layer. To achieve this goal, the Feature Difference Analysis Layer can use specific algorithms or operations to calculate the difference between the two image feature matrices. For example, the degree of difference between the two image feature matrices can be measured by calculating metrics such as the Euclidean distance, cosine similarity, or the structural similarity index (SSIM). This feature difference information between the original and extended image samples can reflect the similarities and differences between the original and extended image samples at the feature level, providing an important basis for subsequent feature evaluation.

[0166] Feature Evaluation Layer: As the output of the initial feature evaluation model, the feature evaluation layer determines the feature evaluation results for the extended image samples based on the feature difference information between the original and extended image samples output by the feature difference analysis layer. To achieve this, the feature evaluation layer can be constructed based on an MLP (Multilayer Perceptron). An MLP is a feedforward neural network that further processes and judges the feature difference information and outputs the feature evaluation results for the extended image samples. For example, thresholds or rules can be set to determine whether the features of the extended image samples meet expectations or standards. If the feature difference information indicates that the extended image samples maintain consistency or similarity with the original image samples at the feature level and meet pre-set standards or requirements, the feature evaluation layer will output a positive feature evaluation result. Conversely, if the feature difference information indicates that the extended image samples differ significantly from the original image samples at the feature level or do not meet pre-set standards or requirements, the feature evaluation layer will output a negative feature evaluation result. Since the differences between the extended and original image samples primarily lie in the background, the feature evaluation results can be used to characterize the harmony between the extended and original image samples at the background level. In some possible implementations, the feature evaluation layer outputs a specific numerical value, for example, a value between 0 and 1, where a higher numerical value indicates that the extended image sample is more coordinated with the original image sample at the background level, and a lower numerical value indicates that the extended image sample is less coordinated with the original image sample at the background level.

[0167] To understand the hierarchical structure of the initial feature evaluation model in this embodiment, please refer to Figure 11 , Figure 11 This is an example diagram of a training feature analysis model provided in an embodiment of this specification.

[0168] Original image sample 008a is processed by feature saliency to obtain original image sample 008A, extended image sample 008c is processed by feature saliency to obtain extended image sample 008C, and extended image sample 008c is processed by feature saliency to obtain extended image sample 008C. Among them, extended image sample 008B is of better quality, while extended image sample 008C is of worse quality.

[0169] Original image sample 008A and extended image sample 008B actually constitute an image pair sample AB. Since extended image sample 008B is of higher quality, the feature evaluation result annotation information for this image pair sample AB can be assigned a higher value (for example, 0.9 within the possible value range of 0 to 1). This image pair sample AB serves as input to the initial feature evaluation model 009 and is first received by the image feature extraction layer 009A. Image feature extraction layer 009A extracts the image feature matrix of original image sample 008A and the image feature matrix of extended image sample 008B, respectively. Feature difference analysis layer 009B then determines feature difference information between original image sample 008A and extended image sample 008B based on the image feature matrix of original image sample 008A and the image feature matrix of extended image sample 008B. Finally, feature evaluation layer 009C determines a predicted feature evaluation result for extended image sample 008B based on the feature difference information between original image sample 008A and extended image sample 008B. The loss function is then calculated based on the difference between the predicted feature evaluation results and the annotated information. This loss function reflects the prediction error of the initial feature evaluation model 009 under the current parameter settings. By adjusting the model parameters using an optimization algorithm (such as gradient descent), the loss function value can be gradually reduced, thereby improving the prediction accuracy of the initial feature evaluation model 009.

[0170] Original image sample 008A and extended image sample 008C actually constitute an image pair sample AC. Since extended image sample 008C is of lower quality, the feature evaluation result annotation information for this image pair sample AC may be assigned a lower value (for example, 0.3 within the possible range of 0 to 1). This image pair sample AC serves as input to the initial feature evaluation model 009 and is first received by the image feature extraction layer 009A. Image feature extraction layer 009A extracts the image feature matrix of original image sample 008A and the image feature matrix of extended image sample 008C, respectively. Feature difference analysis layer 009B then determines feature difference information between original image sample 008A and extended image sample 008C based on the image feature matrix of original image sample 008A and the image feature matrix of extended image sample 008C. Finally, feature evaluation layer 009C determines a predicted feature evaluation result for extended image sample 008C based on the feature difference information between original image sample 008A and extended image sample 008C. The loss function is then calculated based on the difference between the predicted feature evaluation results and the annotated information. This loss function reflects the prediction error of the initial feature evaluation model 009 under the current parameter settings. By adjusting the model parameters using an optimization algorithm (such as gradient descent), the loss function value can be gradually reduced, thereby improving the prediction accuracy of the initial feature evaluation model 009.

[0171] Figure 11 Only some examples of image samples are given. In fact, the initial feature evaluation model is trained by relying on a large number of extended image samples with good quality, extended image samples with poor quality, and original image samples, thus having a certain generalization ability.

[0172] In this embodiment, the initial feature evaluation model is designed as a deep learning model with a multi-level structure, including an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer. This multi-level structure design enables the model to process and analyze image data more efficiently. The image feature extraction layer can automatically extract rich feature information from the original image samples and the extended image samples, providing a basis for subsequent feature difference analysis. The feature difference analysis layer can accurately calculate the feature difference information between the original image samples and the extended image samples, reflecting their similarities and differences at the feature level. Finally, the feature evaluation layer evaluates the features of the extended image samples based on the feature difference information and outputs a feature evaluation result with practical significance. This multi-level structure design not only improves the prediction accuracy of the model, but also enhances the generalization ability of the model, enabling it to be better applied to actual image feature evaluation tasks.

[0173] In one embodiment, the feature difference analysis layer is further used to extract the global feature vector of the original image sample from the image feature matrix of the original image sample, extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample, and determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample.

[0174] Specifically, the Feature Difference Analysis Layer receives the image feature matrix of the original image sample and the image feature matrix of the expanded image sample from the Image Feature Extraction Layer. These two image feature matrices contain rich information at the deep feature level of the original image and the expanded image, respectively. To more comprehensively analyze the differences between the original and expanded image samples, the Feature Difference Analysis Layer further extracts global feature vectors from these two image feature matrices.

[0175] When extracting the global feature vector, the feature difference analysis layer may focus on specific elements or regions in the image feature matrix that are believed to represent the global characteristics of the image sample. For example, in models based on the Transformer architecture, the global feature vector is often associated with a Classification Token (CLS). The CLS is a special token that serves to aggregate global information within the Transformer's self-attention mechanism. Therefore, the feature difference analysis layer can extract the feature vector corresponding to the CLS from the image feature matrix as the global feature vector.

[0176] Once the global feature vectors of the original image sample and the global feature vectors of the extended image sample are extracted, the feature difference analysis layer determines the feature difference information between the original image sample and the extended image sample based on the global feature vectors of the original image sample and the global feature vectors of the extended image sample. This process can be achieved by calculating the similarity or distance between the two global feature vectors. For example, cosine similarity can be used to measure the directional similarity of two global feature vectors, or Euclidean distance can be used to measure the distance between two global feature vectors in space. These similarity or distance indicators can reflect the similarities and differences between the original image sample and the extended image sample at the global feature level.

[0177] In addition to calculating the similarity or distance between global feature vectors, the Feature Difference Analysis layer can further analyze specific elements or dimensions within the two global feature vectors. For example, it can compare the values of the two global feature vectors along various dimensions to determine which dimensions have the most significant feature differences. This fine-grained analysis helps to gain a deeper understanding of the feature differences between the original and expanded image samples. Ultimately, the Feature Difference Analysis layer outputs the feature difference information in an appropriate format for use by the subsequent Feature Evaluation layer.

[0178] In this embodiment, the process of extracting global feature vectors enables the model to capture the overall features and key information in the image samples, rather than being limited to local details. By calculating the similarity or distance between global feature vectors, the feature difference analysis layer can more accurately measure the similarities and differences between the original image samples and the expanded image samples at the global feature level. This global feature analysis not only improves the accuracy and comprehensiveness of feature difference information, but also provides richer information for the subsequent feature evaluation layer, helping the model to produce more accurate feature evaluation results.

[0179] Based on the above Figure 2 - Figure 11 The embodiment shown in this specification may include the following steps:

[0180] In response to an original image uploaded by a user on a user interface, determining a background template corresponding to the original image;

[0181] Based on the background template corresponding to the original image, performing background expansion on the first foreground object in the original image to generate an extended image;

[0182] In response to an input original image and an extended image, performing image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image;

[0183] performing image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine a foreground object evaluation result of the extended image;

[0184] Calling a pre-trained feature evaluation model to evaluate the feature difference between the original image and the extended image, and determining the feature evaluation result of the extended image;

[0185] Based on the foreground object evaluation result and the feature evaluation result, a comprehensive evaluation result of the extended image is determined.

[0186] Specifically, in response to the original image uploaded by the user in the user interface, a series of processing is first required to determine the background template corresponding to the original image. This step can be achieved by automatically analyzing the content of the original image or by selecting the background template based on the user's operation in the user interface. Specifically, if automatic analysis is adopted, image recognition algorithms or deep learning models can be used to detect background features in the original image. These features may include color, texture, shape, and spatial layout. By analyzing these features, a feature set representing the background of the original image can be constructed, and then a background template that matches the original image can be determined or selected based on this feature set. If it is based on the user's selection operation, the background template corresponding to the original image is determined directly in response to the user's background template selection operation in the user interface.

[0187] Once the background template corresponding to the original image is determined, the next step is to perform background expansion on the first foreground object in the original image based on the background template corresponding to the original image and AIGC technology to generate an extended image. The background expansion process is essentially a reconstruction of the relationship between the foreground object and the background in the original image. In this process, based on the characteristics of the background template, AIGC technology is used to generate new background elements that match the background of the original image, and the first foreground object in the original image is spatially expanded so that its original features remain unchanged in the new background environment while being integrated with the new background elements. In some cases, this fusion involves smoothing the edges of the foreground object, adjusting lighting and shadows, and coordinating the color and texture with the new background elements to ensure the visual consistency and coherence of the extended image.

[0188] For the specific process of determining the comprehensive evaluation result of the extended image in response to the input original image and the extended image, please refer to Figure 2 The explanation of the illustrated embodiment will not be repeated here.

[0189] In this embodiment, users can upload an original image through the user interface. A background template is automatically or user-selected, and AIGC technology is then used to extend the background of the foreground object in the original image to generate an extended image. This process effectively improves the automation level of image editing and enhancement. Combined with the subsequent comprehensive evaluation of the extended image, it can quickly complete the screening and filtering of the extended image, improving the efficiency of extended image generation and evaluation.

[0190] In one embodiment, after determining the comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result, the following steps are further included:

[0191] If the comprehensive evaluation result indicates that the evaluation is passed, first evaluation result information is pushed to the user interface for display, wherein the first evaluation result information is used to prompt the user that the extended image has passed the evaluation;

[0192] If the comprehensive evaluation result indicates that the evaluation fails, the second evaluation result information is pushed to the user interface for display, wherein the second evaluation result information is used to prompt the user that the extended image fails the evaluation.

[0193] Specifically, after completing the comprehensive evaluation of the extended image and obtaining the comprehensive evaluation result, corresponding operations can be further performed based on the comprehensive evaluation result. If the comprehensive evaluation result indicates that the evaluation passed, that is, the comprehensive performance of the extended image met expectations, then the first evaluation result information will be pushed to the user interface for display. The first evaluation result information is intended to clearly indicate to the user that the extended image has successfully passed the evaluation and its quality and effect have met the expected standards. By displaying such evaluation result information, the user can intuitively understand that the generated effect of the extended image meets the requirements and can continue to use this extended image.

[0194] Conversely, if the comprehensive evaluation results indicate a failure, meaning the extended image's overall performance does not meet expectations, a second evaluation result message is pushed to the user interface for display. This second evaluation result message is intended to clearly inform the user that the extended image has failed the evaluation and may have certain issues or deficiencies. By displaying this second evaluation result message, the user can promptly identify any issues that may have occurred during the extended image generation process and make appropriate adjustments or corrections to regenerate an extended image that meets the requirements.

[0195] In this embodiment, after the comprehensive evaluation result of the extended image is determined, the corresponding evaluation result information can be intelligently pushed to the user interface, so that the user can intuitively and promptly understand the evaluation status of the extended image. This timely feedback mechanism helps users perform image editing work more efficiently.

[0196] In one embodiment, after determining the comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result, the following steps are further included:

[0197] If the comprehensive evaluation result indicates that the evaluation is passed, the extended image is pushed to the manual review interface for display; in response to the review passing operation on the manual review interface, the first evaluation result information is pushed to the user interface for display, wherein the first evaluation result information is used to prompt the user that the extended image has passed the evaluation;

[0198] If the comprehensive evaluation result indicates that the evaluation fails, the second evaluation result information is pushed to the user interface for display, wherein the second evaluation result information is used to prompt the user that the extended image fails the evaluation.

[0199] Specifically, after completing the comprehensive evaluation of the extended image and obtaining the comprehensive evaluation results, corresponding operations can be performed based on the comprehensive evaluation results. If the comprehensive evaluation results indicate that the evaluation is passed, that is, the overall performance of the extended image meets expectations, the extended image will be pushed to the manual review interface for display. This step is to ensure a more accurate and comprehensive evaluation of the extended image, as manual review can provide more detailed judgment. In the manual review interface, reviewers can carefully review and analyze the extended image to determine whether it truly meets the evaluation criteria and user needs.

[0200] In the manual review interface, if the reviewer determines that the extended image meets the evaluation criteria and user requirements, they can proceed with the approval process. In response to this approval process, a first evaluation result message is pushed to the user interface for display. This first evaluation result message clearly indicates to the user that the extended image has successfully passed the system's comprehensive evaluation and manual review, and that its quality and effectiveness have met the expected standards. By displaying this evaluation result message, the user can intuitively understand that the generated extended image meets the requirements and can continue to use the extended image.

[0201] Conversely, if the comprehensive evaluation results indicate a failure, meaning the extended image's overall performance does not meet expectations, a second evaluation result message will be pushed directly to the user interface for display. This second evaluation result message is intended to clearly inform the user that the extended image has failed the system's comprehensive evaluation and may contain certain issues or deficiencies. By displaying this second evaluation result message, the user can promptly identify any issues that may have occurred during the extended image generation process and make appropriate adjustments or corrections to regenerate an extended image that meets the requirements.

[0202] In this embodiment, after determining the comprehensive evaluation results of the extended image, a manual review process is further introduced. This dual review mechanism effectively ensures the efficiency and quality of the extended image evaluation. Furthermore, for extended images that pass or fail the evaluation, clear feedback is provided to the user in a timely manner, facilitating their management of the extended image.

[0203] To understand the effects of image evaluation in this specification, please refer to Figure 12 and Figure 13 , Figure 12 This is an example diagram of an image evaluation effect provided in an embodiment of this specification. Figure 13 This is an example schematic diagram of an image evaluation effect provided in an embodiment of this specification.

[0204] like Figure 12As shown, the original image containing the glasses case and glasses corresponds to two extended images, and both extended images are determined to be unapproved due to the inconsistent background; the original image containing the handbag corresponds to two extended images, and both extended images are determined to be unapproved due to the inconsistent background.

[0205] like Figure 13 As shown, the original image containing shoes corresponds to four extended images, and all of these four extended images have passed the evaluation of foreground objects and background features, and the four extended images are ultimately determined to have passed the evaluation; the original image containing a vacuum cleaner corresponds to four extended images, and all of these four extended images have passed the evaluation of foreground objects and background features, and the four extended images are ultimately determined to have passed the evaluation.

[0206] based on Figure 1 The following is a scene diagram of Figure 14 , the image evaluation device provided in the embodiment of this specification is introduced in detail. It should be noted that, Figure 14 The image evaluation device 1 is used to execute the Figure 2 - Figure 13 For the convenience of explanation, only the part related to the embodiment of this specification is shown. For the specific technical details not disclosed, please refer to this specification. Figure 2 - Figure 13 The embodiment shown.

[0207] The image evaluation device 1 may include:

[0208] a segmentation unit 11 configured to, in response to an input original image and an extended image, perform image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and to perform image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image;

[0209] a first evaluation unit 12 for performing image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine an evaluation result of the foreground object of the extended image;

[0210] The second evaluation unit 13 is used to call a pre-trained feature evaluation model to evaluate the feature difference between the original image and the extended image, and determine the feature evaluation result of the extended image;

[0211] The determining unit 14 is configured to determine a comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result.

[0212] Optionally, there is at least one first foreground object and at least one second foreground object; the first evaluation unit 12 is further used to: determine the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object for each first foreground object in the at least one first foreground object and each second foreground object in the at least one second foreground object; determine at least one segmented image group based on the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object, wherein any one segmented image group in the at least one segmented image group includes a corresponding segmented image of a first foreground object and a segmented image of a second foreground object; obtain parameters of each segmented image group under a preset content comparison index; obtain parameters of each segmented image group under a preset layout analysis index; determine the foreground object evaluation result of the extended image based on the parameters of each segmented image group under the preset content comparison index and the parameters under the preset layout analysis index.

[0213] Optionally, the first evaluation unit 12 is further used to: traverse at least one segmented image group; for any target segmented image group in the at least one segmented image group traversed, if the preset content comparison index includes a peak signal-to-noise ratio index, obtain the peak signal-to-noise ratio value of the segmented image of the target first foreground object in the target segmented image group, and the peak signal-to-noise ratio value of the segmented image of the target second foreground object in the target segmented image group; compare the peak signal-to-noise ratio value of the segmented image of the target first foreground object with the peak signal-to-noise ratio value of the segmented image of the target second foreground object to determine the parameters of the target segmented image group under the peak signal-to-noise ratio index; if the preset content comparison index includes a multi-scale structural similarity index, obtain the multi-scale structure of the segmented image of the target first foreground object. similarity index value, and the multi-scale structural similarity index value of the segmented image of the target second foreground object; according to the multi-scale structural similarity index value of the segmented image of the target first foreground object and the multi-scale structural similarity index value of the segmented image of the target second foreground object, the parameters of the target segmented image group under the multi-scale structural similarity index indicator are determined; if the preset content comparison indicator includes a foreground object area overlap rate indicator, the area of the target first foreground object in the segmented image of the target first foreground object and the area of the target second foreground object in the segmented image of the target second foreground object are obtained; according to the area of the target first foreground object and the area of the target second foreground object, the parameters of the target segmented image group under the foreground object area overlap rate indicator are determined.

[0214] Optionally, the first evaluation unit 12 is further used to: traverse at least one segmented image group; for any target segmented image group in the at least one traversed segmented image group, if the preset layout analysis index includes a display position index, obtain the center pixel point of the target second foreground object in the segmented image of the target second foreground object in the target segmented image group; perform analysis based on the center pixel point of the target second foreground object and the preset center pixel reference range to determine the parameters of the target segmented image group under the display position index; if the preset layout analysis index includes an area ratio index, obtain the area ratio of the target second foreground object in the segmented image of the target second foreground object; perform analysis based on the area ratio of the target second foreground object and the preset area ratio reference range to determine the parameters of the target segmented image group under the area ratio index.

[0215] Optionally, the segmentation unit 11 is further used to: obtain segmentation prompt information corresponding to a preset object type; input the segmentation prompt information and the original image into a pre-trained image segmentation model to obtain a segmented image of the first foreground object output by the image segmentation model; input the segmentation prompt information and the extended image into the image segmentation model to obtain a segmented image of the second foreground object output by the image segmentation model.

[0216] The effects that can be achieved by this embodiment can be found in the relevant embodiments of the above-mentioned image evaluation method, which will not be described in detail here.

[0217] based on Figure 1 The following is a scene diagram of Figure 15 , the model training device provided in the embodiment of this specification is introduced in detail. It should be noted that, Figure 15 The model training device 2 in this manual is used to execute Figure 2 - Figure 13 For the convenience of explanation, only the part related to the embodiment of this specification is shown. For the specific technical details not disclosed, please refer to this specification. Figure 2 - Figure 13 The embodiment shown.

[0218] The model training device 2 may include:

[0219] A creation unit 21, used for creating an initial feature evaluation model;

[0220] an acquisition unit 22 configured to acquire an image pair sample set, wherein the image pair sample set includes a plurality of image pair samples annotated with feature evaluation result information, wherein any target image pair sample among the plurality of image pair samples includes an original image sample and an extended image sample, wherein the extended image sample is generated by performing background extension based on a foreground object in the original image sample;

[0221] A training unit 23 is used to iteratively train the initial feature evaluation model based on the image pair sample set;

[0222] The determining unit 24 is configured to determine the initial feature evaluation model as a pre-trained feature evaluation model if a preset convergence condition is met.

[0223] Optionally, the initial feature evaluation model includes an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer; the image feature extraction layer is used to receive the input original image samples and extended image samples, and obtain the image feature matrix of the original image samples and the image feature matrix of the extended image samples, respectively; the feature difference analysis layer is used to determine the feature difference information between the original image samples and the extended image samples based on the image feature matrix of the original image samples and the image feature matrix of the extended image samples; the feature evaluation layer is used to determine the feature evaluation result of the extended image sample based on the feature difference information between the original image samples and the extended image samples.

[0224] Optionally, the feature difference analysis layer is also used to extract the global feature vector of the original image sample from the image feature matrix of the original image sample, extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample, and determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample.

[0225] For the effects that can be achieved by this embodiment, please refer to the relevant embodiments of the above-mentioned model training method, which will not be repeated here.

[0226] See Figure 16 , is a schematic diagram of the structure of an electronic device provided in the embodiment of this specification. Figure 16 As shown, the electronic device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, an input / output interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Figure 16 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, an input and output interface module, an image evaluation application, and a model training application.

[0227] exist Figure 16 In the electronic device 1000 shown, the input / output interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user.

[0228] In one embodiment, the processor 1001 may be configured to call an image evaluation application stored in the memory 1005 and specifically perform the following operations:

[0229] In response to an input original image and an extended image, performing image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image;

[0230] performing image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine a foreground object evaluation result of the extended image;

[0231] Calling a pre-trained feature evaluation model to evaluate the feature difference between the original image and the extended image, and determining the feature evaluation result of the extended image;

[0232] Based on the foreground object evaluation result and the feature evaluation result, a comprehensive evaluation result of the extended image is determined.

[0233] Optionally, there is at least one first foreground object and at least one second foreground object; when the processor 1001 performs image comparison based on the segmented image of the first foreground object and the segmented image of the second foreground object to determine the foreground object evaluation result of the extended image, the following specific operations are performed: for each first foreground object in the at least one first foreground object and each second foreground object in the at least one second foreground object, the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object is determined; based on the pixel similarity between the segmented image of each first foreground object and the segmented image of each second foreground object, at least one segmented image group is determined, wherein any one segmented image group in the at least one segmented image group includes a corresponding segmented image of the first foreground object and a segmented image of the second foreground object; parameters of each segmented image group under a preset content comparison index are obtained respectively; parameters of each segmented image group under a preset layout analysis index are obtained respectively; and based on the parameters of each segmented image group under the preset content comparison index and the parameters under the preset layout analysis index, the foreground object evaluation result of the extended image is determined.

[0234] Optionally, when the processor 1001 executes the execution of respectively obtaining the parameters of each segmented image group under the preset content comparison index, the processor 1001 specifically performs the following operations: traverses at least one segmented image group; for any target segmented image group in the at least one traversed segmented image group, if the preset content comparison index includes a peak signal-to-noise ratio index, obtains the peak signal-to-noise ratio value of the segmented image of the target first foreground object in the target segmented image group, and the peak signal-to-noise ratio value of the segmented image of the target second foreground object in the target segmented image group; determines the parameters of the target segmented image group under the peak signal-to-noise ratio index based on the peak signal-to-noise ratio value of the segmented image of the target first foreground object and the peak signal-to-noise ratio value of the segmented image of the target second foreground object; if the preset content comparison index includes a multi-scale structural similarity index index, obtains the peak signal-to-noise ratio value of the target first foreground object The method comprises the steps of: determining a multi-scale structural similarity index value of a segmented image of a first foreground object and a multi-scale structural similarity index value of a segmented image of a second foreground object; comparing the multi-scale structural similarity index value of the segmented image of the first foreground object and the multi-scale structural similarity index value of the segmented image of the second foreground object to determine parameters of the target segmented image group under the multi-scale structural similarity index indicator; if the preset content comparison indicator includes a foreground object area overlap ratio indicator, obtaining the area of the target first foreground object in the segmented image of the first foreground object and the area of the target second foreground object in the segmented image of the second foreground object; and determining parameters of the target segmented image group under the foreground object area overlap ratio indicator based on the comparison of the area of the first foreground object and the area of the second foreground object.

[0235] Optionally, when the processor 1001 executes the operation of respectively obtaining the parameters of each segmented image group under the preset layout analysis index, the following specific operations are performed: traversing at least one segmented image group; for any target segmented image group in the at least one traversed segmented image group, if the preset layout analysis index includes a display position index, obtaining the center pixel point of the target second foreground object in the segmented image of the target second foreground object in the target segmented image group; performing analysis based on the center pixel point of the target second foreground object and the preset center pixel reference range to determine the parameters of the target segmented image group under the display position index; if the preset layout analysis index includes an area ratio index, obtaining the area ratio of the target second foreground object in the segmented image of the target second foreground object; performing analysis based on the area ratio of the target second foreground object and the preset area ratio reference range to determine the parameters of the target segmented image group under the area ratio index.

[0236] Optionally, when performing image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that conforms to a preset object type, the processor 1001 specifically performs the following operations: obtaining segmentation prompt information corresponding to the preset object type; inputting the segmentation prompt information and the original image into a pre-trained image segmentation model to obtain a segmented image of the first foreground object output by the image segmentation model; inputting the segmentation prompt information and the extended image into the image segmentation model to obtain a segmented image of the second foreground object output by the image segmentation model.

[0237] In one embodiment, the processor 1001 may be configured to call a model training application stored in the memory 1005 and specifically perform the following operations:

[0238] Create an initial feature evaluation model;

[0239] Acquire an image pair sample set, wherein the image pair sample set includes multiple image pair samples with feature evaluation result annotation information, and any target image pair sample in the multiple image pair samples includes an original image sample and an extended image sample, and the extended image sample is generated by background extension based on a foreground object in the original image sample;

[0240] Iteratively train the initial feature evaluation model based on the image pair sample set;

[0241] If the preset convergence condition is met, the initial feature evaluation model is determined as the pre-trained feature evaluation model.

[0242] Optionally, the initial feature evaluation model includes an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer; the image feature extraction layer is used to receive the input original image samples and extended image samples, and obtain the image feature matrix of the original image samples and the image feature matrix of the extended image samples, respectively; the feature difference analysis layer is used to determine the feature difference information between the original image samples and the extended image samples based on the image feature matrix of the original image samples and the image feature matrix of the extended image samples; the feature evaluation layer is used to determine the feature evaluation result of the extended image sample based on the feature difference information between the original image samples and the extended image samples.

[0243] Optionally, the feature difference analysis layer is also used to extract the global feature vector of the original image sample from the image feature matrix of the original image sample, extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample, and determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample.

[0244] The effects that can be achieved by this embodiment can be found in the relevant embodiments of the above-mentioned image evaluation method and model training method, which will not be repeated here.

[0245] The embodiment of this specification also provides a computer storage medium that can store multiple program instructions, which are suitable for being loaded and executed by a processor as described above. Figure 2 - Figure 13 The method steps of the embodiment shown, the specific execution process can be found in Figure 2 - Figure 13 The detailed description of the illustrated embodiment will not be repeated here.

[0246] The embodiment of this specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by a processor as described above. Figure 2 - Figure 13 The image evaluation method of the embodiment shown, the specific execution process can be found in Figure 2 - Figure 13 The detailed description of the illustrated embodiment will not be repeated here.

[0247] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0248] The above disclosure is only a preferred embodiment of this specification, and certainly cannot be used to limit the scope of rights of this specification. Therefore, equivalent changes made according to the claims of this specification are still within the scope covered by this specification.

Claims

1. An image evaluation method, comprising: In response to an input original image and an extended image, performing image segmentation on the original image to obtain a segmented image of a first foreground object that meets a preset object type, and performing image segmentation on the extended image to obtain a segmented image of a second foreground object that meets the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image; Obtaining parameters of each segmented image group in at least one segmented image group under a preset content comparison index and a preset layout analysis index, and determining a foreground object evaluation result of the extended image based on the parameters of each segmented image group under the preset content comparison index and the preset layout analysis index, wherein there is at least one first foreground object and at least one second foreground object, and any one of the at least one segmented image group includes a corresponding segmented image of the first foreground object and a corresponding segmented image of the second foreground object; Calling a pre-trained feature evaluation model to perform feature difference evaluation on the original image and the extended image, and determining a feature evaluation result of the extended image; determining a comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result; The pre-trained feature evaluation model is obtained by iterative training based on the initial feature evaluation model; The initial feature evaluation model is used to: obtain and extract the global feature vector of the original image sample from the image feature matrix of the original image sample; obtain and extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample; determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample; and determine the feature evaluation result of the extended image sample based on the feature difference information between the original image sample and the extended image sample.

2. The method according to claim 1, wherein obtaining parameters of each of the at least one segmented image group under a preset content comparison index and a preset layout analysis index, and determining a foreground object evaluation result of the expanded image based on the parameters of each of the segmented image groups under the preset content comparison index and the preset layout analysis index, comprises: for each first foreground object in at least one first foreground object and each second foreground object in at least one second foreground object, determining pixel similarities between a segmented image of each first foreground object and a segmented image of each second foreground object; determining at least one segmented image group based on pixel similarities between the segmented images of each of the first foreground objects and the segmented images of each of the second foreground objects; respectively obtaining parameters of each of the segmented image groups under a preset content comparison index; respectively obtaining parameters of each of the segmented image groups under a preset layout analysis index; An evaluation result of the foreground object of the expanded image is determined based on the parameters of each of the segmented image groups under the preset content comparison index and the parameters under the preset layout analysis index.

3. The method according to claim 2, wherein the step of respectively obtaining parameters of each of the segmented image groups under a preset content comparison index comprises: traversing at least one of the segmented image groups; For any one target segmented image group in at least one of the traversed segmented image groups, if the preset content comparison indicator includes a peak signal-to-noise ratio indicator, obtaining a peak signal-to-noise ratio value of a segmented image of a target first foreground object in the target segmented image group, and a peak signal-to-noise ratio value of a segmented image of a target second foreground object in the target segmented image group; Determining parameters of the target segmented image group under the peak signal-to-noise ratio indicator by comparing a peak signal-to-noise ratio value of the segmented image of the target first foreground object with a peak signal-to-noise ratio value of the segmented image of the target second foreground object; If the preset content comparison index includes a multi-scale structural similarity index, obtaining a multi-scale structural similarity index value of the segmented image of the target first foreground object and a multi-scale structural similarity index value of the segmented image of the target second foreground object; Determining parameters of the target segmented image group under the multi-scale structural similarity index indicator based on a comparison of a multi-scale structural similarity index value of the segmented image of the target first foreground object and a multi-scale structural similarity index value of the segmented image of the target second foreground object; If the preset content comparison index includes a foreground object area overlap rate index, obtaining the area of the target first foreground object in the segmented image of the target first foreground object and the area of the target second foreground object in the segmented image of the target second foreground object; The parameters of the target segmented image group under the foreground object area overlap ratio indicator are determined based on a comparison between the area of the target first foreground object and the area of the target second foreground object.

4. The method according to claim 2, wherein the step of respectively obtaining parameters of each of the segmented image groups under a preset layout analysis index comprises: traversing at least one of the segmented image groups; For any one target segmented image group in at least one of the traversed segmented image groups, if the preset layout analysis index includes a display position index, obtaining a center pixel point of the target second foreground object in a segmented image of the target second foreground object in the target segmented image group; performing analysis based on the center pixel point of the target second foreground object and a preset center pixel reference range to determine parameters of the target segmented image group under the display position index; If the preset layout analysis index includes an area ratio index, the area ratio of the target second foreground object in the segmented image of the target second foreground object is obtained; analysis is performed based on the area ratio of the target second foreground object and the preset area ratio reference range to determine the parameters of the target segmented image group under the area ratio index.

5. The method according to claim 1, wherein performing image segmentation on the original image to obtain a segmented image of a first foreground object that conforms to a preset object type, and performing image segmentation on the expanded image to obtain a segmented image of a second foreground object that conforms to the preset object type, comprises: Obtain segmentation prompt information corresponding to the preset object type; Inputting the segmentation prompt information and the original image into a pre-trained image segmentation model to obtain a segmented image of the first foreground object output by the image segmentation model; The segmentation prompt information and the expanded image are input into the image segmentation model to obtain a segmented image of the second foreground object output by the image segmentation model.

6. A model training method comprising: Create an initial feature evaluation model; Acquire an image pair sample set, wherein the image pair sample set includes a plurality of image pair samples with feature evaluation result annotation information, and any target image pair sample among the plurality of image pair samples includes an original image sample and an extended image sample, wherein the extended image sample is generated by performing background extension based on a foreground object in the original image sample; Iteratively training the initial feature evaluation model based on the image pair sample set; If the preset convergence condition is met, the initial feature evaluation model is determined as the pre-trained feature evaluation model, and the pre-trained feature evaluation model is applied to the image evaluation method according to any one of claims 1 to 5.

7. The method according to claim 6, wherein the initial feature evaluation model comprises an image feature extraction layer, a feature difference analysis layer, and a feature evaluation layer; The image feature extraction layer is configured to receive the input original image sample and the extended image sample, and respectively obtain an image feature matrix of the original image sample and an image feature matrix of the extended image sample; The feature difference analysis layer is configured to determine feature difference information between the original image sample and the extended image sample based on the image feature matrix of the original image sample and the image feature matrix of the extended image sample; The feature evaluation layer is used to determine a feature evaluation result of the extended image sample based on feature difference information between the original image sample and the extended image sample.

8. According to the method according to claim 7, the feature difference analysis layer is further used to extract the global feature vector of the original image sample from the image feature matrix of the original image sample, extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample, and determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample.

9. An image evaluation device comprising: a segmentation unit configured to, in response to an input original image and an extended image, perform image segmentation on the original image to obtain a segmented image of a first foreground object that meets a preset object type, and to perform image segmentation on the extended image to obtain a segmented image of a second foreground object that meets the preset object type, wherein the extended image is generated by performing background extension based on the first foreground object in the original image; a first evaluation unit, configured to obtain parameters of each segmented image group in at least one segmented image group under a preset content comparison index and a preset layout analysis index, and determine an evaluation result of a foreground object of the extended image based on the parameters of each segmented image group under the preset content comparison index and the preset layout analysis index, wherein there is at least one first foreground object and at least one second foreground object, and any one of the at least one segmented image group includes a corresponding segmented image of the first foreground object and a corresponding segmented image of the second foreground object; a second evaluation unit, configured to call a pre-trained feature evaluation model to evaluate feature differences between the original image and the extended image, and determine a feature evaluation result of the extended image; a determining unit, configured to determine a comprehensive evaluation result of the extended image based on the foreground object evaluation result and the feature evaluation result; The pre-trained feature evaluation model is obtained by iterative training based on the initial feature evaluation model; The initial feature evaluation model is used to: obtain and extract the global feature vector of the original image sample from the image feature matrix of the original image sample; obtain and extract the global feature vector of the extended image sample from the image feature matrix of the extended image sample; determine the feature difference information between the original image sample and the extended image sample based on the global feature vector of the original image sample and the global feature vector of the extended image sample; and determine the feature evaluation result of the extended image sample based on the feature difference information between the original image sample and the extended image sample.

10. A model training device comprising: A creation unit for creating an initial feature evaluation model; an acquiring unit, configured to acquire an image pair sample set, wherein the image pair sample set includes a plurality of image pair samples annotated with feature evaluation result information, and any target image pair sample among the plurality of image pair samples includes an original image sample and an extended image sample, wherein the extended image sample is generated by performing background extension based on a foreground object in the original image sample; A training unit, configured to iteratively train the initial feature evaluation model based on the image pair sample set; The determining unit is configured to determine the initial feature evaluation model as the pre-trained feature evaluation model if a preset convergence condition is met, and the pre-trained feature evaluation model is applied to the image evaluation method according to any one of claims 1 to 5.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program code, and when the computer program code is executed, the method according to any one of claims 1 to 8 is implemented.

12. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method according to any one of claims 1 to 8.

13. A computer program product having at least one instruction stored thereon, wherein when the at least one instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images