Image editing method, virtual fitting method, model training method and equipment

Abstract

The embodiment of the invention provides an image editing method, a virtual fitting method, a model training method and equipment. In the method, the image editing model can be called to edit the initial image to obtain the intermediate image, and whether the editing effect of the intermediate image is matched with the image editing instruction can be automatically determined, so that self-evaluation is realized; and when the editing effect of the intermediate image is not matched with the image editing instruction, the image correction instruction can be generated to correct the intermediate image until the target image conforming to the image editing instruction is obtained, so that self-correction is realized. The image editing model forms a closed loop of'editing-evaluating-correcting ', initial editing errors of the image editing model can be actively recognized and corrected, the image editing effect is improved, and therefore the accuracy of generating the target image through the image editing model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image editing method, a virtual fitting method, a model training method and equipment. BACKGROUND

[0002] In a virtual fitting scene, an image of a virtual person wearing different clothes can be generated by a model to show the fitting effect of different clothes to a user. In the process, the user can input an image editing instruction, and the model can modify an initial image based on the image editing instruction input by the user to generate a target image. For example, the initial image can be an image of a virtual person wearing a white short-sleeved shirt, and the user can input an image editing instruction of "changing the white short-sleeved shirt to a green short-sleeved shirt". The model can modify the white short-sleeved shirt worn by the virtual person in the initial image to a green short-sleeved shirt based on the image editing instruction to obtain a target image, and the target image can show the fitting effect of the virtual person wearing a green short-sleeved shirt.

[0003] However, in actual application, the editing effect of the target image can not meet the image editing instruction, resulting in that the target image cannot well show the fitting effect. Therefore, a solution is urgently needed to improve the image editing effect and thus improve the image quality. SUMMARY

[0004] Embodiments of the present application provide an image editing method, a virtual fitting method, a model training method and equipment to improve the image editing effect and thus improve the image quality.

[0005] An image editing method is provided in embodiments of the present application, which includes: in response to an image editing instruction, determining an initial image to be processed; according to the image editing instruction, calling an image editing model to edit the initial image to obtain an intermediate image; if the editing effect of the intermediate image does not match the image editing instruction, generating an image correction instruction according to the image editing instruction and the intermediate image; and correcting the intermediate image according to the image correction instruction until a target image matching the image editing instruction is obtained.

[0006] The embodiment of the present application provides a virtual fitting method, comprising: displaying an initial image including an initial garment worn by a virtual person on a fitting interface; calling an image editing model according to an image editing instruction to edit the initial image to obtain an intermediate image; the image editing instruction is used to indicate to replace the initial garment in the initial image; if the editing effect of the intermediate image does not match the image editing instruction, generating an image correction instruction according to the image editing instruction and the intermediate image; correcting the intermediate image according to the image correction instruction until a target image matching the image editing instruction is obtained; displaying the target image on the fitting interface, and the target image is used to show the effect of the virtual person wearing a target garment.

[0007] The embodiment of the present application provides a model training method, comprising: obtaining a training sample, wherein the training sample includes an initial image, an image editing instruction, a first labeled image, a labeled matching result, a labeled instruction for image correction and a second labeled image, and the second labeled image matches the image editing instruction; generating a predicted matching result and a predicted instruction for image correction by an initial model based on the initial image, the image editing instruction and the first labeled image; correcting the first labeled image according to the predicted instruction by the initial model to generate a predicted image; updating model parameters of the initial model based on the predicted matching result and the labeled matching result, the predicted instruction and the labeled instruction, and the predicted image and the second labeled image to obtain an image editing model.

[0008] The embodiment of the present application further provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled with the memory and is used to execute the computer program in the memory to implement steps in the image editing method, the virtual fitting method and the model training method.

[0009] The embodiment of the present application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor can implement steps in the image editing method, the virtual fitting method and the model training method.

[0010] The embodiment of the present application further provides a computer program product, comprising a computer program / instruction, when the computer program / instruction is executed by a processor, the processor can implement steps in the image editing method, the virtual fitting method and the model training method.

[0011] In the embodiment of the present application, the image editing model can be called to edit the initial image to obtain an intermediate image, and whether the editing effect of the intermediate image matches the image editing instruction can be automatically determined, so that self-evaluation is realized. When the editing effect of the intermediate image does not match the image editing instruction, an image correction instruction can be generated to correct the intermediate image until a target image that meets the image editing instruction is obtained, so that self-correction is realized. The image editing model forms a closed loop of "editing-evaluating-correcting", which can actively identify and correct the initial editing error of itself, improves the image editing effect, and thus improves the accuracy of generating the target image through the image editing model. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application, illustrate the exemplary embodiments of the present application and their descriptions serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings: Figure 1 A flowchart of an image editing method provided for the exemplary embodiments of the present application; Figure 2 A schematic diagram of an image editing process provided for the exemplary embodiments of the present application; Figure 3a A process schematic of an image editing method provided for the exemplary embodiments of the present application Figure 1 ; Figure 3b A process schematic of an image editing method provided for the exemplary embodiments of the present application Figure 2 ; Figure 3c A process schematic of an image editing method provided for the exemplary embodiments of the present application Figure 3d A process schematic of an image editing method provided for the exemplary embodiments of the present application Figure 4 ; Figure 3e A process schematic of an image editing method provided for the exemplary embodiments of the present application Figure 5 ; Figure 4 A flowchart of a virtual fitting method provided for the exemplary embodiments of the present application; Figure 5 A schematic diagram of a fitting interface provided for the exemplary embodiments of the present application; Figure 6 A flowchart of a model training method provided for the embodiments of the present application; Figure 7 A structural schematic of an image editing device provided for the exemplary embodiments of the present application; Figure 8A schematic diagram of the structure of a virtual fitting device provided for an exemplary embodiment of this application; Figure 9 A schematic diagram of the structure of a model training device provided for an exemplary embodiment of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0015] Additionally, it should be noted that when user interaction operations or triggering operations are involved in the embodiments of this application, these operations include, but are not limited to, various interaction methods such as touch operations, gesture operations, voice operations, head movement operations, and eye movement operations. Touch operations include, but are not limited to, click operations, double-click operations, long-press operations, swipe operations, pinch operations, or mouse hover operations. Swipe operations include, but are not limited to, straight-line swipes and curved-line swipes.

[0016] In image editing technologies, image editing models can modify an initial image based on user-inputted image editing instructions to generate a target image, or vice versa. For example, the initial image could be a virtual person wearing a white short-sleeved shirt. The user could input the image editing instruction "replace the white short-sleeved shirt with a green one." The model could then modify the white short-sleeved shirt in the initial image to a green one, obtaining the target image that shows the virtual person wearing the green shirt. However, in practical applications, the edited result of the target image may not match the image editing instructions, resulting in the target image not accurately displaying the fitting effect. Therefore, in image editing technologies, the image editing effect is often poor, and the accuracy of the edited image is low.

[0017] To address the aforementioned technical problems, in this embodiment, the image editing model possesses an "edit-evaluate-correct" capability. It can be invoked to edit an initial image to obtain an intermediate image, and can automatically determine whether the editing effect of the intermediate image matches the image editing instructions, thus achieving self-evaluation. Furthermore, when the editing effect of the intermediate image does not match the image editing instructions, it can generate image correction instructions to correct the intermediate image until a target image conforming to the image editing instructions is obtained, achieving self-correction. The image editing model forms a closed loop of "edit-evaluate-correct," actively identifying and correcting its own initial editing errors, thereby improving the accuracy of generating target images through the image editing model.

[0018] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0019] Figure 1 This is a flowchart illustrating an image editing method provided for an exemplary embodiment of this application. Figure 1 As shown, the method includes: S11. In response to an image editing instruction, determine the initial image to be processed.

[0020] S12. According to the image editing instructions, call the image editing model to edit the initial image to obtain the intermediate image.

[0021] S13. If the editing effect of the intermediate image does not match the image editing instructions, then generate image correction instructions based on the image editing instructions and the intermediate image.

[0022] S14. Correct the intermediate image according to the image correction instructions until the target image that matches the image editing instructions is obtained.

[0023] In this embodiment, the executing entity of the image editing method is not limited; it can be an electronic device or an image editing device installed in an electronic device. The image editing device can be implemented through software or a combination of software and hardware. The image editing device can be a processor in an electronic device. For ease of understanding, the following description uses an electronic device as the executing entity.

[0024] In this embodiment, image editing instructions can be used to instruct the editing of an initial image. Image editing instructions can be in text or voice format.

[0025] For example, image editing instruction 1 could be "Change the beige cotton dress worn by the virtual person to dark blue silk fabric, enhance the flow of the skirt, and remove the decoration at the waist."

[0026] Electronic devices can display an input interface. Users can input image editing commands via text or voice on the input interface. The electronic device can then respond to these commands and determine the initial image to be processed.

[0027] The initial image can be either an image input by the user or an image pre-generated using a text-based image model.

[0028] Below, in conjunction with Figure 2 The image editing process is explained. Figure 2 This is a schematic diagram illustrating an image editing process provided for an exemplary embodiment of this application. For example... Figure 2 As shown, it includes images 201, 202 and 203.

[0029] The user can input image editing command 1 and image 201 into the electronic device. If image editing command 1 is as shown in the example above, the electronic device can respond to image editing command 1, obtain image 201, and use image 201 as initial image 1. In the initial image 1, the virtual person is wearing a beige cotton dress and has decorations at the waist.

[0030] Image editing models can be deployed in electronic devices. These models can possess "edit-evaluation-correction capabilities." For example, an image editing model could be a unified generative understanding model.

[0031] The electronic device can call the image editing model according to the image editing instructions to execute steps S12 to S14, that is, steps S12 to S14 are executed by the image editing model.

[0032] The image editing model can edit an initial image according to image editing instructions to obtain an intermediate image. After obtaining the intermediate image, it can be determined whether the edited effect of the intermediate image matches the image editing instructions. If the edited effect of the intermediate image matches the image editing instructions, no further correction is needed, and the intermediate image can be used as the target image. If the edited effect of the intermediate image does not match the image editing instructions, the intermediate image can be corrected at least once until the target image that conforms to the image editing instructions is obtained.

[0033] For any given correction, the electronic device can determine whether the editing effect of the intermediate image matches the image editing instruction through an image editing model. If the editing effect of the intermediate image does not match the image editing instruction, it identifies the discrepancies in the intermediate image that do not match the image editing instruction, and then generates an image correction instruction based on these discrepancies. The image correction instruction is used to instruct the correction of features in the intermediate image that do not match the image editing instruction.

[0034] It's important to note that after each correction, the focus is on determining whether the corrected intermediate image conforms to the image editing instructions, rather than simply whether it conforms to the image correction instructions used in the current correction process. This is to prevent the correction process from getting stuck in local optima. The image editing instructions represent the initial user intent; therefore, after each correction, it's necessary to determine whether the corrected intermediate image conforms to those instructions.

[0035] For example, consider a three-stage correction process. An electronic device can edit an initial image using an image editing model according to image editing instructions to obtain a first intermediate image. If the edited effect of the first intermediate image does not match the image editing instructions, a first image correction instruction can be generated based on the image editing instructions and the first intermediate image. This first correction instruction is then used to modify the first intermediate image, resulting in a second intermediate image. If the edited effect of the second intermediate image still does not match the image editing instructions, a second image correction instruction can be generated based on the image editing instructions and the second intermediate image. This second correction instruction is then used to modify the second intermediate image, resulting in a third intermediate image. If the edited effect of the third intermediate image still does not match the image editing instructions, a third image correction instruction can be generated based on the image editing instructions and the third intermediate image. This third correction instruction is then used to modify the third intermediate image, resulting in a fourth intermediate image. If the fourth intermediate image matches the image editing instructions, it can be used as the target image.

[0036] It should be noted that before obtaining the target image, each image obtained through correction is called an intermediate image.

[0037] For example, if image editing instruction 1 is as shown in the example above, and the initial image is as shown in image 201, the electronic device can edit image 201 according to image editing instruction 1 to obtain an intermediate image as shown in image 202. If the image editing model determines that the editing effect of the intermediate image does not match image editing instruction 1, then the image editing model can determine the discrepancies in the intermediate image that do not match image editing instruction 1 as follows: the skirt material is not a glossy silk texture, the skirt hem has poor flow, and the waist decoration has not been removed. Therefore, image correction instruction 1 can be generated based on these discrepancies, such as: adjusting the skirt material to a glossy silk texture, increasing the flow of the skirt hem, and removing the waist decoration. Then, the intermediate image (i.e., image 202) can be corrected according to this image correction instruction 1 to obtain the next intermediate image. Assuming the next intermediate image is as follows... Figure 2 As shown in image 203: the skirt is made of dark blue silk fabric, has a flowing texture, and the waist decoration has been removed. If the editing effect of the next intermediate image matches image editing instruction 1, then the next intermediate image (i.e., image 203) can be used as the target image.

[0038] After the target image is generated, the electronic device can display the target image on the display interface.

[0039] In this embodiment, the electronic device can call an image editing model to edit an initial image to obtain an intermediate image, and can automatically determine whether the editing effect of the intermediate image matches the image editing instructions, thus achieving self-evaluation. Furthermore, when the editing effect of the intermediate image does not match the image editing instructions, the image editing model can generate image correction instructions to correct the intermediate image until a target image conforming to the image editing instructions is obtained, thus achieving self-correction. The image editing model forms a closed loop of "editing-evaluation-correction," enabling it to proactively identify and correct its own initial editing errors, thereby improving the accuracy of generating target images through the image editing model.

[0040] In this application embodiment, there are no restrictions on the specific implementation of the above step S12 "according to the image editing instructions, call the image editing model to edit the initial image to obtain the intermediate image".

[0041] In an exemplary embodiment, the following steps S121-S123 can be used to call an image editing model according to image editing instructions to edit the initial image to obtain an intermediate image: S121. Parse and process the image editing instructions to obtain multiple sub-editing instructions.

[0042] S122. Based on the editing objects in the multiple sub-editing instructions, determine the sub-regions corresponding to each of the multiple sub-editing instructions from the initial image.

[0043] S123. Based on the editing requirements in multiple sub-editing instructions, edit the sub-regions corresponding to each of the multiple sub-editing instructions to obtain an intermediate image.

[0044] Optionally, in step S121, the electronic device can perform semantic analysis processing on the image editing instructions through the image editing model to identify multiple entities in the image editing instructions, determine the relationship between the multiple entities and attributes and actions, obtain semantic analysis results (including multiple entities, the attributes of each entity, and the relationship between entities and actions), and decompose the image editing instructions based on the semantic analysis results to obtain multiple sub-editing instructions.

[0045] For any given sub-editing instruction, the sub-editing instruction can include the object to be edited and the editing requirements.

[0046] Optionally, the multiple sub-editing instructions include a first sub-editing instruction and a second sub-editing instruction. The first sub-editing instruction instructs modification of relevant features of the object to be edited in the initial image; the second sub-editing instruction instructs that the relevant features in the initial image be kept unchanged.

[0047] For example, if image editing instruction 1 is as illustrated above, it can be semantically analyzed using an image editing model to obtain semantic analysis result 1. In semantic analysis result 1, multiple entities are identified as: skirt, hem, and waist decoration. The relationships between entities, attributes, and actions include: the skirt's color is beige → changed to dark blue; the skirt's material is pure cotton → changed to silk; the hem has low fluidity → increased fluidity; a waist decoration exists → deleted the waist decoration.

[0048] The electronic device can use the image editing model to decompose the image editing instruction 1 according to the semantic analysis result 1 above, resulting in multiple sub-editing instructions, as shown in Table 1: Table 1 For example, in sub-edit instruction 1, the object to be edited is a skirt, and the editing requirement is to change the color of the skirt from beige to dark blue.

[0049] Among them, sub-editing instructions 1 to 4 are all first sub-editing instructions, and sub-editing instruction 5 is the second sub-editing instruction.

[0050] In steps S122-S123, for any sub-editing instruction, the electronic device can use the image editing model to determine the sub-region corresponding to the sub-editing instruction from the initial image according to the editing object in the sub-editing instruction, and edit the sub-region corresponding to the sub-editing instruction according to the editing requirements in the sub-editing instruction.

[0051] Since multiple sub-editing instructions include a first sub-editing instruction and a second sub-editing instruction, the first sub-editing instruction is used to instruct the modification of relevant features of the object to be edited in the initial image; the second sub-editing instruction is used to instruct the preservation of relevant features in the initial image. Therefore, in the initial image, the sub-region corresponding to the first sub-editing instruction can be called the edited region, and the sub-region corresponding to the second sub-editing instruction can be called the non-edited region.

[0052] Optionally, the electronic device can use an image editing model to edit the sub-region corresponding to the first sub-editing instruction in the initial image according to the editing requirements in the first sub-editing instruction, to obtain the first region in the intermediate image (the first region differs from the sub-region corresponding to the first sub-editing instruction in the initial image), thus modifying the features related to the first sub-editing instruction in the initial image; according to the second sub-editing instruction, based on the sub-region corresponding to the second sub-editing instruction in the initial image, edit to obtain the second region in the intermediate image (the second region is consistent with the sub-region corresponding to the second sub-editing instruction in the initial image), thus reconstructing the sub-region (i.e., the non-edited region) corresponding to the second sub-editing instruction in the initial image in the intermediate image.

[0053] For example, if multiple sub-editing instructions are as shown in Table 1 above, with sub-editing instructions 1-4 being the first sub-editing instruction and sub-editing instruction 5 being the second sub-editing instruction, then the electronic device can use the image editing model to execute sub-editing instruction 1 to edit the skirt in the initial image, changing the pixel color of the skirt in the skirt area of ​​the initial image from beige to dark blue, thus obtaining the first region 1 in the intermediate image (i.e., the skirt color in the skirt area of ​​the intermediate image is dark blue); execute sub-editing instruction 2 to edit the skirt in the initial image, changing the texture of the skirt in the skirt area of ​​the initial image from pure cotton texture to smooth silk texture, thus obtaining the first region 2 in the intermediate image (i.e., the skirt material in the skirt area of ​​the intermediate image is silk); ...; execute sub-editing instruction 5, based on the second region in the initial image corresponding to the second sub-editing instruction, i.e., the virtual person's upper garment, background environment, etc., to edit and obtain the second region in the intermediate image (i.e., the virtual person's upper garment, background environment, etc.), so that the virtual person's upper garment and background environment, etc., are reconstructed in the intermediate image.

[0054] In this embodiment, by executing the first sub-editing instruction, the image editing model can accurately modify the sub-region (i.e., the editing region) corresponding to the first sub-editing instruction in the initial image; by executing the second sub-editing instruction, the image editing model can accurately modify the sub-region (i.e., the non-editing region) corresponding to the second sub-editing instruction in the initial image without destroying it, thus preserving the non-editing region, avoiding erroneous modification of the initial image, and improving the accuracy of generating the target image.

[0055] It should be noted that in practical applications, electronic devices may not be able to completely execute multiple sub-editing instructions through the image editing model. As a result, the generated intermediate image may not conform to the image editing instructions and may not be the expected target image. Therefore, after editing the initial image according to the image editing instructions to obtain the intermediate image, step S15 can be executed: based on the image editing instructions and the intermediate image, determine the matching result of the intermediate image with the image editing instructions.

[0056] In the embodiments of this application, there are no restrictions on the specific implementation of the above-mentioned "determining the matching result of the intermediate image with the image editing instruction based on the image editing instruction and the intermediate image".

[0057] In an exemplary embodiment, the matching result of the intermediate image to the image editing instruction can be determined based on the image editing instruction and the intermediate image through the following steps S151-S152: S151. Calculate the semantic alignment between the intermediate image and multiple sub-editing instructions.

[0058] S152. Determine whether the intermediate image matches the image editing instructions based on the semantic alignment between the intermediate image and multiple sub-editing instructions.

[0059] In one exemplary embodiment, the image editing model may include an image encoder and a text encoder. The image encoder can be used to process an image to generate an image feature vector; the text encoder can process text to generate a text feature vector. The image editing model may be a multimodal model capable of processing multimodal information.

[0060] Optionally, in step S151, for any sub-editing instruction, the electronic device can process the instruction content of the sub-editing instruction through the text encoder in the image editing model to obtain the instruction feature vector; the electronic device can determine the sub-region corresponding to the sub-editing instruction in the intermediate image through the image editing model, and process the sub-region through the image encoder in the image editing model to obtain the region feature vector corresponding to the sub-editing instruction.

[0061] Optionally, the image editing model may also include a cross-attention mechanism and a regression prediction layer. For any sub-editing instruction, the electronic device can generate a fused feature vector of region feature vector and instruction feature vector through the cross-attention mechanism in the image editing model. Then, based on the fused feature vector, the semantic alignment between the sub-editing instruction and the intermediate image can be determined through the regression prediction layer. Optionally, the semantic alignment can be represented by a score of 0 to 10.

[0062] Furthermore, after obtaining the semantic alignment between the intermediate image and multiple sub-editing instructions, it can be determined whether the intermediate image matches the image editing instructions based on the semantic alignment between the intermediate image and the multiple sub-editing instructions. The implementation method for determining whether the intermediate image matches the image editing instructions based on the semantic alignment between the intermediate image and the multiple sub-editing instructions is not limited; examples are given below: In an optional embodiment A1, within the image editing model, a probability value can be generated based on the semantic alignment between the intermediate image and multiple sub-editing instructions. This probability value represents the probability of whether the intermediate image matches the image editing instructions. For example, if the image editing model outputs a probability value of 0.95 for a match between the intermediate image and the image editing command, and a probability value of 0.05 for a mismatch, it can be determined that the intermediate image matches the image editing instructions (or is qualified). Conversely, if the image editing model outputs a probability value of 0.10 for a match between the intermediate image and the image editing command, and a probability value of 0.90 for a mismatch, it can be determined that the intermediate image does not match the image editing instructions (or is unqualified).

[0063] In another optional embodiment A2, in the image editing model, the matching degree between the intermediate image and the image editing instructions can be calculated based on the semantic alignment degree between the intermediate image and multiple sub-editing instructions; if the matching degree is less than the matching degree threshold, it is determined that the intermediate image and the image editing instructions do not match; if the matching degree is greater than or equal to the matching degree threshold, it is determined that the intermediate image and the image editing instructions match.

[0064] Optionally, the matching degree can be a statistical value calculated based on the semantic alignment between the intermediate image and multiple sub-editing instructions. The statistical value can be a weighted average, a sum, or the minimum value among multiple semantic alignment degrees, etc.

[0065] It should be noted that if the matching degree is a weighted average of multiple semantic alignment degrees, then the weights corresponding to multiple sub-editing instructions can be dynamically determined through the image editing model.

[0066] Optionally, the matching threshold can be preset manually or dynamically determined by the image editing model.

[0067] If the matching degree is greater than or equal to the matching degree threshold, it can be determined that the editing effect of the intermediate image matches the image editing instructions; if the matching degree is less than the matching degree threshold, it can be determined that the editing effect of the intermediate image does not match the image editing instructions.

[0068] For example, if multiple sub-editing instructions are as shown in Table 1 above, and assuming that the semantic alignment between these multiple sub-editing instructions and the intermediate image is as shown in Table 2: Table 2 If the matching degree is a weighted average of multiple semantic alignment degrees, and the weights of each sub-editing instruction are shown in Table 2 above, then the matching degree can be determined to be 19.9 based on the multiple semantic alignment degrees and their corresponding weights in Table 2. If the matching degree threshold is 30, since the matching degree of 19.9 is less than the matching degree threshold, it can be determined that the image editing effect of the intermediate image does not match the image editing instructions.

[0069] In an exemplary embodiment, during the process of determining whether the intermediate image matches the image editing instructions, step S16 may also be performed: determining the matching description information of the intermediate image. The matching description information describes the matching status between the intermediate image and the image editing instructions.

[0070] For example, if the intermediate image is as shown in image 202, and image editing instruction 1 is as illustrated in the example above, then the matching description information for the intermediate image could be: "The color of the skirt has changed from beige to dark blue, which is consistent with the image editing instruction. However, the material still resembles matte fabric rather than silk, lacking luster and texture. Furthermore, the skirt appears stiff and has not improved its fluidity according to the image editing instruction; its outline remains as structurally strong as the skirt in the initial image. The metal decoration at the waist, which should have been removed, is still present. The pose, body shape, upper garment, and background environment of the virtual human in the intermediate image are consistent with the initial image." The matching description information for the intermediate image can describe the features that the intermediate image already meets the requirements of the image editing instruction, such as the skirt color, and the features that the intermediate image does not yet meet the requirements of the image editing instruction, such as the skirt's material, luster, and fluidity.

[0071] Optionally, the matching description information can be determined based on multiple semantic alignment degrees through the following steps S161~S163: S161. If there is a first semantic alignment degree that is greater than or equal to the alignment degree threshold among the semantic alignment degrees corresponding to multiple sub-editing instructions, then generate the first feature description information according to the sub-editing instruction corresponding to the first semantic alignment degree.

[0072] S162. If there is a second semantic alignment degree that is less than the alignment degree threshold among the semantic alignment degrees corresponding to multiple sub-editing instructions, then generate second feature description information according to the sub-editing instruction corresponding to the second semantic alignment degree.

[0073] S163. Determine that the matching description information includes the first feature description information and / or the second feature description information.

[0074] For any given sub-editing instruction, if the semantic alignment degree corresponding to the sub-editing instruction is greater than or equal to the alignment degree threshold, it indicates that the sub-editing instruction was executed successfully. The relevant features involved in the sub-editing instruction match the sub-editing instruction, i.e., match the image editing instruction. Therefore, first feature description information can be generated based on the sub-editing instruction. This first feature description information can be used to describe the features in the initial image that match the image editing instruction.

[0075] If the semantic alignment of a sub-editing instruction is less than the alignment threshold, it indicates that the sub-editing instruction failed to execute. The relevant features involved in the sub-editing instruction do not match the sub-editing instruction, i.e., they do not match the image editing instruction. In this case, second feature description information can be generated based on the sub-editing instruction. This second feature description information can be used to describe features in the initial image that do not match the image editing instruction.

[0076] After determining the first feature description information and / or the second feature description information, the first feature description information and / or the second feature description information can be combined to obtain matching description information.

[0077] For example, if multiple sub-editing instructions are as shown in Table 1 above, and their corresponding semantic alignment degrees are as shown in Table 2 above, then the semantic alignment degrees corresponding to sub-editing instructions 1 and 5 are greater than the alignment threshold, and the semantic alignment degrees corresponding to sub-editing instructions 1 and 5 are the first semantic alignment degree, indicating that sub-editing instructions 1 and 5 were executed successfully. First feature description information 1 can be generated based on sub-editing instruction 1, and first feature description information 2 can be generated based on sub-editing instruction 5, as shown in Table 3. However, the semantic alignment degrees corresponding to sub-editing instructions 2, 3, and 4 are less than the alignment threshold, and the semantic alignment degrees corresponding to sub-editing instructions 2, 3, and 4 are the second semantic alignment degree, indicating that sub-editing instructions 2, 3, and 4 failed to execute. Second feature description information 1 can be generated based on sub-editing instruction 2, second feature description information 2 can be generated based on sub-editing instruction 3, and second feature description information 3 can be generated based on sub-editing instruction 4, as shown in Table 3. Table 3 The first feature description information 1, the second feature description information 1, the second feature description information 2, the second feature description information 3 and the first feature description information 2 in Table 3 can be combined to obtain matching description information, which can be as shown in the example above.

[0078] Since the second feature description information is a description of the difference features in the initial image that do not match the image editing instructions, in an exemplary embodiment, if the second feature description information exists, the image correction instructions can be generated based on the second feature description information.

[0079] For example, if there are three second feature descriptions, as shown in Table 3 above, an image correction instruction can be generated based on these three second feature descriptions. The image correction instruction could be "adjust the skirt material to a glossy silk texture, increase the flow of the skirt hem, and remove the decoration at the waist".

[0080] In an exemplary embodiment, before determining the matching result of the intermediate image with respect to the image editing instruction based on the image editing instruction and the intermediate image, the evaluation function of the image editing model can be triggered through the following steps S17-S18: S17. Obtain evaluation prompts.

[0081] S18. Based on the evaluation prompt, trigger the evaluation function of the image editing model to determine whether the editing effect of the intermediate image matches the image editing instructions.

[0082] Optionally, the evaluation prompts can be preset in the image editing model or manually input. For example, the evaluation prompts could be "Act as a visual editing evaluation expert and evaluate the initial image."

[0083] The image editing model can trigger its evaluation function based on evaluation prompts. If the evaluation prompts are preset in the image editing model, the model can automatically obtain them and trigger the evaluation function, meaning the evaluation function can be automatically enabled. If the evaluation prompts are manually input, the model can respond to the user's input, obtain the prompts, and trigger the evaluation function, meaning the evaluation function can be manually enabled.

[0084] Below, in conjunction with Figures 3a-3e The above image editing methods will be further explained through specific examples.

[0085] Figure 3a A schematic diagram of an image editing method provided for an exemplary embodiment of this application Figure 1 .like Figure 3a As shown, it includes images 301, 302, and 303. Among them, image 301 is the initial image, image 302 is the intermediate image, and image 303 is the target image.

[0086] The user can input image editing instruction 2 and image 301 into the electronic device. Image editing instruction 2 is "modify the white short-sleeved shirt into a slim fit made of light mint green linen, slightly lengthen the shirt, roll up the sleeves once, and slightly enlarge the collar". The electronic device can respond to image editing instruction 2, determine the initial image to be processed as image 301, and call the image editing model based on image editing instruction 2 and image 301.

[0087] The electronic device can edit image 301 according to image editing instruction 2 using the image editing model to obtain an intermediate image. Specifically, image editing instruction 2 can be parsed and processed by the image editing model to obtain 7 sub-editing instructions, as shown in Table 4: Table 4 The image editing model can be used to determine the short-sleeved area in image 301 based on the editable object "short-sleeved" in sub-editing instruction a1. Then, based on the editing requirement in sub-editing instruction a1, "change the color from white to light mint green," the pixel color of the short-sleeved area in image 301 can be changed from white to light mint green, resulting in a light mint green short-sleeved area in the intermediate image. Similarly, based on sub-editing instruction a2, the material of the short-sleeved area can be changed from pure cotton texture to linen texture, resulting in a linen texture short-sleeved area in the intermediate image; and so on. Furthermore, based on sub-editing instruction a7, other areas besides the short-sleeved area can be determined in image 301, and these other areas can be reconstructed in the intermediate image. Assuming that sub-editing instructions a1~a7 have been executed, the intermediate image shown in image 302 can be obtained.

[0088] The electronic device can determine whether the editing effect of image 302 matches the image editing instruction 2 through the image editing model.

[0089] Specifically, the electronic device can determine the instruction feature vector corresponding to each sub-editing instruction, and can determine the region feature vector of the corresponding area in image 302 for each sub-editing instruction. Then, based on the instruction feature vector and region feature vector corresponding to each sub-editing instruction, it can calculate the semantic alignment degree between each sub-editing instruction and image 302, resulting in seven semantic alignment degrees. The electronic device can determine whether image 302 matches image editing instruction 2 based on the semantic alignment degrees between image 302 and multiple sub-editing instructions.

[0090] In an optional embodiment, the electronic device can generate a probability value based on the semantic alignment between image 302 and multiple sub-editing instructions. This probability value represents the probability that image 302 matches image editing instruction 2. For example, if the probability value of image 302 matching image editing instruction 2 is 0.95 and the probability value of image 302 not matching image editing instruction 2 is 0.05, it can be determined that image 302 matches image editing instruction 2 (or is qualified). Conversely, if the probability value of image 302 matching image editing instruction 2 is 0.10 and the probability value of image 302 not matching image editing instruction 2 is 0.90, it can be determined that image 302 does not match image editing instruction 2 (or is unqualified).

[0091] In another alternative embodiment, the electronic device can calculate the weighted average of the seven semantic alignments as the matching degree between image 302 and image editing instruction 2. Assuming the matching degree is 30 and the matching degree threshold is 50, it can be determined that image 302 and image editing instruction 2 do not match.

[0092] The electronic device can generate matching description information 2 for image 302 based on image editing instruction 2 and image 302: the short-sleeved shirt color has been changed to light mint green, the fabric is consistent with linen, the collar is slightly larger, maintaining the integrity of the content and visual naturalness. However, the short-sleeved shirt does not look well-fitting (the silhouette is still loose), the length has not been significantly extended, and the sleeves have not been rolled up as required, thus failing to fully execute image editing instruction 2.

[0093] The electronic device can generate image correction instruction 2 based on image editing instruction 2 and image 302. Image correction instruction 2 can be "adjust the shirt cut to fit the body better, slightly lengthen the hem, and add a single-layer visible rolled edge to each sleeve." The electronic device can correct image 302 according to image correction instruction 2 to obtain image 303. Furthermore, the electronic device can determine whether image 303 matches image editing instruction 2. The method for determining whether image 303 matches image editing instruction 2 is the same as or similar to the method for determining whether image 302 matches image editing instruction 2. For example, a probability value indicating whether image 303 matches image editing instruction 2 can be output, representing whether image 303 matches image editing instruction 2. Alternatively, the matching degree between image 303 and image editing instruction 2 can be calculated. Assuming the matching degree is 60% and the matching degree threshold is 50%, it can be determined that image 303 matches image editing instruction 2. Therefore, image 303 can be used as the target image and displayed.

[0094] Figure 3b A schematic diagram of an image editing method provided for an exemplary embodiment of this application Figure 2 .like Figure 3bAs shown, this includes image 304 and image 305. Image 304 is the initial image, and image 305 is the target image.

[0095] The user can input image editing command 3 and image 304 into the electronic device. Image editing command 3 is "replace sequined shorts with high-waisted pure black leather shorts". The electronic device can respond to image editing command 3, determine the initial image to be processed as image 304, and call the image editing model based on image editing command 3 and image 304.

[0096] The electronic device can edit image 304 according to image editing instruction 3 using the image editing model to obtain an intermediate image. Specifically, image editing instruction 3 can be parsed and processed by the image editing model to obtain four sub-editing instructions, as shown in Table 5: Table 5 The image editing model can be used to determine the shorts region in image 304 based on the editable object "shorts" in sub-editing instruction b1, and then, according to the editing requirement "change the color from white to black" in sub-editing instruction b1, change the pixel color of the shorts region in image 304 from white to black, resulting in a black shorts region in the intermediate image. Similarly, according to sub-editing instruction b2, the material of the shorts region can be changed from sequin texture to leather texture, resulting in a leather texture for the shorts region in the intermediate image; and so on; according to sub-editing instruction b4, other regions besides the shorts can be determined in image 304, and based on these other regions, the other regions besides the shorts can be reconstructed in the intermediate image. Assuming that sub-editing instructions b1~b4 have been executed, the intermediate image shown in image 305 can be obtained.

[0097] The electronic device can determine whether the editing effect of image 305 matches the image editing instruction 3 through the image editing model.

[0098] Specifically, the electronic device can determine the instruction feature vector corresponding to each sub-editing instruction, and can determine the region feature vector of the corresponding area in image 305 for each sub-editing instruction. Then, based on the instruction feature vector and region feature vector corresponding to each sub-editing instruction, it can calculate the semantic alignment degree between each sub-editing instruction and image 305, thus obtaining four semantic alignment degrees. The electronic device can determine whether image 305 matches image editing instruction 3 based on the semantic alignment degrees between image 305 and multiple sub-editing instructions.

[0099] In an optional embodiment, the electronic device can generate a probability value based on the semantic alignment between image 305 and multiple sub-editing instructions. This probability value represents the probability that image 305 matches image editing instruction 3. For example, if the probability value of image 302 matching image editing instruction 3 is 0.95, and the probability value of image 302 not matching image editing instruction 3 is 0.05, then image 305 can be determined to match image editing instruction 3 (or be qualified). Conversely, if the probability value of image 305 matching image editing instruction 2 is 0.10, and the probability value of image 302 not matching image editing instruction 3 is 0.90, then image 305 can be determined to not match image editing instruction 2 (or be unqualified).

[0100] In another alternative embodiment, the electronic device can calculate the weighted average of the four semantic alignments as the matching degree between image 305 and image editing instruction 3. Assuming the matching degree is 40 and the matching degree threshold is 30, it can be determined that image 305 matches image editing instruction 3. Therefore, image 305 can be used as the target image and displayed.

[0101] The electronic device can also generate matching description information 3 for image 305 based on image editing instruction 3 and image 305: The sequined shorts in image 304 have been accurately changed into high-waisted solid black leather shorts according to image editing instruction 3. The new shorts maintain the appropriate shape, fit, and realistic leather texture, seamlessly blending with the rest of the image. No important visual elements are lost, and there are no visible artifacts or detail issues in the edited area. The result is natural and consistent with the fashion image. Therefore, image 305 matches image editing instruction 3.

[0102] Figure 3c A schematic diagram three illustrates an image editing method provided for an exemplary embodiment of this application. For example... Figure 3c As shown, the images include 306, 307, and 308. Among them, image 306 is the initial image, image 307 is the intermediate image, and image 308 is the target image.

[0103] The user can input image editing command 4 and image 306 into the electronic device. Image editing command 4 is "change the color scheme of the black daisy knit sweater to a gray-white background with pink flower patterns". The electronic device can respond to image editing command 4, determine the initial image to be processed as image 306, and call the image editing model based on image editing command 4 and image 306.

[0104] The electronic device can edit image 306 according to image editing instruction 4 using the image editing model to obtain an intermediate image. Specifically, image editing instruction 4 can be parsed and processed by the image editing model to obtain three sub-editing instructions, as shown in Table 5: Table 5 Using the image editing model, based on the editable object "sweater" in sub-editing instruction c1, the sweater area in image 306 is determined. Then, based on the editing requirement in sub-editing instruction c1, "change the background color from black to grayscale," the pixel color of the sweater area in image 306 is changed from black to grayscale, resulting in a grayscale sweater area in the intermediate image. Similarly, based on sub-editing instruction c2, the pattern on the sweater area is changed from daisies and grass to pink flowers, resulting in a pink flower pattern on the sweater in the intermediate image. Furthermore, based on sub-editing instruction c3, other areas besides the sweater are determined in image 306, and based on these other areas, the remaining areas besides the sweater are reconstructed in the intermediate image. Assuming that sub-editing instructions c1~c3 are executed, the intermediate image shown in image 307 is obtained.

[0105] The electronic device can determine whether the editing effect of image 307 matches the image editing instruction 4 through the image editing model.

[0106] Specifically, the electronic device can determine the instruction feature vector corresponding to each sub-editing instruction, and can determine the region feature vector of the corresponding area in image 307 for each sub-editing instruction. Then, based on the instruction feature vector and region feature vector corresponding to each sub-editing instruction, it can calculate the semantic alignment degree between each sub-editing instruction and image 307, thus obtaining three semantic alignment degrees. The electronic device can determine whether image 307 matches image editing instruction 4 based on the semantic alignment degrees between image 307 and multiple sub-editing instructions.

[0107] In an optional embodiment, the electronic device can generate a probability value based on the semantic alignment between image 307 and multiple sub-editing instructions. This probability value represents the probability that image 307 matches image editing instruction 4. For example, if the probability value of image 307 matching image editing instruction 4 is 0.95 and the probability value of image 307 not matching image editing instruction 4 is 0.05, it can be determined that image 307 matches image editing instruction 4 (or is qualified). Conversely, if the probability value of image 307 matching image editing instruction 4 is 0.10 and the probability value of image 307 not matching image editing instruction 4 is 0.90, it can be determined that image 307 does not match image editing instruction 4 (or is unqualified).

[0108] In another alternative embodiment, the electronic device can calculate the weighted average of the three semantic alignments as the matching degree between image 307 and image editing instruction 4. Assuming the matching degree is 18 and the matching degree threshold is 25, it can be determined that image 307 and image editing instruction 4 do not match.

[0109] The electronic device can generate a matching description 4 for image 307 based on image editing instruction 4 and image 307, which is: the pattern on the sweater has been changed from daisies and grass to pink flowers, while other areas remain unchanged. However, the background color of the sweater remains black instead of being changed to grayish-white, therefore it does not match image editing instruction 4.

[0110] The electronic device can generate an image correction instruction 4 based on image editing instruction 4 and image 307. Image correction instruction 4 could be "change the background color of the sweater from black to grayscale." The electronic device can then correct image 307 according to image correction instruction 4, resulting in image 308. Furthermore, the electronic device can determine whether image 308 matches image editing instruction 4. The method for determining whether image 308 matches image editing instruction 4 is the same as or similar to the method for determining whether image 307 matches image editing instruction 4. For example, a probability value indicating whether image 308 matches image editing instruction 4 can be output; this probability value represents whether image 308 matches image editing instruction 4. Alternatively, the matching degree between image 308 and image editing instruction 4 can be calculated. Assuming the matching degree is 30% and the matching degree threshold is 45%, it can be determined that image 308 matches image editing instruction 4. Therefore, image 308 can be used as the target image and displayed.

[0111] Figure 3d A schematic diagram of an image editing method provided for an exemplary embodiment of this application Figure 4 .like Figure 3d As shown, the images include 309, 310, and 311. Among them, 309 is the initial image, 310 is the intermediate image, and 311 is the target image.

[0112] The user can input image editing instruction 5 and image 309 into the electronic device. Image editing instruction 5 is "change the background color of the dress from white to light purple, change the pattern to a small purple flower pattern, and remove the diagonal flower decoration design on the chest". The electronic device can respond to image editing instruction 5, determine the initial image to be processed as image 309, and call the image editing model based on image editing instruction 5 and image 309.

[0113] The electronic device can edit image 309 according to image editing instruction 5 using the image editing model to obtain an intermediate image. Specifically, image editing instruction 5 can be parsed and processed by the image editing model to obtain three sub-editing instructions, as shown in Table 6: Table 6 Using the image editing model, based on the editable object "dress" in sub-editing instruction d1, the dress area in image 309 is determined. Then, based on the editing requirement in sub-editing instruction d1, "change the background color from white to light purple," the pixel color of the dress area in image 309 is changed from white to light purple, resulting in a light purple dress area in the intermediate image. Similarly, based on sub-editing instruction d2, the pattern on the dress in the dress area is changed to small purple flowers, resulting in a small purple flower pattern on the dress in the intermediate image. Based on sub-editing instruction d3, the diagonal flower decoration design on the dress in the dress area is deleted, resulting in a dress without diagonal flower decoration design in the intermediate image. Based on sub-editing instruction d4, other areas besides the dress are determined in image 309, and based on these other areas, the other areas besides the dress are reconstructed in the intermediate image. Assuming that sub-editing instructions d1~d4 are executed, the intermediate image shown in image 310 can be obtained.

[0114] The electronic device can determine whether the editing effect of image 310 matches the image editing instruction 5 through the image editing model.

[0115] Specifically, the electronic device can determine the instruction feature vector corresponding to each sub-editing instruction, and can determine the region feature vector of the corresponding area in image 310 for each sub-editing instruction. Then, based on the instruction feature vector and region feature vector corresponding to each sub-editing instruction, it can calculate the semantic alignment degree between each sub-editing instruction and image 310, thus obtaining three semantic alignment degrees. The electronic device can determine whether image 310 matches image editing instruction 5 based on the semantic alignment degrees between image 310 and multiple sub-editing instructions.

[0116] In an optional embodiment, the electronic device can generate a probability value based on the semantic alignment between image 310 and multiple sub-editing instructions. This probability value represents the probability that image 310 matches image editing instruction 5. For example, if the probability value of image 310 matching image editing instruction 5 is 0.95 and the probability value of image 310 not matching image editing instruction 5 is 0.05, then image 310 can be determined to match image editing instruction 5 (or be qualified). Conversely, if the probability value of image 310 matching image editing instruction 5 is 0.10 and the probability value of image 310 not matching image editing instruction 5 is 0.90, then image 310 can be determined to not match image editing instruction 5 (or be unqualified).

[0117] In another alternative embodiment, the electronic device can calculate the weighted average of the three semantic alignments as the matching degree between image 310 and image editing instruction 5. Assuming the matching degree is 28 and the matching degree threshold is 30, it can be determined that image 310 and image editing instruction 5 do not match.

[0118] The electronic device can generate matching description information 5 for image 310 based on image editing instruction 5 and image 310: The edited image successfully changed the background color of the dress to light purple and replaced the original pattern with a small purple flower design. Other areas remain unchanged. However, the diagonal flower decoration design on the chest has not been completely removed, and a prominent diagonal stripe can still be seen, thus not matching image editing instruction 5.

[0119] The electronic device can generate an image correction instruction 5 based on image editing instruction 5 and image 310. Image correction instruction 5 can be "completely remove the diagonal flower decoration design on the chest, ensuring there are no visible diagonal stripes." The electronic device can correct image 310 according to image correction instruction 5 to obtain image 311. Furthermore, the electronic device can determine whether image 311 matches image editing instruction 5. The method for determining whether image 311 matches image editing instruction 5 is the same as or similar to the method for determining whether image 310 matches image editing instruction 5. For example, a probability value indicating whether image 311 matches image editing instruction 2 can be output, representing whether image 311 matches image editing instruction 5. Alternatively, the matching degree between image 311 and image editing instruction 5 can be calculated. Assuming the matching degree is 35 and the matching degree threshold is 30, it can be determined that image 311 matches image editing instruction 5. Therefore, image 311 can be used as the target image and displayed.

[0120] Figure 3e A schematic diagram of an image editing method provided for an exemplary embodiment of this application Figure 5 .like Figure 3e As shown, it includes images 312, 313, and 314. Among them, image 312 is the initial image, image 313 is the intermediate image, and image 314 is the target image.

[0121] The user can input image editing instruction 6 and image 312 into the electronic device. Image editing instruction 6 is "change loose straight-leg jeans into high-waisted slim-fit flared pants". The electronic device can respond to image editing instruction 6, determine the initial image to be processed as image 312, and call the image editing model based on image editing instruction 6 and image 312.

[0122] The electronic device can edit image 312 according to image editing instruction 6 using the image editing model to obtain an intermediate image. Specifically, image editing instruction 6 can be parsed and processed by the image editing model to obtain three sub-editing instructions, as shown in Table 7: Table 7 Using the image editing model, based on the editable object "jeans" in sub-editing instruction e1, the jeans region in image 312 is determined. Then, based on the editing requirement in sub-editing instruction e1, "change from loose fit to slim fit," the style of the jeans in the jeans region of image 312 is changed from loose to slim fit, resulting in slim-fit jeans in the intermediate image. Similarly, based on sub-editing instruction e2, the leg length of the jeans in the jeans region is changed from straight leg to flared, resulting in flared jeans in the intermediate image. Furthermore, based on sub-editing instruction e3, other regions besides the jeans are determined in image 312, and based on these other regions, they are reconstructed in the intermediate image. Assuming that sub-editing instructions e1~e3 are executed, the intermediate image shown in image 313 is obtained.

[0123] The electronic device can determine whether the editing effect of image 313 matches the image editing instruction 6 through the image editing model.

[0124] Specifically, the electronic device can determine the instruction feature vector corresponding to each sub-editing instruction, and can determine the region feature vector of the corresponding area in image 313 for each sub-editing instruction. Then, based on the instruction feature vector and region feature vector corresponding to each sub-editing instruction, it can calculate the semantic alignment degree between each sub-editing instruction and image 313, thus obtaining three semantic alignment degrees. The electronic device can determine whether image 313 matches image editing instruction 6 based on the semantic alignment degrees between image 313 and multiple sub-editing instructions.

[0125] In an optional embodiment, the electronic device can generate a probability value based on the semantic alignment between image 313 and multiple sub-editing instructions. This probability value represents the probability that image 313 matches image editing instruction 6. For example, if the probability value of image 313 matching image editing instruction 6 is 0.95 and the probability value of image 313 not matching image editing instruction 6 is 0.05, it can be determined that image 313 matches image editing instruction 6 (or is qualified). Conversely, if the probability value of image 313 matching image editing instruction 6 is 0.10 and the probability value of image 313 not matching image editing instruction 6 is 0.90, it can be determined that image 313 does not match image editing instruction 6 (or is unqualified).

[0126] In another alternative embodiment, the electronic device can calculate the weighted average of the three semantic alignments as the matching degree between image 313 and image editing instruction 6. Assuming the matching degree is 26 and the matching degree threshold is 30, it can be determined that image 313 and image editing instruction 6 do not match.

[0127] The electronic device can generate matching description information 6 for image 313 based on image editing instruction 6 and image 313: Although the waist and hips of the jeans appear slightly fitted, the jeans are still straight-legged, not flared. The leg silhouette remains loose and straight, with no obvious curvature below the knee. Furthermore, compared to the original jeans, the jeans have not become significantly higher-waisted. The overall shape does not match the required slim-fit flared shape, and therefore does not match image editing instruction 6.

[0128] The electronic device can generate an image correction instruction 6 based on image editing instruction 6 and image 313. Image correction instruction 6 could be something like "reducing the leg fit from the hips to the knees to create a slimmer effect, then widening the calves below the knees to achieve a defined flared silhouette, and raising the waistline to ensure a high-waisted appearance." The electronic device can then correct image 313 according to image correction instruction 6, resulting in image 314. Furthermore, the electronic device can determine whether image 314 matches image editing instruction 6. The method for determining whether image 314 matches image editing instruction 6 is the same as or similar to the method for determining whether image 313 matches image editing instruction 6. For example, a probability value indicating whether image 314 matches image editing instruction 6 can be output, representing whether image 314 matches image editing instruction 6. Alternatively, the matching degree between image 314 and image editing instruction 6 can be calculated. Assuming the matching degree is 39 and the matching degree threshold is 30, then image 314 can be determined to match image editing instruction 6, and therefore image 314 can be used as the target image and displayed.

[0129] In practical applications, the above image editing method can be applied to virtual try-on scenarios. Below, based on the above embodiments, combined with... Figure 4 A virtual try-on method is provided.

[0130] Figure 4 This is a flowchart illustrating a virtual try-on method provided as an exemplary embodiment of this application. Figure 4 As shown, the method may include: S41. Display an initial image of a virtual person wearing the initial clothing on the fitting interface.

[0131] S42. According to the image editing instructions, call the image editing model to edit the initial image to obtain the intermediate image.

[0132] S43. If the editing effect of the intermediate image does not match the image editing instructions, then generate image correction instructions based on the image editing instructions and the intermediate image.

[0133] S44. According to the image correction instructions, correct the intermediate image until the target image that matches the image editing instructions is obtained.

[0134] S45. Display the target image on the fitting interface.

[0135] Electronic devices can be devices with a display screen. For example, electronic devices can be terminals such as mobile phones, computers, and tablets.

[0136] Optionally, shopping software may be deployed in the electronic device, and the shopping software may integrate an image editing model. Optionally, the image editing model may be deployed locally on the electronic device or on a cloud server; there is no limitation on this.

[0137] Virtual avatars can be generated based on the image of clothing models, or they can be personalized avatars generated based on the user's own data or photos. Personalized virtual avatars can help users better showcase how wearing a target garment will look.

[0138] Electronic devices can display a fitting room interface. The fitting room interface can display an initial image, including a virtual person wearing the initial clothing.

[0139] Below, in conjunction with Figure 5 The fitting room interface will be explained.

[0140] Figure 5 This is a schematic diagram of a fitting room interface provided for an exemplary embodiment of this application. Figure 5 As shown, there are interfaces 501 and 502. Both interfaces 501 and 502 are try-on interfaces.

[0141] like Figure 5 As shown, interface 501 can display an initial image in which the virtual person is wearing initial clothing (i.e., a beige top and a beige cotton skirt). The initial image can be input by the user or provided by the merchant in the shopping software.

[0142] Interface 501 can also display an input box, which is used by the user to input image editing commands. Figure 4 In this embodiment, image editing instructions can be used to instruct the replacement of the initial clothing in the initial image.

[0143] For example, a user can enter image editing instruction 1 in interface 501. Image editing instruction 1 can be "Change the beige cotton dress worn by the virtual person to dark blue silk fabric, enhance the flow of the skirt, and remove the decoration at the waist".

[0144] The electronic device can respond to the image editing command input by the user on the fitting interface, call the image editing model to execute steps S42~S44, so as to obtain the target image based on the image editing command and the initial image.

[0145] It should be noted that the specific execution process of S42~S44 can be found in [reference needed].Figure 1 Examples are not described in detail here.

[0146] After obtaining the target image in step S44, the electronic device can display the target image on the fitting interface. The target image can be used to demonstrate the effect of a virtual person wearing the target clothing.

[0147] For example, such as Figure 5 In interface 502, the virtual person in the target image is wearing a beige top and a dark blue silk skirt with a flowing hem, and the waist decoration has been removed.

[0148] In this embodiment, the electronic device can display an initial image, including a virtual person wearing the initial clothing, on the fitting interface. Responding to image editing instructions and the initial image, it calls an image editing model to obtain a target image based on the image editing instructions and the initial image, and then displays the target image of the virtual person wearing the target clothing on the fitting interface. Since the image editing model has editing-evaluation-correction capabilities, the electronic device can call the image editing model to edit the initial image to obtain an intermediate image, and can automatically determine whether the editing effect of the intermediate image matches the image editing instructions, achieving self-evaluation. Furthermore, when the editing effect of the intermediate image does not match the image editing instructions, it can generate image correction instructions to correct the intermediate image until a target image that conforms to the image editing instructions is obtained, achieving self-correction. The image editing model forms a closed loop of "editing-evaluation-correction," which can actively identify and correct its own initial editing errors, thus improving the accuracy of generating the target image through the image editing model. This allows the target image to better showcase the fitting effect, thereby improving the user's virtual fitting experience when purchasing clothing in shopping software. Among them, the target image can display a more realistic and personalized virtual try-on experience, which is conducive to improving user decision-making efficiency, increasing purchase conversion rate and reducing return rate.

[0149] Before using the image editing model, the image editing model can be trained using the model training method provided in the following embodiments. Figure 6 This is a flowchart illustrating a model training method provided in an embodiment of this application. Figure 6 As shown, the method may include: S61. Obtain training samples, which include an initial image, image editing instructions, a first labeled image, label matching results, labeling instructions for image correction, and a second labeled image.

[0150] S62. Using the initial model, based on the initial image, image editing instructions, and the first labeled image, generate prediction matching results and prediction instructions for image correction.

[0151] S63. Using the initial model, the first labeled image is corrected according to the prediction instructions to generate a predicted image.

[0152] S64. Based on the predicted matching results and the labeled matching results, the predicted instructions and the labeled instructions, and the predicted image and the second labeled image, the model parameters of the initial model are updated to obtain the image editing model.

[0153] For any given training sample, the training sample may include an initial image, image editing instructions, a first labeled image, a labeled matching result, another labeled instruction, and a second labeled image. The first labeled image is the resulting image adapted to the image editing instructions; the second labeled image is also the resulting image adapted to the labeled instructions, and this image simultaneously conforms to the image editing instructions. Optionally, the labeled matching result may also include labeled matching descriptive information. This training sample includes five types of data; therefore, based on this training sample, a training data construction process based on "five-tuples" can be implemented, and the model's reflective ability can be stimulated through specific training strategies.

[0154] During the training of the initial model, in order to enable the initial model to learn self-evaluation, it can generate predicted matching results based on the initial image, image editing instructions, and the first labeled image. The initial model executes the image editing instructions to edit the initial image to obtain an edited image, and compares this edited image with the first labeled image to obtain a predicted matching result. The predicted matching result can be that the edited image obtained by executing the image editing instructions matches the edited effect of the first labeled image (or matches the image editing instructions), or that the edited image obtained by executing the image editing instructions does not match the edited effect of the first labeled image (or does not match the image editing instructions).

[0155] Optionally, the predicted matching results may also include predicted matching description information.

[0156] For example, if image editing instruction 1 is as shown in the example above, the initial image is as follows: Figure 2 As shown in image 201, the first labeled image is as follows. Figure 2As shown in image 202, a predictive matching result can be generated by the initial model based on the initial image, image editing instruction 1, and the first annotated image. Assume the predicted matching result is that the edited image obtained by executing the image editing instruction does not match the edited effect of the first annotated image. Alternatively, a predictive matching description can be generated by the initial model based on the initial image, image editing instruction, and the first annotated image. Assume the predictive matching description is: "The color of the skirt has been changed from white to dark blue, consistent with the image editing instruction. However, the material still resembles matte fabric rather than silk." This predictive matching description emphasizes the preservation of non-edited areas, enabling the model to learn to accurately modify without damaging the rest of the initial image, thus meeting the core requirements of visual editing.

[0157] In order to enable the initial model to learn self-correction, prediction instructions can be generated based on the initial image, image editing instructions and the first labeled image, and the first labeled image can be corrected according to the prediction instructions to generate the prediction image.

[0158] For example, if image editing instruction 1 is as shown in the example above, the initial image is as follows: Figure 2 As shown in image 201, the first labeled image is as follows. Figure 2 As shown in image 202, a prediction instruction can be generated by the initial model based on the initial image, image editing instruction 1, and the first labeled image. Let's assume the prediction instruction is "adjust the skirt material to a glossy silk texture." The initial model can then modify the first labeled image according to this prediction instruction to generate a prediction image. Assume the skirt in the prediction image is dark blue, made of silk fabric, but lacks fluidity, and the waist decoration has not been removed.

[0159] Since there are differences between the predicted matching results and the labeled matching results, differences between the predicted instructions and the labeled instructions, and differences between the predicted image and the second labeled image, the model parameters of the initial model can be updated through the following steps S641~S645 to obtain the image editing model: S641. Determine the first semantic similarity loss between the predicted matching result and the labeled matching result.

[0160] S642. Determine the second semantic similarity loss between the predicted instruction and the labeled instruction.

[0161] S643. Determine the image similarity loss between the predicted image and the second labeled image.

[0162] S644. Construct a loss function based on the first semantic similarity loss, the second semantic similarity loss, and the image similarity loss.

[0163] S645. Update the model parameters of the initial model according to the loss function to obtain the image editing model.

[0164] Optionally, the predicted matching vector corresponding to the predicted matching result can be determined, and the labeled matching vector corresponding to the labeled matching result can be determined. Then, based on the predicted matching vector and the labeled matching vector, the first semantic similarity loss between the predicted matching result and the labeled matching result can be calculated. For example, the first cosine similarity between the predicted matching vector and the labeled matching vector can be calculated using a cosine similarity algorithm, and thus the first semantic similarity loss can be determined as 1 - first cosine similarity.

[0165] Similarly, the predicted instruction vector corresponding to the predicted instruction can be determined, and the labeled instruction vector corresponding to the labeled instruction can be determined. Then, the second semantic similarity loss between the predicted instruction vector and the labeled instruction vector can be calculated. For example, the second cosine similarity between the predicted instruction vector and the labeled instruction vector can be calculated using a cosine similarity algorithm, thus determining that the second semantic similarity loss = 1 - second cosine similarity.

[0166] Optionally, the predicted image feature vector of the predicted image can be determined, the labeled image feature vector of the second labeled image can be determined, and then the image similarity loss between the predicted image and the second labeled image can be determined based on the predicted image feature vector and the labeled image feature vector.

[0167] In an exemplary embodiment, different weights can be set for the first semantic similarity loss, the second semantic similarity loss, and the image similarity loss, and the loss function can be constructed by weighted summation of the first semantic similarity loss, the second semantic similarity loss, and the image similarity loss.

[0168] Each time the initial model is trained with training samples, the loss value of the loss function can be calculated. The model parameters of the initial model are then updated based on the loss value until the loss value no longer decreases. At this point, the training can be terminated, and the image editing model is obtained.

[0169] The technical solutions of this application embodiment do not limit the implementation form of the image editing model, and can be various deep learning-based neural network models. Optionally, the image editing model can be a deep learning model with a relatively small number of model parameters, or it can be a deep learning model with a relatively large number of model parameters. The large model is merely an example, and this application embodiment does not limit the number of model parameters supported by the image editing model, aiming to meet actual needs. The image editing model involved in this application embodiment can be an artificial intelligence-based language model (LM) or multimodal model (MM), and there is no limitation on this.

[0170] In this embodiment, when training the initial model using training samples, the initial model generates a predicted matching result based on the initial image, image editing instructions, and the first labeled image to train the initial model to learn self-evaluation; the initial model generates a predicted instruction based on the initial image, image editing instructions, and the first labeled image, and corrects the first labeled image according to the predicted instruction to generate a predicted image to train the initial model to learn self-correction. This enables the image editing model trained based on the initial model to have the ability to edit, evaluate, and correct, thereby improving the accuracy of the image editing model.

[0171] In practical applications, the target image generated by the image editing model in this embodiment is superior in semantic alignment and image quality compared to target images generated by other models. The test data is shown in Table 8. Table 8 In Table 8 above, SC-score refers to semantic consistency score, PQ-score refers to image quality score, and Q-score is comprehensive score. Level-1, Level-2, and Level-3 refer to test tasks of different difficulty.

[0172] Among them, Model 1-1 is the version of Model 1 without evaluation-correction capability, Model 1-2 is the version of Model 1 with evaluation-correction capability; Model 2-1 is the version of Model 2 without evaluation-correction capability, Model 2-2 is the version of Model 2 with evaluation-correction capability; the image editing model has evaluation-correction capability.

[0173] As can be seen from the test data in Table 8, the image editing model in this application is superior to Model 1 and Model 2. Combining the above test data and the visualization figures provided in the above embodiments, it can be seen that in the embodiments of this application, the model can accurately understand and execute complex editing instructions containing multiple constraints (such as color, material, style, detail addition and deletion, etc.), greatly improving the accuracy of instruction understanding and execution.

[0174] In the above embodiments, the evaluation and correction instruction generation process of the image editing model can also be referred to as a "reflection" process. In this embodiment, the "reflection" mechanism is systematically applied to the highly constrained task of "visual editing" for the first time. Based on this, this application provides a reflective training paradigm for visual editing, which proposes a novel method for constructing sample data of a "five-tuple" (source image, editing instruction, edited image, reflection information, target image) and specifically designed to train the model's ability to self-evaluate and correct in highly constrained visual editing tasks. Furthermore, by consistently emphasizing the preservation of non-edited regions during training data construction and model training, the model learns to accurately modify images without damaging the rest of the initial image, thus adapting to the core requirements of visual editing.

[0175] Based on the image editing model trained using the reflective training paradigm provided in this application, the inference process can leverage the iterative optimization of the "generation-reflection-regeneration" process. This is equivalent to introducing a cognitive feedback loop in the model inference stage, enabling the image editing model to proactively identify and correct its editing errors in each round. This breaks down complex editing tasks into easier-to-execute "trial and error-correction" steps, improving the effect of visual editing and solving the technical challenges faced in the field of visual editing that requires high fidelity and local modifications.

[0176] It should be noted that the execution subject of each step in the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps S11 to S18 can be device A; or the execution subject of step S11 can be device A, and the execution subject of steps S12 to S18 can be device B; and so on.

[0177] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0178] Figure 7 This is a schematic diagram of the structure of an image editing apparatus provided for an exemplary embodiment of this application. For example... Figure 7 As shown, the image editing device 70 may include: a response module 71, a first editing module 72, and a first correction module 73, wherein, The response module 71 is used to determine the initial image to be processed in response to an image editing instruction; The first editing module 72 is used to call an image editing model according to the image editing instructions to edit the initial image to obtain an intermediate image; The first correction module 73 is used to generate an image correction instruction based on the image editing instruction and the intermediate image if the editing effect of the intermediate image does not match the image editing instruction. The first correction module 73 is further configured to correct the intermediate image according to the image correction instruction until a target image matching the image editing instruction is obtained.

[0179] Optionally, the first editing module 72 is specifically used to: parse and process the image editing instructions to obtain multiple sub-editing instructions, the sub-editing instructions including editing objects and editing requirements; determine the sub-regions corresponding to each of the multiple sub-editing instructions from the initial image according to the editing objects in the multiple sub-editing instructions; and edit the sub-regions corresponding to each of the multiple sub-editing instructions according to the editing requirements in the multiple sub-editing instructions to obtain the intermediate image.

[0180] Optionally, the plurality of sub-editing instructions includes a first sub-editing instruction and a second sub-editing instruction; the first editing module 72 is specifically used to: edit the sub-region corresponding to the first sub-editing instruction in the initial image according to the editing requirements in the first sub-editing instruction to obtain a first region in the intermediate image, wherein the first region differs from the sub-region corresponding to the first sub-editing instruction in the initial image; and edit the second region in the intermediate image based on the sub-region corresponding to the second sub-editing instruction in the initial image according to the second sub-editing instruction to obtain a second region, wherein the second region is consistent with the sub-region corresponding to the second sub-editing instruction in the initial image.

[0181] Optionally, the image editing device 70 further includes a matching module 74, which is used to: calculate the semantic alignment between the intermediate image and the plurality of sub-editing instructions; and determine whether the intermediate image matches the image editing instructions based on the semantic alignment between the intermediate image and the plurality of sub-editing instructions.

[0182] Optionally, the matching module 74 is further configured to: if there is a first semantic alignment degree greater than or equal to an alignment threshold among the semantic alignment degrees corresponding to the plurality of sub-editing instructions, then generate first feature description information according to the sub-editing instruction corresponding to the first semantic alignment degree, wherein the first feature description information is used to describe features in the intermediate image that match the image editing instruction; if there is a second semantic alignment degree less than the alignment threshold among the semantic alignment degrees corresponding to the plurality of sub-editing instructions, then generate second feature description information according to the sub-editing instruction corresponding to the second semantic alignment degree, wherein the second feature description information is used to describe features in the initial image that do not match the image editing instruction; and determine that the matching description information of the intermediate image includes the first feature description information and / or the second feature description information.

[0183] Optionally, the first correction module 73 is specifically used to: generate the image correction instruction based on the second feature description information, wherein the image correction instruction is used to instruct the correction of features in the intermediate image that do not match the image editing instruction.

[0184] Optionally, the image editing device 70 further includes a trigger module 75, which is used to: acquire evaluation prompt words; and trigger the evaluation function of the image editing model based on the evaluation prompt words, so as to determine whether the editing effect of the intermediate image matches the image editing instruction through the evaluation function of the image editing model.

[0185] Optionally, the image editing device 70 further includes a first display module 76, which is used to display the target image.

[0186] The image editing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0187] Figure 8 This is a schematic diagram of the structure of a virtual fitting device provided for an exemplary embodiment of this application. Figure 8 As shown, the virtual fitting device 80 may include: a second display module 81, a second editing module 82, and a second correction module 83, wherein, The second display module 81 is used to display an initial image, including a virtual person wearing the initial clothing, on the fitting interface; The second editing module 82 is used to call an image editing model according to the image editing instructions to edit the initial image to obtain an intermediate image; the image editing instructions are used to instruct the replacement of the initial clothing in the initial image; The second correction module 83 is used to generate an image correction instruction based on the image editing instruction and the intermediate image if the editing effect of the intermediate image does not match the image editing instruction; and to correct the intermediate image according to the image correction instruction until a target image matching the image editing instruction is obtained.

[0188] The second display module 81 is further configured to: display the target image on the fitting interface, the target image being used to demonstrate the effect of the virtual person wearing the target clothing.

[0189] The virtual fitting device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0190] Figure 9 This is a schematic diagram of a model training apparatus provided for an exemplary embodiment of this application. Figure 9 As shown, the model training device 90 may include: an acquisition module 91 and a training module 92, wherein, The acquisition module 91 is used to acquire training samples, which include an initial image, image editing instructions, a first labeled image, label matching results, labeling instructions for image correction, and a second labeled image, wherein the second labeled image conforms to the image editing instructions. The training module 92 is used to generate a prediction matching result and a prediction instruction for image correction based on the initial image, the image editing instruction and the first labeled image, using an initial model; The training module 92 is used to correct the first labeled image according to the prediction instruction using the initial model to generate a predicted image. The training module 92 is used to update the model parameters of the initial model based on the prediction matching result and the annotation matching result, the prediction instruction and the annotation instruction, and the prediction image and the second annotation image, so as to obtain an image editing model.

[0191] Optionally, the training module 92 is specifically used to: determine a first semantic similarity loss between the predicted matching result and the labeled matching result; determine a second semantic similarity loss between the predicted instruction and the labeled instruction; determine an image similarity loss between the predicted image and the second labeled image; construct a loss function based on the first semantic similarity loss, the second semantic similarity loss, and the image similarity loss; and update the model parameters of the initial model according to the loss function to obtain the image editing model.

[0192] The model training device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0193] Figure 10 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Please refer to... Figure 10 The electronic device 1000 may include a memory 1001 and a processor 1002.

[0194] Memory 1001 is used to store computer programs and can be configured to store various other data to support operation on the computing platform. Examples of this data include instructions for any application or method operating on the computing platform, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0195] Processor 1002, coupled to memory 1001, is configured to execute a computer program in memory 1001 for: determining an initial image to be processed in response to an image editing instruction; invoking an image editing model according to the image editing instruction to edit the initial image to obtain an intermediate image; if the editing effect of the intermediate image does not match the image editing instruction, generating an image correction instruction according to the image editing instruction and the intermediate image; and correcting the intermediate image according to the image correction instruction until a target image matching the image editing instruction is obtained.

[0196] Optionally, when the processor 1002 calls an image editing model to edit the initial image according to the image editing instructions to obtain an intermediate image, it specifically performs the following steps: parsing the image editing instructions to obtain multiple sub-editing instructions, each sub-editing instruction including an editing object and editing requirements; determining the sub-regions corresponding to each of the multiple sub-editing instructions from the initial image based on the editing objects in the multiple sub-editing instructions; and editing the sub-regions corresponding to each of the multiple sub-editing instructions according to the editing requirements in the multiple sub-editing instructions to obtain the intermediate image.

[0197] Optionally, the plurality of sub-editing instructions includes a first sub-editing instruction and a second sub-editing instruction; when the processor 1002 edits the sub-regions corresponding to each of the plurality of sub-editing instructions according to the editing requirements in the plurality of sub-editing instructions to obtain the intermediate image, it is specifically used to: edit the sub-region corresponding to the first sub-editing instruction in the initial image according to the editing requirements in the first sub-editing instruction to obtain a first region in the intermediate image, wherein the first region differs from the sub-region corresponding to the first sub-editing instruction in the initial image; and edit the second region in the intermediate image according to the second sub-editing instruction, based on the sub-region corresponding to the second sub-editing instruction in the initial image, wherein the second region is consistent with the sub-region corresponding to the second sub-editing instruction in the initial image.

[0198] Optionally, the processor 1002 is further configured to: calculate the semantic alignment between the intermediate image and the plurality of sub-editing instructions; and determine whether the intermediate image matches the image editing instructions based on the semantic alignment between the intermediate image and the plurality of sub-editing instructions.

[0199] Optionally, the processor 1002 is further configured to: if there is a first semantic alignment degree greater than or equal to an alignment threshold among the semantic alignment degrees corresponding to the plurality of sub-editing instructions, then generate first feature description information according to the sub-editing instruction corresponding to the first semantic alignment degree, wherein the first feature description information is used to describe features in the intermediate image that match the image editing instruction; if there is a second semantic alignment degree less than the alignment threshold among the semantic alignment degrees corresponding to the plurality of sub-editing instructions, then generate second feature description information according to the sub-editing instruction corresponding to the second semantic alignment degree, wherein the second feature description information is used to describe features in the initial image that do not match the image editing instruction; and determine that the matching description information of the intermediate image includes the first feature description information and / or the second feature description information.

[0200] Optionally, when the processor 1002 generates an image correction instruction based on the image editing instruction and the intermediate image, it is specifically used to: generate the image correction instruction based on the second feature description information, wherein the image correction instruction is used to instruct the correction of features in the intermediate image that do not match the image editing instruction.

[0201] Optionally, the processor 1002 is further configured to: acquire evaluation prompt words; and, based on the evaluation prompt words, trigger the evaluation function of the image editing model to determine whether the editing effect of the intermediate image matches the image editing instruction through the evaluation function of the image editing model.

[0202] Furthermore, such as Figure 10As shown, the electronic device also includes other components such as a communication component 1003, a display 1004, and a power supply component 1005.

[0203] The display 1004 is used to display the target image.

[0204] Optionally, the display 1004 is used to: display an initial image, including a virtual human wearing the initial clothing, on the fitting interface; Optionally, the processor 1002 is configured to: invoke an image editing model according to the image editing instruction to edit the initial image to obtain an intermediate image; the image editing instruction is used to instruct the replacement of the initial clothing in the initial image; Optionally, the processor 1002 is configured to: if the editing effect of the intermediate image does not match the image editing instruction, generate an image correction instruction based on the image editing instruction and the intermediate image; Optionally, the processor 1002 is configured to: correct the intermediate image according to the image correction instruction until a target image matching the image editing instruction is obtained; Optionally, the display 1004 is used to: display the target image on the fitting interface, the target image being used to demonstrate the effect of the virtual person wearing the target clothing.

[0205] Optionally, the processor 1002 is configured to: acquire training samples, the training samples including an initial image, image editing instructions, a first labeled image, a label matching result, a labeling instruction for image correction, and a second labeled image, the second labeled image conforming to the image editing instructions; Optionally, the processor 1002 is configured to: generate a prediction matching result and a prediction instruction for image correction based on the initial image, the image editing instruction, and the first labeled image using an initial model; Optionally, the processor 1002 is configured to: perform correction processing on the first labeled image according to the prediction instruction using the initial model to generate a predicted image; Optionally, the processor 1002 is configured to: update the model parameters of the initial model based on the prediction matching result and the annotation matching result, the prediction instruction and the annotation instruction, and the prediction image and the second annotation image, to obtain an image editing model.

[0206] Optionally, when the processor 1002 updates the model parameters of the initial model based on the predicted matching result and the labeled matching result, the predicted instruction and the labeled instruction, and the predicted image and the second labeled image to obtain an image editing model, it specifically performs the following steps: determining a first semantic similarity loss between the predicted matching result and the labeled matching result; determining a second semantic similarity loss between the predicted instruction and the labeled instruction; determining an image similarity loss between the predicted image and the second labeled image; constructing a loss function based on the first semantic similarity loss, the second semantic similarity loss, and the image similarity loss; and updating the model parameters of the initial model based on the loss function to obtain the image editing model.

[0207] Figure 10 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 10 The components shown. Additionally... Figure 10 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the work node. In this embodiment, the work node can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server-side device such as a conventional server, cloud server, or server array. If the work node in this embodiment is implemented as a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 10 The components within the dashed box; if the working node in this embodiment is implemented as a server-side device such as a conventional server, cloud server, or server array, it may be omitted. Figure 10 The component within the dashed box.

[0208] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0209] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0210] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0211] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0212] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0213] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium. Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0214] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0215] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image editing method, characterized in that, include: In response to image editing instructions, determine the initial image to be processed; According to the image editing instructions, the image editing model is invoked to edit the initial image to obtain an intermediate image; If the editing effect of the intermediate image does not match the image editing instruction, then an image correction instruction is generated based on the image editing instruction and the intermediate image; The intermediate image is corrected according to the image correction instructions until a target image matching the image editing instructions is obtained.

2. The method according to claim 1, characterized in that, According to the image editing instructions, the image editing model is invoked to edit the initial image to obtain an intermediate image, including: The image editing instructions are parsed to obtain multiple sub-editing instructions, each sub-editing instruction including the editing object and the editing requirements; Based on the editable objects in the plurality of sub-editing instructions, determine the sub-regions corresponding to each of the plurality of sub-editing instructions from the initial image; Based on the editing requirements in the plurality of sub-editing instructions, the sub-regions corresponding to each of the plurality of sub-editing instructions are edited to obtain the intermediate image.

3. The method according to claim 2, characterized in that, The plurality of sub-editing instructions includes a first sub-editing instruction and a second sub-editing instruction; according to the editing requirements in the plurality of sub-editing instructions, the sub-regions corresponding to each of the plurality of sub-editing instructions are edited to obtain the intermediate image, including: According to the editing requirements in the first sub-editing instruction, the sub-region corresponding to the first sub-editing instruction in the initial image is edited to obtain the first region in the intermediate image. The first region is different from the sub-region corresponding to the first sub-editing instruction in the initial image. According to the second sub-edit instruction, a second region in the intermediate image is obtained by editing based on the sub-region corresponding to the second sub-edit instruction in the initial image. The second region is consistent with the sub-region corresponding to the second sub-edit instruction in the initial image.

4. The method according to claim 2 or 3, characterized in that, The method further includes: Calculate the semantic alignment between the intermediate image and the plurality of sub-editing instructions; Based on the semantic alignment between the intermediate image and the plurality of sub-editing instructions, it is determined whether the intermediate image matches the image editing instructions.

5. The method according to claim 4, characterized in that, The method further includes: If there is a first semantic alignment degree that is greater than or equal to the alignment degree threshold among the semantic alignment degrees corresponding to the plurality of sub-editing instructions, then a first feature description information is generated according to the sub-editing instruction corresponding to the first semantic alignment degree. The first feature description information is used to describe the features in the intermediate image that match the image editing instruction. If there is a second semantic alignment degree that is less than the alignment degree threshold among the semantic alignment degrees corresponding to the plurality of sub-editing instructions, then a second feature description information is generated according to the sub-editing instruction corresponding to the second semantic alignment degree. The second feature description information is used to describe the features in the initial image that do not match the image editing instructions. The matching description information of the intermediate image is determined to include the first feature description information and / or the second feature description information.

6. The method according to claim 5, characterized in that, Based on the image editing instructions and the intermediate image, an image correction instruction is generated, including: The image correction instruction is generated based on the second feature description information. The image correction instruction is used to instruct the correction of features in the intermediate image that do not match the image editing instruction.

7. The method according to any one of claims 1-3 or 5-6, characterized in that, The method further includes: Get evaluation prompts; Based on the evaluation prompt, the evaluation function of the image editing model is triggered to determine whether the editing effect of the intermediate image matches the image editing instruction.

8. A virtual try-on method, characterized in that, include: The fitting room interface displays an initial image, including a virtual person wearing the initial clothing. According to the image editing instructions, the image editing model is invoked to edit the initial image to obtain an intermediate image; the image editing instructions are used to instruct the replacement of the initial clothing in the initial image; If the editing effect of the intermediate image does not match the image editing instruction, then an image correction instruction is generated based on the image editing instruction and the intermediate image; The intermediate image is corrected according to the image correction instructions until a target image matching the image editing instructions is obtained; The target image is displayed on the fitting interface, and the target image is used to demonstrate the effect of the virtual person wearing the target clothing.

9. A model training method, characterized in that, include: Obtain training samples, which include an initial image, image editing instructions, a first labeled image, label matching results, labeling instructions for image correction, and a second labeled image, wherein the second labeled image conforms to the image editing instructions; Based on the initial image, the image editing instructions, and the first labeled image, a prediction matching result and prediction instructions for image correction are generated using the initial model. Using the initial model, the first labeled image is corrected according to the prediction instructions to generate a predicted image; Based on the predicted matching result and the labeled matching result, the predicted instruction and the labeled instruction, and the predicted image and the second labeled image, the model parameters of the initial model are updated to obtain the image editing model.

10. The method according to claim 9, characterized in that, Based on the predicted matching result and the labeled matching result, the predicted instruction and the labeled instruction, and the predicted image and the second labeled image, the model parameters of the initial model are updated to obtain an image editing model, including: Determine the first semantic similarity loss between the predicted matching result and the labeled matching result; Determine the second semantic similarity loss between the predicted instruction and the labeled instruction; Determine the image similarity loss between the predicted image and the second labeled image; A loss function is constructed based on the first semantic similarity loss, the second semantic similarity loss, and the image similarity loss; The model parameters of the initial model are updated according to the loss function to obtain the image editing model.

11. An electronic device, characterized in that, include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program in the memory to implement the steps of the method according to any one of claims 1-10.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-10.

13. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-10.

Images