Virtual fitting image processing method, generation method, device, equipment and medium

Abstract

This application provides a virtual try-on image processing method, generation method, apparatus, device, and medium. The virtual try-on image processing method includes: acquiring a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is a low-quality image; based on the first clothing image, performing a quality assessment on the first virtual try-on image to obtain a first assessment result, and generating a first optimization instruction based on the first assessment result; processing the first virtual try-on image based on the first optimization instruction to obtain a second virtual try-on image; wherein the second virtual try-on image is a high-quality image. The method provided in this application, after obtaining a low-quality virtual try-on image, performs a quality assessment on it, and then processes the low-quality virtual try-on image according to the assessment result to obtain a final virtual try-on image, ensuring that the final virtual try-on image can highly reproduce clothing details and guaranteeing the effect when processing complex human postures.

Description

Technical Field

[0001] This application relates to the field of virtual try-on technology, and in particular to a virtual try-on image processing method, generation method, apparatus, device, and medium. Background Technology

[0002] Image-based Virtual Try-On (VTON) is an important research direction in the field of computer vision. It is mainly used to generate a composite image of a person wearing specified clothing based on an input image of a person and an image of the clothing to be tried on. This technology provides an innovative solution for optimizing the online shopping experience and expanding the application scenarios of fashion e-commerce by enabling virtual previews without the need for physical try-ons.

[0003] In terms of technical implementation, the early VTON method based on Generative Adversarial Networks (GAN) usually adopted an implementation scheme that includes the following two processing stages: first, the clothing is deformed based on human pose information, and then the deformed clothing is fused with the human image.

[0004] However, due to insufficient deformation processing precision, this method suffers from the technical defect of low reproduction of clothing details and is not effective when dealing with complex human postures. Summary of the Invention

[0005] To address one of the aforementioned technical deficiencies, this application provides a virtual try-on image processing method, generation method, apparatus, device, and medium.

[0006] In a first aspect, this application provides a virtual try-on image processing method, the method comprising:

[0007] Obtain a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image;

[0008] Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result;

[0009] The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image; wherein the second virtual try-on image is a high-quality image.

[0010] A second aspect of this application provides a method for generating virtual try-on images, the method comprising:

[0011] Generate a first virtual try-on image based on the person image and the first clothing image;

[0012] The second virtual try-on image is obtained by using the virtual try-on image processing method described in the first aspect above;

[0013] The second virtual try-on image is determined as the generated virtual try-on image.

[0014] A third aspect of this application provides a virtual try-on image processing apparatus, the apparatus comprising:

[0015] The acquisition module is used to acquire a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image;

[0016] The first processing module is used to perform a quality assessment on the first virtual try-on image based on the first clothing image, obtain a first assessment result, and generate a first optimization instruction based on the first assessment result.

[0017] The second processing module is used to process the first virtual try-on image based on the first optimization instruction to obtain the second virtual try-on image; wherein the second virtual try-on image is a high-quality image.

[0018] In a fourth aspect, this application provides a virtual try-on image generation apparatus, the apparatus comprising:

[0019] The first generation module is used to generate a first virtual try-on image based on the image of the person and the first image of the clothing.

[0020] A virtual try-on image processing apparatus is used to obtain a second virtual try-on image through the virtual try-on image processing method described in the first aspect above;

[0021] The second generation module is used to determine the obtained second virtual try-on image as the generated virtual try-on image.

[0022] A fifth aspect of this application provides an electronic device, comprising:

[0023] Memory;

[0024] Processor; and

[0025] Computer programs;

[0026] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect above; or the computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the second aspect above.

[0027] A sixth aspect of this application provides a computer-readable storage medium having a computer program stored thereon; the computer program being executed by a processor to implement the method as described in the first aspect above; or, the computer program being executed by a processor to implement the method as described in the second aspect above.

[0028] This application provides a virtual try-on image processing method, generation method, apparatus, device, and medium. The virtual try-on image processing method includes: acquiring a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is generated based on the first clothing image and is a low-quality image; performing a quality assessment on the first virtual try-on image based on the first clothing image to obtain a first assessment result, and generating a first optimization instruction based on the first assessment result; processing the first virtual try-on image based on the first optimization instruction to obtain a second virtual try-on image; wherein the second virtual try-on image is a high-quality image. The method provided in this application, after obtaining a low-quality virtual try-on image, performs a quality assessment on it, and then processes the low-quality virtual try-on image based on the assessment result to obtain a final virtual try-on image, ensuring that the final virtual try-on image can highly reproduce clothing details and guaranteeing the effect when processing complex human postures. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 A flowchart illustrating a virtual try-on image processing method provided in an embodiment of this application;

[0031] Figure 2 A schematic diagram illustrating the implementation principle of a virtual try-on image processing method provided in this application embodiment;

[0032] Figure 3 This is a schematic diagram of the structure of a virtual try-on image processing device provided in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0034] Figure 5 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0035] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0036] In the process of developing this application, the inventors discovered that, in terms of technical implementation, early VTON methods based on Generative Adversarial Networks (GANs) typically employed a scheme involving two processing stages: first, deforming the clothing based on human pose information, and then fusing the deformed clothing with the human image. However, due to insufficient precision in deformation processing, such methods suffer from low detail reproduction of clothing and are ineffective in handling complex human poses.

[0037] To address the aforementioned problems, this application provides a virtual try-on image processing method, generation method, apparatus, device, and medium. The virtual try-on image processing method includes: acquiring a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is generated based on the first clothing image and is a low-quality image; performing a quality assessment on the first virtual try-on image based on the first clothing image to obtain a first assessment result, and generating a first optimization instruction based on the first assessment result; processing the first virtual try-on image based on the first optimization instruction to obtain a second virtual try-on image; wherein the second virtual try-on image is a high-quality image. The method provided in this application, after obtaining a low-quality virtual try-on image, performs a quality assessment on it, and then processes the low-quality virtual try-on image according to the assessment result to obtain a final virtual try-on image, ensuring that the final virtual try-on image can highly reproduce clothing details and guaranteeing the effect when processing complex human postures.

[0038] This embodiment provides a virtual try-on image processing method, the implementation process of which is as follows: Figure 1 As shown, the implementation principle of this method is as follows: Figure 2 As shown.

[0039] See Figure 1 The implementation process of the virtual try-on image processing method provided in this embodiment is as follows:

[0040] 101. Obtain the first virtual try-on image and the first clothing image.

[0041] The first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image.

[0042] For example, the image of the person and the first image of the clothing are input into any existing virtual try-on generation model (such as a generative adversarial network), and the virtual try-on generation model generates a first virtual try-on image based on the image of the person and the first image of the clothing. In step 101, the first virtual try-on image and the first image of the clothing are obtained.

[0043] 102. Based on the first clothing image, perform a quality assessment on the first virtual try-on image to obtain a first assessment result, and generate a first optimization instruction based on the first assessment result.

[0044] Step 102 can be achieved using a well-trained quality assessment model.

[0045] The quality assessment model can be a human preference perception model based on a visual large language model, such as the quality assessment model as follows: Figure 2 The VTON-VLLM (Virtual Try On - Visual Language Large Model) shown is a quality assessment model. This model can: 1) quantitatively assess the overall quality of the input virtual try-on image (e.g., the first virtual try-on image); 2) perform multi-dimensional analysis of the degree of preservation of clothing detail features in the input virtual try-on image (e.g., the first virtual try-on image); and 3) comprehensively judge whether the try-on effect of the input virtual try-on image (e.g., the first virtual try-on image) meets the expected requirements based on human visual preference standards.

[0046] In order to evaluate the quality of the input virtual try-on image (such as the first virtual try-on image) according to human judgment standards, the quality assessment model (such as VTON-VLLM) evaluates the input virtual try-on image (such as the first virtual try-on image) according to a pre-constructed evaluation system.

[0047] The evaluation system comprises multiple evaluation dimensions based on two core indicators: clothing consistency and image quality. Regarding clothing consistency, the evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, and accuracy of clothing silhouette. Regarding image quality, the evaluation dimensions include: edge artifacts and the reasonableness of the target person's pose. In other words, the evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's pose.

[0048] 1. Visual Pattern Feature Accuracy: This focuses on evaluating whether the input virtual try-on image (such as the first virtual try-on image) accurately preserves the visual pattern features of the original garment. Visual pattern features include texture features and markings.

[0049] 2. Character Accuracy: This focuses on evaluating the accuracy of text elements in the input virtual try-on image (such as the first virtual try-on image).

[0050] 3. Sleeve Feature Accuracy: This focuses on evaluating the accuracy of specific sleeve features in the input virtual try-on image (such as the first virtual try-on image).

[0051] 4. Clothing Shape Accuracy: This focuses on evaluating the accuracy of the overall clothing shape in the input virtual try-on image (such as the first virtual try-on image).

[0052] 5. Edge Artifact: This function focuses on detecting edge artifacts at the junction of clothing and the human body in the input virtual try-on image (such as the first virtual try-on image).

[0053] 6. Human Pose: This section focuses on evaluating the reasonableness of the human pose in the input virtual try-on image (such as the first virtual try-on image).

[0054] After constructing the evaluation system, a set of questions will be generated for each evaluation dimension, and the questions in the set of questions for each evaluation dimension will be different.

[0055] User research was conducted based on evaluation dimensions, and several key questions were set for each evaluation dimension based on the research results. In other words, each question in the question set for any given evaluation dimension was predetermined based on that dimension.

[0056] For example, user research based on evaluation dimensions could include three questions on the "accuracy of visual pattern features." Each question would be different but related (i.e., the three questions would progressively delve deeper into the subject). Question 11 would ask whether the input virtual try-on image accurately retained the original garment's visual pattern features; Question 12 would ask which visual pattern features were not retained in the input virtual try-on image; and Question 13 would ask for the accuracy score of the input virtual try-on image's visual pattern features. Similarly, three questions could be set regarding the "accuracy of text elements," each different but related (i.e., the three questions would progressively delve deeper into the subject), such as Questions 21, 22, and 23. Three questions could also be set regarding the "accuracy of sleeve features," each different but related (i.e., the three questions would progressively delve deeper into the subject), such as Questions 31, 32, and 33. Finally, three questions could be set regarding the "accuracy of garment silhouette," each different but related (i.e., the three questions would progressively delve deeper into the subject), such as Questions 41, 42, and 43. Set three questions about "edge artifacts," each different but related (i.e., the three questions gradually delve deeper), such as questions 51, 52, and 53. Set three questions about "the plausibility of the target character's posture," each different but related (i.e., the three questions gradually delve deeper), such as questions 61, 62, and 63.

[0057] This generates the following problem sets: "Accuracy of visual pattern features" {Problem 11, Problem 12, Problem 13} (hereinafter referred to as Problem Set 1 for ease of description), "Accuracy of text elements" {Problem 21, Problem 22, Problem 23} (hereinafter referred to as Problem Set 2 for ease of description), "Accuracy of sleeve features" {Problem 31, Problem 32, Problem 33} (hereinafter referred to as Problem Set 3 for ease of description), "Accuracy of clothing silhouette" {Problem 41, Problem 42, Problem 43} (hereinafter referred to as Problem Set 4 for ease of description), "Edge artifacts" {Problem 51, Problem 52, Problem 53} (hereinafter referred to as Problem Set 6 for ease of description), and "Reasonableness of target person's posture" {Problem 61, Problem 62, Problem 63} (hereinafter referred to as Problem Set 6 for ease of description).

[0058] After constructing the evaluation system and generating a set of questions for each evaluation dimension, the first dataset for training the quality evaluation model will be constructed.

[0059] The first dataset includes multiple first sample data.

[0060] Each first sample data point includes: a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0061] 1) Second image pair

[0062] The second image pair consists of a second clothing image and a third virtual try-on image.

[0063] The third virtual try-on image is generated based on the second clothing image, and the third virtual try-on image is a low-quality image.

[0064] The generation process of the third virtual try-on image can be implemented using existing processes. For example, the first sample person image and the second clothing image can be input into any existing virtual try-on generation model (such as a generative adversarial network), and the virtual try-on generation model can generate the third virtual try-on image based on the first sample person image and the second clothing image. Further details are omitted here.

[0065] 2) Question and Answer Sequence

[0066] In this context, the question-answer sequence for any evaluation dimension includes multiple first question-answer pairs.

[0067] Each first question answer pair includes one question from the set of questions for any evaluation dimension and the second answer to that question.

[0068] The questions included in each pair of answers to the first question are different.

[0069] The second answer to any question is generated by the quality assessment model before training, based on the second image, and is confirmed by human review.

[0070] For example, the first dataset can be constructed using the following steps:

[0071] 201. Obtain 40,000 different low-quality virtual try-on images (i.e., obtain 40,000 third virtual try-on images) and generate clothing images for each low-quality virtual try-on image (i.e., 40,000 second clothing images).

[0072] 202. Each low-quality virtual try-on image and the clothing image that generated it are combined into an image pair (i.e., 40,000 second image pairs).

[0073] For example, a low-quality virtual try-on image 1 (i.e., the third virtual try-on image) and a clothing image 1 (i.e., the second clothing image) from which the low-quality virtual try-on image 1 is generated are obtained. The low-quality virtual try-on image 1 (i.e., the third virtual try-on image) and clothing image 1 (i.e., the second clothing image) are then combined to form image pair 1 (i.e., the second image pair). Similarly, a low-quality virtual try-on image 2 (i.e., the third virtual try-on image) and a clothing image 2 (i.e., the second clothing image) from which the low-quality virtual try-on image 2 is generated are obtained. The low-quality virtual try-on image 2 (i.e., the third virtual try-on image) and clothing image 2 (i.e., the second clothing image) are then combined to form image pair 2 (i.e., the second image pair). ... Acquire a low-quality virtual try-on image 40000 (i.e., the third virtual try-on image) and generate a clothing image 40000 (i.e., the second clothing image) from the low-quality virtual try-on image 40000. Combine the low-quality virtual try-on image 40000 (i.e., the third virtual try-on image) and the clothing image 40000 (i.e., the second clothing image) to form an image pair 40000 (i.e., the second image pair).

[0074] This results in 40,000 second image pairs.

[0075] 203. Input 40,000 second image pairs and problem sets 1, 2, 3, 4, 5, and 6 into the quality assessment model (since the quality assessment model has not been trained yet, the quality assessment model here is the pre-training quality assessment model).

[0076] 204. Because the quality assessment model is a human preference perception model based on a visual large language model, it inherently possesses the ability to answer questions based on images. Therefore, the quality assessment model will:

[0077] 1) Based on image pair 1, answer the three questions in question set 1 to obtain three answers: answer 11(1) for question 11, answer 12(1) for question 12, and answer 13(1) for question 13. Based on image pair 2, answer the three questions in question set 1 to obtain three answers: answer 11(2) for question 11, answer 12(2) for question 12, and answer 13(2) for question 13. … Based on image pair 40000, answer the three questions in question set 1 to obtain three answers: answer 11(40000) for question 11, answer 12(40000) for question 12, and answer 13(40000) for question 13.

[0078] 2) Based on image pair 1, answer the three questions in question set 2 to obtain three answers: answer 21(1) for question 21, answer 22(1) for question 22, and answer 23(1) for question 23. Based on image pair 2, answer the three questions in question set 2 to obtain three answers: answer 21(2) for question 21, answer 22(2) for question 22, and answer 23(2) for question 23. … Based on image pair 40000, answer the three questions in question set 2 to obtain three answers: answer 21(40000) for question 21, answer 22(40000) for question 22, and answer 23(40000) for question 23.

[0079] …

[0080] 6) Based on image pair 1, answer the three questions in question set 6 to obtain three answers: answer 61(1) for question 61, answer 22(1) for question 62, and answer 63(1) for question 63. Based on image pair 2, answer the three questions in question set 6 to obtain three answers: answer 61(2) for question 61, answer 22(2) for question 62, and answer 63(2) for question 63. … Based on image pair 40000, answer the three questions in question set 6 to obtain three answers: answer 61(40000) for question 61, answer 22(40000) for question 62, and answer 63(40000) for question 63.

[0081] Upon completion, the answers to each question in the set of questions for each of the six evaluation dimensions for each image pair are obtained. For example, for image pair 1, the answers to questions 11(1) for "Accuracy of visual pattern features", 12(1) for question 12, and 13(1) for question 13 are as follows: for "Accuracy of text elements", the answers to questions 21(1), 22(1), and 23(1) for question 21 are as follows: for "Accuracy of text elements", the answers to questions 22(1), 23(1) for question 22 are as follows: for "Accuracy of sleeve features", the answers to questions 31(1), 32(1), and 33(1) for question 33 are as follows: for "Accuracy of clothing silhouette", the answers to questions 41(1), 42(1), and 43(1) for question 43 are as follows: for "Edge artifacts", the answers to questions 51(1), 52(1), and 53(1) for question 51 are as follows: for "Edge artifacts", the answers to questions 52(1), 53(1) for question 52 are as follows: for "Reasonableness of target person's posture", the answers to questions 61(1), 62(1), and 63(1) for question 63 are as follows: Image pair 2 provides the following answers to questions 11(2), 12(2), and 13(2) for "Accuracy of visual pattern features"; 21(2), 22(2), and 23(2) for "Accuracy of text elements"; 31(2), 32(2), and 33(2) for "Accuracy of sleeve features"; 41(2), 42(2), and 43(2) for "Accuracy of clothing silhouette"; 51(2), 52(2), and 53(2) for "Edge artifacts"; and 61(2), 62(2), and 63(2) for "Reasonableness of target person's posture". ...The image pairs provide the following answers to questions 11, 12, and 13 regarding the accuracy of visual pattern features: 11(40000), 12(40000), and 13(40000) for "Accuracy of Visual Pattern Features"; 21(40000), 22(40000), and 23(40000) for "Accuracy of Text Elements"; and 31(40000), 32(40000), and 33(40000) for "Accuracy of Sleeve Features"; and 33(40000), 31(40000), 32(40000), and 33(40000) for "Accuracy of Sleeve Features". 000); Answers to questions 41 (40000), 42 (40000), and 43 (40000) regarding the accuracy of clothing silhouette; Answers to questions 51 (40000), 52 (40000), and 53 (40000) regarding edge artifacts; Answers to questions 61 (40000), 62 (40000), and 63 (40000) regarding the rationality of the target person's posture.

[0082] All of the above answers are the second answers as described in this embodiment and subsequent embodiments.

[0083] like Figure 2 The answer to "Letters on garment are not correct; they should generate the letters 'LEVIS'" is an example.

[0084] like Figure 2 The answer “Score:Pattern(4.6) / Sleeve(4.7) / Characters(1.5) / Shape(4.7) / Edge Artifact(4.8) / Human Pose(4.8)” is another example answer.

[0085] 205. Fifteen annotators with different professional backgrounds were organized to manually review and confirm all answers for all image pairs. If the answer was correct, it was kept; otherwise, it was deleted.

[0086] like Figure 2 The phrase "Letters on garment are correct" indicates that the answer "Score: Pattern(4.6) / Sleeve(4.7) / Characters(1.5) / Shape(4.7) / Edge Artifact(4.8) / HumanPose(4.8)" was manually verified and confirmed to be correct.

[0087] Furthermore, since the multiple questions in each evaluation dimension are interconnected, if any answer to a particular evaluation dimension is incorrect, then all answers to questions under that dimension are deleted. For example, if it is manually confirmed that answer 11(1) to question 11 regarding "accuracy of visual pattern features" for image pair 1 is incorrect, and all other answers (i.e., answers 12(1), 13(1), 21(1), 22(1), 23(1), 31(1), 32(1), 33(1), 41(1), 42(1), 43(1), 51(1), 52(1), 53(1), 61(1), 62(1), 63(1)) are correct, then all answers to questions regarding "accuracy of visual pattern features" for image pair 1 are deleted, resulting in answer 21 to question 21 regarding "accuracy of text elements" for image pair 1. Answer 21(1), Answer 22(1) to Question 22, Answer 23(1) to Question 23; Answer 31(1) to Question 31 regarding “accuracy of sleeve features”, Answer 32(1) to Question 32, Answer 33(1) to Question 33; Answer 41(1) to Question 41 regarding “accuracy of garment silhouette”, Answer 42(1) to Question 42, Answer 43(1) to Question 43; Answer 51(1) to Question 51 regarding “edge artifacts”, Answer 52(1) to Question 52, Answer 53(1) to Question 53; Answer 61(1) to Question 61 regarding “reasonableness of the target person’s posture”, Answer 62(1) to Question 62, Answer 63(1) to Question 63.

[0088] At the same time, each question and its corresponding answer will be paired to form a first question answer pair.

[0089] For example, for image pair 1, there are 15 answer pairs for the first question after manual verification, namely <Question 21, Answer 21(1)>, <Question 22, Answer 22(1)>, <Question 23, Answer 23(1)>, <Question 31, Answer 31(1)>, <Question 32, Answer 32(1)>, <Question 33, Answer 33(1)>, <Question 41, Answer 41(1)>, <Question 42, Answer 42(1)>, <Question 43, Answer 43(1)>, <Question 51, Answer 51(1)>, <Question 52, Answer 52(1)>, <Question 53, Answer 53(1)>, <Question 61, Answer 61(1)>, <Question 62, Answer 62(1)>, <Question 63, Answer 63(1)>.

[0090] This ensures that all remaining answers are correct. Furthermore, for any image pair, after executing step 205, it's possible to retain 18 answers for 6 evaluation dimensions, resulting in 18 answer pairs for the first question; it's also possible to retain 15 answers for 5 evaluation dimensions, resulting in 15 answer pairs for the first question; it's also possible to retain 12 answers for 4 evaluation dimensions, resulting in 12 answer pairs for the first question; it's also possible to retain 9 answers for 3 evaluation dimensions, resulting in 9 answer pairs for the first question; it's also possible to retain 6 answers for 2 evaluation dimensions, resulting in 6 answer pairs for the first question; it's also possible to retain 3 answers for 1 evaluation dimension, resulting in 3 answer pairs for the first question; and it's also possible to retain 0 answers for 0 evaluation dimensions (in this case, the image pair can be deleted, or it can remain empty with no answers).

[0091] 206. Each image pair and its corresponding first question answer pair constitute the first sample data.

[0092] For example, for image pair 1, its first sample data is: low-quality virtual try-on image 1 - clothing image 1, {<Question 21, Answer 21(1)>, <Question 22, Answer 22(1)>, <Question 23, Answer 23(1)>}, {<Question 31, Answer 31(1)>, <Question 32, Answer 32(1)>, <Question 33, Answer 33(1)>}, {<Question 41, Answer 41(1)>, <Question 42, Answer 42(1)>, <Question 43, Answer 43(1)>}, {<Question 51, Answer 51(1)>, <Question 52, Answer 52(1)>, <Question 53, Answer 53(1)>}, {<Question 61, Answer 61(1)>, <Question 62, Answer 62(1)>, <Question 63, Answer 63(1)>}.

[0093] Among them, low-quality virtual try-on image 1-clothing image 1 is image pair 1 (i.e., the second image pair), {<Question 21, Answer 21(1)>, <Question 22, Answer 22(1)>, <Question 23, Answer 23(1)>} is the question-answer sequence for "accuracy of text elements", {<Question 31, Answer 31(1)>, <Question 32, Answer 32(1)>, <Question 33, Answer 33(1)>} is the question-answer sequence for "accuracy of sleeve features", and {<Question 41, Answer 41(1)> The sequence of questions and answers for “Accuracy of Clothing Silhouette” is {<Question 42, Answer 42(1)>, <Question 43, Answer 43(1)>}. The sequence of questions and answers for “Edge Artifacts” is {<Question 51, Answer 51(1)>, <Question 52, Answer 52(1)>, <Question 53, Answer 53(1)>}. The sequence of questions and answers for “Rationality of Target Person’s Posture” is {<Question 61, Answer 61(1)>, <Question 62, Answer 62(1)>, <Question 63, Answer 63(1)>}.

[0094] In other words, each third virtual try-on image obtained through steps 202 to 206 corresponds to a first sample data point. Each first sample data point contains an image pair consisting of the third virtual try-on image and the clothing image that generated the third virtual try-on image, and may also contain a question-answer sequence. Each question-answer sequence corresponds to an evaluation dimension, and each question-answer sequence consists of three first question-answer pairs. Each first question-answer pair includes one question from the question set of that evaluation dimension and the answer to that question (i.e., the second answer). That is to say, the questions included in each first question-answer pair are different, and the answer has been manually verified and is the correct answer.

[0095] 207. All the first sample data are used to form the first dataset.

[0096] The rigorous quality control process from steps 201 to 207 ensures the accuracy of the answers in the first dataset, ultimately resulting in a high-quality dataset containing virtual try-on image-clothing image pairs, fine-grained human feedback, and six evaluation dimensions.

[0097] After obtaining the first dataset, a quality assessment model will be trained based on the first dataset; that is, the quality assessment model is trained based on the first dataset.

[0098] The training process for the quality assessment model is as follows:

[0099] 601. Generate data subsets for each evaluation dimension based on the first dataset.

[0100] Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question answers for any evaluation dimension of the second image pair.

[0101] For ease of explanation, the first dataset includes three first sample data sets: First Sample Data 1, First Sample Data 2, and First Sample Data 3. First Sample Data 1 includes the second image pair 1 and answer sequences B1 ("Accuracy of Text Elements"), C1 ("Accuracy of Sleeve Features"), D1 ("Accuracy of Clothing Silhouette"), E1 ("Accuracy of Edge Artifacts"), and F1 ("Reasonableness of Target Person's Posture"). First Sample Data 2 includes the second image pair 2 and answer sequences A2 ("Accuracy of Visual Pattern Features"), B2 ("Accuracy of Text Elements"), C2 ("Accuracy of Sleeve Features"), D2 ("Accuracy of Clothing Silhouette"), and E2 ("Accuracy of Edge Artifacts"). For example, the first sample data 3 includes the second image pair 3 and the question answer sequences A3 ("Accuracy of visual pattern features"), B3 ("Accuracy of text elements"), C3 ("Accuracy of sleeve features"), D3 ("Accuracy of clothing silhouette"), E3 ("Accuracy of edge artifacts"), and F3 ("Accuracy of target person's posture"). In step 601, six data subsets will be obtained, namely, data subset A ("Accuracy of visual pattern features"), data subset B ("Accuracy of text elements"), data subset C ("Accuracy of sleeve features"), data subset D ("Accuracy of clothing silhouette"), data subset E ("Accuracy of edge artifacts"), and data subset F ("Accuracy of target person's posture").

[0102] The data subset A for "visual pattern feature accuracy" includes two sample sub-data sets, namely sample sub-data A2 and sample sub-data A3. Sample sub-data A2 consists of the second image pair 2 obtained from the first sample data 2 and the question-answer sequence A2 for "visual pattern feature accuracy"; sample sub-data A3 consists of the second image pair 3 obtained from the first sample data 3 and the question-answer sequence A3 for "visual pattern feature accuracy".

[0103] The data subset B for "text element accuracy" includes three sample sub-data sets: sample sub-data B1, sample sub-data B2, and sample sub-data B3. Specifically, sample sub-data B1 consists of the second image pair 1 obtained from the first sample data 1 and the question-answer sequence B1 for "text element accuracy"; sample sub-data B2 consists of the second image pair 2 obtained from the first sample data 2 and the question-answer sequence B2 for "text element accuracy"; and sample sub-data B3 consists of the second image pair 3 obtained from the first sample data 3 and the question-answer sequence B3 for "text element accuracy".

[0104] The data subset C for "Sleeve Feature Accuracy" includes three sample sub-data sets: sample sub-data C1, sample sub-data C2, and sample sub-data C3. Specifically, sample sub-data C1 consists of the second image pair 1 obtained from the first sample data 1 and the question-answer sequence C1 for "Sleeve Feature Accuracy"; sample sub-data C2 consists of the second image pair 2 obtained from the first sample data 2 and the question-answer sequence C2 for "Sleeve Feature Accuracy"; and sample sub-data C3 consists of the second image pair 3 obtained from the first sample data 3 and the question-answer sequence C3 for "Sleeve Feature Accuracy".

[0105] The data subset D for "Clothing Silhouette Accuracy" includes three sample sub-data sets: sample sub-data set D1, sample sub-data set D2, and sample sub-data set D3. Specifically, sample sub-data set D1 consists of the second image pair 1 obtained from the first sample data set 1 and the question-answer sequence D1 for "Clothing Silhouette Accuracy"; sample sub-data set D2 consists of the second image pair 2 obtained from the first sample data set 2 and the question-answer sequence D2 for "Clothing Silhouette Accuracy"; and sample sub-data set D3 consists of the second image pair 3 obtained from the first sample data set 3 and the question-answer sequence D3 for "Clothing Silhouette Accuracy".

[0106] The data subset E for "edge artifacts" includes three sample sub-data sets: sample sub-data E1, sample sub-data E2, and sample sub-data E3. Specifically, sample sub-data E1 consists of the second image pair 1 obtained from the first sample data 1 and the question-and-answer sequence E1 for "edge artifacts"; sample sub-data E2 consists of the second image pair 2 obtained from the first sample data 2 and the question-and-answer sequence E2 for "edge artifacts"; and sample sub-data E3 consists of the second image pair 3 obtained from the first sample data 3 and the question-and-answer sequence E3 for "edge artifacts".

[0107] The data subset F for "the rationality of the target person's posture" includes three sample sub-data sets: sample sub-data F1, sample sub-data F2, and sample sub-data F3. Specifically, sample sub-data F1 consists of the second image pair 1 obtained from the first sample data 1 and the question-answer sequence F1 for "the rationality of the target person's posture"; sample sub-data F2 consists of the second image pair 2 obtained from the first sample data 2 and the question-answer sequence F2 for "the rationality of the target person's posture"; and sample sub-data F3 consists of the second image pair 3 obtained from the first sample data 3 and the question-answer sequence F3 for "the rationality of the target person's posture".

[0108] 602. Input the data subsets of each evaluation dimension into the quality assessment model, so that the quality assessment model generates the third answer for each question in the question set of each evaluation dimension based on the second image, and uses the formula... Conduct training.

[0109] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0110] Taking the example from step 601 as an example, in step 602, the following steps are taken: 1) The data subset A of "visual pattern feature accuracy" is input into the quality assessment model, so that the quality assessment model generates third answers to each question in the question set of each assessment dimension based on the second image, and is then trained. 2) The data subset B of "text element accuracy" is input into the quality assessment model, so that the quality assessment model generates third answers to each question in the question set of each assessment dimension based on the second image, and is then trained. 3) The data subset C of "sleeve shape feature accuracy" is input into the quality assessment model, so that the quality assessment model generates third answers to each question in the question set of each assessment dimension based on the second image, and is then trained. 4) The data subset D of "garment silhouette accuracy" is input into the quality assessment model, so that the quality assessment model generates third answers to each question in the question set of each assessment dimension based on the second image, and is then trained. 5) The data subset E of "edge artifacts" is input into the quality assessment model, so that the quality assessment model generates third answers to each question in the question set of each assessment dimension based on the second image, and is then trained. 6) Input the data subset E of “the rationality of the target person’s posture” into the quality assessment model so that the quality assessment model can generate the third answer of each question in the question set of each assessment dimension based on the second image and train it.

[0111] In other words, in step 602, the training is performed using the evaluation dimension, that is, one evaluation dimension at a time.

[0112] The training process of "visual pattern feature accuracy" will be introduced below. Other dimensions can be referred to "visual pattern feature accuracy" and will not be elaborated here.

[0113] The data subset A of "visual pattern feature accuracy" is input into the quality assessment model. Since the data subset A of "visual pattern feature accuracy" includes two sample sub-data, namely sample sub-data A2 and sample sub-data A3, in step 602, the model will first be trained based on sample sub-data A2 to adjust the parameters of the quality assessment model, and then trained based on sample sub-data A3.

[0114] (1) Training based on sample sub-data A2

[0115] Since sample subdata A2 includes: the second image pair 2 and the question-answer sequence A2 of "visual pattern feature accuracy", such as the question-answer sequence A2 of "visual pattern feature accuracy" being {<question 11, answer 11(A2)>, <question 12, answer 12(A2)>, <question 13, answer 13(A2)>}, the training process is as follows:

[0116] a. Input the second image pair 2 and the set of "visual pattern feature accuracy" questions (such as including questions 11, 12 and 13) into the quality assessment model.

[0117] b. The quality assessment model obtains the third answer 11 (second image pair 2) by answering question 11 based on the second image pair 2, obtains the third answer 12 (second image pair 2) by answering question 12 based on the second image pair 2, and obtains the third answer 13 (second image pair 2) by answering question 13 based on the second image pair 2.

[0118] c. Through formula Train the system so that the third answer 11 (second image pair 2) is infinitely close to answer 11 (A2), the third answer 12 (second image pair 2) is infinitely close to answer 12 (A2), and the third answer 13 (second image pair 2) is infinitely close to answer 13 (A2).

[0119] The specific values ​​for each parameter are as follows:

[0120] m represents sample sub-data A2, and j represents "visual pattern feature accuracy".

[0121] The answers are: 11 (second image pair 2), 12 (second image pair 2), and 13 (second image pair 2).

[0122] For the second image pair 2, X q (j) represents {problems 11, 12, and 13}, and ∏ represents the multiplication operator.

[0123] L is {<Question 11, Answer 11(A2)>, <Question 12, Answer 12(A2)>, <Question 13, Answer 13(A2)>}. In the actual implementation, a compact form will be used, that is, L is the string corresponding to the answer to question 11, answer to question 12, answer to question 12(A2), and answer to question 13(A2).

[0124] 'i' represents a word identifier, where 'word' refers to a word in a standard vocabulary. A standard vocabulary is constructed and maintained, containing all known words. L is segmented based on this standard vocabulary, and 'i' represents each word. i (j) is the i-th word in L.

[0125] For example, question 11 is ab, answer 11 (A2) is cd; question 12 is ef, answer 12 (A2) is gh; question 13 is km, answer 13 (A2) is xy. Then L is abcdefghkmxy. If a is a word, b is a word, c is a word, d is a word, e is a word, f is a word, gh is a word, k is a word, m is a word, x is a word, and y is a word, then L has a total of 11 words, i = 1, 2, ..., 11.

[0126] X q,＜i (j) represents the complete question preceding the i-th word in L. For example, when i = 2, X q,＜i (j) is the question before the second word in abcdefghkmxy, and the second word is b, which is not preceded by a complete question. In this case, X q,＜i (j) is empty. When i = 3, it is X. q,＜i (j) is the question before the third word in abcdefghkmxy, and the third word is c, which is preceded by a complete question (i.e., ab). At this time, X q,＜i (j) represents ab. Other values ​​of i represent X. q,＜i The value of (j) can be found in the two examples above, and will not be repeated here.

[0127] X a,＜i (j) represents the complete answer before the i-th word in L. For example, when i = 4, X q,＜i (j) represents the answer before the 4th word in the sequence abcdefghkmxy, where the 4th word is d, and there is no complete answer before it. In this case, X q,＜i (j) is empty. When i = 5, it is X. q,＜i (j) represents the answer before the 5th word in the sequence abcdefghkmxy, where the 5th word is 'e', ​​and there is a complete answer before it (i.e., 'cd'). In this case, X... q,＜i (j) represents cd. Other values ​​of i represent X. a,＜i The value of (j) can be found in the two examples above, and will not be repeated here.

[0128] θ represents the current parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0129] (2) Training based on sample sub-data A3

[0130] Since sample subdata A3 includes: the second image pair 3 and the question-answer sequence A3 of "visual pattern feature accuracy", if the question-answer sequence A2 of "visual pattern feature accuracy" is {<question 11, answer 11(A3)>, <question 12, answer 12(A3)>, <question 13, answer 13(A3)>}, then the training process is as follows:

[0131] a. Input the second image pair 3 and the set of "visual pattern feature accuracy" questions (such as including questions 11, 12 and 13) into the quality assessment model.

[0132] b. The quality assessment model obtains the third answer 11 (second image pair 3) by answering question 11 based on the second image pair 2, obtains the third answer 12 (second image pair 3) by answering question 12 based on the second image pair 2, and obtains the third answer 13 (second image pair 3) by answering question 13 based on the second image pair 2.

[0133] c. Through formula Train the system so that the third answer 11 (second image pair 3) is infinitely close to answer 11 (A3), the third answer 12 (second image pair 3) is infinitely close to answer 12 (A3), and the third answer 13 (second image pair 3) is infinitely close to answer 13 (A3).

[0134] The specific values ​​of each parameter can be found in the training process based on sample subdata A2, and will not be repeated here.

[0135] During the training process described above, L enables the trained model to decompose complex evaluation tasks into multiple intermediate reasoning steps through chain-like reasoning, thereby gaining a deeper understanding of the characteristics of virtual try-on technology and generating more reliable quality scoring results, thus improving the performance of the quality evaluation model.

[0136] In addition, by modeling the joint probability distribution of dialogue history and visual features, a multi-dimensional and accurate evaluation of the quality of virtual try-on images was achieved.

[0137] Furthermore, since there are a sufficient number of virtual try-on images (e.g., 40,000) when constructing the first dataset, each virtual try-on image corresponds to a question-and-answer sequence across six dimensions. This ensures a sufficient number (e.g., 40,000) of question-and-answer sequences for each dimension. Even if erroneous question-and-answer sequences are removed during subsequent manual review, the remaining correct question-and-answer sequences (e.g., 30,000) are sufficient to meet the requirements for subsequent training rounds. That is, in step 602, each question-and-answer sequence will be trained once, resulting in 30,000 training iterations. The resulting model parameters are sufficient to meet the accuracy requirements for virtual try-on image processing.

[0138] This concludes the detailed explanation of the training process for the quality assessment model.

[0139] In step 102, the first answer to each question in the question set of each evaluation dimension can be generated based on the first image using the trained quality assessment model. The first answer is determined as the first evaluation result, and the text of the first answer is determined as the first optimization instruction.

[0140] The first image pair consists of a first clothing image and a first virtual try-on image.

[0141] Additionally, the first optimization instruction is essentially the first answer in text form. For example, the first optimization instruction is text formed by the answers of the trained quality assessment model to each question in the set of questions for each assessment dimension based on the first image. This text can reflect the quality of the first virtual try-on image, such as indicating specific areas in the first virtual try-on image that do not conform to human preferences in terms of clothing consistency or image quality dimensions.

[0142] 103. The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image.

[0143] The second virtual try-on image is a high-quality image.

[0144] Step 103 can be achieved using a trained optimization model.

[0145] The optimization model can be a model that generates high-quality images from text and low-quality images, such as the optimization model being... Figure 2 The VTON Refine-model is used in this context. This VTON Refine-model can be a VRM-Instruct model, which integrates the inherent visual prior knowledge of a pre-trained text-to-image diffusion transformer (i.e., FLUX-Fill), thereby transforming the virtual try-on task into a conditional image inpainting problem.

[0146] The optimized model was trained based on the second dataset.

[0147] The second dataset includes multiple second sample data.

[0148] Each second sample data entry includes: a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0149] 1) Fourth image pair

[0150] The fourth image pair consists of the third clothing image and the fifth virtual try-on image.

[0151] The fifth virtual try-on image was generated based on the third clothing image, and the fifth virtual try-on image is a low-quality image.

[0152] The generation process of the fifth virtual try-on image can be implemented using existing processes. For example, the second sample person image and the third clothing image can be input into any existing virtual try-on generation model (such as a generative adversarial network), and the virtual try-on generation model can generate the fifth virtual try-on image based on the second sample person image and the third clothing image. Further details are omitted here.

[0153] 2) Fourth virtual try-on image

[0154] The fourth virtual try-on image is generated based on the third clothing image, and the fourth virtual try-on image is a high-quality image.

[0155] The generation process of the fourth virtual try-on image can be implemented using existing processes. For example, the second sample person image and the third clothing image can be input into any existing virtual try-on generation model (such as a generative adversarial network). At the same time, the clothing features can be effectively fused by integrating dedicated functional modules (including ControlNet and ReferenceNet), and a progressive generation strategy can be used to output a high-fidelity fourth virtual try-on image. Further details are omitted here.

[0156] In other words, the fourth and fifth virtual try-on images are virtual try-on images generated from the same clothing image and sample person image, but the two virtual try-on images are of different quality. The fourth virtual try-on image is a high-quality image, while the fifth virtual try-on image is a low-quality image.

[0157] 3) Third optimization instruction

[0158] The third optimization instruction can be obtained from the trained quality assessment model.

[0159] For example, the fourth image is input into the trained quality assessment model, which then uses the trained model to assess the quality of the fifth virtual try-on image based on the third clothing image, obtaining a third assessment result. A third optimization instruction is then generated based on this third assessment result. This process can be referred to in step 102, and will not be elaborated further here.

[0160] Each second sample data entry includes three images and an optimization instruction. The three images are: the third clothing image I... g Fifth virtual try-on image Fourth virtual try-on image An optimization instruction is the third optimization instruction O.

[0161] The optimized model will then be trained on the second dataset, using the third optimization instruction O (which is text) and the fifth virtual try-on image. (This is a low-quality image) Generate the fourth virtual try-on image. (It's a high-quality image). In other words, the optimization model will use the fifth virtual try-on image based on the third optimization instruction O. Optimized and upgraded to the fourth virtual try-on image

[0162] The loss function during model training is:

[0163] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0164] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image (i.e., a pure black image), and I1 represents a white image (i.e., a pure white image).

[0165] If the third clothing image Fifth virtual try-on image in, Let H be the set of real numbers, H be the height of the third clothing image and the fifth virtual try-on image, W be the width of the third clothing image and the fifth virtual try-on image, and 3 be the number of channels of the third clothing image and the fifth virtual try-on image. Then, through... The third clothing image And the fifth virtual try-on image splicing to form visual condition I cond Define the real target image as A repair mask is generated by combining the black image I0 and the white image I1. If the preserved region and the repaired region are identified separately, then the loss function during model training is:

[0166] The process of implementing step 103 based on the trained optimization model is as follows:

[0167] 103-1, an intermediate virtual try-on image is generated from the first virtual try-on image using a trained optimization model based on the first optimization instruction.

[0168] In step 103-1, the first virtual try-on image will be optimized and upgraded into a high-quality virtual try-on image (i.e., an intermediate virtual try-on image) according to the first optimization instruction.

[0169] However, the intermediate virtual try-on image obtained may not necessarily meet the requirements for clothing detail reproduction and human posture effect. Therefore, it will be adjusted through steps 103-2 and 103-3 to obtain a virtual try-on image (i.e., the second virtual try-on image) that meets the requirements for clothing detail reproduction and human posture effect.

[0170] 103-2, evaluate the fourth answer of each question in the question set of each evaluation dimension based on the third image pair using the quality assessment model, and determine the fourth answer as the second evaluation result.

[0171] The third image pair consists of the first clothing image and the intermediate virtual try-on image.

[0172] The implementation process of step 103-2 can be referred to the implementation process of step 102, and will not be repeated here.

[0173] 103-3 If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result, and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image.

[0174] If the second evaluation result meets the preset conditions, or if the second optimization instruction is repeatedly generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image, and the number of times reaches the preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0175] The preset conditions are pre-defined conditions for images that meet the requirements for clothing detail reproduction and human posture effect. In other words, if the second evaluation result shows that the current intermediate virtual try-on image has met the requirements for clothing detail reproduction and human posture effect, then the intermediate virtual try-on image that meets the requirements for clothing detail reproduction and human posture effect (i.e., the latest intermediate virtual try-on image) will be determined as the second virtual try-on image.

[0176] If the second evaluation result indicates that the current intermediate virtual try-on image does not meet the requirements for clothing detail reproduction and human posture effect, then the intermediate virtual try-on image is evaluated again through the quality evaluation model to obtain optimization instructions. The intermediate virtual try-on image is then optimized based on the optimization instructions through the optimization model until an intermediate virtual try-on image that meets the requirements for clothing detail reproduction and human posture effect is obtained, or the number of optimization repetitions reaches a preset number, then the repeated optimization process is stopped, and the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0177] The adjustment process in steps 103-2 and 103-3 is a dynamic adjustment process based on human preference-perceived reward (i.e. Figure 2The scaling with human-preference-aware reward process is implemented through the collaboration of a quality assessment model and an optimization model. After the optimization model generates intermediate virtual try-on images, the quality assessment model, acting as a reward model based on human preference perception, evaluates the quality of these images. If the evaluation result does not fully meet human preference standards, the optimization model is executed again, using the intermediate virtual try-on images and corresponding optimization instructions as conditions, to generate progressively optimized intermediate virtual try-on images. This iterative process continues until the intermediate virtual try-on images reach the preset quality standard or the maximum number of iterations is reached.

[0178] The virtual try-on image processing method provided in this embodiment can evaluate and optimize low-quality virtual try-on images based on human preferences.

[0179] The virtual try-on image processing method provided in this embodiment can be fine-tuned based on human feedback on the virtual try-on images, enabling a comprehensive evaluation of whether the virtual try-on images meet user expectations. The quality assessment model can be optimized in two ways: first, by providing fine-grained supervision signals to the optimization model; second, by dynamically adjusting based on a human preference-aware reward mechanism, supporting adaptive inference.

[0180] In addition, to establish a more comprehensive evaluation system, this embodiment constructs a challenging evaluation system that includes realistic and complex virtual try-on scenarios. Through six evaluation dimensions, the virtual try-on synthesis effect can be significantly improved.

[0181] This embodiment provides a virtual try-on image processing method, which acquires a first virtual try-on image and a first clothing image. The first virtual try-on image is generated based on the first clothing image and is a low-quality image. Based on the first clothing image, a quality assessment is performed on the first virtual try-on image to obtain a first assessment result, and a first optimization instruction is generated based on the first assessment result. The first virtual try-on image is then processed based on the first optimization instruction to obtain a second virtual try-on image, which is a high-quality image. The method provided in this embodiment, after obtaining a low-quality virtual try-on image, performs a quality assessment on it, and then processes the low-quality virtual try-on image based on the assessment result to obtain a final virtual try-on image. This ensures that the final virtual try-on image can highly reproduce clothing details and guarantees the effectiveness when handling complex human postures.

[0182] Based on the same inventive concept as the virtual try-on image processing method, this embodiment provides a virtual try-on image generation method, the implementation process of which is as follows:

[0183] 701. Generate a first virtual try-on image based on the person image and the first clothing image.

[0184] 702, through Figure 1 The virtual try-on image processing method shown is used to obtain the second virtual try-on image.

[0185] This section provides only a brief explanation and does not elaborate in detail. For the implementation details of step 702, please refer to [link / reference]. Figure 1 The implementation process of the virtual try-on image processing method shown.

[0186] For example, a first virtual try-on image and a first clothing image are obtained. The first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image.

[0187] Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result.

[0188] The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image. The second virtual try-on image is a high-quality image.

[0189] Specifically, based on the first clothing image, a quality assessment is performed on the first virtual try-on image to obtain a first assessment result, and a first optimization instruction is generated based on the first assessment result, including:

[0190] The quality assessment model generates the first answer to each question in the question set of each assessment dimension based on the first image, determines the first answer as the first assessment result, and determines the text of the first answer as the first optimization instruction.

[0191] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0192] The quality assessment model was trained based on the first dataset.

[0193] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0194] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0195] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0196] The quality assessment model is trained through the following steps:

[0197] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0198] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0199] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0200] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0201] Specifically, the first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image.

[0202] An intermediate virtual try-on image is generated from the first virtual try-on image using an optimized model based on the first optimization instruction.

[0203] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0204] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0205] The optimized model was trained based on the second dataset.

[0206] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0207] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0208] The loss function during model training is:

[0209] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0210] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0211] 703, the second virtual try-on image is determined as the generated virtual try-on image.

[0212] This embodiment provides a method for generating virtual try-on images. After obtaining a low-quality virtual try-on image, its quality is evaluated, and then the low-quality virtual try-on image is processed according to the evaluation results to obtain the final virtual try-on image. This ensures that the final virtual try-on image can highly restore the details of the clothing and ensure the effect when handling complex human postures.

[0213] Based on the same inventive concept as the virtual try-on image processing method, this embodiment provides a virtual try-on image processing device, see [link to relevant documentation]. Figure 3 The device includes:

[0214] The acquisition module 301 is used to acquire a first virtual try-on image and a first clothing image. The first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image.

[0215] The first processing module 302 is used to perform a quality assessment on the first virtual try-on image based on the first clothing image, obtain a first assessment result, and generate a first optimization instruction based on the first assessment result.

[0216] The second processing module 303 is used to process the first virtual try-on image based on the first optimization instruction to obtain a second virtual try-on image. The second virtual try-on image is a high-quality image.

[0217] The first processing module 302 is used to generate the first answer to each question in the question set of each evaluation dimension based on the first image using the quality assessment model, determine the first answer as the first evaluation result, and determine the text of the first answer as the first optimization instruction.

[0218] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0219] The quality assessment model was trained based on the first dataset.

[0220] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0221] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0222] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0223] The quality assessment model is trained through the following steps:

[0224] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0225] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0226] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0227] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0228] The second processing module 303 is used to generate an intermediate virtual try-on image from the first virtual try-on image based on the first optimization instruction using an optimization model.

[0229] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0230] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0231] The optimized model was trained based on the second dataset.

[0232] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0233] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0234] The loss function during model training is:

[0235] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0236] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0237] The device provided in this embodiment evaluates the quality of a low-quality virtual try-on image, and then processes the low-quality virtual try-on image based on the evaluation results to obtain a final virtual try-on image. This ensures that the final virtual try-on image can highly reproduce clothing details and guarantees the effectiveness when handling complex human postures.

[0238] Based on the same inventive concept of virtual try-on image generation method, this embodiment provides a virtual try-on image generation device, which includes: a first generation module, a virtual try-on image processing device, and a second generation module.

[0239] 1. First generation module

[0240] The first generation module is used to generate a first virtual try-on image based on the image of the person and the first image of the clothing.

[0241] 2. Virtual try-on image processing device

[0242] Virtual try-on image processing device, used for... Figure 1 The virtual try-on image processing method shown is used to obtain the second virtual try-on image.

[0243] Among them, the virtual try-on image processing device can, for example, Figure 3 As shown.

[0244] The virtual try-on image processing device includes: an acquisition module, a first processing module, and a second processing module.

[0245] The acquisition module is used to acquire a first virtual try-on image and a first garment image. The first virtual try-on image is generated based on the first garment image, and the first virtual try-on image is a low-quality image.

[0246] The first processing module is used to perform a quality assessment on the first virtual try-on image based on the first clothing image, obtain a first assessment result, and generate a first optimization instruction based on the first assessment result.

[0247] The second processing module is used to process the first virtual try-on image based on the first optimization instruction to obtain a second virtual try-on image. The second virtual try-on image is a high-quality image.

[0248] The first processing module is used to generate the first answer to each question in the question set of each evaluation dimension based on the first image using the quality assessment model, determine the first answer as the first evaluation result, and determine the text of the first answer as the first optimization instruction.

[0249] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0250] The quality assessment model was trained based on the first dataset.

[0251] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0252] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0253] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0254] The quality assessment model is trained through the following steps:

[0255] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0256] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0257] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i(j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0258] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0259] The second processing module is used to generate an intermediate virtual try-on image from the first virtual try-on image based on the first optimization instruction using an optimization model.

[0260] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0261] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0262] The optimized model was trained based on the second dataset.

[0263] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0264] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0265] The loss function during model training is:

[0266] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, xmask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0267] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0268] 3. Second generation module

[0269] The second generation module is used to determine the obtained second virtual try-on image as the generated virtual try-on image.

[0270] The device provided in this embodiment evaluates the quality of a low-quality virtual try-on image, and then processes the low-quality virtual try-on image based on the evaluation results to obtain a final virtual try-on image. This ensures that the final virtual try-on image can highly reproduce clothing details and guarantees the effectiveness when handling complex human postures.

[0271] Based on the same inventive concept of virtual try-on image processing method, this embodiment provides an electronic device, which is as follows: Figure 4 As shown, it includes: a memory 401, a processor 402, and a computer program.

[0272] The computer program is stored in memory 401 and configured to be executed by processor 402 to implement the virtual try-on image processing method described above.

[0273] Specifically,

[0274] Obtain a first virtual try-on image and a first clothing image. The first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image.

[0275] Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result.

[0276] The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image. The second virtual try-on image is a high-quality image.

[0277] Specifically, based on the first clothing image, a quality assessment is performed on the first virtual try-on image to obtain a first assessment result, and a first optimization instruction is generated based on the first assessment result, including:

[0278] The quality assessment model generates the first answer to each question in the question set of each assessment dimension based on the first image, determines the first answer as the first assessment result, and determines the text of the first answer as the first optimization instruction.

[0279] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0280] The quality assessment model was trained based on the first dataset.

[0281] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0282] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0283] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0284] The quality assessment model is trained through the following steps:

[0285] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0286] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0287] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0288] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0289] Specifically, the first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image.

[0290] An intermediate virtual try-on image is generated from the first virtual try-on image using an optimized model based on the first optimization instruction.

[0291] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0292] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0293] The optimized model was trained based on the second dataset.

[0294] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0295] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0296] The loss function during model training is:

[0297] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0298] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0299] The electronic device provided in this embodiment has a computer program executed by a processor to evaluate the quality of a low-quality virtual try-on image after it is obtained. Then, based on the evaluation result, the low-quality virtual try-on image is processed to obtain the final virtual try-on image, ensuring that the final virtual try-on image can highly restore the details of the clothing and guarantee the effect when handling complex human postures.

[0300] Based on the same inventive concept as the virtual try-on image generation method, this embodiment provides an electronic device, which is as follows: Figure 5 As shown, it includes: memory 501, processor 502, and computer program.

[0301] The computer program is stored in memory 501 and configured to be executed by processor 502 to implement the above-described virtual try-on image generation method.

[0302] Specifically,

[0303] A first virtual try-on image is generated based on the image of the person and the first image of the clothing.

[0304] pass Figure 1 The virtual try-on image processing method shown is used to obtain the second virtual try-on image.

[0305] The second virtual try-on image is determined as the generated virtual try-on image.

[0306] Among them, through Figure 1 The virtual try-on image processing method shown involves obtaining a first virtual try-on image and a first garment image when obtaining a second virtual try-on image. The first virtual try-on image is generated based on the first garment image and is a low-quality image.

[0307] Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result.

[0308] The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image. The second virtual try-on image is a high-quality image.

[0309] Specifically, based on the first clothing image, a quality assessment is performed on the first virtual try-on image to obtain a first assessment result, and a first optimization instruction is generated based on the first assessment result, including:

[0310] The quality assessment model generates the first answer to each question in the question set of each assessment dimension based on the first image, determines the first answer as the first assessment result, and determines the text of the first answer as the first optimization instruction.

[0311] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0312] The quality assessment model was trained based on the first dataset.

[0313] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0314] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0315] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0316] The quality assessment model is trained through the following steps:

[0317] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0318] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0319] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,<i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0320] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0321] Specifically, the first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image.

[0322] An intermediate virtual try-on image is generated from the first virtual try-on image using an optimized model based on the first optimization instruction.

[0323] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0324] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0325] The optimized model was trained based on the second dataset.

[0326] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0327] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0328] The loss function during model training is:

[0329] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0330] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0331] The electronic device provided in this embodiment has a computer program executed by a processor to evaluate the quality of a low-quality virtual try-on image after it is obtained. Then, based on the evaluation result, the low-quality virtual try-on image is processed to obtain the final virtual try-on image, ensuring that the final virtual try-on image can highly restore the details of the clothing and guarantee the effect when handling complex human postures.

[0332] Based on the same inventive concept as the virtual try-on image processing method, this embodiment provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the above-described virtual try-on image processing method.

[0333] Specifically,

[0334] Obtain a first virtual try-on image and a first clothing image. The first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image.

[0335] Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result.

[0336] The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image. The second virtual try-on image is a high-quality image.

[0337] Specifically, based on the first clothing image, a quality assessment is performed on the first virtual try-on image to obtain a first assessment result, and a first optimization instruction is generated based on the first assessment result, including:

[0338] The quality assessment model generates the first answer to each question in the question set of each assessment dimension based on the first image, determines the first answer as the first assessment result, and determines the text of the first answer as the first optimization instruction.

[0339] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0340] The quality assessment model was trained based on the first dataset.

[0341] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0342] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0343] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0344] The quality assessment model is trained through the following steps:

[0345] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0346] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0347] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0348] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0349] Specifically, the first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image.

[0350] An intermediate virtual try-on image is generated from the first virtual try-on image using an optimized model based on the first optimization instruction.

[0351] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0352] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0353] The optimized model was trained based on the second dataset.

[0354] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0355] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0356] The loss function during model training is:

[0357] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0358] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0359] The computer-readable storage medium provided in this embodiment has a computer program thereon that is executed by a processor to perform a quality assessment on a low-quality virtual try-on image, and then process the low-quality virtual try-on image based on the assessment results to obtain a final virtual try-on image. This ensures that the final virtual try-on image can highly reproduce clothing details and guarantees the effectiveness when handling complex human postures.

[0360] Based on the same inventive concept as the virtual try-on image generation method, this embodiment provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the above-described virtual try-on image generation method.

[0361] Specifically,

[0362] A first virtual try-on image is generated based on the image of the person and the first image of the clothing.

[0363] pass Figure 1 The virtual try-on image processing method shown is used to obtain the second virtual try-on image.

[0364] The second virtual try-on image is determined as the generated virtual try-on image.

[0365] Among them, through Figure 1 The virtual try-on image processing method shown involves obtaining a first virtual try-on image and a first garment image when obtaining a second virtual try-on image. The first virtual try-on image is generated based on the first garment image and is a low-quality image.

[0366] Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result.

[0367] The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image. The second virtual try-on image is a high-quality image.

[0368] Specifically, based on the first clothing image, a quality assessment is performed on the first virtual try-on image to obtain a first assessment result, and a first optimization instruction is generated based on the first assessment result, including:

[0369] The quality assessment model generates the first answer to each question in the question set of each assessment dimension based on the first image, determines the first answer as the first assessment result, and determines the text of the first answer as the first optimization instruction.

[0370] The questions in each evaluation dimension's question set are different. The first image pair consists of a first clothing image and a first virtual try-on image. Each question in any evaluation dimension's question set is predetermined based on that evaluation dimension.

[0371] The quality assessment model was trained based on the first dataset.

[0372] The first dataset includes multiple first sample data sets, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair.

[0373] The second image pair consists of a second clothing image and a third virtual try-on image. The third virtual try-on image is generated based on the second clothing image and is a low-quality image.

[0374] Each evaluation dimension's question-answer sequence includes multiple first question-answer pairs. Each first question-answer pair includes one question from the question set for any evaluation dimension and its second answer. The questions included in each first question-answer pair are different. The second answer to any question is generated by the pre-trained quality evaluation model based on the second image pair and is manually verified.

[0375] The quality assessment model is trained through the following steps:

[0376] Data subsets for each evaluation dimension are generated based on the first dataset. Each evaluation dimension's data subset includes multiple sample sub-data sets, and each sample sub-data set includes a second image pair and a sequence of question-answer pairs for any evaluation dimension of the second image pair.

[0377] The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training.

[0378] Where m represents the sample sub-data, and j represents the evaluation dimension identifier. This is the third answer to each question in the generated evaluation dimension j question set for the second image of the sample subdata m. X is the second image pair of sample subdata m. q (j) represents the set of questions for evaluation dimension j, ∏ is the multiplication operator, i is the word identifier, L is the sequence of answers to questions for evaluation dimension j in the sample subdata m, and x i (j) is the i-th word in L, X q,＜i (j) represents the question preceding the i-th word in L, X a,＜i (j) represents the answer before the i-th word in L, θ represents the parameters of the quality assessment model, and p[·|·] represents the joint probability distribution.

[0379] The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture.

[0380] Specifically, the first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image.

[0381] An intermediate virtual try-on image is generated from the first virtual try-on image using an optimized model based on the first optimization instruction.

[0382] The fourth answer to each question in the set of questions for each evaluation dimension generated based on the third image pair is evaluated using a quality assessment model, and the fourth answer is determined as the second evaluation result. The third image pair consists of a first clothing image and an intermediate virtual try-on image.

[0383] If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result. The intermediate virtual try-on image is then processed based on the second optimization instruction to obtain a new intermediate virtual try-on image. If the second evaluation result meets the preset conditions, or if the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image.

[0384] The optimized model was trained based on the second dataset.

[0385] The second dataset includes multiple second sample data sets, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair.

[0386] The fourth image pair consists of the third clothing image and the fifth virtual try-on image. The fourth virtual try-on image is generated based on the third clothing image and is a high-quality image. The fifth virtual try-on image is generated based on the third clothing image and is a low-quality image.

[0387] The loss function during model training is:

[0388] in, Let be the expectation function, t be the time step identifier, and ∈ be Gaussian noise. For Gaussian noise ∈ conforming to the standard distribution, v γ To optimize the model parameters, x0 represents the real target image I. gt The latent encoding, x cond For visual condition I cond The latent encoding, x mask For the real target image I mask The latent encoding, x t The result of adding noise ∈ to x0 at time step t, where O is the third optimization instruction.

[0389] I g For the third clothing image, This is the fourth virtual try-on image. This is the concatenation operator. This is the fifth virtual try-on image. I0 represents a black image, and I1 represents a white image.

[0390] The computer-readable storage medium provided in this embodiment has a computer program thereon that is executed by a processor to perform a quality assessment on a low-quality virtual try-on image, and then process the low-quality virtual try-on image based on the assessment results to obtain a final virtual try-on image. This ensures that the final virtual try-on image can highly reproduce clothing details and guarantees the effectiveness when handling complex human postures.

[0391] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0392] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0393] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0394] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0395] Furthermore, the terms "first," "second," "third," "fourth," and "fifth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. In the description of this application, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0396] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0397] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A virtual try-on image processing method, characterized in that, The method includes: Acquire a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image; Based on the first clothing image, the quality of the first virtual try-on image is evaluated to obtain a first evaluation result, and a first optimization instruction is generated based on the first evaluation result; The first virtual try-on image is processed based on the first optimization instruction to obtain a second virtual try-on image; wherein, the second virtual try-on image is a high-quality image; The step of performing a quality assessment on the first virtual try-on image based on the first clothing image to obtain a first assessment result, and generating a first optimization instruction based on the first assessment result, includes: The quality assessment model generates the first answer to each question in the question set of each assessment dimension based on the first image, determines the first answer as the first assessment result, and determines the text of the first answer as the first optimization instruction; The questions in each evaluation dimension's question set are different; the first image pair consists of a first clothing image and a first virtual try-on image; each question in any evaluation dimension's question set is predetermined based on that evaluation dimension. The quality assessment model is trained based on the first dataset. The first dataset includes multiple first sample data, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair. The second image pair consists of a second clothing image and a third virtual try-on image; the third virtual try-on image is generated based on the second clothing image and is a low-quality image. The question-answer sequence for any evaluation dimension includes multiple first question-answer pairs; each first question-answer pair includes a question from the question set for any evaluation dimension and a second answer to that question; the questions included in each first question-answer pair are different; the second answer to any question is the answer to any question generated by the quality evaluation model before training based on the second image pair, and is confirmed by manual review; The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture; The quality assessment model is trained through the following steps: Data subsets for each evaluation dimension are generated based on the first dataset; wherein, the data subset for any evaluation dimension includes multiple sample sub-data, and each sample sub-data includes a second image pair and a sequence of question answers for the second image pair for any evaluation dimension; The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training; in, For sample sub-data, For evaluation dimension identification, For sample sub-data The second image pairs the generated evaluation dimensions The third answer to each question in the question set. For sample subdata The second image pair, For evaluation dimensions Problem set For multiplication operators, For word identification, For sample subdata Evaluation dimensions The sequence of questions and answers for The first in One word, for The first in The question before each word, for The first in The answer before the word, For the parameters of the quality assessment model, It is a joint probability distribution.

2. The method according to claim 1, characterized in that, The first virtual try-on image is processed based on the first optimization instruction to obtain the second virtual try-on image; An intermediate virtual try-on image is generated from the first virtual try-on image using an optimized model based on the first optimization instruction. The fourth answer to each question in the question set of each evaluation dimension generated based on the third image pair is evaluated by a quality assessment model, and the fourth answer is determined as the second evaluation result; wherein, the third image pair consists of a first clothing image and an intermediate virtual try-on image; If the second evaluation result does not meet the preset conditions, a second optimization instruction is generated based on the second evaluation result, and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image; if the second evaluation result meets the preset conditions, or the number of times the second optimization instruction is generated based on the second evaluation result and the intermediate virtual try-on image is processed based on the second optimization instruction to obtain a new intermediate virtual try-on image reaches a preset number, then the latest intermediate virtual try-on image is determined as the second virtual try-on image; The optimization model is trained based on the second dataset; The second dataset includes multiple second sample data, each of which includes a fourth image pair, a fourth virtual try-on image, and a third optimization instruction for the fourth image pair. The fourth image pair consists of a third clothing image and a fifth virtual try-on image; the fourth virtual try-on image is generated based on the third clothing image and is a high-quality image; the fifth virtual try-on image is generated based on the third clothing image and is a low-quality image. The loss function during the training of the optimization model is: ; in, Let be the expected function. For time step identifier, It is Gaussian noise. Gaussian noise Conforms to a standard distribution. To optimize the model parameters, For real target image The potential encoding, For visual conditions The potential encoding, For real target image The potential encoding, for At time step Add noise The result after that, This is the third optimization instruction; , For the third clothing image, This is the fourth virtual try-on image. For concatenation operators; , This is the fifth virtual try-on image; , The image is black and white. The image is white.

3. A method for generating virtual try-on images, characterized in that, The method includes: Generate a first virtual try-on image based on the person image and the first clothing image; The second virtual try-on image is obtained by the virtual try-on image processing method according to claim 1 or 2; The obtained second virtual try-on image is determined as the generated virtual try-on image.

4. A virtual try-on image processing device, characterized in that, The device includes: The acquisition module is used to acquire a first virtual try-on image and a first clothing image; wherein the first virtual try-on image is generated based on the first clothing image, and the first virtual try-on image is a low-quality image; The first processing module is used to perform a quality assessment on the first virtual try-on image based on the first clothing image, obtain a first assessment result, and generate a first optimization instruction based on the first assessment result. The second processing module is used to process the first virtual try-on image based on the first optimization instruction to obtain a second virtual try-on image; wherein the second virtual try-on image is a high-quality image; The first processing module is used to generate the first answer to each question in the question set of each evaluation dimension based on the first image using the quality assessment model, determine the first answer as the first evaluation result, and determine the text of the first answer as the first optimization instruction; The questions in each evaluation dimension's question set are different; the first image pair consists of a first clothing image and a first virtual try-on image; each question in any evaluation dimension's question set is predetermined based on that evaluation dimension. The quality assessment model is trained based on the first dataset. The first dataset includes multiple first sample data, each of which includes a second image pair and a sequence of question answers for each evaluation dimension of the second image pair. The second image pair consists of a second clothing image and a third virtual try-on image; the third virtual try-on image is generated based on the second clothing image and is a low-quality image. The question-answer sequence for any evaluation dimension includes multiple first question-answer pairs; each first question-answer pair includes a question from the question set for any evaluation dimension and a second answer to that question; the questions included in each first question-answer pair are different; the second answer to any question is the answer to any question generated by the quality evaluation model before training based on the second image pair, and is confirmed by manual review; The evaluation dimensions include: accuracy of visual pattern features, accuracy of text elements, accuracy of sleeve features, accuracy of clothing silhouette, edge artifacts, and the reasonableness of the target person's posture; The quality assessment model is trained through the following steps: Data subsets for each evaluation dimension are generated based on the first dataset; wherein, the data subset for any evaluation dimension includes multiple sample sub-data, and each sample sub-data includes a second image pair and a sequence of question answers for the second image pair for any evaluation dimension; The data subsets for each evaluation dimension are input into the quality assessment model, enabling the model to generate third answers for each question in the question set for each evaluation dimension based on the second image, and then using the formula... Conduct training; in, For sample sub-data, For evaluation dimension identification, For sample sub-data The second image pairs the generated evaluation dimensions The third answer to each question in the question set. For sample subdata The second image pair, For evaluation dimensions Problem set For multiplication operators, For word identification, For sample subdata Evaluation dimensions The sequence of questions and answers for The first in One word, for The first in The question before each word, for The first in The answer before the word, For the parameters of the quality assessment model, It is a joint probability distribution.

5. A virtual try-on image generation device, characterized in that, The device includes: The first generation module is used to generate a first virtual try-on image based on the image of the person and the first image of the clothing. A virtual try-on image processing device is used to obtain a second virtual try-on image by means of the virtual try-on image processing method according to claim 1 or 2; The second generation module is used to determine the obtained second virtual try-on image as the generated virtual try-on image.

6. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as claimed in claim 1 or 2; or the computer program is stored in the memory and configured to be executed by the processor to implement the method as claimed in claim 3.

7. A computer-readable storage medium, characterized in that, It stores a computer program thereon; the computer program is executed by a processor to implement the method as claimed in claim 1 or 2; or, the computer program is executed by a processor to implement the method as claimed in claim 3.

Images