A method and system for generating a virtual fitting image by generating clothing wrinkles and clothing deformation

By combining diffusion model and Deformable attention technology, virtual trial-on-one pictures are generated, which solves the problem of poor wrinkles and deformation effects in the existing technology, and achieves high-quality, low-cost, and strong sense of real virtual trial-on-one pictures.

CN117745881BActive Publication Date: 2024-11-15NANJING YINGQI INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311706389.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-13
Publication Date
2024-11-15
Estimated Expiration
2043-12-13

AI Technical Summary

Technical Problem

The existing virtual try-on image generation technology is difficult to effectively generate the wrinkles and deformation effects of clothes, resulting in the generated images lacking realism and requiring manual intervention, which is costly.

Method used

Using the virtual try-on diagram of wrinkles based on diffusion model and the deformation is generated based on Deformable attention, and combining the two through semantic segmentation and image overlay technology, using models such as stable-diffusion and Controlnet to optimize edge and background processing to achieve high-quality virtual Try on the picture to generate.

Benefits of technology

It greatly improves the authenticity of the wrinkles and deformation characteristics of the dress, reduces the cost of manual intervention, improves the generation efficiency, and the generated images are close to the real trial-on effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117745881B_ABST
    Figure CN117745881B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and a system for generating a virtual try-on picture by generating clothes wrinkles and clothes deformation. By combining the latest virtual try-on generation model, the authenticity of features such as wrinkles and deformation of clothes being tried on is greatly improved. The process does not require human intervention, the cost is lower, the efficiency of outputting pictures is very high, and the unreality of virtual try-on pictures caused by lack of wrinkles, detail changes, unreasonable deformation and the like is overcome, and an effect very close to a real try-on picture is achieved. The method is simple to use, and the corresponding picture can be generated by simply specifying the background picture, clothes, model postures and the like to generate the picture, and is easy to use.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of virtual fitting images, and in particular to a method and system for generating virtual fitting images by generating clothing wrinkles and clothing deformation. Background Art

[0002] The development of computer technology and the widespread application of computer vision principles have made the research on using computer image processing technology to generate virtual try-on images increasingly popular. Virtual clothing try-on aims to visualize the effect of clothing on a person based on the image of the person and the image of the clothing. It is of great significance to improving the user consumption experience of e-commerce platforms and reducing merchants' model image costs.

[0003] The challenges of generating virtual try-on images currently lie primarily in two areas: deforming the garment to the target pose and preserving the original garment's details. Some commonly used virtual try-on image generation technologies suffer from the randomness of the generated models, leading to changes or loss of detail in the original garment. Others simply deform the garment, eliminating effects like wrinkles and rendering the pose unrealistic. Still others require extensive manual screening and adjustments using techniques like Photoshop, resulting in significant costs. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and a system for generating virtual try-on images by generating clothing wrinkles and clothing deformations, and to generate high-quality virtual try-on images by generating wrinkles based on a diffusion model, generating deformable clothing based on deformable attention, and superimposing the two.

[0005] To solve the above technical problems, the present invention provides a method for generating a virtual fitting image by generating clothing wrinkles and clothing deformation, comprising the following steps:

[0006] Step 1: Input the clothes and models for virtual fitting images into the data preprocessing module to obtain the posture skeleton image and model mask image model input required by the two generation models;

[0007] Step 2: Input the corresponding data into the wrinkled clothing generation model and the deformable clothing generation model respectively to generate images of a model wearing white clothing, clothing with only wrinkles, and a model wearing a try-on clothing;

[0008] Step 3: Consider the white wrinkled clothes image as the base image and the other image as the image to be overlaid, and obtain the mask images of the clothes part of the two images through semantic segmentation;

[0009] Step 4: Perform a bitwise AND operation on the clothing parts of the two mask images to obtain the overlapping part of the masks; superimpose the values ​​of the pixels at corresponding positions of the two images according to the overlapping masks; amplify the edge part (non-overlapping part) of the mask to be superimposed by a small proportion, and use the pixels of the base image in this part as the pixels of the corresponding part of the new superimposed image, and use the pixels of the remaining part of the superimposed image as the pixels of the corresponding part of the new superimposed image;

[0010] Step 5: Use the stable-diffusion inpainting model to deal with defects on the edges of the clothes, and use ControlNet to control the generation of the image;

[0011] Step 6: Use the semantic mask obtained by SAM (Segment Anything Model) to fix the character, switch the background, adjust the character's proportion and position, and generate the final virtual model image.

[0012] Preferably, in step 2, the wrinkled clothing generation model is based on the inpainting diffusion model of stable diffusion, and specifically includes the following steps:

[0013] Step 21: The white clothes corresponding to the input clothes shape are converted into the conditional input form required by stable diffusion through the textual inversion module, and the noise generation is controlled by the conditional input;

[0014] Step 22: Randomly generate a Gaussian noise image and aggregate it with the mask image of the scaled-down model, the pose information image, the white clothing deformation image generated by the simple model encoded by the VAE encoder, and the image of the model with the clothing removed. Input the trained stable diffusion unet, calculate the noise to be removed at this time step, and subtract the noise from the noise image.

[0015] Step 23: Repeat the denoising process and generate the target wrinkled clothing image through the VAE decoder.

[0016] Preferably, in step 2, the deformable clothing generation model is based on an end-to-end model of deformable attention, specifically comprising the following steps:

[0017] Step 24: Input the model's clothing-agnostic image and posture skeleton image into a Feature Pyramid Network (FPN) to extract features of different scales, and input the processed clothing image into another FPN to extract features of different scales.

[0018] Step 25: Use a layered cascade method to fuse the features extracted by the two FPNs starting from the smallest scale feature. This layered cascade method consists of multiple layers of deformable attention wrap modules. These modules estimate the deformation of the clothes from coarse to fine through self-deformable attention and cross-deformable attention. Each layer of deformable attention wrap module fuses the information of the corresponding scales of the two FPNs and the information from the deformable attention wrap module of the previous level to obtain the final clothing deformation information.

[0019] Step 26: The final estimated clothing deformation information is integrated into an encoder-decoder model. The clothing and model agnostic images are input into the encoder, the clothing deformation information is integrated, and the virtual fitting image is output through the decoder.

[0020] Preferably, in step 3, the values ​​of the pixels at corresponding positions of the two images are superimposed according to the overlapping mask, and the RGB brightness values ​​of the corresponding pixels of the two images are subjected to the following operations respectively:

[0021] L new =r*L2+(1-r)(L1*L2)

[0022]

[0023] Where L1 and L2 are the brightness of wrinkle and deformation images respectively, a is a hyperparameter, L new The brightness of the new overlay image.

[0024] Preferably, in step 4, a bitwise AND operation is performed on the mask image of the image to be overlaid and the mask image of the base image to obtain an overlapping mask image of the two images; the edge portion of the image to be overlaid is enlarged at a small ratio to obtain an edge mask image. The final overlaid image consists of three parts. The part covered by the overlapping mask image superimposes the values ​​of the pixels at corresponding positions of the base image and the image to be overlaid. The pixels covered by the edge mask image are the pixels at corresponding positions of the base image, and the remaining parts are the pixels at corresponding positions of the image to be overlaid.

[0025] Preferably, in step 5, the edge of the superimposed effect image is extracted by the Canny edge detection method, and the edge is input into Controlnet to control the generation process of stable diffusion; the superimposed effect image is input into the stable-diffusion model, and the mask of the edge part is used as the mask of the inpainting model to remove the defects on the edge of the superimposed effect image and beautify the model.

[0026] Preferably, in step 6, the semantic mask obtained by SAM is used to cut out the character from the background of the overlay image, the background is switched, the proportion or position of the character is adjusted, and the final avatar image is generated.

[0027] Correspondingly, a virtual try-on image generation system for generating clothing wrinkles and clothing deformation includes: a data preprocessing module, a wrinkled clothing generation module, a deformed clothing generation module, a semantic segmentation module, an image overlay module and a detail optimization module; the clothing and model for generating the virtual try-on image are input into the data preprocessing module to obtain the posture skeleton image and the model mask image model input required by the two generation models; the corresponding data are input into the wrinkled clothing generation module and the deformed clothing generation module respectively to generate a model image of a model wearing white clothing with only wrinkles on the clothing and a model image of a model wearing try-on clothing; the white wrinkled clothing image is regarded as a base image and the other image is regarded as an image to be overlaid, and the mask images of the clothing parts of the two images are obtained through the semantic segmentation module; the clothing parts of the two mask images are overlaid through the image overlay module to obtain a try-on image; the detail optimization module is used to process the defects on the edge of the clothing, and Controlnet is used to control the generation of the image, and the semantic mask obtained by SAM is used to fix the character, switch the background, adjust the character's proportion and position, and generate the final virtual model image.

[0028] Preferably, in the wrinkled clothing generation module, the wrinkled clothing generation model is based on the inpainting diffusion model of stable diffusion, and the white clothing corresponding to the input clothing shape is converted into the conditional input form required by stable diffusion through the textual inversion module, and the noise generation is controlled by the conditional input; a Gaussian noise image is randomly generated, and it is aggregated with the mask image of the reduced model, the pose information image, the white clothing deformation image generated by the simple model encoded by the VAE encoder, and the model's clothing removal part image, and the trained stable diffusion UNET is input to calculate the noise that needs to be removed at this time step, and the noise is subtracted from the noise image; the denoising process is repeated continuously, and the target wrinkled clothing image is generated through the VAE decoder.

[0029] Preferably, in the deformable clothing generation module, the deformable clothing generation model is an end-to-end model based on deformable attention, and the model's clothing agnostic graph and posture skeleton graph are input into an FPN (Feature Pyramid Network) to extract features of different scales, and the clothing processing graph is input into another FPN to extract features of different scales; a layered cascade method is adopted to fuse the features extracted by the two FPNs starting from the features with the smallest scale. The layered cascade method is composed of multiple layers of deformable attention wrap modules, which estimate the deformation of the clothes from coarse to fine through self-deformable attention and cross-deformable attention. Each layer of deformable attention wrap module fuses the information of the corresponding scales of the two FPNs and the information from the previous level deformable attention wrap module to obtain the final clothing deformation information; the final estimated clothing deformation information is fused into an encoder-decoder model, the clothing and model agnostic graphs are input into the encoder, the clothing deformation information is fused, and then the virtual try-on image is output through the decoder.

[0030] The beneficial effects of the present invention are as follows: by combining the latest virtual try-on generation model, the present invention greatly improves the authenticity of features such as wrinkles and deformation of tried-on clothes. The process does not require human intervention, the cost is lower, the image output efficiency is very high, and it overcomes the unreality of virtual try-on images due to the lack of wrinkles, changes in details, unreasonable deformation, etc., and achieves an effect that is very close to the real try-on image; the method is simple to use, and only needs to specify the background image, clothes, model posture, etc. to generate the corresponding image, which is easy to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the method of the present invention.

[0032] FIG2( a ) is an original picture of the model of the present invention.

[0033] FIG2( b ) is a clothing agnostic diagram of the present invention.

[0034] FIG2( c ) is a posture skeleton diagram of the present invention.

[0035] FIG. 2( d ) is a diagram showing the clothes processing of the present invention.

[0036] Figure 2(e) is a diagram of white clothing according to the present invention.

[0037] FIG2( f ) is a white clothing wrinkle image generated by the present invention based on the stable diffusion inpainting model.

[0038] Figure 2(g) is a virtual try-on image generated by the deformable attention-based model of the present invention.

[0039] FIG2( h ) is a mask image of the image to be superimposed according to the present invention.

[0040] FIG2(i) is a mask image of the base image of the present invention.

[0041] FIG2(j) is a superimposed mask diagram of the present invention.

[0042] FIG2( k ) is an edge mask diagram of the present invention.

[0043] FIG2( l ) is a diagram showing the superposition effect of the present invention.

[0044] FIG2( m ) is a diagram showing the final result of the present invention. DETAILED DESCRIPTION

[0045] like Figure 1 As shown, a method for generating a virtual try-on image by generating clothing wrinkles and clothing deformation includes the following steps:

[0046] Step 1: Input the clothes and models for virtual fitting images into the data preprocessing module to obtain the posture skeleton image and model mask image model input required by the two generation models;

[0047] Step 2: Input the corresponding data into the wrinkled clothing generation model and the deformable clothing generation model respectively to generate images of a model wearing white clothing, clothing with only wrinkles, and a model wearing a try-on clothing;

[0048] Step 3: Consider the white wrinkled clothing image as the base image and the other image as the image to be overlapped. Obtain the mask images of the clothing parts of the two images through semantic segmentation. Perform a bitwise AND operation on the clothing parts of the two masks to obtain the overlapping part of the masks. Superimpose the values ​​of the corresponding pixels in the two images based on the overlapping masks.

[0049] Step 4: Enlarge the edge of the mask (non-overlapping portion) of the image to be overlaid at a slight ratio, use the pixels of the base image in this portion as the pixels of the corresponding portion of the new overlaid image, and use the pixels of the remaining portion of the overlaid image as the pixels of the corresponding portion of the new overlaid image;

[0050] Step 5: Use the stable-diffusion inpainting model to deal with defects on the edges of the clothes, and use ControlNet to control the generation of the image;

[0051] Step 6: Use the semantic mask obtained by SAM (Segment Anything Model) to fix the character, switch the background, adjust the character's proportion and position, and generate the final virtual model image.

[0052] Input the model image to be virtualized for try-on, obtain the model's pose key points and pose sketch through Openpose, and use SAM or other semantic segmentation models to obtain the clothing mask of the model image. This is processed to obtain the clothing-agnostic cloth mask (pixels such as arms and face remain unchanged, and the clothing areas are replaced with gray).

[0053] Input the cloth image to be generated for virtual try-on, remove the background, trademark and other redundant information to obtain the cloth image, and remove the color to obtain a pure white cloth image of the corresponding style. Finally, organize the preprocessed data into a format similar to the Vitonhd dataset, such as Figure 2(a)-Figure 2(e) As shown;

[0054] The pure white image of the clothing, the cloth agnostic image, and the model's pose key points image are input into a virtual fitting model based on the stable-diffusion inpainting model. The original clothing image is converted into the CLIP embedding space using the textual inversion module based on ViT and CLIP, and then used as the conditional input for SD. The remaining data is scaled to a low-dimensional space using VAE or directly diffused, resulting in a white wrinkled fitting image, as shown in Figure 2(f).

[0055] The clothing image, the model's cloth agnostic image, and the pose sketch are fed into a stream model based on deformable attention. Features are extracted layer by layer, and then information is combined using a wrapblock based on deformable attention. The encoder-decoder model then outputs the result, resulting in a virtual try-on image to be superimposed, as shown in Figure 2(g).

[0056] Use the SAM model to perform semantic segmentation on the two pictures and obtain the semantic mask of the clothes.

[0057] The pixel values ​​of the overlapping parts of the semantic masks of the two images are superimposed. The superposition formula is:

[0058] L new =r*L2+(1-r)(L1*L2),

[0059] Subtract the overlapping part from the mask of the image to be overlaid (the virtual try-on image generated in the fourth step) and slightly expand it outward. The pixels of this part use the pixels of the base image (the white clothes image generated in the third step), and the remaining part uses the pixels of the overlaid image (the virtual try-on image generated in the fourth step), as shown in the following example: Figure 2(h)-Figure 2(l) As shown;

[0060] Put the superimposed result image into the inpainting model of stable diffusion, mask the superimposed edge part, optimize and replace the background, adjust the position and size, and obtain the final fitting image, as shown in Figure 2(m);

[0061] The effect comparison is to compare the generation quality and success rate of the proposed method with those of the two base models. The comparison results are shown in the following table:

[0062] Table 1 Comparison of virtual try-on performance

[0063]

[0064] The success rate is the number of images whose quality meets the requirements of real model images / the total number of generated images, and the generated quality is the result of comparison with the real model image.

[0065] Correspondingly, a virtual try-on image generation system for generating clothing wrinkles and clothing deformation includes: a data preprocessing module, a wrinkled clothing generation module, a deformed clothing generation module, a semantic segmentation module, an image overlay module and a detail optimization module; the clothing and model for generating the virtual try-on image are input into the data preprocessing module to obtain the posture skeleton image and the model mask image model input required by the two generation models; the corresponding data are input into the wrinkled clothing generation module and the deformed clothing generation module respectively to generate a model image of a model wearing white clothing with only wrinkles on the clothing and a model image of a model wearing try-on clothing; the white wrinkled clothing image is regarded as a base image and the other image is regarded as an image to be overlaid, and the mask images of the clothing parts of the two images are obtained through the semantic segmentation module; the clothing parts of the two mask images are overlaid through the image overlay module to obtain a try-on image; the detail optimization module is used to process the defects on the edge of the clothing, and Controlnet is used to control the generation of the image, and the semantic mask obtained by SAM is used to fix the character, switch the background, adjust the character's proportion and position, and generate the final virtual model image.

Claims

1. A method for generating a virtual fitting image by generating clothing wrinkles and clothing deformation, characterized in that: The steps include: Step 1: Input the clothes and models for which virtual fitting images need to be generated into a data preprocessing module to obtain the posture skeleton image and model mask image model input required by the two generation models; Step 2, input the corresponding data into the wrinkled clothes generation model and the deformable clothes generation model respectively, and generate images of models wearing white clothes, clothes with only wrinkles, and models wearing tried-on clothes; The wrinkled clothing generation model is based on the inpainting diffusion model of stable diffusion, which includes the following steps: Step 21, convert the white clothes corresponding to the input clothes shape into the conditional input form required by stable diffusion through the textual inversion module, and control the noise generation through the conditional input; Step 22: randomly generate a Gaussian noise image, aggregate it with the mask image of the reduced model, the pose information image, the white clothes deformation image generated by the simple model encoded by the VAE encoder, and the model's clothes removal part image, input the trained stable diffusion unet, calculate the noise to be removed, and subtract the noise from the noise image; Step 23, the denoising process is repeated continuously, and the target wrinkled clothing image is generated through the VAE decoder; The deformable clothing generation model is based on an end-to-end model of deformable attention, which includes the following steps: Step 24, input the model's clothing agnostic image and posture skeleton image into one FPN to extract features of different scales, and input the clothing processed image into another FPN to extract features of different scales; Step 25: Use a layered cascade method to fuse the features extracted by the two FPNs starting from the features with the smallest scale. The layered cascade method consists of multiple layers of deformable attention wrap modules. These modules estimate the deformation of the clothes from coarse to fine through self-deformable attention and cross-deformable attention. Each layer of deformable attention wrap module fuses the information of the corresponding scales of the two FPNs and the information from the previous level deformable attention wrap module to obtain the final clothing deformation information. Step 26: The final estimated clothing deformation information is integrated into an encoder-decoder model. The clothing and model agnostic images are input into the encoder, the clothing deformation information is integrated, and then the virtual fitting image is output through the decoder. Step 3: Consider the white wrinkled clothing image as the base image and the other image as the image to be overlapped. Obtain the mask images of the clothing parts of the two images through semantic segmentation. Superimpose the values ​​of the pixels at the corresponding positions of the two images according to the overlapping mask, and perform the following operations on the RGB brightness values ​​of the corresponding pixels of the two images: L new =r*L2+(1-r)(L1*L2) Where L1 and L2 are the brightness of the wrinkle and deformation images respectively, a is a hyperparameter, L new is the brightness of the new superimposed image; Step 4: Perform a bitwise AND operation on the clothing parts of the two mask images to obtain the overlapping part of the mask; superimpose the values ​​of the pixels at corresponding positions of the two images according to the overlapping mask; The edge of the mask of the image to be overlaid is enlarged at a small ratio, and the pixels of the base image of this part are used as the pixels of the corresponding part of the new overlaid image, and the pixels of the remaining part of the overlaid image are used as the pixels of the corresponding part of the new overlaid image; Step 5. Use the inpainting model of stable-diffusion to deal with the defects on the edge of the clothes, and use Controlnet to control the generation of the picture; extract the edge of the superimposed effect picture through the Canny edge detection method, and input the edge into Controlnet to control the generation process of stable diffusion; input the superimposed effect picture into the stable-diffusion model, use the mask of the edge part as the mask of the inpainting model, remove the defects on the edge of the superimposed effect picture, and beautify the model; Step 6: Use the semantic mask obtained by SAM to fix the character, switch the background, adjust the character's proportion and position, and generate the final virtual model image.

2. The method for generating a virtual fitting image by generating clothing wrinkles and clothing deformation according to claim 1, characterized in that: In step 4, the mask image of the image to be overlapped and the mask image of the base image are subjected to a bitwise AND operation to obtain an overlapping mask image of the two images; The edge of the image to be overlaid is enlarged at a small scale to obtain an edge mask image. The final overlaid image consists of three parts. The part covered by the overlapping mask image superimposes the values ​​of the pixels at corresponding positions of the base image and the image to be overlaid. The pixels covered by the edge mask image are the pixels at corresponding positions of the base image, and the remaining parts are the pixels at corresponding positions of the image to be overlaid.

3. The method for generating a virtual fitting image by generating clothing wrinkles and clothing deformation according to claim 1, characterized in that: In step 6, the semantic mask obtained by SAM is used to crop the person from the background of the overlay image, the background is switched, the scale or position of the person is adjusted, and the final avatar image is generated.

4. A system for generating virtual fitting images by generating clothing wrinkles and clothing deformation, characterized in that: include: Data preprocessing module, wrinkled clothing generation module, deformable clothing generation module, semantic segmentation module, image overlay module and detail optimization module; The clothes and models to be generated for virtual fitting images are input into the data preprocessing module to obtain the posture skeleton image and model mask image model input required by the two generation models; the corresponding data are respectively input into the wrinkled clothes generation module and the deformable clothes generation module to generate images of models wearing white clothes, clothes with only wrinkles, and models wearing fitting clothes; In the wrinkled clothes generation module, the wrinkled clothes generation model is based on the inpainting diffusion model of stable diffusion. The white clothes corresponding to the input clothes shape are converted into the conditional input form required by stable diffusion through the textual inversion module, and the noise generation is controlled by the conditional input. A Gaussian noise map is randomly generated, and it is aggregated with the mask map of the reduced model, the pose information map, the white clothes deformation map generated by the simple model encoded by the vae encoder, and the model's clothes removal part map. The trained stable diffusion unet is input to calculate the noise to be removed, and the noise map is subtracted from the noise map. The denoising process is repeated continuously, and the target wrinkled clothing image is generated through the VAE decoder; In the deformable clothing generation module, the deformable clothing generation model is based on an end-to-end model of deformable attention. The clothing agnostic map and posture skeleton map of the model are input into an FPN to extract features of different scales, and the clothing processing map is input into another FPN to extract features of different scales. A layered cascade method is used to fuse the features extracted by the two FPNs starting from the features with the smallest scale. The layered cascade method consists of multiple layers of deformable attention wrap modules. These modules estimate the deformation of the clothes from coarse to fine through self-deformable attention and cross-deformable attention. Each layer of deformable attention wrap module fuses the information of the corresponding scales of the two FPNs and the information from the deformable attention wrap module of the previous level to obtain the final clothing deformation information. The final estimated clothing deformation information is integrated into an encoder-decoder model. The clothing and model agnostic images are input into the encoder, the clothing deformation information is integrated, and then the virtual fitting image is output through the decoder. The white wrinkled clothes picture is regarded as the base picture, and the other picture is regarded as the picture to be overlapped. The mask pictures of the clothes part of the two pictures are obtained through the semantic segmentation module. The values ​​of the pixels at the corresponding positions of the two pictures are superimposed according to the overlapping mask, and the RGB brightness values ​​of the corresponding pixels of the two pictures are respectively subjected to the following operations: L new =r*L2+(1-r)(L1*L2) Where L1 and L2 are the brightness of the wrinkle and deformation images respectively, a is a hyperparameter, L new is the brightness of the new superimposed image; the clothing parts of the two mask images are superimposed through the image superposition module to obtain the fitting image; the detail optimization module is used to deal with the defects on the edge of the clothes, and Controlnet is used to control the generation of the image; the edge of the superimposed effect image is extracted through the Canny edge detection method, and the edge is input into Controlnet to control the generation process of stable diffusion; the superimposed effect image is input into the stable-diffusion model, and the mask of the edge part is used as the mask of the inpainting model to remove the defects on the edge of the superimposed effect image and beautify the model; the semantic mask obtained by SAM is used to fix the character, switch the background, adjust the character's proportion and position, and generate the final virtual model image.