Virtual clothes changing system based on AI algorithm

Abstract

The invention relates to the technical field of image processing, and particularly discloses a virtual clothes changing system based on an AI algorithm, and the system comprises a preprocessing module which carries out the filling, size adjustment and normalization processing of a first image; obtaining a user portrait and obtaining a target image; the visual large model processing module is used for mapping the second clothing image, the noise image and the target image to a high-dimensional space; performing feature extraction on the second clothing image in the high-dimensional space to obtain a first feature; splicing the noise image and the target image in the high-dimensional space to obtain a standard image, and performing feature extraction on the standard image to obtain a second feature; fusing the first feature and the second feature, and obtaining a virtual fitting image; and the post-processing module is used for obtaining a fitting portrait and obtaining a virtual fitting effect image according to the fitting portrait and the user background. According to the method, various types of clothes can be stably tried on, and meanwhile, natural textures, wrinkles and actual lengths of the clothes in a real dressing state are presented.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a virtual clothing-changing system based on an AI algorithm. Background Art

[0002] With the booming development of e-commerce and the digital economy, the demand for innovation in user experience is becoming increasingly urgent. Take the clothing retail industry as an example. In traditional online shopping, users often face the limitation of not being able to try on clothes directly, which weakens the shopping experience to a certain extent. Driven by the new technology wave, the emergence of the innovative form of virtual fitting has completely changed this situation. Through virtual fitting technology, users can preview the effect of clothing on the body in real time in a digital environment, and experience the clothing in person as if they were in a physical store, which greatly enhances the interactivity and immersion of shopping. This change is not only popular among consumers, but also provides clothing merchants with a new way to display goods and promote sales, jointly promoting the digital transformation and upgrading of the industry.

[0003] 3D virtual try-on based on 3D template involves converting 2D images into 3D textures of clothing mesh models. The key to this process is to establish an accurate correspondence between catalog images and textures. Methods to solve this problem, such as manual calculation, generative adversarial networks and diffusion models, etc., the manual calculation method is limited by the need for a specific grid, which makes it difficult for this method to achieve the diversity required for various types of clothing try-on; the generative adversarial network method is difficult to generate clear images, due to its own characteristics, the clothing is seriously lacking in consistency, and there will also be a problem of mode collapse, making the output image difficult to use in actual production; the diffusion model-based method relies on mask generation, and it is difficult to complete accurate non-rigid conversion when the original clothing and the try-on clothing are very different, so as to meet various body shapes, actions, and scenes, and it is easy to cause deformation of clothing patterns and textures as well as actual try-on lengths. Therefore, how to stably try on various types of clothing while presenting the natural texture, wrinkles and actual length of clothing in real dressing state has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a virtual clothing-changing system based on AI algorithm to solve the following technical problems:

[0005] Stable fit of multiple types of clothing while showing the natural texture, folds and actual length of the clothing in real-life wearing state.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] Virtual dressing system based on AI algorithm, including:

[0008] Preprocessing module: obtaining a first image, the first image including a clothing image and a user image, padding the first image so that the ratio of the first image is 3:4, adjusting the padded first image to the same size, and normalizing the scaled first image to obtain a second image, and mapping the pixel value of the scaled first image to a preset pixel value range through the normalization process;

[0009] Performing portrait segmentation on the second image corresponding to the user image to obtain a user portrait and a user background, acquiring a dense pose estimation image, a pose estimation image, and a full mask estimation image retaining only the user's face based on the user portrait, and using them as target images;

[0010] Visual large model processing module: generating a noise image based on random Gaussian noise, mapping the second clothing image, the noise image and the target image to a preset high-dimensional space through a variational autoencoder, wherein the second clothing image is a second image corresponding to the clothing image;

[0011] Inputting the second clothing image in the high-dimensional space into the clothing prompt unet module, extracting the features of the second clothing image in the high-dimensional space in the clothing prompt unet module to obtain a first feature, and sending the first feature to the virtual clothing change unet module;

[0012] The noise image and the target image in the high-dimensional space are spliced ​​on the pixel channel to obtain a standard image, and the standard image is sent to the virtual clothing-changing unet module to extract features of the standard image to obtain a second feature;

[0013] In the virtual dressing unet module, the first feature and the second feature are fused based on the cross attention mechanism to obtain a high-dimensional feature space representation of the virtual fitting, and a virtual fitting image is obtained based on the decoder;

[0014] Post-processing module: performing portrait segmentation on the virtual fitting image to obtain a fitting portrait and a fitting background, and embedding the fitting portrait into the user background to obtain a final virtual fitting effect image.

[0015] As a further solution of the present invention: in the preprocessing module, the ratio of the width to the height of the first image is 3:4.

[0016] As a further solution of the present invention: in the preprocessing module, based on Seam Carvin g The algorithm resizes the padded first image to the same size.

[0017] As a further solution of the present invention: in the preprocessing module, the dense pose estimation image, the pose estimation image and the full mask estimation image retaining only the user's face are obtained based on the dense pose estimation module, the pose estimation module and the full mask estimation module respectively; in the dense pose estimation module, the dense pose estimation image is obtained based on the dense pose estimation algorithm.

[0018] As a further solution of the present invention: portrait segmentation is performed based on the Segformer algorithm.

[0019] As a further solution of the present invention: the features of the second clothing image in the high-dimensional space are obtained based on a cascade feature extraction method.

[0020] As a further solution of the present invention: in the post-processing module, the fitting portrait is embedded into the user background based on the Poisson ImageEditing algorithm.

[0021] As a further solution of the present invention: in the visual large model processing module, a double-stream diffusion model is used as the basis of the visual large model processing module.

[0022] Beneficial effects of the present invention: In the present invention, the first image is first padded to a 3:4 ratio and adjusted to a uniform size, which is to ensure that the sizes of all input images are consistent, so that the subsequent model can be calculated on a uniform image dimension, reducing processing complexity. Images of the same size will increase calculation difficulty and uncertainty; then, the user image is divided into a portrait and a background, so as to focus on the posture and position of the portrait and reduce the interference of background noise on the clothing display effect; the user's posture details are captured by dense posture estimation images and posture estimation images, and only the user's face is retained using a full mask image. These operations help the system better simulate the wearing effect of clothes, ensure that the clothes can be adjusted according to the user's posture and face, and provide a realistic trying-on experience; the clothing prompt UNet module extracts the features of the clothing image and uses it as the first feature to ensure that the model can capture the details of the clothing, including texture, shape, color, etc., to ensure the realism of the clothing when trying on; in the figure In image processing, stitching on pixel channels usually refers to merging the pixel channels of two or more images (such as red channel, green channel, and blue channel). This process can be understood as "superimposing" different image features together to form a "standard image" containing richer information, thereby ensuring that the model can simultaneously obtain information on clothing, posture, and other aspects during subsequent processing; through the cross-attention mechanism, the characteristics of clothing are combined with the user's posture, facial information and other features. This mechanism can strengthen the connection between different features of the model, making the effect of virtual clothing changing more in line with the user's actual posture and facial contour, and enhancing the realism of trying on; which are the first features and / or the second features can be pre-set manually; the cross-attention mechanism is a mechanism used to process multimodal (multiple types of data) or multi-input data in deep learning models. It focuses on relevant information and fuses features from different inputs by establishing associations between different inputs. In virtual fitting technology or other complex applications, the cross-attention mechanism can help the model more intelligently match, combine and process data from different sources; after generating the virtual fitting image, it is segmented twice to separate the user portrait after the fitting from the background, ensuring that the person and the background can be accurately segmented, avoiding blur or misalignment of the edge part, and re-embedding the fitting portrait into the original background, perfectly integrating the user and the background, and generating the final virtual fitting effect picture. The present invention ensures better restoration of the clothing texture features of the image and the actual length of the clothing, avoiding the problem that the virtual fitting method is difficult to express the actual length of the clothing fitting. At the same time, the method of pre-processing and post-processing using portrait segmentation can help the virtual fitting model better avoid the influence of the environment and avoid the effect limitation caused by the mask. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention will be further described below in conjunction with the accompanying drawings.

[0024] Figure 1 It is a flow chart of a virtual clothing changing system based on AI algorithm of the present invention;

[0025] Figure 2 It is a structural schematic diagram of a preprocessing module of a virtual clothing changing system based on an AI algorithm of the present invention;

[0026] Figure 3 It is a structural schematic diagram of the visual large model processing module of the virtual clothing changing system based on AI algorithm of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1-3 As shown, the present invention is a virtual clothing changing system based on AI algorithm, comprising:

[0029] Preprocessing module: obtaining a first image, the first image including a clothing image and a user image, padding the first image so that the ratio of the first image is 3:4, adjusting the padded first image to the same size, and normalizing the scaled first image to obtain a second image, and mapping the pixel value of the scaled first image to a preset pixel value range through the normalization process;

[0030] Performing portrait segmentation on the second image corresponding to the user image to obtain a user portrait and a user background, acquiring a dense pose estimation image, a pose estimation image, and a full mask estimation image retaining only the user's face based on the user portrait, and using them as target images;

[0031] Visual large model processing module: generating a noise image based on random Gaussian noise, mapping the second clothing image, the noise image and the target image to a preset high-dimensional space through a variational autoencoder, wherein the second clothing image is a second image corresponding to the clothing image;

[0032] Inputting the second clothing image in the high-dimensional space into the clothing prompt unet module, extracting the features of the second clothing image in the high-dimensional space in the clothing prompt unet module to obtain a first feature, and sending the first feature to the virtual clothing change unet module;

[0033] The noise image and the target image in the high-dimensional space are spliced ​​on the pixel channel to obtain a standard image, and the standard image is sent to the virtual clothing-changing unet module to extract features of the standard image to obtain a second feature;

[0034] In the virtual dressing unet module, the first feature and the second feature are fused based on the cross attention mechanism to obtain a high-dimensional feature space representation of the virtual fitting, and a virtual fitting image is obtained based on the decoder;

[0035] Post-processing module: performing portrait segmentation on the virtual fitting image to obtain a fitting portrait and a fitting background, and embedding the fitting portrait into the user background to obtain a final virtual fitting effect image.

[0036] It should be noted that the first image is first padded to a 3:4 ratio and adjusted to a uniform size. This is to ensure that the sizes of all input images are consistent, so that the subsequent models can be calculated on a uniform image dimension and reduce processing complexity. Images of the same size will increase computational difficulty and uncertainty. After that, the user image is divided into a portrait and a background, so as to focus on the posture and position of the portrait and reduce the interference of background noise on the clothing display effect. The user's posture details are captured through dense posture estimation images and posture estimation images, and only the user's face is retained using a full mask image. These operations help the system better simulate the wearing effect of clothes, ensure that the clothes can be adjusted according to the user's posture and face, and provide a realistic trying-on experience. The clothing prompt UNet module extracts the features of the clothing image and uses it as the first feature to ensure that the model can capture the details of the clothing, including texture, shape, color, etc., to ensure the realism of the clothing when trying on. In image processing , stitching on pixel channels usually refers to merging the pixel channels of two or more images (such as red channel, green channel, blue channel). This process can be understood as "superimposing" different image features together to form a "standard image" containing richer information, thereby ensuring that the model can simultaneously obtain information such as clothing and posture during subsequent processing; through the cross-attention mechanism, the characteristics of clothing are combined with the user's posture, facial information and other features. This mechanism can strengthen the connection between different features of the model, making the effect of virtual clothing changing more consistent with the user's actual posture and facial contour, and enhancing the realism of trying on; which are the first features and / or the second features can be pre-set manually; the cross-attention mechanism is a mechanism used to process multimodal (multiple types of data) or multi-input data in deep learning models. It focuses on relevant information by establishing associations between different inputs and fuses features from different inputs. In virtual fitting technology or other complex applications, the cross-attention mechanism can help the model more intelligently match, combine and process data from different sources; after generating the virtual fitting image, it is segmented twice to separate the user's portrait after trying on from the background, ensuring that the person and the background can be accurately segmented to avoid blurring or misalignment of the edges, and the fitting portrait is re-embedded into the original background, perfectly blending the user and the background to generate the final virtual fitting effect image.

[0037] In another preferred embodiment of the present invention, in the preprocessing module, the ratio of the width to the height of the first image is 3:4.

[0038] In another preferred embodiment of the present invention, in the preprocessing module, the filled first images are adjusted to the same size based on the SeamCarving algorithm.

[0039] It can be understood that the Seam Carving algorithm is a content-aware scaling method that adjusts the size of the image without destroying the important parts of the image by identifying unimportant areas (or "seamless paths") in the image. The Seam Carving algorithm dynamically determines which pixels can be removed or expanded based on the image content. Compared with traditional global scaling, it is more intelligent and suitable for detail preservation in clothing images. In the fitting scene, different clothing materials, patterns, designs, etc. are very important for the fitting effect. The Seam Carving algorithm can make more flexible adjustments based on the differences in these contents.

[0040] In another preferred embodiment of the present invention, in the preprocessing module, the dense pose estimation image, the pose estimation image and the full mask estimation image retaining only the user's face are acquired based on the dense pose estimation module, the pose estimation module and the full mask estimation module, respectively. In the dense pose estimation module, the dense pose estimation image is acquired based on the dense pose estimation algorithm.

[0041] It should be noted that in the pose estimation module, the pose estimation image is obtained based on the pose estimation algorithm; in the full mask estimation module, the full mask estimation image that only retains the user's face is obtained based on the full mask estimation algorithm; dense pose estimation algorithms, such as DensePos e The algorithm constructs the three-dimensional posture information of the whole body by mapping each pixel in the two-dimensional image to a specific position of the three-dimensional human body surface model. The algorithm will first identify the various parts of the human body, and then accurately project the pixels of these parts onto a known 3D human body model; the steps in the posture estimation module and the full mask estimation module are analogous to those in the dense posture estimation module and will not be repeated here.

[0042] In another preferred embodiment of the present invention, in the preprocessing module, portrait segmentation is performed based on the Segformer algorithm.

[0043] It should be noted that Segformer is a multi-head attention mechanism architecture, which is specially used to segment the foreground of the image and remove the background. It can achieve high-precision and fast portrait segmentation, especially in portrait images (such as faces and half-length photos). Segformer can process portrait images well and accurately segment the outline of the person, especially with high accuracy at the edge of the portrait, which is very important for virtual fitting, because during virtual fitting, the clothing needs to fit the person's body accurately, and the error of the segmentation boundary will affect the final fitting effect.

[0044] In another preferred embodiment of the present invention, in the large visual model processing module, the features of the second clothing image in the high-dimensional space are acquired based on a cascade feature extraction method.

[0045] It can be understood that cascade feature extraction is a progressive feature extraction method. It processes the input image in a step-by-step manner through multiple levels of feature extraction modules. Each layer performs more complex analysis based on the features extracted from the previous layer, thereby obtaining richer and higher-level feature information. The detail features (such as texture, pattern, folds) and overall features (such as contour, shape) of clothing are hierarchical. Extracting all these features directly from the image may lose details or cause confusion. Through the cascade method, low-level features are extracted first, and then high-level features are gradually extracted to ensure that every level of clothing can be fully captured, such as Cascad e The R-CNN algorithm cascades multiple detectors, and each detector further optimizes and screens according to the output of the previous detector, thereby gradually improving the detection accuracy. This method not only improves the accuracy of target detection, but also effectively avoids the error accumulation problem that may be caused by a single detector.

[0046] In another preferred embodiment of the present invention, in the post-processing module, the fitting portrait is embedded into the user background based on the Poisson Image Editing algorithm.

[0047] It is worth noting that Poisson Image Editing is an image processing algorithm mainly used for seamless image synthesis, also known as image fusion. It cleverly blends the selected area of ​​one image into another image seamlessly by solving the partial differential equation based on the Poisson equation, ensuring smooth and natural edge transitions, ensuring the natural fusion of the portrait and the background, and eliminating the unnatural feeling caused by background differences or lighting changes.

[0048] In another preferred embodiment of the present invention, in the visual large model processing module, a double-stream diffusion model is used as the basis of the visual large model processing module.

[0049] It is worth noting that the diffusion model is a generative model that gradually restores the target data (such as images) from random noise through a "reverse process". This process simulates the phenomenon of diffusion and de-diffusion in nature, that is, gradually adding noise from an ordered object (such as an image) until it becomes a completely random state. Then, through the reverse process, the model gradually removes the noise and finally reconstructs the original data. The two-stream diffusion model is an extension of the traditional diffusion model. It uses two independent streams (two streams) to process different information flows. The two-stream diffusion model processes the global information and local information in the image through the structure flow and the detail flow respectively, avoiding the unnecessary complexity brought by the mixing of all information, which makes the generated image more reasonable in overall structure while retaining rich local details.

[0050] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A virtual dressing system based on AI algorithm, characterized by: include: Preprocessing module: obtaining a first image, the first image including a clothing image and a user image, padding the first image so that the ratio of the first image is 3:4, adjusting the padded first image to the same size, and normalizing the scaled first image to obtain a second image, and mapping the pixel value of the scaled first image to a preset pixel value range through the normalization process; Performing portrait segmentation on the second image corresponding to the user image to obtain a user portrait and a user background, acquiring a dense pose estimation image, a pose estimation image, and a full mask estimation image retaining only the user's face based on the user portrait, and using them as target images; Visual large model processing module: generating a noise image based on random Gaussian noise, mapping the second clothing image, the noise image and the target image to a preset high-dimensional space through a variational autoencoder, wherein the second clothing image is a second image corresponding to the clothing image; Inputting the second clothing image in the high-dimensional space into the clothing prompt unet module, extracting the features of the second clothing image in the high-dimensional space in the clothing prompt unet module to obtain a first feature, and sending the first feature to the virtual clothing change unet module; The noise image and the target image in the high-dimensional space are spliced ​​on the pixel channel to obtain a standard image, and the standard image is sent to the virtual clothing-changing unet module to extract features of the standard image to obtain a second feature; In the virtual dressing unet module, the first feature and the second feature are fused based on the cross attention mechanism to obtain a high-dimensional feature space representation of the virtual fitting, and a virtual fitting image is obtained based on the decoder; Post-processing module: performing portrait segmentation on the virtual fitting image to obtain a fitting portrait and a fitting background, and embedding the fitting portrait into the user background to obtain a final virtual fitting effect image.

2. The virtual clothing changing system based on AI algorithm according to claim 1 is characterized in that: In the preprocessing module, the ratio of the width to the height of the first image is 3:

4.

3. The virtual clothing-changing system based on AI algorithm according to claim 1, characterized in that: In the preprocessing module, the filled first image is adjusted to the same size based on the Seam Carving algorithm.

4. The virtual clothing-changing system based on AI algorithm according to claim 1, characterized in that: In the preprocessing module, the dense pose estimation image, the pose estimation image and the full mask estimation image retaining only the user's face are respectively acquired based on the dense pose estimation module, the pose estimation module and the full mask estimation module. In the dense pose estimation module, the dense pose estimation image is acquired based on the dense pose estimation algorithm.

5. The virtual clothing-changing system based on AI algorithm according to claim 1, characterized in that: In the preprocessing module, portrait segmentation is performed based on the Segformer algorithm.

6. The virtual clothing-changing system based on AI algorithm according to claim 1, characterized in that: In the large visual model processing module, the features of the second clothing image in the high-dimensional space are acquired based on a cascade feature extraction method.

7. The AI ​​algorithm-based virtual clothing-changing system according to claim 1, characterized in that: In the post-processing module, the fitting portrait is embedded into the user background based on the Poisson Image Editing algorithm.

8. The AI ​​algorithm-based virtual clothing-changing system according to claim 1, characterized in that: In the visual large model processing module, a dual-stream diffusion model is used as the basis of the visual large model processing module.

Images