A method and system for generating virtual fitting image display based on AI technology

Abstract

The present invention relates to the field of image display technology, and in particular to a method and system for generating virtual fitting image display based on AI technology. First, a target reference image and a target clothing image are obtained; then a mask image required for trying on the target clothing image is calculated based on the target reference image; finally, a fitting image is generated based on the mask image, the target reference image and the target clothing image to obtain a fitting result image, which is fused based on features in multiple images to solve the problems existing in the current method, such as insufficient clothing restoration, unnatural wearing and unrealistic generation results.

Description

Technical Field

[0001] The present invention relates to the field of image display technology, and in particular to a method and system for generating virtual fitting image display based on AI technology. Background Art

[0002] Image-based virtual fitting technology refers to the technology that can generate pictures of any person wearing the clothing based on clothing images. In the field of e-commerce, pictures of clothing on the body can show the texture of clothing and intuitively show the wearing experience to potential consumers, which is valued by a large number of consumers. Merchants often need to spend a lot of money to rent venues, hire shooting teams, models and post-production personnel to shoot and produce exquisite pictures of clothing on the body. Therefore, low-cost image generation technology that can truly restore the texture of clothing and can perform virtual fitting on any scene and any model has attracted widespread attention. At present, image-based virtual fitting methods generally decompose the task into clothing bending mapping module and clothing fitting module. The clothing bending mapping module estimates the posture and surface of the human body and deforms the clothing accordingly in order to fit the human body. The clothing fitting module focuses on better integrating the deformed clothing with the person. This type of method is generally limited by the accuracy of the clothing bending mapping module and cannot fit the human body well. At the same time, the clothing fitting module also has problems such as unnatural and unrealistic upper body. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for generating virtual fitting image display based on AI technology, so as to solve the problems existing in the current method such as insufficient clothing restoration, unnatural wearing and unrealistic generation results.

[0004] To achieve the above object, the present invention provides a method for generating a virtual fitting image display based on AI technology, the method comprising:

[0005] Obtain a target reference image and a target clothing image; the target clothing image is a picture of clothing that the user is interested in trying on; the target reference image is a photo of the person trying on the clothing;

[0006] Calculating a mask image required for trying on the target clothing image based on the target reference image;

[0007] Generating a fitting image based on the mask image, the target reference image, and the target clothing image to obtain a fitting result image specifically includes:

[0008] Preprocessing the target reference image, the mask image and the target clothing image respectively;

[0009] Extracting key point features of the human body based on the preprocessed target reference image to obtain human body posture features;

[0010] The preprocessed target reference image and the mask image are fused to obtain a mask mark reference image;

[0011] Inputting the preprocessed target clothing image into a multimodal encoder and a hierarchical encoder respectively to obtain clothing global features and clothing local hierarchical features respectively;

[0012] The human body posture features, the mask mark reference image, the clothing global features and the clothing local hierarchical features are all input into the image diffusion model to generate a fitting image, and obtain a fitting result image.

[0013] Optionally, the calculating a mask image required for trying on the target clothing image based on the target reference image specifically includes:

[0014] Inputting the target reference image into a semantic segmentation module to segment it according to the semantic labels to obtain the main segmentation areas;

[0015] Inputting the clothing category selected by the user and the main segmented area into the mask calculation module to calculate the mask area and obtain an initial mask area;

[0016] The initial mask area is smeared, adjusted and marked, and a mask image is output.

[0017] Optionally, extracting key point features of a human body based on the preprocessed target reference image to obtain human body posture features specifically includes:

[0018] Extract key points of the human body and obtain key point images of the human body;

[0019] The human body key point image is input into a human body posture feature encoder to extract posture features and obtain human body posture features.

[0020] Optionally, the multimodal encoder adopts an image encoder of a CLIP model, including a pixel block embedding layer, a self-attention encoding layer, and an output mapping layer; the hierarchical encoder adopts a hierarchical encoder of a UNet structure, including a latent space representation module, an input downsampling module, an intermediate module, and an output upsampling module.

[0021] Optionally, the step of inputting the human body posture feature, the mask mark reference image, the clothing global feature and the clothing local hierarchical feature into an image diffusion model to generate a fitting image and obtain a fitting result image specifically includes:

[0022] Step S351: inputting the mask mark reference image into a latent space feature encoder for latent space feature encoding to obtain mask mark features;

[0023] Step S352: inputting the random Gaussian noise image, the mask image and the mask mark feature into the model input part to perform feature channel dimension splicing to obtain splicing features;

[0024] Step S353: Let the concatenated feature be the input noise latent space feature xt, and t=T in the initial iteration;

[0025] Step S354: after inputting the input noise latent space feature xt into the input feature mapping layer, introducing the human body posture feature to enhance the channel dimension and obtain the superposition feature;

[0026] Step S355: inputting the superimposed features and the clothing global features into a convolution downsampling self-attention module for feature extraction to obtain downsampling features;

[0027] Step S356: inputting the down-sampled features and the clothing global features into the intermediate self-attention module for stacking operation to obtain stacked features;

[0028] Step S357: inputting the stacked features, the clothing global features and the clothing local hierarchical features into the convolution upsampling self-attention module for feature extraction to obtain the input noise latent space feature xt-1;

[0029] Step S358: Determine whether t-1 is equal to 0; if t-1 is greater than 0, let the input noise latent space feature xt-1 be the input noise latent space feature xt, and return to "Step S354"; if t-1 is equal to 0, then based on the input noise latent space feature x0, input it into the latent space image decoder for latent space image decoding to obtain the fitting result image.

[0030] The present invention also provides a virtual fitting image display system based on AI technology, the system comprising:

[0031] An acquisition unit is used to acquire a target reference image and a target clothing image; the target clothing image is a picture of clothing to be tried on that the user is interested in; the target reference image is a photo of the person trying on the clothing;

[0032] A mask image extraction unit, used for calculating a mask image required for trying on the target clothing image based on the target reference image;

[0033] A fitting result image generating unit, configured to generate a fitting image based on the mask image, the target reference image and the target clothing image, and obtain a fitting result image;

[0034] The fitting result graph generating unit comprises:

[0035] A preprocessing module, used for preprocessing the target reference image, the mask image and the target clothing image respectively;

[0036] A human body key point feature extraction module is used to extract human body key point features based on the preprocessed target reference image to obtain human body posture features;

[0037] A fusion module, used for fusing the preprocessed target reference image and the mask image to obtain a mask mark reference image;

[0038] A clothing feature extraction module is used to input the preprocessed target clothing image into a multimodal encoder and a hierarchical encoder to obtain clothing global features and clothing local hierarchical features respectively;

[0039] The fitting image generation module is used to input the human body posture features, the mask mark reference image, the clothing global features and the clothing local hierarchical features into the image diffusion model to generate a fitting image and obtain a fitting result image.

[0040] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for generating a virtual fitting image display based on AI technology are implemented.

[0041] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method for generating a virtual fitting image display based on AI technology are implemented.

[0042] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0043] The present invention discloses a method and system for generating a virtual fitting image display based on AI technology. First, a target reference image and a target clothing image are obtained; then a mask image required for trying on the target clothing image is calculated based on the target reference image; finally, a fitting image is generated based on the mask image, the target reference image and the target clothing image to obtain a fitting result image, which is fused based on features in multiple images to solve the problems existing in the current method, such as insufficient clothing restoration, unnatural wearing and unrealistic generation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 This is a flow chart of a method for generating a virtual fitting image display based on AI technology according to an embodiment of the present invention;

[0046] Figure 2 A flowchart of calculating a mask image required for trying on a target clothing image according to an embodiment of the present invention;

[0047] Figure 3 This is a general flow chart of generating a fitting result image from a fitting image according to an embodiment of the present invention;

[0048] Figure 4 This is a flow chart of generating a fitting result image from a fitting image using an image diffusion model in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The purpose of the present invention is to provide a method and system for generating virtual fitting image display based on AI technology, so as to solve the problems existing in the current method such as insufficient clothing restoration, unnatural wearing and unrealistic generation results.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, the present invention discloses a method for generating a virtual fitting image display based on AI technology, the method comprising:

[0053] Step S1: Obtain a target reference image and a target clothing image.

[0054] Step S2: Calculating a mask image required for trying on the target clothing image based on the target reference image.

[0055] Step S3: Generate a fitting image based on the mask image, the target reference image and the target clothing image to obtain a fitting result image.

[0056] The following is a detailed discussion of each step:

[0057] Step S1: Obtain a target reference image and a target clothing image; the target clothing image is a picture of clothing that the user is interested in trying on; the target reference image is a photo of a person trying on clothes, who may be a fitting model and / or a purchaser selected by the user, and the person trying on clothes includes information such as gender, skin color, appearance, posture, and scene.

[0058] Step S2: Calculate the mask image required for trying on the target clothing image based on the target reference image; the mask image is usually represented by a binary image, such as using a value of 0 to represent the retained area and a value of 1 to represent the area to be regenerated.

[0059] like Figure 2 As shown, the mask image required for trying on the target clothing image is calculated based on the target reference image, specifically including:

[0060] Step S21: Input the target reference image into the semantic segmentation module and segment it according to the semantic labels to obtain the main segmentation areas; the main segmentation areas include the areas corresponding to the background, hair, face, tops, arms, palms, bottoms, legs and feet; in this embodiment, the semantic segmentation module is a neural network model trained based on a deep learning method, trained using a large amount of manually annotated data, and has the ability to segment human body parts and foreground and background, and can subsequently be segmented according to the semantic labels.

[0061] Step S22: Input the clothing category and main segmentation area selected by the user into the mask calculation module to calculate the mask area and obtain the initial mask area, also known as the initial mask map; for example, if the user selects the clothing category as tops, then the tops, arms and palms are selected from the main segmentation area to form the initial mask area. If the user selects the clothing category as bottoms, then the legs and feet of the bottoms are selected from the main segmentation area to form the initial mask area.

[0062] Step S23: Smear, adjust and mark the initial mask area, and output the mask image.

[0063] like Figure 3 As shown, a fitting image is generated based on the mask image, the target reference image and the target clothing image to obtain a fitting result image, which specifically includes:

[0064] Step S31: pre-processing the target reference image, the mask image and the target clothing image respectively; the pre-processing includes image scaling and edge filling to achieve the resolution requirement of subsequent image generation.

[0065] Step S32: extracting key point features of the human body based on the preprocessed target reference image to obtain human body posture features; in this embodiment, the human body posture features are used to control the degree of restoration of the human body posture in the fitting result to the reference image.

[0066] Step S32: extracting key point features of the human body based on the preprocessed target reference image to obtain human body posture features, specifically including:

[0067] Step S321: extracting key points of the human body and obtaining a key point image of the human body; the key points of the human body include at least one of the neck, shoulders, elbows, wrists, torso center, hips, knees and ankles, and the key points of the human body are usually represented by coordinate images.

[0068] In this embodiment, a general human key point extraction algorithm (such as OpenPose) can be used to extract human key points to obtain a human key point image. It is also possible to extract human key points based on a human key point extraction module to obtain a human key point image; the human key point extraction module is a neural network model trained based on a deep learning method, trained using a large amount of manually annotated data, and has a general human key point extraction capability.

[0069] Step S322: inputting the human body key point image into the human body posture feature encoder to extract posture features and obtain human body posture features.

[0070] Furthermore, the human posture feature encoder is a convolutional neural network, which is divided into an input convolution module, an intermediate convolution module and an output convolution module. Among them, the input convolution module extracts the input human key point image as the input feature image; the intermediate convolution module is composed of multiple convolution layers and nonlinear activation layers stacked alternately, and the intermediate convolution module downsamples the input feature image to obtain an intermediate feature image; this step increases the feature dimension while downsampling the feature image to align it with the input resolution required by the subsequent image diffusion model; the output convolution module linearly maps the intermediate feature image to align its feature dimension with the human key point feature input dimension required by the subsequent image diffusion model. The input human key point image is Matrix, the output human posture feature is , Represents the image height, Represents the image width.

[0071] Step S33: Fusing the preprocessed target reference image and the mask image to obtain a mask mark reference image; in this embodiment, the mask mark reference image is used to mark the area that needs to be redrawn during the image generation stage and the area information that needs to be retained.

[0072] Step S34: Input the preprocessed target clothing image into the multimodal encoder and the hierarchical encoder respectively to obtain the global features of the clothing and the local hierarchical features of the clothing respectively; the global features of the clothing are used to control the degree of restoration of the clothing in the fitting results. The global features generally include information such as the type, color, and attributes of the clothing, which helps to better restore the characteristics of the clothing in image generation. Therefore, the present invention uses a multimodal encoder to extract the global features of clothing. The local hierarchical features of clothing are used to control the degree of detail restoration of the clothing in the fitting results. The local features contain the spatial distribution information of the clothing texture, and the use of the hierarchical encoder can cover the texture characteristics at different scales.

[0073] When extracting clothing global features, the present invention uses the image encoder of the CLIP model to extract clothing global features. The image encoder consists of a pixel block embedding layer, a self-attention encoding layer and an output mapping layer.

[0074] The pixel block embedding layer takes the preprocessed target clothing image as input, divides the image into 14x14 pixel blocks evenly, and maps each pixel to a pixel block feature. When the matrix is ​​, the pixel block embedding layer output is The matrix of .

[0075] The self-attention encoding layer uses a stacked Self-Attention module, which can provide powerful image feature expression capabilities through training with a large amount of data. Its input is The output is also The matrix of Indicates the number of image tokens, each token corresponds to the above blocks of pixels.

[0076] The output mapping layer linearly maps the features so that their feature dimensions are aligned with the image global feature input dimensions required by the subsequent image diffusion model.

[0077] The present invention uses the first image token feature of the output mapping layer as the global image feature.

[0078] When extracting local hierarchical features of clothing, the present invention uses a hierarchical encoder of a UNet structure, including a latent space representation module, an input downsampling module, an intermediate module, and an output upsampling module.

[0079] The latent space representation module maps the preprocessed target clothing image to the latent space to obtain the latent space image features; when the input is When the clothing image is Latent space image features.

[0080] The input downsampling module takes latent space image features as input and contains 4 levels. Levels 1 to 3 each contain two convolutional attention modules and one convolutional downsampling module, and level 4 contains one convolutional module. The convolutional attention module is composed of a residual convolutional module and a self-attention module stacked together. The convolutional downsampling module uses convolution operations to reduce the feature image resolution. The input of the residual convolution module is The feature image is output as The feature image of , and then before the self-attention module is calculated, The characteristic image of The characteristic matrix of Indicates the number of image tokens, the self-attention module calculates the output The feature matrix is ​​then transformed again before the convolution downsampling calculation. The feature image of the final convolution downsampling module outputs The downsampled feature image.

[0081] The intermediate module is composed of multiple convolutional attention modules stacked together, which can further enhance the feature representation output by the input downsampling module.

[0082] The output upsampling module also contains 4 layers. Layers 1-3 contain three convolutional attention modules and one convolutional upsampling module, and layer 4 contains one convolutional module. The input of the convolutional upsampling module is The output of the convolution upsampling module is Upsample the feature image.

[0083] The local hierarchical features of clothing used in the present invention are taken from the input feature matrix of all self-attention modules in the output upsampling module, that is, it contains three levels, each level contains 3 feature matrices with the same number of image tokens. When the hierarchical encoder input is a 3xHxW clothing image, the local hierarchical features of clothing extracted by the present invention contain 3 Feature matrix, 3 Feature matrix, 3 Feature matrix.

[0084] Step S35: inputting the human body posture features, the mask mark reference image, the clothing global features and the clothing local hierarchical features into the image diffusion model to generate a fitting image, and obtaining a fitting result image.

[0085] The present invention uses an image diffusion model combined with the above features to generate images, and can redraw the masked area into the upper body effect of the target clothing, while the masked area retains the original content of the reference image. The image diffusion model used in the present invention is trained by a large amount of clothing-person pairing data, and can perform natural area redrawing, while restoring the characteristics of the target clothing to the greatest extent, and generating a realistic fitting result image.

[0086] Specifically, the present invention uses a latent space image diffusion model, that is, the sampling of the generated image is performed in the latent space, which can save a considerable amount of calculation, and the quality of the final generated image is almost unaffected. The present invention restores the latent space image features to the real image through the latent space image decoder. The calculation process of the latent space image diffusion model is actually an iterative latent space image feature denoising. The present invention starts from the steps Start by estimating the noise step by step until step , get the sampled latent space image features. Calculate from a random Gaussian noise image Start in step Above, the latent space image diffusion model input is , output estimated noise , the sampler can calculate , then start step There are many ways to choose a sampler. Taking DDIM as an example, it can be calculated by the following formula

[0087]

[0088] in, and The hyperparameters chosen when training the model.

[0089] The present invention uses a latent space image diffusion model of a UNet structure. The latent space image diffusion model includes: a latent space feature encoder, a model input part, an input feature mapping layer, a convolution downsampling self-attention module, an intermediate self-attention module, a convolution upsampling self-attention module and a latent space image decoder.

[0090] like Figure 4 As shown in the figure, the human body posture features, mask mark reference image, clothing global features and clothing local hierarchical features are input into the image diffusion model to generate fitting images, and the fitting result image is obtained, which specifically includes:

[0091] Step S351: input the mask mark reference image into the latent space feature encoder for latent space feature encoding to obtain the mask mark feature.

[0092] Step S352: Input the random Gaussian noise image, the mask image and the mask mark feature into the model input part for feature channel dimension splicing to obtain splicing features; in this embodiment, the model input part is spliced ​​in the feature channel dimension by the random Gaussian noise image, the mask image and the mask mark feature obtained by latent space feature encoding, and the input is The present invention performs T rounds of iterations in total according to the splicing features. Figure 4 The full English name corresponding to the letter "C" in Chinese is CONCAT, and the corresponding Chinese name means dimensional splicing, that is, dimensional splicing (CONCAT, abbreviated as C).

[0093] Step S353: Let the concatenated feature be the input noise latent space feature xt, and in the initial iteration, t=T, where T is a positive integer greater than 1.

[0094] Step S354: After the input noise latent space feature xt is input to the input feature mapping layer, the human posture feature is introduced to increase the channel dimension to obtain the superposition feature. In this embodiment, the feature mapping layer introduces the human posture feature to increase the channel dimension of the input feature to 320, which is aligned with the input requirement of the convolution downsampling self-attention module. The present invention increases the channel dimension of the input feature to 320 by superposition.

[0095] Step S355: Input the superimposed features and the global clothing features into the convolutional downsampling self-attention module for feature extraction to obtain downsampling features; in this embodiment, the convolutional downsampling self-attention module includes 4 levels, levels 1 to 3 each include two convolutional attention modules and one convolutional downsampling module, level 4 includes a convolution module, and the convolutional attention module includes a cross-attention module and a self-attention module.

[0096] Step S356: Input the downsampled features and the global clothing features into the intermediate self-attention module for stacking operation to obtain stacked features. In this embodiment, the intermediate self-attention module is formed by alternately stacking a plurality of convolutional layers and attention modules.

[0097] Step S357: The stacked features, clothing global features and clothing local hierarchical features are input into the convolutional upsampling self-attention module for feature extraction to obtain the input noise latent space feature xt-1; in this embodiment, the convolutional upsampling self-attention module also includes 4 levels, levels 1 to 3 each include two convolutional attention modules and one convolutional upsampling module, level 4 includes a convolution module, and the convolutional attention module includes a cross-attention module and a self-attention module.

[0098] The core calculation logic of the attention module is as follows: the input is divided into query features , key features , value feature The above features can be generally expressed as The matrix of Represents the number of tokens, Represents the dimension of the feature. First, query the feature and key features To calculate similarity, for each query feature token, use Calculate the normalized similarity weights of the normalized query feature and all key features , the specific formula is:

[0099]

[0100] in, Represents the dimension of K features.

[0101] Then, according to the similarity weight A, the value features Perform a weighted sum to get O.

[0102]

[0103] Among them, O represents the output of the attention module.

[0104] The input of the general attention module is the query feature In particular, in the cross-attention module, the present invention uses the global clothing feature as the key feature And the value feature ; In the self-attention module, the present invention concatenates the query feature and the local level feature of clothing in the token dimension to obtain the key feature And the value feature In the above manner, the present invention introduces the global features of clothing in the cross-attention module and the local hierarchical features of clothing in the self-attention module, so that the generated results can simultaneously refer to the global information (such as clothing style, color, etc.) and local information (such as pattern, pocket, etc.) of clothing.

[0105] Step S358: Determine whether t-1 is equal to 0; if t-1 is greater than 0, let the input noise latent space feature xt-1 be the input noise latent space feature xt, and return to "Step S354"; if t-1 is equal to 0, then based on the input noise latent space feature x0, input it into the latent space image decoder for latent space image decoding to obtain the fitting result image.

[0106] The technical solution of the present invention can generate realistically restored fitting effect pictures, which are not restricted by the character's skin color, gender, posture and scene, and is suitable for generating most daily clothing wearing effects.

[0107] The present invention combines automatic means such as semantic segmentation and clothing type acquisition with interactive adjustment means with the user to obtain accurate clothing redrawing areas, thereby ensuring a good virtual fitting effect.

[0108] The technical solution of the present invention is based on the image diffusion model, which integrates the key point features of the human body, the mask area features, the global and hierarchical local features of the clothing to achieve a highly restored, realistic and natural virtual fitting effect, and solves the problems of insufficient clothing restoration, unnatural wearing and unrealistic generated results existing in the current method.

[0109] The present invention also discloses a virtual fitting image display system generated based on AI technology, the system comprising:

[0110] The acquisition unit is used to acquire a target reference image and a target clothing image; the target clothing image is a picture of clothing to be tried on that the user is interested in; and the target reference image is a photo of the person trying on the clothing.

[0111] The mask image extraction unit is used to calculate the mask image required for trying on the target clothing image based on the target reference image.

[0112] The fitting result image generating unit is used to generate a fitting image based on the mask image, the target reference image and the target clothing image to obtain a fitting result image.

[0113] As an optional implementation manner, the fitting result graph generating unit of the present invention includes:

[0114] The preprocessing module is used to preprocess the target reference image, the mask image and the target clothing image respectively.

[0115] The human body key point feature extraction module is used to extract the human body key point features based on the preprocessed target reference image to obtain human body posture features.

[0116] The fusion module is used to fuse the preprocessed target reference image and the mask image to obtain a mask mark reference image.

[0117] The clothing feature extraction module is used to input the preprocessed target clothing image into a multimodal encoder and a hierarchical encoder respectively to obtain clothing global features and clothing local hierarchical features respectively.

[0118] The fitting image generation module is used to input the human body posture features, the mask mark reference image, the clothing global features and the clothing local hierarchical features into the image diffusion model to generate a fitting image and obtain a fitting result image.

[0119] For the parts that are the same as the method for generating a virtual fitting image display based on AI technology, specific reference is made to the method for generating a virtual fitting image display based on AI technology, and no further details will be given here.

[0120] The present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of generating a virtual fitting image display method based on AI technology are implemented.

[0121] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of generating a virtual fitting image display method based on AI technology are implemented.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for generating a virtual fitting image display based on AI technology, characterized in that the method include: Obtain a target reference image and a target clothing image; the target clothing image is a picture of clothing that the user is interested in trying on; The target reference image is a photo of the person trying on clothes; Calculating a mask image required for trying on the target clothing image based on the target reference image; Generating a fitting image based on the mask image, the target reference image, and the target clothing image to obtain a fitting result image specifically includes: Preprocessing the target reference image, the mask image and the target clothing image respectively; Extracting key point features of the human body based on the preprocessed target reference image to obtain human body posture features; The preprocessed target reference image and the mask image are fused to obtain a mask mark reference image; The preprocessed target clothing image is input into a multimodal encoder and a hierarchical encoder respectively to obtain clothing global features and clothing local hierarchical features respectively; the multimodal encoder adopts an image encoder of a CLIP model, including a pixel block embedding layer, a self-attention encoding layer, and an output mapping layer; the hierarchical encoder adopts a hierarchical encoder of a UNet structure, including a latent space representation module, an input downsampling module, an intermediate module, and an output upsampling module; The human body posture feature, the mask mark reference image, the clothing global feature and the clothing local hierarchical feature are input into the image diffusion model to generate a fitting image, and obtain a fitting result image, specifically including: Step S351: inputting the mask mark reference image into a latent space feature encoder for latent space feature encoding to obtain mask mark features; Step S352: inputting the random Gaussian noise image, the mask image and the mask mark feature into the model input part to perform feature channel dimension splicing to obtain splicing features; Step S353: Let the concatenated feature be the input noise latent space feature xt, and t=T in the initial iteration; Step S354: after inputting the input noise latent space feature xt into the input feature mapping layer, introducing the human body posture feature to enhance the channel dimension and obtain the superposition feature; Step S355: inputting the superimposed features and the clothing global features into a convolution downsampling self-attention module for feature extraction to obtain downsampling features; Step S356: inputting the down-sampled features and the clothing global features into the intermediate self-attention module for stacking operation to obtain stacked features; Step S357: inputting the stacked features, the clothing global features and the clothing local hierarchical features into the convolution upsampling self-attention module for feature extraction to obtain the input noise latent space feature xt-1; Step S358: Determine whether t-1 is equal to 0; if t-1 is greater than 0, let the input noise latent space feature xt-1 be the input noise latent space feature xt, and return to "step S354"; if t-1 is equal to 0, then input the input noise latent space feature x0 into the latent space image decoder for latent space image decoding to obtain the fitting result image.

2. The method for generating a virtual fitting image display based on AI technology according to claim 1, characterized in that: The step of calculating the mask image required for trying on the target clothing image based on the target reference image specifically includes: Inputting the target reference image into a semantic segmentation module to segment it according to the semantic labels to obtain the main segmentation areas; Inputting the clothing category selected by the user and the main segmented area into the mask calculation module to calculate the mask area and obtain an initial mask area; The initial mask area is smeared, adjusted and marked, and a mask image is output.

3. The method for generating a virtual fitting image display based on AI technology according to claim 1, characterized in that: The method of extracting key point features of a human body based on the preprocessed target reference image to obtain human body posture features specifically includes: Extract key points of the human body and obtain key point images of the human body; The human body key point image is input into a human body posture feature encoder to extract posture features and obtain human body posture features.

4. A virtual fitting image display system based on AI technology, characterized in that: The system includes: An acquisition unit, used to acquire a target reference image and a target clothing image; the target clothing image is a picture of clothing that the user is interested in and wants to try on; The target reference image is a photo of the person trying on clothes; A mask image extraction unit, used for calculating a mask image required for trying on the target clothing image based on the target reference image; A fitting result image generating unit, configured to generate a fitting image based on the mask image, the target reference image and the target clothing image, and obtain a fitting result image; The fitting result graph generating unit comprises: A preprocessing module, used for preprocessing the target reference image, the mask image and the target clothing image respectively; A human body key point feature extraction module is used to extract human body key point features based on the preprocessed target reference image to obtain human body posture features; A fusion module, used for fusing the preprocessed target reference image and the mask image to obtain a mask mark reference image; A clothing feature extraction module is used to input the preprocessed target clothing image into a multimodal encoder and a hierarchical encoder respectively, to obtain clothing global features and clothing local hierarchical features respectively; the multimodal encoder adopts an image encoder of a CLIP model, including a pixel block embedding layer, a self-attention encoding layer, and an output mapping layer; the hierarchical encoder adopts a hierarchical encoder of a UNet structure, including a latent space representation module, an input downsampling module, an intermediate module, and an output upsampling module; The fitting image generation module is used to input the human body posture features, the mask mark reference image, the clothing global features and the clothing local hierarchical features into the image diffusion model to generate a fitting image and obtain a fitting result image, specifically including: Step S351: inputting the mask mark reference image into a latent space feature encoder for latent space feature encoding to obtain mask mark features; Step S352: inputting the random Gaussian noise image, the mask image and the mask mark feature into the model input part to perform feature channel dimension splicing to obtain splicing features; Step S353: Let the concatenated feature be the input noise latent space feature xt, and t=T in the initial iteration; Step S354: after inputting the input noise latent space feature xt into the input feature mapping layer, introducing the human body posture feature to enhance the channel dimension and obtain the superposition feature; Step S355: inputting the superimposed features and the clothing global features into a convolution downsampling self-attention module for feature extraction to obtain downsampling features; Step S356: inputting the down-sampled features and the clothing global features into the intermediate self-attention module for stacking operation to obtain stacked features; Step S357: inputting the stacked features, the clothing global features and the clothing local hierarchical features into the convolution upsampling self-attention module for feature extraction to obtain the input noise latent space feature xt-1; Step S358: Determine whether t-1 is equal to 0; if t-1 is greater than 0, let the input noise latent space feature xt-1 be the input noise latent space feature xt, and return to "step S354"; if t-1 is equal to 0, then input the input noise latent space feature x0 into the latent space image decoder for latent space image decoding to obtain the fitting result image.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for generating a virtual fitting image display based on AI technology as described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for generating a virtual fitting image display based on AI technology as described in any one of claims 1 to 3 are implemented.

Images