Length-adaptive virtual fitting method for Chinese opera short clothes

Abstract

The invention relates to a virtual fitting method for the adaptive length of a Chinese opera short garment, which can dynamically adjust the length and form of the garment based on a fitting condition generator and a fitting image generator, and can improve the naturalness and reality sense of a fitting image while keeping the integrity of the pattern and texture of the garment. In addition, by adding a skin color adjusting module and a length self-adaptive adjusting module, the problem of difference between different human body skin colors and clothes styles can be solved to a greater extent. When the traditional opera clothes are tried on, optimization is carried out especially according to the complex patterns and form characteristics of the Qincavity clothes, the try-on effect is greatly improved, the fitting experience is closer to the real wearing experience, and the method has high application value and research significance.

Description

Technical Field

[0001] The present application relates to the field of computer graphics processing, and in particular to a virtual fitting method for an opera short coat with adaptive length. Background Art

[0002] With the rapid development of virtual reality (VR), augmented reality (AR) and deep learning technology, virtual try-on technology has gradually become an important application method in the fields of e-commerce platforms, digital clothing design and cultural heritage protection. Virtual try-on can not only help consumers understand the effect of clothing more intuitively before purchasing, but also provide designers with a tool to quickly verify design solutions. However, traditional virtual try-on methods face many challenges when dealing with complex patterns and special forms of clothing (such as opera costumes), including loss of pattern details, inaccurate colors, and poor fit between clothing and the human body.

[0003] Opera costumes, especially those of Qinqiang, a type of opera with a profound cultural background, usually have delicate and complex patterns, unique shapes and bright colors. The design of these costumes should not only reflect the identity and temperament of the characters, but also meet the needs of the movements in the opera performance. In the traditional virtual fitting system, the length and shape of the costumes are fixed, and there is a lack of adaptive adjustment capabilities, which makes it difficult to meet the diverse needs of different characters, especially when trying on these complex opera costumes. Summary of the invention

[0004] In order to overcome at least one shortcoming in the prior art, the present application provides a virtual try-on method for an opera short coat with adaptive length.

[0005] In a first aspect, a virtual fitting method for an opera short coat with adaptive length is provided, comprising:

[0006] Acquire a portrait image and a target clothing image; perform dense human posture prediction based on the portrait image to obtain a posture image; perform semantic segmentation on the portrait image to obtain a semantic segmentation map; use a feature extractor to extract features from the target clothing image to obtain clothing features; use a feature extractor to extract features from the semantic segmentation map and the posture image to obtain human posture features; use a try-on condition generator to obtain a semantic segmentation map of a human wearing the target clothing and a deformed clothing image based on clothing features and human posture features;

[0007] Construct a virtual try-on model, which includes a try-on image generator, a skin color adjustment module, and a length adaptive adjustment module;

[0008] The semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image are input into the virtual try-on model to obtain the virtual try-on image; the try-on image generator obtains the initial try-on image that retains the high-quality features of the human posture and clothing based on the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image; the skin color adjustment module uses the skin color information to perform weighted fusion on the skin area of ​​the initial try-on image to obtain the try-on image after skin color adjustment; the length adaptive adjustment module uses the human skeleton information to adjust the try-on image after skin color adjustment so that the clothing length matches the human posture to obtain the virtual try-on image.

[0009] In one embodiment, the feature extractor includes a plurality of residual blocks connected sequentially.

[0010] In one embodiment, the fitting condition generator includes a plurality of feature fusion modules connected in sequence, wherein the feature fusion module is used to perform a convolution operation on a first input to obtain a first convolution result; perform a gated processing on a second input to obtain a gated processing result; multiply the second convolution result and the gated processing result to obtain a first multiplication result; perform a convolution operation on the first multiplication result to obtain a second convolution result; multiply the second convolution result and the first multiplication result to obtain a second multiplication result, i.e., a first output;

[0011] The second multiplication result is convolved to obtain a third convolution result;

[0012] Performing a convolution operation on the second input to obtain a fourth convolution result;

[0013] The fourth convolution result is multiplied by the third convolution result to obtain a third multiplication result;

[0014] The third multiplication result is added to the third convolution result to obtain a second output;

[0015] The first feature fusion module among the multiple feature fusion modules has a first input of clothing features and a second input of human posture features; the other feature fusion modules have a first input of the result of adding clothing features to the first output of the previous feature fusion module and a second input of the result of optical flow prediction of human posture features and the second output of the previous feature fusion module;

[0016] The first output of the last feature fusion module is the deformed clothing image, and the second output is the semantic segmentation map of the human body wearing the target clothing.

[0017] In one embodiment, the try-on image generator includes a convolutional layer, a plurality of residual blocks, and a convolutional layer connected in sequence.

[0018] In one embodiment, the skin color adjustment module is used to:

[0019] Calculate the average RGB value of all pixels in the skin area of ​​the initial trial image, that is, the skin color information;

[0020] According to the skin color information, the following formula is used to obtain the skin color adjusted trial image:

[0021] I2(x,y)=αI1(x,y)+(1-α)C

[0022] Among them, I1(x, y) is the RGB value of the pixel (x, y) in the initial trial image, C is the skin color information, α is the weighting coefficient, and I2(x, y) is the RGB value of the pixel (x, y) in the trial image after skin color adjustment.

[0023] In one embodiment, the length adaptive adjustment module is used to:

[0024] Obtain the coordinates of skeleton key points based on the posture image;

[0025] The desired clothing length is set according to the distance from the hip to the ankle in the skeleton keypoint coordinates, and the length scaling factor λ of the clothing is determined by the following formula:

[0026]

[0027] Among them, L1 is the expected clothing length, and L2 is the original length of the target clothing;

[0028] The vertical coordinate of each pixel in the skin color adjusted fitting image is multiplied by the length scale factor λ of the clothing to obtain a virtual fitting image.

[0029] In one embodiment, constructing the virtual fitting model includes training the virtual fitting model, and the loss function used in the training process is:

[0030] L=Σ x,y w(x,y)(|I1(x+1,y)-I1(x,y)|+|I1(x,y+1)-I1(x,y)|)

[0031] Where L is the loss function, I1(x,y) is the RGB value of the pixel (x,y) in the initial trial image, I1(x+1,y) is the RGB value of the pixel (x+1,y) in the initial trial image, and I1(x,y+1) is the RGB value of the pixel (x,y+1) in the initial trial image; w(x,y) is the weight coefficient of the pixel (x,y);

[0032]

[0033] in, is the gradient of pixel (x,y) in the x direction, is the gradient of pixel (x, y) in the y direction, and β is a parameter that controls the degree of smoothness.

[0034] In a second aspect, a virtual fitting device for an opera short coat with adaptive length is provided, comprising:

[0035] A preprocessing module is used to obtain a portrait image and a target clothing image; perform dense human posture prediction based on the portrait image to obtain a posture image; perform semantic segmentation on the portrait image to obtain a semantic segmentation map; use a feature extractor to extract features of the target clothing image to obtain clothing features; use a feature extractor to extract features from the semantic segmentation map and the posture image to obtain human posture features; use a try-on condition generator to obtain a semantic segmentation map of a human wearing the target clothing and a deformed clothing image based on clothing features and human posture features;

[0036] A model building module is used to build a virtual try-on model, which includes a try-on image generator, a skin color adjustment module, and a length adaptive adjustment module;

[0037] The virtual try-on module is used to input the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image into the virtual try-on model to obtain the virtual try-on image; the try-on image generator obtains the initial try-on image that retains the high-quality features of the human body posture and clothing based on the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image; the skin color adjustment module uses the skin color information to perform weighted fusion on the skin area of ​​the initial try-on image to obtain the try-on image after the skin color is adjusted; the length adaptive adjustment module uses the human skeleton information to adjust the try-on image after the skin color is adjusted so that the clothing length matches the human body posture to obtain the virtual try-on image.

[0038] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the virtual try-on method for the adaptive length of the opera short jacket is implemented.

[0039] In a fourth aspect, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned virtual try-on method for the adaptive length of a short opera jacket.

[0040] Compared with the prior art, the present application has the following beneficial effects: the virtual try-on method of the adaptive length of opera short jackets of the present application, based on the try-on condition generator and the try-on image generator, can not only dynamically adjust the length and shape of the clothing, but also improve the naturalness and realism of the try-on image while maintaining the integrity of the clothing pattern and texture. In addition, by adding a skin color adjustment module and a length adaptive adjustment module, the problem of differences in skin color and clothing styles of different human bodies can be solved to a greater extent. When trying on opera costumes, the complex patterns and morphological characteristics of Qinqiang costumes are especially optimized, which greatly improves the try-on effect and makes it closer to the real wearing experience, with high application value and research significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present application may be better understood by referring to the following description given in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification. In the drawings:

[0042] Figure 1 A flowchart of a virtual fitting method for an opera short coat with adaptive length is shown;

[0043] Figure 2 A schematic diagram of the structure of a feature extractor is shown;

[0044] Figure 3 A schematic diagram of the structure of a try-on image generator is shown;

[0045] Figure 4 A schematic diagram of the structure of a try-on condition generator is shown;

[0046] Figure 5 A schematic diagram of a virtual try-on effect is shown;

[0047] Figure 6 A schematic diagram of the virtual try-on effect of opera costumes is shown;

[0048] Figure 7 The structural block diagram of the virtual fitting device for the adaptive length of opera short coat is shown. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present application will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many implementation-specific decisions can be made in the process of developing any such actual embodiments in order to achieve the specific goals of the developer, and these decisions may vary from embodiment to embodiment.

[0050] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, only the device structure closely related to the scheme according to the present application is shown in the drawings, while other details that are not closely related to the present application are omitted.

[0051] It should be understood that the present application is not limited to the described implementation forms due to the following description with reference to the accompanying drawings. In this article, where feasible, the embodiments can be combined with each other, features between different embodiments can be replaced or borrowed, and one or more features can be omitted in one embodiment.

[0052] The embodiment of the present application provides a virtual fitting method for an opera short coat with adaptive length. Figure 1 A flowchart of a virtual fitting method for an opera short coat with adaptive length is shown, see Figure 1 , methods include:

[0053] Step S1, obtaining a portrait image and a target clothing image; performing dense human posture prediction based on the portrait image to obtain a posture image; performing semantic segmentation on the portrait image to obtain a semantic segmentation map; using a feature extractor to perform feature extraction on the target clothing image to obtain clothing features; using a feature extractor to perform feature extraction on the semantic segmentation map and the posture image to obtain human posture features; using a fitting condition generator to obtain a semantic segmentation map of a human body wearing the target clothing and a deformed clothing image based on clothing features and human posture features.

[0054] In this step, a dense posture prediction model is used to perform dense posture prediction of the human body based on the portrait image, so as to capture the posture changes of the human body details and obtain a posture image. The dense posture prediction model may use DensePose, for example.

[0055] Then, a pre-trained semantic segmentation model is used to perform semantic segmentation on the portrait image to obtain a semantic segmentation map, which divides different parts of the human body into different categories (such as head, top, pants, etc.) to provide semantic information for subsequent clothing deformation. The semantic segmentation model can use FCN (Fully Convolutional Network) and U-Net, for example. FCN improves the classification network into a semantic segmentation network and uses a fully convolutional layer for end-to-end training. U-Net uses an encoder-decoder architecture to perform segmentation by splicing feature maps.

[0056] Then, the feature extractor is used to extract clothing features and human posture features respectively. Figure 2 The structure diagram of the feature extractor is shown in FIG. Figure 2, the feature extractor includes multiple sequentially connected residual blocks (ResBlock). Here, the residual block can be set to 4 or other numbers. Each residual block is used to implement residual operation and 2x downsampling. The residual block contains multiple convolutional layers (Convolutional Layer), batch normalization layer (Batch Normalization Layer), activation function (Activation Function) and residual connection (Residual Connection).

[0057] Step S2, constructing a virtual try-on model, which includes a try-on image generator, a skin color adjustment module, and a length adaptive adjustment module.

[0058] Step S3, inputting the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image into the virtual try-on model to obtain a virtual try-on image; the try-on image generator obtains an initial try-on image that retains high-quality features of the human body posture and clothing based on the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image; the skin color adjustment module uses the skin color information to perform weighted fusion on the skin area of ​​the initial try-on image to obtain a try-on image after skin color adjustment; the length adaptive adjustment module uses the human skeleton information to adjust the try-on image after skin color adjustment so that the clothing length matches the human body posture to obtain a virtual try-on image.

[0059] Specifically, Figure 3 A schematic diagram of the structure of the try-on image generator is shown in FIG. Figure 3 , the try-on image generator includes a convolutional layer, multiple residual blocks and a convolutional layer connected in sequence. Here, the residual block can be set to 8 or other numbers. Except for the last residual block, other residual blocks are used to implement residual operations and 2x upsampling. The last residual block implements residual operations. Here, a series of residual blocks are used to process the input features. SPADE (hole conditional normalization layer) is used in the residual block to normalize the input image so that the details between the clothing and the human body posture are retained. The input image is resized before each residual block and connected to the activation layer to ensure that the high-quality features of the posture and clothing are retained during the image generation process.

[0060] In this embodiment, based on the fitting condition generator and the fitting image generator, not only can the length and shape of the clothing be adjusted dynamically, but also the naturalness and realism of the fitting image can be improved while maintaining the integrity of the clothing pattern and texture. In addition, by adding a skin color adjustment module and a length adaptive adjustment module, the problem of differences in skin color and clothing styles of different human bodies can be solved to a greater extent. When trying on opera costumes, the complex patterns and morphological characteristics of Qinqiang costumes are optimized, which greatly improves the fitting effect and makes it closer to the real wearing experience, which has high application value and research significance.

[0061] In one embodiment, Figure 4 The schematic diagram of the structure of the try-on condition generator is shown in Figure 4 The fitting condition generator includes a plurality of feature fusion modules connected in sequence, wherein the feature fusion module is used to perform a convolution operation on the first input to obtain a first convolution result; perform a gated processing on the second input to obtain a gated processing result, wherein the gated processing is based on a gated processing unit (Gate), adopts Gating Machinism, and uses Sigmmod to enhance the memory and fusion capabilities of the module; the second convolution result is multiplied with the gated processing result to obtain a first multiplication result; the first multiplication result is convolved to obtain a second convolution result, and the second convolution result is multiplied with the first multiplication result to obtain a second multiplication result, i.e., the first output;

[0062] The second multiplication result is convolved to obtain a third convolution result;

[0063] Performing a convolution operation on the second input to obtain a fourth convolution result;

[0064] The fourth convolution result is multiplied by the third convolution result to obtain a third multiplication result;

[0065] The third multiplication result is added to the third convolution result to obtain a second output;

[0066] The first feature fusion module among the multiple feature fusion modules has a first input of clothing features and a second input of human body posture features; the other feature fusion modules have a first input of the result of adding clothing features to the first output of the previous feature fusion module and a second input of the result of optical flow prediction (Warping) of human body posture features and the second output of the previous feature fusion module;

[0067] The first output of the last feature fusion module is the deformed clothing image, and the second output is the semantic segmentation map of the human body wearing the target clothing.

[0068] In this embodiment, a feature fusion module is used to dynamically learn the relationship between clothing features and human posture features to obtain a deformed clothing image and a semantic segmentation map of a human body wearing the target clothing, so as to ensure that the clothing can still fit the human body naturally in different postures and avoid obvious misalignment or deformation.

[0069] In one embodiment, the skin color adjustment module is used to:

[0070] Calculate the average RGB value of all pixels in the skin area of ​​the initial trial image, that is, the skin color information;

[0071] According to the skin color information, the following formula is used to obtain the skin color adjusted trial image:

[0072] I2(x,y)=αI1(x,y)+(1-α)C

[0073] Among them, I1(x, y) is the RGB value of the pixel (x, y) in the initial trial image, C is the skin color information, α is the weighting coefficient, and I2(x, y) is the RGB value of the pixel (x, y) in the trial image after skin color adjustment.

[0074] In this embodiment, the extracted skin color information is used to perform weighted fusion on the skin area in the initial fitting image, so that the generated skin color is closer to the skin color of the original portrait. This process can effectively solve the impact of different lighting and skin color differences on the fitting results, making the generated portrait image more natural and realistic visually.

[0075] Here, the weighting coefficient α is the optimal weighting coefficient determined by the model training process. The optimal weighting coefficient can be obtained by calculating SSIM (Structural Similarity), FID (Frechet Inception Distance) and LPIPS (Learned Perceptual Image Patch Similarity) between the initial trial image and the trial image after skin color adjustment. LPIPS is an image quality assessment method based on deep learning, and FID is an indicator used to evaluate the performance of image generation models; and calculating the average value between the three indicators as the similarity result; if the similarity result is less than the set threshold, increase α and train again until the similarity result reaches the set threshold, and then get the optimal weighting coefficient.

[0076] In one embodiment, the length adaptive adjustment module is used to:

[0077] Obtain the coordinates of skeleton key points based on the posture image;

[0078] The desired clothing length is set according to the distance from the hip to the ankle in the skeleton keypoint coordinates, and the length scaling factor λ of the clothing is determined by the following formula:

[0079]

[0080] Among them, L1 is the expected clothing length, L2 is the original length of the target clothing; λ is used to control the proportion of longitudinal expansion.

[0081] The vertical coordinate of each pixel in the skin color adjusted fitting image is multiplied by the length scale factor λ of the clothing to obtain a virtual fitting image.

[0082] In this embodiment, for the situation that the style of the target clothing is quite different from the original clothing, the skeleton key point coordinates in the JSON file in the posture image are obtained, and the expected clothing length is set according to the distance from the hip to the ankle in the skeleton key point coordinates. Due to the complex and characteristic postures of the opera characters, their hand postures are relatively complex, but their center points and center of gravity are usually at the hip, so the hip position of the human body is located. The try-on image after skin color adjustment is repositioned and adjusted according to the skeleton key point coordinates so that the length of the clothes matches the human body posture. By adaptively adjusting the shape of the clothing, the generated try-on result is not only limited to the length of the original clothing, but can achieve a virtual try-on effect of any length on the basis of ensuring the integrity of the clothing texture and style. Figure 5 A schematic diagram of the virtual try-on effect is shown. Figure 6 A schematic diagram of the virtual try-on effect of a short opera dress is shown.

[0083] In one embodiment, constructing a virtual try-on model includes training the virtual try-on model. The data set used in the model training is the VITON-HD data set. The loss function used in the training process is:

[0084] L=∑ x,y w(x,y)(|I1(x+1,y)-I1(x,y)|+|I1(x,y+1)-I1(x,y)|)

[0085] Where L is the loss function, I1(x,y) is the RGB value of the pixel (x,y) in the initial trial image, I1(x+1,y) is the RGB value of the pixel (x+1,y) in the initial trial image, and I1(x,y+1) is the RGB value of the pixel (x,y+1) in the initial trial image; w(x,y) is the weight coefficient of the pixel (x,y);

[0086]

[0087] in, is the gradient of pixel (x,y) in the x direction, is the gradient of pixel (x, y) in the y direction, and β is a parameter that controls the degree of smoothness.

[0088] Here, a larger β will give a lower weight to the edge area, thereby retaining more details. The weight coefficient w(x,y) will be close to zero in the edge area, reducing the smoothing of the area, and close to 1 in the smooth area, performing more smoothing. Using the loss function in the above embodiment for model training can constrain the try-on image generator to reduce exaggerated deformation and maintain good clothing details.

[0089] Based on the same inventive concept as the method for virtual fitting of an opera short coat with self-adaptive length, this embodiment also provides a corresponding virtual fitting device for virtual fitting of an opera short coat with self-adaptive length. Figure 7 The structure block diagram of the virtual fitting device for the adaptive length of opera short coat is shown, including:

[0090] The preprocessing module 71 is used to obtain a portrait image and a target clothing image; perform dense human posture prediction based on the portrait image to obtain a posture image; perform semantic segmentation on the portrait image to obtain a semantic segmentation map; use a feature extractor to extract features of the target clothing image to obtain clothing features; use a feature extractor to extract features from the semantic segmentation map and the posture image to obtain human posture features; use a fitting condition generator to obtain a semantic segmentation map of a human body wearing the target clothing and a deformed clothing image based on clothing features and human posture features;

[0091] A model building module 72 is used to build a virtual try-on model, which includes a try-on image generator, a skin color adjustment module, and a length adaptive adjustment module;

[0092] The virtual try-on module 73 is used to input the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image into the virtual try-on model to obtain a virtual try-on image; the try-on image generator obtains an initial try-on image that retains high-quality features of the human body posture and clothing based on the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image; the skin color adjustment module uses the skin color information to perform weighted fusion on the skin area of ​​the initial try-on image to obtain a try-on image after skin color adjustment; the length adaptive adjustment module uses the human skeleton information to adjust the try-on image after skin color adjustment so that the clothing length matches the human body posture to obtain a virtual try-on image.

[0093] The virtual trying-on device for adaptive length of opera short jackets of this embodiment has the same inventive concept as the virtual trying-on method for adaptive length of opera short jackets described above. Therefore, the specific implementation method of the device can be seen in the implementation example part of the virtual trying-on method for adaptive length of opera short jackets in the above text, and its technical effect corresponds to the technical effect of the above method, which will not be repeated here.

[0094] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for virtual trying on the adaptive length of a short opera jacket is implemented.

[0095] The embodiment of the present application provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the method for virtual trying on the adaptive length of the opera short jacket is implemented.

[0096] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A virtual fitting method for an opera short coat with adaptive length, characterized in that: include: Obtain a portrait image and a target clothing image; Performing dense human posture prediction based on the portrait image to obtain a posture image; Performing semantic segmentation on the portrait image to obtain a semantic segmentation map; using a feature extractor to perform feature extraction on the target clothing image to obtain clothing features; A feature extractor is used to extract features from the semantic segmentation map and the posture image to obtain human posture features; a fitting condition generator is used to obtain a human semantic segmentation map of a human body wearing target clothing and a deformed clothing image based on the clothing features and the human posture features; Constructing a virtual try-on model, the virtual try-on model includes a try-on image generator, a skin color adjustment module, and a length adaptive adjustment module; The semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image are input into the virtual fitting model to obtain a virtual fitting image; the fitting image generator obtains an initial fitting image that retains high-quality features of human posture and clothing based on the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image; The skin color adjustment module uses skin color information to perform weighted fusion on the skin area of ​​the initial try-on image to obtain a try-on image after skin color adjustment; the length adaptive adjustment module uses human skeleton information to adjust the try-on image after skin color adjustment so that the length of the clothing matches the human body posture to obtain a virtual try-on image.

2. The method according to claim 1, characterized in that The feature extractor includes a plurality of residual blocks connected in sequence.

3. The method according to claim 1, characterized in that The fitting condition generator comprises a plurality of feature fusion modules connected in sequence, wherein the feature fusion module is used to perform a convolution operation on a first input to obtain a first convolution result; perform a gated processing on a second input to obtain a gated processing result; multiply the second convolution result and the gated processing result to obtain a first multiplication result; perform a convolution operation on the first multiplication result to obtain a second convolution result; and multiply the second convolution result and the first multiplication result to obtain a second multiplication result, i.e., a first output; Performing a convolution operation on the second multiplication result to obtain a third convolution result; Performing a convolution operation on the second input to obtain a fourth convolution result; Multiplying the fourth convolution result by the third convolution result to obtain a third multiplication result; Adding the third multiplication result to the third convolution result to obtain a second output; The first feature fusion module among the multiple feature fusion modules has the clothing feature as the first input and the human body posture feature as the second input; the other feature fusion modules have the clothing feature as the first input and the result of adding the clothing feature to the first output of the previous feature fusion module as the second input and the result of optical flow prediction of the human body posture feature and the second output of the previous feature fusion module as the second input; The first output of the last feature fusion module is the deformed clothing image, and the second output is the semantic segmentation map of the human body wearing the target clothing.

4. The method according to claim 1, characterized in that The try-on image generator includes a convolutional layer, a plurality of residual blocks and a convolutional layer connected in sequence.

5. The method according to claim 1, characterized in that The skin color adjustment module is used for: Calculate the average RGB value of all pixels in the skin area of ​​the initial fitting image, that is, the skin color information; According to the skin color information, the following formula is used to obtain the try-on image after skin color adjustment: I2(x,y)=αI1(x,y)+(1-α)C Among them, I1(x, y) is the RGB value of the pixel (x, y) in the initial trial image, C is the skin color information, α is the weighting coefficient, and I2(x, y) is the RGB value of the pixel (x, y) in the trial image after skin color adjustment.

6. The method according to claim 1, characterized in that The length adaptive adjustment module is used for: Obtain the coordinates of skeleton key points based on the posture image; The desired clothing length is set according to the distance from the hip to the ankle in the skeleton key point coordinates, and the length scale factor λ of the clothing is determined by the following formula: Among them, L1 is the expected clothing length, and L2 is the original length of the target clothing; The vertical coordinate of each pixel in the skin color adjusted try-on image is multiplied by the length scale factor λ of the garment to obtain a virtual try-on image.

7. The method according to claim 1, characterized in that The construction of the virtual fitting model includes training the virtual fitting model, and the loss function used in the training process is: L=∑ x,y w(x,y)(|I1(x+1,y)-I1(x,y)|+|I1(x,y+1)-I1(x,y)|) Where L is the loss function, I1(x,y) is the RGB value of the pixel (x,y) in the initial trial image, I1(x+1,y) is the RGB value of the pixel (x+1,y) in the initial trial image, and I1(x,y+1) is the RGB value of the pixel (x,y+1) in the initial trial image; w(x,y) is the weight coefficient of the pixel (x,y); in, is the gradient of pixel (x,y) in the x direction, is the gradient of pixel (x, y) in the y direction, and β is a parameter that controls the degree of smoothness.

8. A virtual fitting device for opera short coats with adaptive length, characterized in that: include: A preprocessing module, used to obtain a portrait image and a target clothing image; Performing dense human posture prediction based on the portrait image to obtain a posture image; Performing semantic segmentation on the portrait image to obtain a semantic segmentation map; using a feature extractor to perform feature extraction on the target clothing image to obtain clothing features; A feature extractor is used to extract features from the semantic segmentation map and the posture image to obtain human posture features; a fitting condition generator is used to obtain a human semantic segmentation map of a human body wearing target clothing and a deformed clothing image based on the clothing features and the human posture features; A model building module, used to build a virtual try-on model, the virtual try-on model includes a try-on image generator, a skin color adjustment module, and a length adaptive adjustment module; The virtual fitting module is used to input the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image into the virtual fitting model to obtain a virtual fitting image; the fitting image generator obtains an initial fitting image that retains high-quality features of the human body posture and clothing based on the semantic segmentation map of the human body wearing the target clothing, the deformed clothing image, the portrait image, and the posture image; The skin color adjustment module uses skin color information to perform weighted fusion on the skin area of ​​the initial try-on image to obtain a try-on image after skin color adjustment; the length adaptive adjustment module uses human skeleton information to adjust the try-on image after skin color adjustment so that the length of the clothing matches the human body posture to obtain a virtual try-on image.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for virtual fitting of an opera short coat with adaptive length as described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the virtual fitting method for the adaptive length of an opera short coat as described in any one of claims 1 to 7.

Images