A hierarchical context iterative and dense fusion refined virtual fitting system

Abstract

This invention discloses a refined virtual try-on system based on hierarchical context iteration and dense fusion, relating to the field of computer vision. It includes a hierarchical context iteration refinement warping module and a dense fusion generation module. The hierarchical context iteration refinement warping module warps the target garment to obtain warped clothing, and includes a feature pyramid network, a hierarchical context flow estimation module, and an iterative attention refinement module. The dense fusion generation module matches the warped clothing to a person's image for virtual try-on. This dense fusion generation module is based on the Res-UNet network, introducing DenseNet style connections and SE attention mechanisms into the residual blocks. This invention uses a refined virtual try-on system based on hierarchical context iteration and dense fusion to address appearance flow prediction bias in complex textures and areas of significant deformation. It can effectively reduce texture distortion and artifacts, better fuse latent features, and capture local details to generate higher-quality try-on results.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a refined virtual try-on system based on hierarchical context iteration and dense fusion. Background Technology

[0002] The warping method based on thin plate spline (TPS) currently realizes clothing deformation by calculating global smooth transformation through control points. Improvements such as learnable TPS, semantic layout, and higher-order constraints have been introduced. However, the TPS method assumes that the deformation is globally smooth and continuous. When it is necessary to deal with complex shapes or highly nonlinear deformations, it cannot accurately simulate the real deformation situation and is prone to problems such as local inaccuracy, loss of details, or excessive distortion.

[0003] The appearance flow-based warping method uses pixel-level appearance flow fields to directly map pixels of the source image to the target location. Compared with TPS-based methods, this method can better capture local details and nonlinear deformations. Current methods optimize flow estimation through techniques such as self-supervised training, global feature matching, global feature fusion, and variable attention flow. However, existing flow estimation modules lack multi-scale context modeling capabilities, resulting in insufficient prediction accuracy for complex textures and regions with large deformations. At the same time, the global clothing parsing module struggles to focus on key structural regions (such as collars and cuffs), leading to parsing ambiguity.

[0004] Current improvements to flow-based warping methods primarily target the warping module. Most virtual try-on models use Res-UNet-based generators to synthesize try-on results. While these generators offer advantages such as simple structure and high efficiency, their feature fusion capabilities are limited, and they lack dynamic attention mechanisms for key information in channel and spatial dimensions. This results in the generator struggling to effectively integrate information when synthesizing the final try-on image, often leading to issues like blurred textures and lost details.

[0005] To address these issues, there is an urgent need for a refined virtual try-on system that integrates hierarchical context iteration and dense fusion. Summary of the Invention

[0006] To address the aforementioned issues, this application proposes a refined virtual try-on system based on hierarchical context iteration and dense fusion. The model takes clothing images, clothing masks, clothing analytical images, person images, dense poses of the person, key points of the person, and analytical images of the person as input, and adopts a self-supervised training method, including a hierarchical context iteration refinement warp module and a dense fusion generation module.

[0007] The hierarchical context iterative refinement warping module is used to warp the target clothing based on the person image to obtain a global clothing analysis map, i.e., warped clothing.

[0008] The dense fusion generation module is used to match the global clothing analysis image to the person image to perform virtual try-on and obtain a clothing matching image.

[0009] Preferably, the hierarchical context iterative refinement warp module includes:

[0010] Feature Pyramid Network: The feature pyramid network is used to extract multi-level features. There are two networks: a human feature pyramid network for extracting multi-scale human features from human images and a clothing feature pyramid network for extracting multi-scale clothing features from target clothing.

[0011] The human feature pyramid network takes human pose, dense human pose, and preserved region mask as input, and outputs multi-scale human features.

[0012] The clothing feature pyramid network takes clothing and its corresponding analytical image as input and outputs multi-scale clothing features.

[0013] The multi-scale human features and multi-scale clothing features constitute multi-level features;

[0014] Hierarchical context flow estimation module: The hierarchical context flow estimation module is used to extract optical flow information, i.e., preliminary optical flow field, through multi-level features, then optimize optical flow information using context information, and then perform feature integration to obtain refined optical flow information, i.e. refined optical flow field.

[0015] Iterative Attention Refinement Module: The iterative attention refinement module is used to perform feature fusion on the refined optical flow information after clothing feature warping, and then splice it with the character features to obtain a global fused feature. This is then used to generate and refine the attention map multiple times in order to gradually enhance the feature representation of important regions and suppress the interference of irrelevant regions to obtain a global clothing analysis map.

[0016] Preferably, the hierarchical context flow estimation module includes a hierarchical flow estimator and a context flow estimator;

[0017] The hierarchical flow estimator is used to generate a preliminary optical flow field through layer-by-layer feature extraction and fusion, including 5 convolutional layers and 1 flow field prediction layer;

[0018] The first convolutional layer maps the multi-level features of the input to a 128-dimensional feature space, providing a rich information foundation for subsequent feature processing and fusion;

[0019] The second convolutional layer further extracts deep features while maintaining the number of feature channels;

[0020] In the 3rd to 5th convolutional layers, each layer reduces the number of feature channels from 128 to 32 through feature concatenation and progressive dimensionality reduction. The concatenation operation combines the features of different layers, enhances the expressive power of the module, and preserves the features of earlier layers, avoiding information loss during the convolution process.

[0021] The flow field prediction layer uses a single convolution operation to map the concatenated features into a two-dimensional flow field, which represents the displacement of each pixel.

[0022] Preferably, the context flow estimator is used to extract multi-scale contextual information from the input multi-level features to generate a predicted refined optical flow field;

[0023] The kernel size of the context flow estimator is fixed at 3×3, and the expansion rates are set to 1, 2, 4, 8, 16, and 1, respectively.

[0024] The context flow estimator includes 7 convolutional layers, of which 4 convolutional layers constitute the multi-scale feature extraction part and 3 convolutional layers constitute the refined flow prediction head;

[0025] The convolutional layer is used to progressively fuse spatial information to extract multi-scale contextual information;

[0026] The refined flow prediction head is used to receive multi-scale context information and generate a refined optical flow field.

[0027] Preferably, the dilation rate of the first convolution in the refined flow prediction head is set to 16, and its output is used to provide information support across a wide range;

[0028] The dilation rate of the second convolution in the refine flow prediction head is set to 1 to focus on refining local details and the final refine flow prediction.

[0029] Preferably, the iterative attention refining module includes an iterative attention layer;

[0030] The iterative attention layer is formed by multiple attention layers. Each attention layer generates an attention map and weights the input features based on the current attention map.

[0031] The attention layer consists of multiple convolutional layers with the same structure, and each convolutional layer consists of two convolutional operations and an activation function.

[0032] The attention layer is used to generate an attention map and to weight and adjust the input features based on the current attention map.

[0033] Preferably, when adjusting the input features based on the current attention map, the multiplicative weighting adjustment method is selected.

[0034] Preferably, the dense fusion generation module is based on the Res-UNet network and introduces DenseNet-style connections and SE attention mechanisms into the residual blocks;

[0035] Through DenseNet-style connections, the features of each layer can be directly passed to the final output;

[0036] In the innermost layer of Res-UNet, there is a hybrid feature fusion layer, which captures multi-scale spatial context information of the input features by combining multiple convolutional kernel sizes;

[0037] The hybrid feature fusion layer operates on the input features using convolutional kernels of different sizes, then adds and fuses the resulting features point by point, and finally processes them through a non-linear activation function to obtain the final output.

[0038] Preferably, the dense fusion generation module includes four different convolution kernel sizes: 1×1 convolution kernel size, 3×3 convolution kernel size, 5×5 convolution kernel size, and 7×7 convolution kernel size.

[0039] Convolutions with a 1×1 kernel size are used for cross-channel information integration;

[0040] Convolutions with a 3×3 kernel size are used to capture small local features;

[0041] Convolutions with a 5×5 kernel size are used to expand the receptive field;

[0042] Convolutions with a 7×7 kernel size are used to cover a larger receptive field, capture global context, and fuse data through direct addition.

[0043] In summary, the refined virtual try-on system based on hierarchical context iteration and dense fusion of the present invention has the following advantages compared with traditional technologies:

[0044] 1. The hierarchical context flow estimation module solves the appearance flow prediction bias of complex textures and large deformation areas through multi-scale context iterative optimization, which can effectively reduce texture distortion and artifacts;

[0045] 2. The iterative attention module can weight the input features according to the current attention map. After multiple iterations, the network can gradually adjust the attention of each region, thereby helping to focus on important details in the image and generate a more accurate semantic map.

[0046] 3. The dense fusion generation module enhances feature representation capabilities through dense connections and attention mechanisms. Combined with a hybrid feature fusion layer, it can better integrate latent features and capture local details to generate higher-quality try-on results.

[0047] The technical method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the hierarchical context flow estimation module of the refined virtual try-on system based on hierarchical context iteration and dense fusion according to the present invention;

[0049] Figure 2 This is a schematic diagram of the iterative attention refinement module of a refined virtual try-on system based on hierarchical context iteration and dense fusion according to the present invention;

[0050] Figure 3 This invention provides an overall architecture for a densely fused generation module in a refined virtual try-on system based on hierarchical context iteration and dense fusion.

[0051] Figure 4 This is an improved residual block diagram of a refined virtual try-on system based on hierarchical context iteration and dense fusion according to the present invention.

[0052] Figure 5 This is a hybrid feature fusion layer diagram of a refined virtual try-on system based on hierarchical context iteration and dense fusion according to the present invention. Detailed Implementation

[0053] The technical method of the present invention will be further described below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application.

[0054] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0055] Techniques, systems, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the instruction manual.

[0056] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0057] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0058] Example 1

[0059] This invention provides a refined virtual fitting system model based on hierarchical context iteration and dense fusion. It takes clothing images, clothing masks, clothing analytical images, human images, dense human poses, human key points, and human analytical images as inputs and adopts a self-supervised training method, including a hierarchical context iteration refinement warp module and a dense fusion generation module.

[0060] The use of virtual try-on technology is similar to the early Taobao fitting room. Users select clothing from a store, upload a photo of themselves, and a picture of them wearing the clothing is generated, allowing buyers to make better purchasing decisions. The two-stage try-on model works as follows: First, the warping module iteratively refines the warping of the clothing according to the pose of the person's photo (the warping process involves extracting features, estimating optical flow information using a hierarchical context flow estimation module, and warping the clothing image in parts according to the estimated optical flow information; the warped parts are then combined under the guidance of the global clothing analysis output by the iterative attention refinement module to form a complete warped garment). Second, the dense fusion generation module puts the warped garment on the person's photo, outputting a picture of them wearing the clothes.

[0061] During the training phase, the hierarchical context iterative refinement of the warp module warps the corresponding clothing in the character image according to the shape in the character image (to improve warping accuracy, the entire garment is not warped, but the local flow of three parts (left sleeve, torso, and right sleeve) is estimated separately to guide the warping of local garment parts, and then the three warped parts are assembled together to obtain a complete warped garment). The warped garment is obtained, and the warping quality is evaluated by comparing the warped garment in the refined virtual fitting system model with the garment in the character image, thereby learning the warp module.

[0062] The dense fusion generation module puts the warped clothing onto the person image. By comparing the person image generated after the model try-on with the original person image, the effect of the dense fusion generation module is evaluated, and the generation module learns from this.

[0063] The refined virtual try-on system model is a two-stage model, namely, a hierarchical context iterative refinement of the warp module, such as... Figure 1 The diagram shows the dense fusion generation module as follows: Figure 3 The diagram shows training performed separately.

[0064] The hierarchical context iterative refinement warping module is used to warp the target clothing (i.e., in-store clothing) based on the person image to obtain a global clothing analysis image of the warped clothing (i.e., in-store clothing warped according to the posture of the uploaded person image).

[0065] Furthermore, the specific workflow of the hierarchical context iterative refinement warping module is as follows: First, five multi-scale character features and five multi-scale clothing features are extracted using two feature pyramid networks. The correlation between the character features and clothing features is calculated and used as input to the flow estimation module to estimate the local flow. The local flow is then used to warp each part of the clothing, resulting in warped components for each part. Simultaneously, the character features, clothing features, and local flow are used as input to the attention module to estimate the global clothing parsing. The obtained global clothing parsing is then used to guide the assembly of the warped components to obtain the complete warped clothing.

[0066] The hierarchical context iterative refinement warp module includes:

[0067] Feature Pyramid Network: The feature pyramid network is used to extract multi-level features. There are two networks: a human feature pyramid network for extracting multi-scale human features from human images and a clothing feature pyramid network for extracting multi-scale clothing features from target clothing.

[0068] The human feature pyramid network takes human pose, dense human pose, and preserved region mask as input, and outputs multi-scale human features.

[0069] The clothing feature pyramid network takes clothing and its corresponding analytical image as input and outputs multi-scale clothing features.

[0070] The multi-scale human features and multi-scale clothing features constitute multi-level features;

[0071] Hierarchical context flow estimation module: The hierarchical context flow estimation module is used to gradually extract optical flow information, i.e., the preliminary optical flow field, through multi-level feature abstraction, and then optimize the details of optical flow information by using the context information of different receptive fields, and then perform feature integration to obtain the refined optical flow information, i.e., the refined optical flow field, thus realizing the progressive feature integration from global to local.

[0072] Since the local flows of the three parts need to be estimated separately, the hierarchical context flow estimation module first copies the character features and clothing features output by the feature pyramid and performs correlation calculation. Then it is input into the hierarchical flow prediction module to predict the coarse appearance flow, and then input into the context flow prediction module to finally obtain the refined local flow output.

[0073] Furthermore, the core task of the hierarchical context flow estimation module is to generate a two-dimensional flow field, including a hierarchical flow estimator and a context flow estimator.

[0074] The hierarchical flow estimator adopts a layer-by-layer feature extraction and fusion mode, which can effectively reduce redundant computation and feature representation ability. The hierarchical flow estimator is used to generate a preliminary optical flow field through layer-by-layer feature extraction and fusion, including 5 convolutional layers and 1 flow field prediction layer.

[0075] The first convolutional layer maps the multi-level features of the input to a 128-dimensional feature space, providing a rich information foundation for subsequent feature processing and fusion;

[0076] The second convolutional layer further extracts deep features while maintaining the number of feature channels;

[0077] In the 3rd to 5th convolutional layers, each layer reduces the number of feature channels from 128 to 32 through feature concatenation and progressive dimensionality reduction. The concatenation operation can combine features from different layers, enhance the expressive power of the module, and preserve the features of earlier layers, avoiding information loss during the convolution process.

[0078] The flow field prediction layer uses a single convolution operation to map the concatenated features into a two-dimensional flow field, which represents the displacement of each pixel.

[0079] Furthermore, the context flow estimator aims to extract multi-scale contextual information from the input features to generate a refined optical flow field. By cascading multiple convolutional layers with different receptive fields, spatial information is gradually fused, and the final output is the predicted refined optical flow field.

[0080] The context flow estimator is used to extract multi-scale contextual information from the input multi-level features to generate a predicted refined optical flow field.

[0081] To capture feature information under different receptive fields, the context flow estimator uses a series of convolution operations. The kernel size of the context flow estimator is fixed at 3×3, and the dilation rate is set to 1, 2, 4, 8, 16, and 1, respectively.

[0082] The context flow estimator comprises seven convolutional layers, with the first four layers forming a multi-scale feature extraction part and the last three layers forming a refined flow prediction head.

[0083] The contextual features obtained from the multi-scale feature extraction part are input into the refined flow prediction head to further generate the final optical flow field.

[0084] The convolutional layer is used to progressively fuse spatial information to extract multi-scale contextual information;

[0085] In deep learning, the convolution process is equivalent to extracting relevant information from an image full of information points through the movement of the convolution kernel. The information provided by the relative positions, geometric structures, contours, textures, and other relationships between pixels in the input image feature map is spatial information. Dilated convolution expands the receptive field by setting holes in the convolution kernel, capturing more spatial information. Different dilation rates extract different information. The first four dilation rates are used to extract different features, and the final dilation rate of 16 extracts global information. Through this gradual extraction and fusion, the network obtains a feature map rich in spatial information from all levels, from details to the global picture. Subsequently, based on this feature map that has fused rich spatial information, the motion of each pixel, i.e., optical flow, is predicted. Therefore, it is called fusion of spatial information.

[0086] The refined flow prediction head is used to receive multi-scale context information and generate a refined optical flow field.

[0087] Furthermore, the dilation rate of the first convolution in the refined flow prediction head is set to 16, and its output is used to provide information support across a wide range.

[0088] The dilation rate of the second convolution in the refined flow prediction head is set to 1 to focus on refining local details and the final refined flow prediction. This enables progressive feature integration from global to local, improving the accuracy of flow estimation.

[0089] Iterative attention refining module, such as Figure 2 As shown: The iterative attention refinement module is used to perform feature fusion on the refined optical flow information after clothing feature warping, and then splice it with the character features to obtain a global fused feature. This is then used to generate and refine the attention map through multiple iterations, so as to gradually enhance the feature representation of important regions and suppress the interference of irrelevant regions to obtain a global clothing analysis map.

[0090] An iterative attention mechanism is employed to progressively enhance the feature representations of important regions and suppress interference from irrelevant regions, thereby strengthening the focus on small, important parts. The core idea of ​​the iterative attention refinement module is to generate and refine the attention map through multiple iterations to progressively enhance the feature representations of important regions and suppress interference from irrelevant regions.

[0091] First, the clothing features are copied three times. Then, the refined local flow output by the local flow module is used to warp the three clothing features respectively, thus obtaining locally warped clothing features. The locally warped clothing features are then fused through a convolutional layer to obtain global clothing features. The global clothing features are then concatenated with the character features to obtain global fused features. The global fused features are then input into an iterative attention refinement layer, and finally, the global clothing parsing is output.

[0092] Furthermore, the iterative attention refining module includes multiple iterative attention layers;

[0093] The iterative attention layer is formed by multiple attention layers. Each attention layer generates an attention map and weights the input features according to the current attention map. After multiple iterations, the network can gradually adjust the attention of each region, thereby improving the accuracy of feature representation and the performance of the model.

[0094] The attention layer consists of multiple convolutional layers with the same structure, and each convolutional layer consists of two convolutional operations and an activation function.

[0095] The attention layer is used to generate an attention map and adjust the input features according to the attention map. After multiple iterations, the network can gradually adjust the attention of each region, thereby improving the accuracy of feature representation and the performance of the model.

[0096] Furthermore, when adjusting the input features based on the current attention map, a multiplicative weighting adjustment method is used. Multiplicative weighting can dynamically amplify or reduce certain features according to the weights, or even completely suppress some unimportant features. For the attention mechanism, this weighting can more significantly highlight important regions while ignoring irrelevant regions.

[0097] Two-stage virtual fitting models typically use Res-UNet-based generators to synthesize fitting results, but they still have problems such as insufficient ability to capture local details, insufficient attention to salient areas, and insufficient ability to fuse potential features. As a result, the fitting results may be blurry and distorted in the details of the connection between clothing and the human body (such as cuffs and necklines) and texture restoration.

[0098] In this application, the dense fusion generation module is used to match the global clothing analysis map to the person image to perform virtual try-on and obtain a clothing matching image. This is achieved by introducing DenseNet style connections and SE attention mechanisms into the residual blocks, such as... Figure 4 As shown, feature reuse and gradient flow are enhanced, and channel feature weights are dynamically calibrated. At the same time, a hybrid feature fusion layer is combined to integrate low-level texture and high-level semantic features, thereby improving the ability to generate details.

[0099] Therefore, in order to enhance the ability to capture local details and increase attention to salient regions, the dense fusion generation module of this application is based on the Res-UNet network and introduces DenseNet-style connections and SE attention mechanism in the residual block;

[0100] Through DenseNet-style connections, features from each layer can be directly passed to the final output, thereby improving the efficiency of feature reuse. Furthermore, through concatenation operations, efficient fusion of multi-level features is achieved, enhancing the ability to model local details and global structures.

[0101] DenseNet-style connections provide multi-level features, and the introduction of an attention mechanism can more accurately allocate weights. Therefore, the introduction of the SE attention mechanism significantly enhances the ability to focus on key features by weighting the importance of the channel dimension, effectively improving the network's feature representation ability, especially in complex scenes where it pays more attention to key regions.

[0102] Meanwhile, to enhance the latent feature fusion capability, a hybrid feature fusion layer is set in the innermost layer of Res-UNet. The hybrid feature fusion layer captures multi-scale spatial context information of input features through a combination of various convolutional kernel sizes, thereby improving the model's feature extraction capability, enhancing its adaptability to different receptive fields, and preserving global and local information during feature fusion.

[0103] The hybrid feature fusion layer is as follows Figure 5 As shown, the input features are processed by convolution kernels of different sizes, the resulting features are then added together point by point and fused, and finally the output is obtained by processing with a non-linear activation function.

[0104] Furthermore, the dense fusion generation module includes four different convolution kernel sizes: 1×1, 3×3, 5×5, and 7×7, each corresponding to a different receptive field.

[0105] Convolutions with a 1×1 kernel size are used for cross-channel information integration, which can quickly reduce the dimension of the feature map or adjust the number of channels;

[0106] Convolutions with a 3×3 kernel size are used to capture small local features and are suitable for extracting fine-grained edge information;

[0107] Convolutions with a 5×5 kernel size are used to expand the receptive field and capture contextual information at a medium scale.

[0108] Convolutions with a 7×7 kernel size are used to cover a larger receptive field, capturing global context, while fusion is performed by direct addition, preserving the feature contribution of each receptive field.

[0109] Multiplicative weighting can dynamically amplify or reduce certain features based on their weights, or even completely suppress some unimportant features. For attention mechanisms, this weighting can more significantly highlight important regions while ignoring irrelevant regions.

[0110] Example 2

[0111] From the code perspective, the data flow of the refined virtual try-on system model, which features hierarchical context iteration and dense fusion, is as follows:

[0112] The feature pyramid network extracts features at five scales and is trained in a multi-scale loop during warp module training, starting from the smallest scale and traversing each scale.

[0113] Taking a specific scale as an example, firstly, for the hierarchical context flow estimation module, the clothing feature x_warp and the character feature x_cond at the current scale are obtained. x_warp and x_cond are then copied three times along the batch dimension to obtain x_warp_concate and x_cond_concate. If a flow field last_flow from the previous scale exists, it is used to sample x_warp_concate to obtain x_warp_after; otherwise, x_warp_concate is directly assigned to x_warp_after. Next, the correlation between x_warp_after and x_cond_concate is calculated to obtain tenCorrelation. tenCorrelation is then divided into three parts (left sleeve, torso, right sleeve), namely left_tenCorrelation, torso_tenCorrelation, and right_tenCorrelation. Each part is input into the hierarchical flow prediction module (flow_estimator) to predict the local coarse flow field (left_flow, torso_flow, right_flow), and the three local flows are concatenated into a flow. If a previous-scale flow field (last_flow) exists, the current flow field flow is used to sample and update the previous-scale flow field last_flow; otherwise, the flow is directly assigned to last_flow. The current flow field last_flow is used to sample and update x_warp_concate. The x_warp_concate and x_cond_concate from different batches are concatenated to obtain the concatenation features (left_concat, torso_concat, right_concat) for each part. These concatenation features are input into the context flow prediction module (context_networks) to predict the refined flow field (left_flow, torso_flow, right_flow), and the three local refined flows are concatenated into a flow. The current flow field flow is then used again to sample and update the last_flow.

[0114] Secondly, for the iterative attention refinement module, the clothing feature x_warp at the current scale is copied three times to update x_warp. Then, the final refined flow obtained from the hierarchical context flow estimation module is used to sample and update x_warp. x_warp is split into three local features (x_warp_left, x_warp_torso, x_warp_right). Each local feature is multiplied by its corresponding edge feature and the preserved region to obtain new x_warp_left, x_warp_torso, and x_warp_right. The three parts are concatenated and then used to perform feature fusion and update x_warp through a convolutional layer. x_warp is then concatenated with the character feature x_cond and input into the iterative attention refinement layer to obtain the global clothing parsing.

[0115] This module contains four different convolutional kernel sizes: 1×1, 3×3, 5×5, and 7×7, each corresponding to a different receptive field. The 1×1 convolution is used for cross-channel information integration, which can quickly reduce the dimensionality of the feature map or adjust the number of channels; the 3×3 convolution is used to capture small-scale local features, suitable for extracting fine-grained edge information; the 5×5 convolution is used to expand the receptive field and capture medium-scale contextual information; and the 7×7 convolution is used to cover a larger receptive field and capture global context. At the same time, the features are fused by direct addition, preserving the feature contribution of each receptive field.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical methods of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical methods of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical methods to deviate from the spirit and scope of the technical methods of the present invention.

Claims

1. A hierarchical context iterative and dense fusion refined virtual fitting system, characterized in that, The refining virtual fitting system takes a clothing picture, a clothing mask, a clothing parse map, a figure picture, a figure dense pose, a figure key point and a figure parse map as inputs, adopts a self-supervised training manner, and comprises a hierarchical context iterative refining warping module and a dense fusion generation module. The in-store clothing is warped according to the pose of the figure picture through the hierarchical context iterative refining warping module. The warping process is to extract features, estimate optical flow information, warp the clothing picture in parts according to the estimated optical flow information, and complete splicing of the parts under the guidance of the global clothing parse output by the iterative attention refining module to form a complete warped clothing. The warped clothing is worn on the figure picture through the dense fusion generation module to output a picture of a well-fitted clothing. The hierarchical context iterative refining warping module is used to warp a target clothing according to a figure image to obtain a global clothing parse map. The dense fusion generation module is used to match the global clothing parse map to the figure image to obtain a clothing matching picture. The hierarchical context iterative refining warping module comprises: a feature pyramid network, which is used to extract multi-level features and is provided with two feature pyramid networks, i.e., a figure feature pyramid network used to extract multi-scale figure features of the figure image and a clothing feature pyramid network used to extract multi-scale clothing features of the target clothing; the figure feature pyramid network takes a human body pose, a human body dense pose and a reserved region mask as inputs and outputs multi-scale figure features; the clothing feature pyramid network takes a clothing and a corresponding clothing parse map as inputs and outputs multi-scale clothing features; the multi-scale figure features and the multi-scale clothing features constitute the multi-level features; a hierarchical context flow estimation module, which is used to extract optical flow information, i.e., a preliminary optical flow field, from the multi-level features, optimize the optical flow information by using context information, and then perform feature integration to obtain refined optical flow information, i.e., a refined optical flow field; an iterative attention refining module, which is used to perform feature fusion after clothing feature warping of the refined optical flow information, splice the global fusion features obtained by the feature fusion with figure features, generate and refine an attention map multiple times, gradually enhance feature representation of important regions, suppress interference of irrelevant regions, and obtain a global clothing parse map; the dense fusion generation module is based on a Res-UNet network and introduces a DenseNet style connection and an SE attention mechanism in a residual block; through the DenseNet style connection, features of each layer can be directly transmitted to the final output; in the innermost layer of the Res-UNet, a mixed feature fusion layer is arranged, which captures multi-scale spatial context information of input features through combination of multiple convolution kernel sizes; the mixed feature fusion layer operates on the input features through different size convolution kernels, then point-by-point adds and fuses the obtained features, and finally processes the final output through a nonlinear activation function.

2. The hierarchical context iterative and dense fusion refined virtual fitting system according to claim 1, wherein, The hierarchical context flow estimation module comprises a hierarchical flow estimator and a context flow estimator; The hierarchical flow estimator is configured to generate a preliminary optical flow field by layer-by-layer feature extraction and fusion, and comprises five convolutional layers and one flow field prediction layer; The first convolutional layer maps the input multi-level features to a 128-dimensional feature space, providing a rich information base for subsequent feature processing and fusion; The second convolutional layer further extracts deep features while maintaining the number of feature channels; The third to fifth convolutional layers each reduce the number of feature channels from 128 to 32 through feature concatenation and step-by-step dimension reduction. The concatenation operation combines features from different layers, enhancing the expression ability of the module and preserving early layer features to avoid information loss during layer-by-layer convolution. The flow field prediction layer maps the concatenated features to a two-dimensional flow field through a convolution operation, which represents the displacement of each pixel.

3. The hierarchical context iterative and dense fusion refined virtual fitting system according to claim 2, wherein, The context flow estimator is configured to extract multi-scale context information from the input multi-level features to generate a predicted refined optical flow field; The kernel size of the context flow estimator is fixed at 3x3, and the dilation rates are set to 1, 2, 4, 8, 16, and 1, respectively. The context flow estimator comprises seven convolutional layers, four of which constitute a multi-scale feature extraction part, and three of which constitute a refined flow prediction head. The convolutional layers are used to gradually fuse spatial information to extract multi-scale context information. The refined flow prediction head is configured to receive multi-scale context information and generate a refined optical flow field.

4. The hierarchical context iterative and dense fusion refined virtual fitting system according to claim 3, wherein, The dilation rate of the first convolution in the refined flow prediction head is set to 16, and its output is used to provide support for a large range of information. The dilation rate of the second convolution in the refined flow prediction head is set to 1 to focus on refining local details and the final refined flow prediction.

5. The hierarchical context iterative and dense fusion refined virtual fitting system according to claim 4, wherein, The iterative attention refinement module comprises an iterative attention layer. The iterative attention layer is formed by iterating multiple attention layers, each of which generates an attention map and weights the input features based on the current attention map. The attention layer is composed of multiple convolutional layers with the same structure, each of which consists of two convolutional operations and an activation function. The attention layer is configured to generate an attention map and weight the input features based on the current attention map.

6. The hierarchical context iterative and dense fusion refined virtual fitting system according to claim 5, wherein, When weighting the input features based on the current attention map, the multiplication weighting adjustment method is selected.

7. The hierarchical context iterative and dense fusion refined virtual fitting system according to claim 1, wherein, The dense fusion generation module contains four different convolution kernel sizes, namely 1x1, 3x3, 5x5, and 7x7. The 1x1 convolution kernel size is used for cross-channel information integration. The 3x3 convolution kernel size is used to capture small-range local features. The 5x5 convolution kernel size is used to expand the receptive field once. The 7x7 convolution kernel size is used to expand the receptive field twice to capture global context and fuse through direct addition.

Images