A virtual fitting method and a virtual fitting image generation method of fusing a multi-view and a multi-level sampling distortion module

Abstract

This invention belongs to the field of virtual reality and discloses a virtual try-on method that integrates multi-viewpoint and multi-level sampling distortion modules. Specifically, it includes: multi-viewpoint image acquisition and preprocessing; extracting multi-angle human body features using a multi-branch residual fusion network and achieving spatial consistency fusion through a cross-attention mechanism; inputting human body features and clothing images into a multi-level sampling distortion module to perform local and global deformation on different parts of the clothing; using a BoTNet-based generator to generate high-fidelity virtual try-on images through a multi-head self-attention mechanism and a bottleneck residual structure; and finally, achieving multi-angle visualization through a rendering module. This invention solves the problems of single viewpoint, unnatural clothing deformation, and low texture fidelity in existing virtual try-on technologies, achieving a virtual try-on effect that is natural-fitting, detailed, and can be displayed from multiple angles, with high real-time performance and broad application prospects.

Description

Technical Field

[0001] This invention belongs to the field of virtual reality, and particularly relates to a virtual try-on method and a virtual try-on image generation method that integrates multi-view and multi-level sampling distortion modules. Background Technology

[0002] In recent years, the rapid development of e-commerce has led to a significant increase in online clothing purchase demand, and users are increasingly seeking immersive and realistic virtual try-on experiences. As a result, virtual try-on technology has become a research hotspot. Existing virtual try-on methods are mainly divided into four categories: image-based, 3D model-based, generative adversarial networks (GANs)-based, and diffusion model-based algorithms.

[0003] Image-based algorithms achieve a virtual try-on effect by overlaying clothing onto a person's image, offering convenient deployment but limited by a single viewpoint and struggling to handle complex poses and occlusions. 3D model-based algorithms can render the three-dimensional structure and details of clothing, but require multi-view data and robust hardware, resulting in high training costs. GAN-based algorithms utilize adversarial training between the generator and discriminator, generating natural-looking results, but are still prone to distortion under complex poses and textures. Diffusion model-based algorithms excel in detail fidelity and body shape adaptation, but are computationally intensive, slow, and difficult to apply in real-time.

[0004] Existing virtual fitting technologies generally face the following engineering bottlenecks in practical industrial deployments: First, single-view or weak multi-view modeling cannot accurately restore the true 3D contours and posture changes of the human body, resulting in inconsistencies between the clothing fit position and the human body structure. This is especially true in non-frontal postures such as sideways or turning, where clothing is prone to floating, misalignment, and proportional distortion. Second, existing clothing deformation methods based on overall geometric mapping cannot simultaneously take into account large-scale posture changes and fine-grained local structures such as clothing wrinkles, collars, and cuffs, causing blurring, breakage, or semantic mismatch in clothing textures during stretching. Third, generative networks rely heavily on local convolutional operations, making it difficult to model the long-range dependencies and cross-regional consistency of clothing textures, leading to pattern breaks, edge artifacts, and visual discontinuities. Fourth, in low-to-medium computing power environments, existing methods cannot meet real-time interaction requirements while ensuring resolution and realism, thus limiting their deployment capabilities in scenarios such as e-commerce shopping guides, offline fitting mirrors, and self-service terminals. Summary of the Invention

[0005] The purpose of this invention is to propose a virtual try-on method that integrates multiple user perspectives and multi-level sampling distortion modules, addressing issues such as single perspective, unnatural clothing deformation, insufficient texture preservation, and poor pose adaptability in existing virtual try-on technologies. This invention significantly improves the realism, structural consistency, and detail fidelity of the generated virtual try-on images through multi-view feature fusion, local-global multi-level distortion modeling, and the introduction of a BoTNet attention generator.

[0006] This invention is implemented as follows: a virtual try-on method integrating multi-view and multi-level sampling distortion modules, specifically including:

[0007] S1, Multi-view image acquisition and preprocessing, acquires multi-view images of the user from the front, side, back and oblique sides, performs size normalization, illumination compensation and background segmentation on the input images, and obtains a set of multi-view human images.

[0008] S2, Multi-branch residual fusion network feature extraction: The multi-view human image is input into the multi-branch residual fusion network. Each branch extracts local features based on the residual structure and fuses the features from each angle through a cross-attention mechanism to obtain the fused multi-view human feature map.

[0009] S3, Multi-level sampling distortion module processing, inputs the image of the clothing to be tried on and the multi-view human body feature map into the multi-level sampling distortion module, uses multiple local geometric distortion modules to estimate the local deformation of different clothing areas respectively, and combines the global pose information to achieve overall deformation fusion to obtain the clothing deformation result image;

[0010] S4, Image generation based on BoTNet generator: Input the clothing deformation result image, human body analysis image, skin color image and preserved area image into BoTNet generator, extract texture and edge features through multi-head self-attention mechanism and bottleneck residual structure to generate virtual fitting image;

[0011] S5, the virtual fitting result output and visualization display, inputs the generated virtual fitting image into the rendering module, and realizes multi-angle visualization display through 3D reconstruction and stitching, outputting the virtual fitting effect.

[0012] Furthermore, each branch of the multi-branch residual fusion network is built based on the ResNet structure, which uses skip connections to preserve low-level texture information and achieves multi-view spatial alignment through attention weights in the feature fusion stage.

[0013] To ensure that the local deformation is semantically consistent with the human body region, a semantic classification constraint is introduced, expressed by formula (1):

[0014] (1)

[0015] Furthermore, the multi-level sampling distortion module includes several local geometric distortion modules, each responsible for the local deformation of different parts of the upper and lower body. The local and global features are finally fused to generate a complete distorted clothing image, expressed by formula (2):

[0016] (2)

[0017] Furthermore, the BoTNet generator introduces a multi-head self-attention mechanism in the bottleneck layer, and its computation process includes the following steps:

[0018] (1) Calculate the query matrix Q, key matrix K, and value matrix V of the input features;

[0019] (2) Calculate attention weights based on the Softmax function;

[0020] (3) Attention output is obtained through matrix multiplication.

[0021] Furthermore, the BoTNet generator employs a bottleneck residual structure to reduce the number of network parameters, specifically including a combination of convolutional layers, normalization layers, and activation layers, which can reduce memory usage and improve inference speed while ensuring image quality.

[0022] Furthermore, the fitting result output module uses a combination of 3D reconstruction and image stitching to generate a fitting image with a rotatable viewpoint, and achieves 360° visualization display through an interactive interface.

[0023] Furthermore, it further includes a model training step, wherein the model training employs a comprehensive loss function composed of structural similarity loss, texture preservation loss, and perceptual consistency loss.

[0024] Furthermore, when the method is run on a low-to-medium computing power platform, the single-frame generation delay does not exceed 1 second, and the output image resolution reaches 512×512 pixels, which can achieve a real-time virtual try-on effect.

[0025] Another objective of this invention is to provide a virtual fitting system that integrates multi-view and multi-level sampling distortion modules for implementing the virtual fitting method, specifically comprising:

[0026] (1) Input acquisition module, used to acquire and preprocess user multi-view human body images;

[0027] (2) Feature extraction module, used to extract multi-angle human body features through a multi-branch residual fusion network;

[0028] (3) Clothing distortion module, used to achieve local and global deformation of clothing images by adopting a multi-level sampling distortion structure;

[0029] (4) Generation module, used to generate virtual fitting images based on the BoTNet generator;

[0030] (5) Display module, used to output and display the generated virtual try-on results.

[0031] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0032] This invention establishes a collaborative relationship between human semantic structure constraints and local geometric deformation constraints during the clothing deformation process, so that the clothing deformation result is consistent with the human body region in terms of semantic structure and spatial structure, thereby achieving structural matching and visual consistency between clothing and human body. The method includes at least the following mechanisms.

[0033] This invention presents a virtual fitting image generation method and system that integrates multi-viewpoint and multi-level sampling distortion modules. Employing deep learning, it achieves high-fidelity virtual fitting, solving the problems of single viewpoint, unnatural clothing deformation, and low texture fidelity in existing technologies. By capturing multi-angle human body features through a multi-branch residual fusion network, achieving local and global adaptive deformation through a multi-level sampling distortion module, and enhancing texture detail through a BoTNet generator, it generates fitting images with strong structural consistency, natural deformation, and consistent multi-viewpoint performance. This method is computationally efficient, can run in real-time on low-to-medium computing power devices, and has good practicality and promotional value, making it widely applicable in e-commerce, fashion design, human-computer interaction, and virtual character generation.

[0034] This invention addresses the problems of inconsistency between clothing deformation and human body structure, difficulty in aligning body parts from different perspectives, and conflicts between local fine deformation and overall spatial stability in existing virtual try-on processes. It introduces a multi-view human body structure representation and semantic structure constraint mechanism, enabling the same body part to form a unified and stable structural representation from different perspectives. Based on this, a semantic correspondence is established between the clothing area and the human body area, thus constraining clothing deformation within a semantically consistent human body area. Simultaneously, the clothing deformation process is decoupled into multiple local geometric deformations and a global geometric constraint process. By uniformly modulating each local deformation using overall posture information, problems such as overall contour drift, proportional distortion, and structural misalignment caused by local deformation are effectively avoided. This allows the clothing to maintain a precise fit while preserving overall spatial consistency, fundamentally improving the stability, controllability, and cross-posture adaptability of clothing deformation.

[0035] This invention further addresses common issues in virtual try-on generation, such as clear local textures but broken overall textures, inconsistent styles in distant regions, and gradual loss of details during multi-layer semantic processing. It introduces a collaborative mechanism of global correlation modeling and residual representation to establish correspondences between distant regions at the feature level, achieving consistent propagation of texture style and structural semantics in the overall space. At the same time, residual representation preserves the original edges and detailed textures during multi-layer feature transmission, ensuring that high-level semantics do not overwrite low-level structural information. This maintains the stability and continuity of local details while ensuring overall consistency, significantly improving the realism, coherence, and visual naturalness of virtual try-on images.

[0036] By systematically integrating mechanisms such as multi-view structural modeling, semantic region mapping, multi-level geometric deformation and global constraint coordination, as well as global correlation and residual fusion, this invention constructs a virtual fitting processing framework with a closed-loop structure, coordinated functions, and clear hierarchy. This framework enables clothing structure matching, deformation control, and image generation to work collaboratively under the same constraint system. It not only significantly improves the stability and consistency of fitting results under complex postures, complex clothing structures, and multi-view conditions, but also enhances the system's engineering feasibility, scalability, and practical application value. Compared with existing technologies, it has achieved significant progress in structural consistency, visual realism, and system reliability. Attached Figure Description

[0037] Figure 1 This is a general framework diagram of the virtual try-on method that integrates multi-view and multi-level sampling distortion modules provided in an embodiment of the present invention;

[0038] Figure 2 M-LSWM network structure diagram of the virtual try-on method integrating multi-view and multi-level sampling distortion modules provided in this embodiment of the invention.

[0039] Figure 3 The diagram shows the BotNet network structure of the virtual try-on method that integrates multi-view and multi-level sampling distortion modules provided in this embodiment of the invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative.

[0041] This method introduces a multi-level collaborative mechanism among the perception layer, geometric deformation layer, and generation layer, enabling the three stages of human body structure understanding, clothing geometric transformation, and image generation to form a closed-loop constraint and mutual feedback relationship.

[0042] In the perception layer, multi-view human images are simultaneously input into a multi-branch residual fusion network. Each branch models the human contour, limb boundaries, and local details from different perspectives. The residual structure ensures that low-level texture information is not lost during deep semantic extraction, while the cross-attention mechanism achieves weighted alignment of cross-view information in the feature fusion stage, enabling corresponding human regions from different perspectives to form a consistent representation in the feature space, thereby constructing a human feature map with spatial and semantic consistency.

[0043] At the geometric layer, the multi-level sampling distortion module no longer uses a single overall deformation field. Instead, based on the human semantic parsing results, it divides the clothing into multiple semantically consistent regions, such as the upper body, sleeves, and lower body, and configures an independent local geometric distortion module for each type of region. Each local module estimates a local displacement field to match the corresponding human geometric changes in that region. At the same time, a global pose constraint field is introduced to constrain the overall clothing contour, ensuring that local detail deformations remain geometrically consistent with overall structural changes. Semantic classification constraints ensure that local deformations always occur within the semantically corresponding region, avoiding cross-semantic mismatches such as mapping collar textures to the torso or cuffs to the lower body.

[0044] In the generation layer, the BoTNet-based generation network introduces a multi-head self-attention mechanism through a bottleneck layer, enabling the network to model long-distance dependencies between clothing textures globally, thereby ensuring the spatial continuity and consistency of stripes, checks, and patterns. The bottleneck residual structure maintains expressive power while controlling the parameter scale, enabling the model to achieve high-quality inference on low-to-medium computing power platforms.

[0045] Finally, the generated results are consistently stitched together from multiple perspectives through the rendering and 3D reconstruction modules, enabling users to obtain a continuous and rotatable virtual try-on effect during the interaction process, thereby meeting the needs of commercial display and user experience.

[0046] The multi-view residual fusion network provides a highly consistent human semantic and spatial structure benchmark for the geometric distortion module, enabling subsequent geometric deformation to be based on a stable understanding of human structure rather than directly relying on the original image pixels, fundamentally reducing the uncertainty caused by noise, occlusion and viewpoint changes.

[0047] The multi-level sampling distortion module decomposes clothing deformation into multiple semantically consistent local geometric mapping tasks and introduces global posture constraints to achieve overall consistency. This makes the local high-degree-of-freedom deformation and the overall low-degree-of-freedom posture change complementary, thus taking into account both realistic fit and geometric stability.

[0048] The BoTNet generator does not perform geometric correction but focuses on texture restoration, edge reconstruction, and visual consistency enhancement, thus decoupling the geometric structure from the visual representation, thereby reducing the difficulty of network training and improving stability.

[0049] The 3D reconstruction and stitching module utilizes the geometrically consistent results output by the aforementioned modules to make the rendering results from different perspectives naturally close in space, thereby forming an interactive, multi-perspective consistent virtual try-on experience.

[0050] The modules mentioned above form a hierarchical collaborative relationship through semantic constraints, spatial alignment, geometric constraints, and attention modeling, enabling the system to form a continuous constraint chain at the three levels of structural understanding, deformation modeling, and image generation, thereby significantly improving the realism, consistency, and stability of the fitting results.

[0051] This invention achieves stable human body modeling and clothing fitting using only multi-view images captured by ordinary camera equipment without relying on high-precision 3D scanning equipment. While ensuring 512×512 resolution output quality, it can achieve sub-second single-frame generation, making it suitable for e-commerce online fitting, offline smart fitting mirrors, mobile virtual fitting, and digital human modeling scenarios. It can also significantly reduce common industrial problems such as clothing misalignment, texture breakage, and contour drift, improving the usability and reliability of virtual fitting systems in real commercial environments.

[0052] This invention achieves a shift in virtual try-on from the image processing level to the structural understanding level through the collaborative design of structured semantic constraints, multi-level geometric modeling, and global attention generation mechanism. This balances engineering deployability and visual realism, and has clear industrial practical value.

[0053] This invention provides a virtual try-on method based on a fusion of user multi-view and multi-level sampling distortion module, the overall process of which is as follows: Figure 1 As shown, it specifically includes three core steps: multi-view feature construction, local-to-global clothing distortion modeling, and high-fidelity image generation based on BoTNet.

[0054] Step 1: Construct a multi-perspective user input model

[0055] First, multi-view images of the user are read, including different angles such as front view, left micro-side view, side view, right micro-side view, and rear view. Each view image is then input into a multi-branch residual fusion network to achieve joint modeling of human features from multiple perspectives.

[0056] MB-RFN comprises several sub-networks based on residual structures. Each sub-network processes an independent viewpoint, extracting low-level texture features through convolutional layers and employing residual learning to maintain consistency between shallow and deep features. Subsequently, the features generated by each branch are aligned through a cross-attention fusion module, ensuring consistency in the feature space for corresponding regions from multiple viewpoints, such as the shoulders, waist, and limbs.

[0057] By using a jump connection structure, the underlying texture details are fused with the high-level semantic features, thereby obtaining stable and consistent multi-view fusion features of the human body, providing accurate structural information for subsequent clothing deformation processing.

[0058] Step 2: Garment deformation modeling using the Multi-Level Sampling Twist Module (M-LSWM)

[0059] like Figure 2 As shown, the multi-level sampling distortion module constructed in this invention is composed of multiple cascaded LGWM units. Each LGWM unit includes a local thinning module (Ax / Bx) and a global clothing analysis module (Cx), which are used to predict the deformation of clothing images from both local and global scales.

[0060] (1) Local scale clothing distortion prediction

[0061] To achieve a precise fit for different body areas, the garment is divided into multiple local regions (such as the upper arms, forearms, and torso of the upper body, and the thighs, calves, and waist of the lower body). The Ax module first replicates the garment features and, combined with the input local parsing features, independently estimates the local distortion amount for each of the five local regions.

[0062] The deformation offset of a local pixel is expressed by formula (1):

[0063] (1)

[0064] To ensure that the local deformation is consistent with the human body region in terms of semantic structure, a semantic classification constraint is introduced, expressed by formula (2):

[0065] (2)

[0066] The Bx module further integrates human body features and local distortion features on the basis of Ax, making the edges of local areas smoother and the texture more coherent.

[0067] (2) Global-scale clothing analysis and co-modeling (Cx module)

[0068] The Cx module utilizes the local distortion results output by Bx, and through convolutional fusion, forms global clothing parsing features and generates an overall semantic parsing map, which includes semantic labels for various regions of the upper and lower body.

[0069] The local and global features are finally fused to generate a complete distorted clothing image, which is represented by formula (3):

[0070] (3)

[0071] Through a multi-level twisting structure, the details of the garment are preserved at multiple scales, resulting in a natural and believable deformation effect.

[0072] Step 3: Generation of high-quality try-on images based on the BoTNet network

[0073] like Figure 3 As shown, this invention uses a BoTNet network to fuse the user's skin region with the distorted clothing image to generate a final high-quality try-on image. The BoTNet network takes the distorted clothing image G′, skin image J, analytical image K, and preserved region image L as input.

[0074] BoTNet employs a multi-head self-attention mechanism (MHSA) to effectively capture high-frequency information such as texture and edges. The attention weights are calculated using formula (4):

[0075] (4)

[0076] in , These are the query and key vectors, respectively. The dimension of the key vector.

[0077] Attention weights are used to guide the BoTNet network to selectively focus on key areas such as clothing texture and human skin edges, thereby improving the local texture quality of the final synthesized image and making the try-on effect more realistic and natural.

[0078] Step 4: Output the virtual try-on results

[0079] The output image generated by the BoTNet network is used as the final virtual try-on effect image.

[0080] The method of this invention can output the fitting results from multiple perspectives, displaying the overall effect of the clothing from the front, side, back and other angles, which is more suitable for e-commerce fitting and clothing display scenarios.

[0081] The following examples illustrate how the present invention is used on real datasets, but do not limit the scope of protection of the present invention.

[0082] Example 1: DressCode Dataset Try-on Experiment

[0083] Images of the top and bottom garments in DressCode were used as clothing input, and a VITON-HD image of a person was used as user input. The try-on images generated through the above steps outperformed existing FS-VTON, HR-VTON, and GP-VTON models in terms of SSIM, FID, and mIoU.

[0084] Example 2: Multi-view 3D fitting scene display

[0085] Multi-view input was generated using 3D reconstruction, and a five-view fitting effect was generated using the method of this invention. The results show that the consistency of clothing texture and edge structure is good in each viewpoint.

[0086] Example 3: Implementation of Multi-Perspective Consistent Structure Modeling and Semantic Alignment

[0087] In this embodiment, by acquiring user's frontal, left-side, right-side, and oblique side views during the acquisition phase, the human body contour is jointly modeled from different perspectives. After scale normalization and background separation of each perspective image, they are sent to a parallel structural modeling unit to extract spatial representations of key human body regions. Then, each region is weighted and fused through cross-viewpoint correspondence, so that the same human body part forms a unified structural expression under different perspectives, thereby avoiding structural deviations caused by single-viewpoint occlusion or posture changes.

[0088] Based on this, a semantic region mapping map is generated according to the aforementioned structural expression, establishing a one-to-one correspondence between the upper body, lower body, and associated regions and the corresponding regions of the clothing. All subsequent clothing deformation operations are confined to the corresponding semantic regions, thereby ensuring that local clothing deformations do not cross the boundaries of the human body structure and avoiding misalignments such as the neckline mapping to the torso or the cuffs mapping to the lower body.

[0089] Example 4: Local geometric deformation method based on semantic constraints

[0090] In this embodiment, the clothing image is divided into multiple regions according to semantic mapping, and an independent geometric deformation mapping model is established for each region, so that it only responds to the corresponding human body region. When the upper body of the human body bends or rotates, only the local geometric adjustment of the upper body clothing region is triggered, without affecting the lower body region, so that the clothing maintains structural stability while conforming to changes in the human body posture.

[0091] Local geometric mapping is always constrained by semantic regions during execution. The deformation boundary is restricted to the semantic segmentation region, so that the texture stretching will not spread out of the boundary to non-corresponding areas, thus ensuring the continuity and consistency of clothing patterns, edges and structural lines in space.

[0092] Example 5: Cooperative Implementation of Local Geometric Deformation and Global Attitude Constraints

[0093] In this embodiment, while performing local geometric deformations, an overall human posture representation is constructed to describe torso tilt, body rotation, and overall proportion changes. This overall posture representation does not directly participate in local detail deformations, but rather constrains the relative spatial relationships between the results of each local deformation, ensuring that the overall silhouette of the clothing maintains a uniform proportion after deformation in each area.

[0094] When a large deformation occurs in a local area, the global posture constraint will modulate the amplitude of the local mapping to prevent excessive local deformation from causing the overall structure of the garment to drift or become unbalanced, thus achieving a balance between fine fit and structural stability.

[0095] Example 6: Texture Consistency Generation Method Based on Global Association Modeling

[0096] In this embodiment, cross-regional relationships are established for clothing texture features during the image generation stage to ensure that texture changes between distant regions remain synchronized and consistent. For example, the direction of striped clothing remains continuous at corresponding positions on different body parts, avoiding local breaks or directional errors.

[0097] This global correlation mechanism does not change the geometric structure, but only coordinates and adjusts the texture and color distribution, so that the generated result presents a continuous and natural texture effect in the overall visual appearance, while maintaining the clarity and realism of local details.

[0098] Example 7: Detail Preservation and Stable Generation Method Based on Residual Representation

[0099] In this embodiment, a residual propagation mechanism is introduced during the generation process to directly introduce the original texture, edges and low-level detail information into the subsequent generation stage, so that this detail information is not covered or blurred during deep semantic processing.

[0100] This method can maintain high clarity in areas with rich details such as complex textures, lace edges, and print boundaries, while avoiding texture smoothing caused by multi-layer processing, thereby improving the realism of the generated results.

[0101] Example 8: Real-time Virtual Fitting Implementation for Low-to-Medium Computing Power Platforms

[0102] In this embodiment, by controlling the computational scale of the structural representation, geometric mapping and generation modules, the system can complete the fitting room generation in less than 1 second per frame on a medium-to-low computing power platform and output an image result with a resolution of 512×512.

[0103] This implementation can be deployed on e-commerce self-service terminals, shopping mall fitting mirrors, or mobile devices, enabling a real-time interactive fitting experience while ensuring image quality, thereby meeting the needs of actual commercial applications.

[0104] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A virtual fitting method based on multi-view consistent structure constraint and multi-level geometric collaborative deformation, characterized in that, include: Construct a multi-view human body structure representation so that corresponding parts of the human body under different viewpoints form a consistent structural representation in the feature space; Based on the representation of human body structure, semantic regions are divided into clothing images to form a semantic correspondence between clothing regions and human body regions; Local geometric deformation fields are calculated for different semantic regions to cause local geometric deformation of clothing within the corresponding human body region; Global geometric constraints are calculated based on the overall posture information of the human body, so that the geometric deformation of each local area remains consistent in the overall spatial structure. The local geometric deformation is synergistically fused with the global geometric constraints to generate a clothing deformation result with consistent structure.

2. The method of claim 1, wherein, The multi-view human body structure representation is obtained by jointly modeling the frontal, lateral, posterior, and oblique view images, so that the same human body part forms a unified structural representation under different viewpoints.

3. The method according to claim 1, characterized in that, The semantic region includes at least an upper body region, a lower body region, and an auxiliary region, and the local geometric deformation of each region only applies to the corresponding semantic region.

4. A method for deforming clothing images based on multi-level geometric decoupling and global constraint coordination, characterized in that, The method decomposes the garment deformation process into multiple local geometric deformation processes and a global geometric constraint process, and ensures the consistency between the precision of local deformation and the overall structure through the synergistic relationship between the two, thereby achieving stable and controllable garment deformation. The method includes at least the following mechanisms: The clothing image is divided into multiple local deformation regions according to its structural location; Calculate independent local geometric deformation mappings for each local region; Calculate global geometric constraint mapping based on overall human posture information; By using global geometric constraints to uniformly modulate the mapping of each local geometric deformation, the mapping of each local deformation remains consistent in the overall spatial relationship. The overall deformation result of the garment is generated based on the modulated local geometric deformation mapping.

5. The method according to claim 4, characterized in that, The localized deformation area includes at least the upper body area and the lower body area.

6. The method according to claim 4, characterized in that, The global geometric constraints are used to limit the overall silhouette drift and structural misalignment of the garment caused by local deformation.

7. A virtual fitting image generation method based on the synergy of global correlation modeling and residual representation, characterized in that, By introducing the synergistic relationship between a global correlation modeling mechanism and a residual representation mechanism during the generation process, the generated result simultaneously possesses long-range texture consistency and local detail stability, thereby improving the realism and continuity of the fitting image. The method includes at least the following mechanisms: Establish global correlations between input features to create consistent texture representations across distant regions; Based on residual representation, low-level texture and edge information are preserved so that detailed information is not covered by high-level semantics; The global correlation results and residual expression results are synergistically fused to generate a virtual fitting image with global consistency and local continuity.

8. The method according to claim 7, characterized in that, The global association relationship is achieved by establishing a correspondence between input features and calculating the weights between regions based on the correspondence.

9. The method according to claim 7, characterized in that, The residual representation is used to preserve the original texture structure during multi-layer feature transfer.

10. A virtual fitting system for implementing the method according to any one of claims 1 to 9, characterized in that, include: The structural modeling module is used to construct multi-view human body structure representations; The semantic mapping module is used to establish the semantic correspondence between clothing areas and human body areas; The geometry deformation module is used to perform coordinated processing of local geometric deformation and global geometric constraints; The generation module is used to perform image generation processing that combines global correlation modeling and residual representation. The output module is used to output virtual fitting images.

Images