Virtual fitting image synthesis method and system

Abstract

The application provides a virtual fitting image synthesis method, which first acquires a user human body image and a target clothing image; performs key point detection and semantic segmentation on the human body image to construct a human body posture feature space; completes main body segmentation and semantic analysis on the clothing image to construct a clothing texture feature space; and simultaneously separates and extracts background environment features to construct a background environment feature space, wherein the human body posture feature space, the clothing texture feature space and the background environment feature space are independent of each other; the application reduces the information coupling degree among the three types of features through feature decoupling constraint, establishes a clothing and human body semantic region mapping relationship and generates a dense deformation field, completes clothing texture space transformation according to the deformation field to obtain clothing candidate features; combines boundary local features to generate adaptive fusion weight masks, realizes multi-feature adaptive fusion; and finally completes image reconstruction under a consistency constraint condition, thereby stably outputting a high-quality virtual fitting image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for virtual try-on image synthesis. Background Technology

[0002] With the rapid popularization of e-commerce, smart retail, and online outfit matching experiences, virtual try-on technology has become an important technical means to improve the user shopping experience and reduce return and exchange rates. The core objective of virtual try-on image synthesis is to accurately transfer the target clothing image into the user's human body image to generate a visually realistic, structurally reasonable, naturally bordered, and scene-consistent wearing effect image, providing users with an intuitive and efficient outfit preview service.

[0003] Currently, mainstream virtual try-on image synthesis schemes generally adopt the technical route of "human body analysis - clothing deformation - image fusion", or an end-to-end generation scheme based on generative models. The typical implementation process is as follows: First, acquire the user's human body image and the target clothing image. Perform key point detection and semantic segmentation on the human body image to identify the human skeleton, skin region, original clothing region and background region. Perform main body segmentation on the clothing image to extract clothing contour, texture and pattern information. Then, perform spatial deformation on the target clothing through affine transformation, thin plate spline transformation (TPS) or optical flow estimation to make it initially adapt to the human body posture and contour. Finally, perform pixel-level blending, Poisson fusion or mask fusion on the deformed clothing region and the human body image, and complete boundary optimization through an image inpainting network to output the virtual try-on result.

[0004] However, existing virtual try-on image synthesis technologies still have the following obvious drawbacks in practical applications: high coupling of multi-source features and poor stability of the generated results. Existing virtual try-on image synthesis technologies typically encode information such as human pose, clothing texture, and background environment in a mixed manner, which leads to feature confusion during the model generation process, causing problems such as residual original clothing texture, loss of target clothing pattern, distortion of human body contours, and contamination of background areas.

[0005] In summary, the virtual try-on industry urgently needs a high-quality virtual try-on image synthesis solution that can achieve stable generation results through feature decoupling. Summary of the Invention

[0006] This invention provides a virtual fitting image synthesis method and system to achieve feature-decoupled virtual fitting image synthesis, thereby ensuring high stability of the fitting image synthesis results.

[0007] In a first aspect, embodiments of the present invention provide a virtual fitting image synthesis method, comprising: acquiring a user's human body image and a target clothing image; performing human body key point detection and human body semantic segmentation on the user's human body image to extract human posture features and construct a human posture feature space; performing clothing body segmentation and clothing semantic parsing on the target clothing image to extract clothing texture features and construct a clothing texture feature space; separating and extracting background environment features from the user's human body image to construct a background environment feature space; wherein the human posture feature space, clothing texture feature space, and background environment feature space are independent of each other; and reducing the human posture features and clothing texture features through feature decoupling constraints. The information coupling degree between the features and the background environment features; establishing a correspondence between the first clothing semantic region and the first human body semantic region, and generating a dense deformation field based on the correspondence; wherein the first clothing semantic region is extracted from the clothing texture feature space, and the first human body semantic region is extracted from the human body posture feature space; performing spatial transformation on the clothing texture features according to the dense deformation field to obtain clothing candidate features aligned to the human body dressing area; generating an adaptive fusion weight mask based on boundary local features, and adaptively fusing the clothing candidate features, human skin features, and background environment features; performing image reconstruction under the first consistency constraint, and outputting the reconstructed virtual try-on image.

[0008] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the step of reducing the information coupling between the human posture features, the clothing texture features, and the background environment features through feature decoupling constraints includes: inputting the human posture features, clothing texture features, and background environment features into independent encoding branches for feature encoding; applying structure preservation constraints to the human posture features to represent the human skeleton, contour, and posture information; applying appearance preservation constraints to the clothing texture features to represent the color, pattern, material, and pattern information of the target clothing; applying scene preservation constraints to the background environment features to represent the background lighting, shadow, color temperature, and texture information; and reducing the information overlap between the human posture features, clothing texture features, and background environment features through feature orthogonal constraints, mutual information reduction constraints, or contrastive learning constraints.

[0009] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the feature orthogonal constraint adopts the feature orthogonal regularization term to constrain the feature space basis vector, and the orthogonal constraint formula is: in, For human posture feature matrix, For clothing texture feature matrix, The background environment feature matrix, The Frobenius norm is used; the inner product norm of any two feature matrices is constrained to be no greater than 0.05, achieving a pairwise weakly correlated orthogonal distribution of the three feature spaces.

[0010] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, establishing the correspondence between the first clothing semantic region and the first human semantic region, and generating a dense deformation field based on the correspondence, includes: mapping the clothing neckline region to the human neck and upper chest region; mapping the clothing shoulder line region to the region connecting the left and right shoulders of the human body; mapping the clothing cuff region to the human wrist or forearm region; mapping the clothing waistline region to the human waist region; mapping the clothing hem region to the lower boundary region of the human torso; and generating a dense deformation field from clothing texture features to the human target region based on the above correspondence, the location of human key points, the boundary of the human region, and the boundary of the clothing region.

[0011] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the step of generating an adaptive fusion weight mask based on boundary local features and adaptively fusing the clothing candidate features, human skin features, and background environment features includes: determining the neckline boundary, cuff boundary, hem boundary, and outer contour boundary of the clothing as the synthesis boundary; calculating the color difference, texture gradient, and boundary curvature within local windows on both sides of the boundary; generating a pixel-wise fusion weight mask with a value range of 0 to 1 based on the color difference, texture gradient, and boundary curvature; and fusing the clothing candidate features, human skin features, and background environment features based on the adaptive weight mask.

[0012] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the step of fusing clothing candidate features, human skin features, and background environment features according to an adaptive weight mask includes: using the weight value of the corresponding pixel in the adaptive weight mask as the fusion coefficient, weighted summing and fusing the clothing candidate features, human skin features, and background environment features at corresponding pixel positions; reducing the smoothing fusion weight for clothing detail areas such as buttons, pockets, and prints to preserve clothing texture details; increasing the smoothing fusion weight for areas with boundary curvature greater than a preset curvature threshold to eliminate edge jaggedness; and expanding the boundary transition width for boundary areas with color differences greater than a preset color difference threshold to achieve natural color transition.

[0013] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the adaptive fusion weight mask is output by a dynamic edge-aware network, and the mask weights satisfy: Where σ is the Sigmoid function, γ∈0.3~0.7, δ∈0.2~0.5; |Ig−Is| is the boundary color difference, and ||∇I (x,y)|| is the texture gradient, which makes the width of the boundary transition region adaptively range from 3 to 12 pixels.

[0014] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the first consistency constraint includes: human body structure consistency constraint, which ensures that the proportion of the human skeleton, shoulder width, and arm length of the generated image are consistent with the input user's human body image; clothing texture consistency constraint, which ensures that the clothing color, pattern, and material texture of the generated image are consistent with the target clothing image; semantic region consistency constraint, which ensures that the first clothing semantic region and the first human body semantic region are aligned; boundary smoothness consistency constraint, which eliminates seams and breaks between clothing and skin and background; and lighting consistency constraint, which adjusts the brightness, shadow direction, and color temperature of the clothing area according to the background environment characteristics.

[0015] Optionally, in the virtual fitting image synthesis method of this embodiment of the invention, the step of spatially transforming the clothing texture features according to the dense deformation field to obtain clothing candidate features aligned to the human body changing area includes: in a first pose, using affine transformation or thin plate spline transformation to complete the overall alignment of the clothing with the clothing candidate features of the human body changing area; in a second pose including side-facing and / or raising hands and / or occlusion, using dense deformation field to perform pixel-level resampling of the clothing texture features to achieve local precise fitting and complete the clothing candidate features of the clothing to the human body changing area.

[0016] Secondly, embodiments of the present invention provide a virtual fitting image synthesis system, comprising: an image input module configured to acquire a user's human body image and a target clothing image; a multi-dimensional feature encoding module configured to perform human body key point detection and human body semantic segmentation on the user's human body image to extract human posture features and construct a human posture feature space, perform clothing body segmentation and clothing semantic parsing on the target clothing image to extract clothing texture features and construct a clothing texture feature space, and separate and extract background environment features from the user's human body image to construct a background environment feature space; wherein the human posture feature space, clothing texture feature space, and background environment feature space are independent of each other; and a feature decoupling constraint module configured to reduce the human posture features and clothing texture features through feature decoupling constraints. The information coupling degree between the features and the background environment features; a spatial alignment module, configured to establish a correspondence between a first clothing semantic region and a first human semantic region and generate a dense deformation field based on the correspondence, wherein the first clothing semantic region is extracted from the clothing texture feature space and the first human semantic region is extracted from the human pose feature space; and to perform spatial transformation on the clothing texture features according to the dense deformation field to obtain clothing candidate features aligned to the human dressing area; a boundary adaptive fusion module, configured to generate an adaptive fusion weight mask based on boundary local features and adaptively fuse the clothing candidate features, human skin features and background environment features; and an image reconstruction module, configured to perform image reconstruction under a first consistency constraint and output a virtual try-on image.

[0017] This invention constructs a human pose feature space, a clothing texture feature space, and a background environment feature space from human and clothing images, ensuring that these three spaces are independent of each other. The invention reduces the information coupling between these three types of features through feature decoupling constraints, establishes a semantic region mapping relationship between clothing and the human body, and generates a dense deformation field. Based on the deformation field, a clothing texture space transformation is performed to obtain candidate clothing features. An adaptive fusion weight mask is generated by combining local boundary features to achieve adaptive fusion of multiple features. Finally, image reconstruction is completed under consistency constraints, thereby ensuring high-stability output of high-quality virtual try-on images. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the virtual fitting image synthesis method provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the virtual fitting image synthesis system provided in Embodiment 2 of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0022] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0023] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0024] In this embodiment of the invention, a human pose feature space is constructed by performing key point detection and semantic segmentation on human body images; a clothing texture feature space is constructed by performing subject segmentation and semantic parsing on clothing images; simultaneously, background environment features are extracted to construct a background environment feature space, ensuring that the human pose feature space, clothing texture feature space, and background environment feature space are independent of each other; in this invention, the information coupling degree between the above three types of features is reduced by feature decoupling constraints, a mapping relationship between clothing and human body semantic regions is established and a dense deformation field is generated, and clothing texture space transformation is performed based on the deformation field to obtain clothing candidate features; an adaptive fusion weight mask is generated by combining boundary local features to achieve multi-feature adaptive fusion; finally, image reconstruction is completed under consistency constraints, thereby outputting a high-quality virtual try-on image with high stability.

[0025] Example 1: Figure 1 This is a flowchart illustrating the virtual fitting image synthesis method provided in Embodiment 1 of the present invention.

[0026] This embodiment fully illustrates the interactive virtual fitting scenario of a smart fitting mirror in an offline apparel retail store. The smart fitting mirror is a terminal device equipped with a 1080P high-definition RGB acquisition camera, a multi-touch screen, a 4K high-definition display screen, and an edge computing preprocessing unit. It is connected to a high-performance GPU computing server in the cloud via an encrypted link between a local area network and the Internet. The terminal is responsible for real-time acquisition of user human body images, interactive selection of target clothing, image preprocessing, result display, and user feedback. The cloud server is responsible for receiving standardized image data uploaded by the terminal, performing high-computation-intensive tasks such as human key point detection, human semantic segmentation, clothing semantic parsing, multi-dimensional feature encoding, feature decoupling constraint calculation, dense deformation field generation, clothing spatial transformation alignment, adaptive fusion weight mask calculation, feature fusion, and image reconstruction under multiple constraints. The reconstructed virtual fitting result image is then transmitted back to the smart fitting mirror in real time for display to the user.

[0027] This embodiment provides a virtual fitting image synthesis method, which includes the following steps: Step 100: Acquire user's body image and target clothing image In this embodiment, a user enters a physical clothing retail store and stands in a standard standing area in front of the smart fitting mirror. This area is defined by ground markings, with a distance of 0.8m to 1.2m from the mirror, and the height range is adapted to the user's full-body image. The smart fitting mirror has a built-in front-facing 1080P resolution RGB image acquisition camera that captures the user's full-body image in real time at a frame rate of 30fps. The original image resolution is 1920×1080, the pixel format is RGB888, and the color space is sRGB. The smart fitting mirror's edge preprocessing unit performs preliminary cropping on the acquired human body image, scaling the image size to a unified calculated size of 768×1024, while keeping the human body centered, ensuring that the main body area occupies no less than 70% of the overall image.

[0028] Users browse the store's clothing product list via the touch screen of the fitting mirror, click to select the desired clothing to try on, and the smart fitting mirror retrieves the corresponding standard image of the target clothing from the cloud product database. The standard image includes a flat-lay scan of the clothing, a background-free image of the clothing, and a standard model wearing the clothing. The image size is also normalized to 768×1024 to match the user's human body image.

[0029] The terminal performs a unified preprocessing procedure on the two images, as detailed below: Pixel value normalization: linearly maps image pixels from 0 to 255 to the range of 0.0 to 1.0. The calculation formula is as follows: in For the original image, This is the normalized image.

[0030] Color space unification conversion: Convert all input images to the standard sRGB color space to eliminate color differences caused by different acquisition devices.

[0031] Image noise reduction: A 3×3 bilateral filter operator is used for noise reduction, with the following filter parameters: , It removes noise while preserving edge and texture details.

[0032] Data encapsulation and upload: The preprocessed user human body image data and target clothing image data are encapsulated into RGB (Red Green Blue, the three primary color modes) tensor format and transmitted to the cloud server through an encrypted network link for subsequent processing.

[0033] Preferably, in this embodiment, the system incorporates an automatic image quality assessment and feedback mechanism to ensure that the input image meets the requirements for subsequent feature extraction and synthesis. The assessment criteria include: Blur evaluation: The blur score LP (Laplacian Variance Parameter) is calculated using the Laplacian variance algorithm. If LP < 30, the image is considered blurry, and the user is prompted to reposition. Human body integrity assessment: If the number of key points detected on the human body is less than 15, it is determined that the human body is not fully in the frame, and the user is prompted to adjust the position. Brightness validity determination: Calculate the overall average brightness of the image. ,like If the image is determined to be too dark or too bright, the fitting room mirror fill light will be automatically adjusted and re-captured.

[0034] The above steps ensure that the human body and clothing images input to the cloud are clear, complete, appropriately lit, and uniform in size, providing a stable data foundation for subsequent high-precision processing.

[0035] Step 101: Perform human keypoint detection and human semantic segmentation on the user's human image to extract human pose features and construct a human pose feature space. In this embodiment, the cloud server receives a human body image from the terminal and enters the human pose feature extraction process, which is executed in parallel by the human key point detection branch and the human semantic segmentation branch.

[0036] First, human keypoint detection is performed using a pre-trained human keypoint model based on the HRNet-W32 architecture. This model infers the values ​​from the input image and outputs the coordinates and confidence scores of 21 human keypoints. The keypoint definitions are shown in the table below. 0 Neck center point Position the neckline for alignment 1、2 Left and right shoulder points Positioning shoulder line area 3、4 Left and right elbow points Determine arm posture 5、6 Left and right wrist points Positioning cuff area 7、8 Left and right waist points Positioning the waistline area 9、10 Left and right hip points Locating the lower boundary of the torso 11～20 auxiliary points such as trunk, pelvis, and head Improve human body structure constraints The model output format is ,in For pixel coordinates, The confidence level is 0 to 1, and only key points with a confidence level greater than 0.5 are retained.

[0037] Subsequently, human semantic segmentation was performed using the DeepLabV3+ semantic segmentation model. The image was classified at the pixel level, dividing it into 19 non-overlapping semantic regions, and a single-channel semantic mask image was output. Key categories include: 0: background; 1: human skin area; 2: hair area; 3: area of ​​the user's original clothing; 4: pants area; 5: arm area; 6: neck area, etc.

[0038] Based on keypoint coordinates and semantic masks, a human pose feature space is constructed in the cloud. This feature space is a multi-scale, high-dimensional feature space with an output dimension of 128×768×1024, denoted as [missing information]. The content represented in the feature space is strictly limited to information related to human anatomy: Underlying features: human body outline, limb edges, joint positions, occlusion relationships, and limb orientation; High-level characteristics: human skeletal structure, body proportions, posture type, limb angles, and target area location for clothing change.

[0039] This feature space strictly satisfies the structure preservation constraint, encoding only the human skeleton, contour, and pose information, and does not contain irrelevant information such as clothing texture, background color, and lighting distribution. It ensures independence from other feature spaces at the data level, providing a foundation for subsequent feature decoupling.

[0040] Step 102: Perform clothing subject segmentation and clothing semantic parsing on the target clothing image, extract clothing texture features, and construct a clothing texture feature space. In this embodiment, since the target garment needs to be accurately transferred to the user's body surface while maintaining the integrity of the pattern and texture, the cloud server performs garment body segmentation and refined garment semantic parsing processing on the target garment image.

[0041] Step 1: Clothing subject segmentation. A clothing-specific segmentation model based on Mask-RCNN (Mask Region-based Convolutional Neural Network) is used to separate the foreground and background of the target clothing image, outputting a binary mask image of the clothing. A pixel value of 1 represents the clothing area, and 0 represents the non-clothing area, enabling complete extraction of the main body of the clothing and automatically removing interfering information such as hangers, tags, text, logos, and background clutter.

[0042] Step 2: Clothing semantic parsing. Further fine-grained semantic segmentation is performed within the clothing region, dividing the clothing into 12 categories of the first clothing semantic region, and outputting a semantic mask. The regions are defined in the following table: 1 neckline area Neck and upper chest 2、3 Left shoulder line and right shoulder line area left and right shoulder lines 4、5 Left cuff and right cuff area Wrist / Forearm 6 Waistline area waist 7 hem area Lower boundary of the torso 8～11 Placket, pockets, prints, buttons Clothing details area Step 3: Garment Texture Feature Extraction. Based on the semantic regions of the garment, multi-dimensional appearance features are extracted, including: Color characteristics: Calculate the global mean and variance in the Lab color space to ensure color consistency; Texture features: Calculating image gradients Obtain the texture direction and density; Material characteristics: Frequency domain analysis is used to distinguish the properties of knitted, woven, and denim fabrics; Pattern features: Extract the position and structural information of prints, stripes, and checks.

[0043] Based on the above information, a clothing texture feature space is constructed in the cloud, with an output dimension of 128×768×1024, denoted as... This feature space strictly satisfies the appearance preservation constraint, expressing only the color, pattern, material, and pattern information of the target garment, without including irrelevant features such as human posture and background environment, and is aligned with the size of the human posture feature space to facilitate subsequent spatial transformation and fusion.

[0044] Step 103: Separate and extract background environment features from the user's human body image to construct a background environment feature space; wherein the human body pose feature space, clothing texture feature space, and background environment feature space are independent of each other. In the cloud processing flow of this embodiment, in order to ensure that the virtual fitting results are visually consistent with the real-world environment of the store where the smart fitting mirror is located, and to avoid problems such as background blurring, distortion, and loss, it is necessary to independently extract background environment features and construct a dedicated background environment feature space.

[0045] First, based on the human semantic mask output in step 101 Perform background separation on the user's human body image: set the corresponding pixels in the human body area, hair area, and original clothing area to zero, and retain only the pure background area image. The background area includes real-world scene information such as store walls, floors, shelves, lighting, and decorations.

[0046] Subsequently, multi-level background environment features were extracted within the pure background area: Main direction of illumination: Illumination angle obtained by fitting background shadow gradient. ; Brightness characteristics: Calculate the average background brightness ; Color temperature characteristics: Calculate the average ratio of the RGB three channels ; Texture features: Extracting low-frequency structure and high-frequency noise distribution in the background; Color cast characteristics: Calculate the color cast value compared to the standard white field.

[0047] Based on the above characteristics, a background environment feature space is constructed in the cloud, with an output dimension of 64×768×1024, denoted as . This feature space strictly satisfies scene-preserving constraints, encoding only background lighting, shadows, color temperature, and texture information, and does not include human anatomy or clothing appearance information.

[0048] This embodiment strictly guarantees: human posture feature space Clothing texture feature space Background environment feature space The three components are independent of each other. They are encoded by three separate neural network branches, without sharing weights, aliasing channels, cross-sampling, or information leakage. They are only compliantly fused in the final image reconstruction stage, completely avoiding feature coupling at the encoding structure level and laying the foundation for subsequent decoupling constraints.

[0049] Preferably, in this first embodiment, the background environment feature extraction step can be performed as follows: To address industry-wide issues such as garment whitening, color shifts, and inconsistent lighting caused by multiple light sources, mixed warm and cool light, and localized highlights from spotlights in offline stores, this embodiment proposes a bidirectional adaptive normalization technology for store ambient lighting. This technology calculates both the forward brightness distribution and the reverse shadow attenuation distribution of ambient light, constructing a bidirectional lighting fitting formula to achieve a perfect match between clothing lighting and the physical lighting of the store environment. The unique bidirectional normalization calculation formula is as follows: In the formula: To finally fit the ambient light coefficient; The background has a positive highlight brightness distribution; Background inverse shadow attenuation distribution; This refers to the highlight balance factor; This represents the shadow compensation coefficient. In the offline fitting mirror scenario of this embodiment, the parameter values ​​are limited as shown in the table below: Cool white spotlight 5500~6500 0.65 0.25 Suppressing the whitening of highlights on clothing Neutral natural light 4000~5000 0.50 0.40 Keep the original color of the clothing unchanged warm yellow spotlight 2700~3500 0.30 0.60 Compensation for warm color shadows This preferred embodiment employs bidirectional illumination branch fitting, specifically adapted to the mixed light sources of multiple spotlights in offline stores, thus avoiding the beneficial effect of localized exposure white spots on the collar and shoulders of clothing.

[0050] Step 104: Reduce the information coupling between the human posture features, the clothing texture features, and the background environment features through feature decoupling constraints. In this embodiment, to address the issues of clothing residue, human body distortion, and background pollution caused by multi-source feature coupling, the cloud processes the three-way features. , , Apply complete feature decoupling constraints.

[0051] Step 1: Mapping the three independent coding branches. The three features are fed into three independent coding branches, each containing four convolutional layers, Group Normalization (GN), and GELU (Gaussian Error Linear Units) activation functions. The parameters between the branches are completely independent, with no information exchange.

[0052] Step Two: Three Types of Specific Retention Constraints: right Apply structural preservation constraints: keypoint reprojection error < 1.5 pixels to ensure the human body structure remains unchanged; right Apply appearance preservation constraints: If the garment texture similarity (SSIM) (Structural Similarity Index Measure) is > 0.95, the appearance will remain unchanged. right Apply scene preservation constraints: background lighting error < 0.1 to ensure scene remains unchanged.

[0053] Step 3: Decoupling constraint calculation. The orthogonality of the feature space basis vectors is constrained using a feature orthogonality regularization term. The loss function is: in: Human posture feature matrix; Clothing texture feature matrix; Background environment feature matrix; : Matrix Frobenius norm.

[0054] System constraints This ensures that the inner product norm of any two feature matrices is no greater than 0.05, achieving a pairwise weakly correlated orthogonal distribution of the three feature spaces.

[0055] Simultaneously, mutual information is used to reduce constraints: Further reduce the overlap of information between features.

[0056] Through the above constraints, the three features are highly decoupled, each undertaking a single information expression task, thus eliminating feature confusion and interference from the root.

[0057] Preferably, in this first embodiment, the feature decoupling constraint technique step can be performed in the following manner: Based on the orthogonal constraint of features, this preferred embodiment further adds a three-feature space mutually exclusive entropy regularization decoupling process. This preferred embodiment introduces information entropy mutual exclusion penalty, forcing the entropy distributions of the three types of features—human body, clothing, and background—to be staggered, completely removing redundant information at the information distribution level. The mutual exclusion entropy regularization formula is as follows: In the formula: Entropy is the feature information. This is the mutual exclusion entropy loss; the lower the entropy value, the purer and more singular the feature information. This embodiment defines the distribution range of entropy values ​​for three types of features, as shown in the table below: Human Posture Characteristics 1.2~1.6 Only retain low-entropy structural information Clothing texture features 2.4~3.0 Preserve high-entropy texture detail information Background environmental characteristics 1.8~2.2 Preserve isentropic information in the environment Step 105: Establish the correspondence between the first clothing semantic region and the first human body semantic region, and generate a dense deformation field based on the correspondence. In this embodiment, to achieve high-precision local alignment between clothing and the human body, the cloud establishes a precise correspondence based on semantic regions and generates a dense deformation field.

[0058] Step 1: Extract semantic regions. Extract the first semantic region of clothing from the clothing texture feature space: neckline, shoulder line, cuff, waistline, and hem; extract the first semantic region of the human body from the human posture feature space: neck and upper chest, shoulder line, wrist / forearm, waist, and lower boundary of torso.

[0059] Step 2: Establish standardized correspondences: Clothing neckline area ↔ Neck and upper chest area Shoulder line area of ​​clothing ↔ Area connecting the left and right shoulders of the human body Clothing cuff area ↔ Human wrist or forearm area Clothing waistline area ↔ Human waist area Clothing hem area ↔ Lower boundary area of ​​the human torso Step 3: Generate a dense deformation field. Based on semantic correspondences, human body key points, human body boundaries, and clothing outlines, calculate pixel-by-pixel displacement vectors to generate a dual-channel dense deformation field. The dimensions are 2×768×1024.

[0060] Applying smooth regularization constraints to the deformation field ensures continuous deformation without tearing: In this embodiment, a regularization coefficient is set. .

[0061] Dense deformation fields can accurately describe the non-rigid mapping relationship between clothing and the human body surface, and can perfectly adapt to complex postures such as turning to the side, raising an arm, and covering.

[0062] Step 106: Perform a spatial transformation on the clothing texture features based on the dense deformation field to obtain candidate clothing features aligned with the human body changing area. In this embodiment, the cloud performs spatial resampling transformation on the texture features of the clothing based on the dense deformation field, so as to achieve high-precision alignment between the clothing and the human body.

[0063] This invention employs bilinear grid resampling transformation to achieve non-rigid spatial mapping of clothing texture features. The dense deformation field in this invention adopts an inverse deformation field form, conforming to the standard mathematical definition of image deformation in deep learning. Specifically, the target pixel coordinates are defined as follows: The dense deformation field offset vector is The expression for clothing texture feature mapping is as follows: In the formula, These are the bilinear interpolation weighting coefficients, used for smoothing and weighting calculations of deformation sampling locations. To meet the requirements of network discretization engineering operations, the deformation field coordinates are normalized. The normalized mesh sampling formula is as follows: in, This is a grid sampling operation function, with bilinear interpolation as the sampling mode and boundary copying as the boundary fill mode. The height of the feature map is used. After transformation, candidate features for clothing are obtained. It is precisely aligned to the area where the person is changing clothes.

[0064] The system automatically switches alignment strategies based on pose complexity: First posture (upright facing, arms hanging naturally): Affine transformation or TPS thin plate spline transformation is used to achieve rapid global alignment; Second posture (sideways, raising hand, bending over, limb obstruction): Simple transformation is turned off, and pixel-level resampling is performed only by relying on dense deformation field to finely adjust key areas such as neckline, shoulder line, cuff, waistline, and hem, completely eliminating problems such as floating, offset, skew, and stretching.

[0065] After the transformation is completed, the candidate features of the clothing retain all texture details and the pattern structure is free from distortion, so it can directly enter the fusion stage.

[0066] Preferably, in this first embodiment, the following optimization steps can be further added: In the process of completing the spatial transformation of clothing within a dense deformation field, this preferred embodiment introduces a fabric rigidity sensing deformation constraint technology. This preferred embodiment automatically determines the fabric rigidity coefficient based on the clothing texture characteristics, constructs a fabric rigidity constraint formula, and limits the maximum deformation gradient of clothing of different materials, achieving physically realistic deformation. The original fabric rigidity constraint formula is as follows: In the formula: This is due to the loss of fabric rigidity; These are the deformation canonical coefficients; The gradient of the deformation field; This is the fabric stiffness discrimination factor. This embodiment establishes a stiffness discrimination table for common clothing items found in offline stores, as shown below: Denim and suit fabrics 0.85 ≤0.12 Inhibit stretching and maintain the shape's stiffness Cotton, linen, and ordinary knitted fabrics 0.50 ≤0.25 Moderate deformation, retaining slight wrinkles Silk, chiffon soft fabrics 0.20 ≤0.45 Flexible deformation, generating natural draping folds Step 107: Generate an adaptive fusion weight mask based on local boundary features, and adaptively fuse the clothing candidate features, human skin features, and background environment features. In this embodiment, adaptive boundary fusion is performed in the cloud to eliminate seams, jagged edges, color shifts, and ghosting.

[0067] Step 1: Determine the composition boundaries: neckline boundary, cuff boundary, hem boundary, and outer contour boundary of the garment. Step 2: Calculate local boundary features (3×3 window): Boundary color difference: ; Texture gradient: ; Boundary curvature: .

[0068] Step 3: Generate an adaptive fusion mask: in: : Sigmoid function; ; The output mask value ranges from 0 to 1, and the boundary transition width adapts to 3 to 12 pixels.

[0069] Step 4: Weighted fusion

[0070] Differentiated integration strategy: Buttons, pockets, and printed areas: Reduce the emphasis on smoothness and retain details; High curvature boundaries: improve smoothness and eliminate jagged edges; Areas with large color differences: Expand the transition width to achieve a natural transition.

[0071] After blending, the edges are natural and artifact-free, and the clothing, human body, and background transition perfectly.

[0072] Preferably, in this example, specific preferred values ​​are set for the boundary curvature preset threshold and the boundary color difference preset threshold to achieve differentiated adaptive fusion control: A preset optimal threshold for boundary curvature is set: the preferred value for determining boundary curvature is 0.25 rad / px. When the calculated boundary curvature value for a local area is greater than 0.25 rad / px, it is determined to be a high-curvature bending boundary. The smoothing weight of this area is automatically increased to weaken pixel abrupt changes and quickly eliminate edge jaggedness, pixel burrs, and splicing breaks at clothing neckline corners, armpit bends, and cuff bends. When the boundary curvature is less than this threshold, the normal blending weight is maintained to ensure the clarity of the edge contour.

[0073] Preset optimal threshold for boundary color difference: Using the Lab color space color difference calculation method, the optimal threshold for judging color difference is set as follows. When the pixel color difference between the clothing area and the human skin area is greater than 18, it is determined to be a strong color difference boundary area. The boundary fusion transition pixel width is adaptively expanded, and the regular 3 / 12 pixel adaptive range is adjusted to an 8 / 12 pixel wide transition range to achieve a smooth gradient between the clothing color block and the human skin color, eliminating the problems of hard boundary abruptness and abrupt color block splicing. When the color difference is less than this threshold, the default narrow range transition can meet the fusion requirements.

[0074] Step 108: Perform image reconstruction under the first consistency constraint, and output the reconstructed virtual fitting image. In this embodiment, the cloud uses an encoder-decoder architecture to reconstruct the image and generates the final virtual fitting image under the supervision of the first consistency constraint.

[0075] The first consistency constraint includes five strong constraints: Consistency of human anatomy: Skeletal proportions, shoulder width, and arm length are consistent with the original image; Clothing texture consistency: The color, pattern, and material are completely consistent with the target clothing; Semantic region consistency: The semantic region of clothing is precisely aligned with the human body region; Smooth and consistent boundaries: no seams, no jagged edges, no breaks, no afterimages; Lighting consistency: Adjust the brightness, shadows, and color temperature of clothing according to the background environment to achieve uniform lighting.

[0076] Total constraint loss function:

[0077] The image reconstruction outputs a high-resolution RGB virtual fitting image of 768×1024, which is transmitted back to the smart fitting mirror for real-time display via an encrypted link.

[0078] Preferably, in this embodiment, the cloud also performs post-processing optimization: Guided filtering enhances high-frequency details in clothing, with an enhancement coefficient of [missing information]. ; Global color consistency correction, color cast error controlled to <0.05; The edges are slightly smoothed to further enhance the visual texture.

[0079] The final output image shows precise clothing fit, complete details, natural boundaries, uniform lighting, and a complete background, fully meeting the commercial deployment requirements of smart fitting mirrors in offline stores.

[0080] Example 2: To more clearly reveal the structure of the virtual fitting image synthesis system involved in this invention, Figure 2 This is a schematic diagram of the structure of the virtual fitting image synthesis system provided in Embodiment 2 of the present invention.

[0081] Embodiment 2 of the present invention discloses a virtual fitting image synthesis system 200, which includes an image input module 201, a multi-dimensional feature encoding module 202, a feature decoupling constraint module 203, a spatial alignment module 204, a boundary adaptive fusion module 205, and an image reconstruction module 206. The image input module 201 is configured to acquire a user's human body image and a target clothing image; the multi-dimensional feature encoding module 202 is configured to perform human keypoint detection and human semantic segmentation on the user's human body image to extract human pose features and construct a human pose feature space, perform clothing body segmentation and clothing semantic parsing on the target clothing image to extract clothing texture features and construct a clothing texture feature space, and separate and extract background environment features from the user's human body image to construct a background environment feature space; wherein the human pose feature space, clothing texture feature space, and background environment feature space are independent of each other; the feature decoupling constraint module 203 is configured to reduce the differences between the human pose features, clothing texture features, and background environment features through feature decoupling constraints. The information coupling degree; the spatial alignment module 204 is configured to establish a correspondence between the first clothing semantic region and the first human semantic region and generate a dense deformation field based on the correspondence, wherein the first clothing semantic region is extracted from the clothing texture feature space and the first human semantic region is extracted from the human posture feature space; and to perform spatial transformation on the clothing texture features according to the dense deformation field to obtain clothing candidate features aligned to the human dressing area; the boundary adaptive fusion module 205 is configured to generate an adaptive fusion weight mask based on the boundary local features and to adaptively fuse the clothing candidate features, human skin features and background environment features; the image reconstruction module 206 is configured to perform image reconstruction under the first consistency constraint and output a virtual fitting image. The virtual fitting image synthesis system 200, and its components including the image input module 201, multi-dimensional feature encoding module 202, feature decoupling constraint module 203, spatial alignment module 204, boundary adaptive fusion module 205 and image reconstruction module 206, will be described in detail below with a practical and complete example.

[0082] This second embodiment uses a mobile smartphone application (APP) to simulate a virtual clothing try-on scenario at home. The user is positioned in a natural indoor lighting environment with a complex and cluttered background including curtains, a fabric sofa, tabletop decorations, and soft diffused indoor lighting. The user assumes a complex human posture with their body turned at a 45-degree angle and one arm slightly raised. This posture involves complex deformation conditions such as turning sideways, raising the arm, and slight underarm obstruction, used to verify the system's clothing fit capability in non-standard upright postures. The system's hardware architecture consists of mobile hardware and cloud-based inference server hardware. The mobile hardware is a standard consumer smartphone equipped with a rear-facing 48-megapixel high-definition camera, an automatic white balance adjustment module, and an automatic exposure compensation module. The mobile device handles image acquisition, pre-compression, local preprocessing, interactive selection, and result preview. The cloud-based inference graphics card performs all feature calculations, constraint solving, deformation field generation, feature fusion, and image reconstruction calculations. This embodiment uses real-life home portraits and backgroundless e-commerce clothing product images as input samples throughout.

[0083] The specific implementation of the image input module 201 is as follows: The image input module 201 is the initial acquisition module of this system, used to acquire images of the user's human body and the target clothing. In this embodiment, the user holds the smartphone vertically and takes a selfie in a home living room environment. The shooting distance is 1.5m, and the ambient light color temperature is 4300K, which is neutral natural light. The resolution of the original human body image acquired by the mobile phone camera is 3072×4096, and the image format is RGB color image. The image input module 201 has a built-in image compression unit (not shown), a quality screening unit (not shown), and a standardization preprocessing unit (not shown). First, the compression unit compresses the original high-resolution image to a uniform square input size of 1024×1024 to adapt to the transmission bandwidth of the mobile terminal; then, the quality screening unit judges the image sharpness, human body integrity, and lighting rationality. The sharpness judgment adopts the Laplacian variance algorithm, and the sharpness threshold is set to 35, which is higher than the store standard in Embodiment 1, adapting to the slight shaking problem of home selfies; the lighting judgment range is set to 0.20 to 0.80 to expand the compatibility range of home lighting.

[0084] The user selects a solid-color collared long-sleeved shirt as the target garment from the clothing product library on the mobile app. The target garment image is a standard e-commerce product image with a pure white background, and the resolution is uniformly scaled to 1024×1024. The image input module performs standardized preprocessing on both images, including pixel normalization, color gamut alignment, and Gaussian noise reduction. The pixel normalization calculation formula uses a linear normalization method: In the formula Original pixel brightness To normalize pixel brightness, the normalized pixel values ​​are strictly controlled between 0 and 1. This module removes salt-and-pepper noise generated during shooting, using a Gaussian smoothing operator with a radius of 1.5 pixels to weaken ambient noise while preserving the human body's edge contours. After preprocessing, the module uploads the two normalized images to the cloud as encrypted data streams, completing the image input process.

[0085] The specific implementation of the multidimensional feature encoding module 202 is as follows: The multidimensional feature encoding module 202 is the core feature extraction unit of this system. It contains three independent encoding sub-branches, which respectively complete the encoding of human pose features, clothing texture features, and background environment features, strictly adhering to the technical constraint that the three feature spaces are mutually independent. In this embodiment, the multidimensional feature encoding module 202 receives a standardized image transmitted from the image input module and sequentially performs human keypoint detection, human semantic segmentation, clothing subject segmentation, clothing semantic parsing, and background separation and extraction operations.

[0086] The extraction of human posture features and the construction of the human posture feature space are as follows: This module incorporates a high-precision human keypoint detection network. For a home-based side-lying, arm-raising posture, it detects and outputs 21 human keypoints, including key alignment nodes such as the left shoulder, right shoulder, neck, waist, wrist, and elbow joint. The keypoint confidence threshold is set to 0.55 to filter out false detection coordinates from background noise. Simultaneously, a lightweight semantic segmentation model is used to perform pixel-level human body classification. Semantic categories include skin, hair, existing home clothing, exposed arms, and neck skin, generating a single-channel human semantic mask. Combining the keypoint coordinates and the semantic mask, a 128×1024×1024 human pose feature matrix is ​​constructed, forming a human pose feature space. This feature space only encodes the human skeletal structure, limb angles, side-lying deflection angles, and occluded area locations, excluding any clothing textures or background clutter information.

[0087] The extraction of clothing texture features and the construction of the clothing texture feature space are as follows: In this embodiment, since the target garment for this fitting is a long-sleeved collared shirt, the clothing has intricate structures such as stiff shoulder lines, a cinched waistline, cuff buttons, and placket stitching, requiring high precision in semantic parsing. This module uses a dedicated clothing segmentation model to extract the main body of the clothing against a pure white background, removing the background of the product image, watermarks, and annotation text to generate a clean clothing mask. Further refined clothing semantic parsing is performed, dividing the garment into five first-level semantic regions: the neckline region, the left and right shoulder line regions, the cuff region, the waistline region, and the hem region. The module extracts appearance information such as the shirt fabric texture, solid color saturation, placket stitching pattern, and button positions to construct a clothing texture feature space, with a feature matrix dimension of 128×1024×1024. This feature space strictly adheres to appearance preservation constraints, fully preserving the garment's shape, material, color, and detailed textures, without introducing features irrelevant to human posture or ambient lighting.

[0088] The background environment feature extraction and background environment feature space construction are as follows: In the home scene of this embodiment, the background includes a sofa, curtains, a table, and soft diffused natural light, with complex textures and rich color levels. This module relies on human semantic masks to set all areas of the human body within the image to zero, preserving a pure home background image. The module extracts background lighting angle, ambient color temperature, shadow distribution, and fabric texture features to construct a 64×1024×1024 background environment feature space, strictly adhering to scene preservation constraints. This embodiment strictly follows the principle of spatial independence; the encoding network weights of the human posture feature space, clothing texture feature space, and background environment feature space are not shared, feature channels are not mixed, and feature information is not interfering with each other. This ensures that the three feature spaces are mutually independent at the architectural level, providing a foundation for subsequent feature decoupling.

[0089] The specific implementation of the feature decoupling constraint module 203 is as follows: In this embodiment, the feature decoupling constraint module 203 receives three types of feature matrices output by the multi-dimensional feature encoding module and applies multiple constraints to human posture features, clothing texture features, and background environment features to reduce the information coupling between different features and avoid defects such as clothing ghosting, background color mixing, and human body deformation in complex home backgrounds. This module internally includes an independent encoding mapping unit, a feature preservation constraint unit, and an orthogonal constraint calculation unit. First, the three features are fed into convolutional encoding branches with completely identical structures and different parameters to complete high-dimensional feature nonlinear mapping. Then, specific constraints are applied to each feature: a structure preservation constraint is applied to the human posture features, controlling the keypoint reprojection error to be less than 1.2 pixels to ensure no distortion of the side-view human skeleton; an appearance preservation constraint is applied to the clothing texture features, using a structural similarity index to determine clothing fidelity, with a similarity threshold set not lower than 0.96; and a scene preservation constraint is applied to the background environment features, controlling the background illumination error to be less than 0.09 to ensure no distortion or warping of light and shadow in the home environment.

[0090] This feature decoupling constraint module 203 is equipped with a feature orthogonality constraint formula, which uses the Frobenius norm to calculate matrix orthogonality. The orthogonality constraint loss function formula is as follows: In the formula, The human posture feature matrix, For clothing texture feature matrix, The background environment feature matrix, The norm is the Frobenius norm. This embodiment strictly constrains the inner product norm of any two feature matrices to be no greater than 0.05, ensuring that the three feature spaces maintain a weakly correlated orthogonal distribution. Simultaneously, mutual information constraints are added to control the mutual information value of any two features to below 0.07, further weakening the feature coupling.

[0091] The specific implementation of the spatial alignment module 204 is as follows: In this embodiment, the spatial alignment module 204 receives the feature matrix after decoupling and completes the entire process of semantic region matching, dense deformation field generation, and clothing spatial transformation. This module first extracts the first clothing semantic region from the clothing texture feature space, including the shirt collar, shoulder line, cuffs, waistline, and hem; and extracts the first human semantic region from the human posture feature space, including the upper chest region of the neck, the shoulder line region, the forearm wrist region, the waist region, and the lower boundary region of the torso. The following corresponding mapping and binding relationships are established: the clothing collar corresponds to the human neck and upper chest, the clothing shoulder line corresponds to the line connecting the left and right shoulders, the clothing cuffs correspond to the human forearm, the clothing waistline corresponds to the human waist, and the clothing hem corresponds to the lower boundary of the human torso.

[0092] The module combines the coordinates of key human body points, the human body edge contour, and the outer contour boundary of clothing to generate a dual-channel dense deformation field. The deformation field matrix size is 2×1024×1024, and the deformation field stores pixel-level lateral and vertical offsets. To ensure smooth deformation without tearing, a deformation smoothing regularization constraint is applied, and the smoothing loss formula is: In the formula This is the dense deformation field offset vector, with the regularization coefficient set to 0.12.

[0093] In this embodiment, the user's posture is a complex one of sideways movement and raising an arm. The system automatically disables affine transformation and thin-plate spline transformation, and only uses a dense deformation field to complete pixel-level resampling. This embodiment uses the standard spatial transformation formula: In the formula, Candidate features for clothing These are the bilinear interpolation weighting coefficients. This formula completes the non-rigid deformation mapping of clothing texture features, accurately adapting to the underarm stretching, shoulder line offset, and torso deflection deformation caused by the human body turning to the side and raising its arms. Finally, it generates candidate clothing features that fit the human body's dressing area without floating, skewing, or local stretching distortion.

[0094] Preferably, in this second embodiment, the space alignment module 204 can be further implemented in the following manner: For the special posture of a 45-degree sideways body with an arm raised to cover the body in Embodiment 2, there is a perspective shift in the human skeleton. Conventional keypoint detection has problems with shoulder and neck misalignment and armpit displacement. This preferred embodiment introduces a sideways posture skeleton shift compensation constraint technology, constructs a perspective skeleton correction formula, and dynamically corrects the keypoint shift based on the human perspective angle to adapt to clothing fit in the sideways posture. The skeleton shift compensation formula is as follows: In the formula: This is the amount of compensation for bone misalignment; This refers to the angle of the human body's sideways rotation. This is the perspective correction factor; The coordinates of the key points on the left and right shoulders are shown in the table below. The parameters corresponding to the side profile angle in this embodiment are as follows: 15°~30° slight side turn 0.12 Minor correction of shoulder line offset Side profile at a mid-angle of 30° to 60° 0.28 Correcting underarm covering deformation and shoulder and neck misalignment 60°~90° wide angle sideways 0.45 Significantly correct perspective skeletal deviations The specific implementation of the boundary adaptive fusion module 205 is as follows: In this embodiment, the boundary adaptive fusion module 205, tailored to the characteristics of soft boundaries, diffused light, and natural, translucent skin tones in home selfies, performs adaptive fusion of clothing candidate features, human skin features, and background environment features. This module first identifies the synthesized boundaries, including the shirt collar boundary, cuff boundary, hem boundary, and outer contour boundary of the clothing. Feature calculation is performed within a 5×5 local window on both sides of the boundary. Compared to Embodiment 1, this expands the local window to adapt to the blurred boundary characteristics of home selfies. The calculated metrics include boundary color difference, texture gradient, and boundary curvature.

[0095] This module incorporates a dynamic edge-aware network to generate an adaptive fusion weight mask. The mask is calculated using the following formula: In the formula, For the Sigmoid activation function, the hyperparameters in this embodiment are set to... , ; For color difference at the boundary between clothing and skin, This is for texture gradient. In this embodiment, the adaptive transition width of the control boundary is 6 to 11 pixels, suitable for soft lighting in home environments.

[0096] This module performs pixel-level fusion based on weighted fusion rules, where the fusion formula is: In the formula, To fuse feature maps, Characteristics of human skin This module incorporates background environmental features and employs a differentiated blending strategy: for detailed areas such as shirt buttons and placket stitching, it reduces the smoothing blending weight to preserve the sewing texture; for high-curvature curved boundaries under the armpits, it increases the smoothing weight to eliminate jagged edges; and for the neckline area where there is a significant color difference between skin tone and clothing, it widens the transition width to achieve a natural connection between skin tone and shirt fabric. This module addresses the issues of blurred boundaries, harsh transitions, and loss of detail in home selfies.

[0097] Preferably, in this second embodiment, the boundary adaptive fusion module 205 can further perform the following processing: Example 2: In a home setting, there is a complex, high-frequency background with clutter such as curtain textures, fabric patterns, and table clutter. Conventional fusion masks easily mix high-frequency noise from the background into the clothing edges, causing background ghosting and color contamination along the clothing edges. This example adds a high-frequency noise suppression fusion technique for cluttered backgrounds. Based on an adaptive weighted mask, a high-frequency noise suppression factor is introduced to construct a noise reduction fusion correction formula. This distinguishes between high-frequency background textures and the effective texture of the clothing, suppressing the interference of cluttered backgrounds on clothing edges while preserving the fine texture of the clothing itself. The noise suppression fusion formula in this preferred embodiment is as follows: In the formula: This is the fusion mask after noise reduction correction; For the original edge-aware mask; Background suppression weights; This represents the high-frequency gradient response value of the background. This embodiment limits the suppression weight parameters for common home backgrounds; the preferred parameter values ​​are shown in the table below. Fabric and textured curtains ≥0.35 0.75 Strong suppression of background texture crosstalk Solid color walls, matte furniture 0.15~0.35 0.40 Moderate suppression of weak noise Simple solid color background ≤0.15 0.10 Preserve the original blending effect The specific implementation of the image reconstruction module 206 is as follows: In this embodiment, the image reconstruction module 206 receives the fused feature map, completes image decoding and reconstruction under the first consistency constraint, and outputs the final virtual try-on image. This module adopts a symmetric encoder-decoder network structure and incorporates five types of consistency constraint supervision units, using the following constraint types: human body structure consistency constraint, clothing texture consistency constraint, semantic region consistency constraint, boundary smoothness consistency constraint, and illumination consistency constraint. This embodiment optimizes the constraint weights based on the home scene, and the total loss function expression is: In the formula, For the loss of human body structure, For the loss of clothing texture, For semantic alignment loss, For boundary smoothing loss, This represents the lighting matching loss.

[0098] For the side-lying, arm-raising posture in this embodiment, the human body structure consistency constraint strictly locks the shoulder width, arm length, and torso rotation angle, prohibiting deformation and distortion of the human skeleton; the clothing texture consistency constraint ensures that the pure white color of the shirt, the placket texture, and the button position do not shift or become distorted; the semantic region consistency constraint ensures that the shirt shoulder line and collar always fit the corresponding semantic region of the human body; the boundary smoothness consistency constraint eliminates the discontinuity of the seams at the collar, armpits, and cuffs; the lighting consistency constraint extracts the diffuse reflection lighting parameters of the home environment, performs brightness fine-tuning, shadow correction, and color temperature adaptation on the shirt surface, so that the clothing lighting and shadow are completely unified with the indoor environment, avoiding a stiff and floating feeling in the clothing.

[0099] The image reconstruction module 206 ultimately outputs a 1024×1024 high-resolution virtual try-on image, which is compressed and then sent back to a mobile app for display. Preferably, in this embodiment, after image reconstruction, a mobile-optimized post-processing unit performs minor color white balance correction, home-ambience color adjustment, and edge softening to reduce slight noise generated in the cloud, further enhancing the realistic look of home-based try-on photos. The output image features natural human anatomy, complete clothing details, smooth boundary transitions, distortion-free backgrounds, and lighting that matches the home environment, fully meeting the civilian needs of ordinary consumers for self-trying at home, comparing outfits, and online shopping references.

[0100] The virtual fitting image synthesis system 200 disclosed in this embodiment relies on the coordinated operation of six functional modules. This embodiment selects a novel application scenario different from Embodiment 1, employing mobile home selfies, complex indoor backgrounds, and complex human postures such as side-facing and arm-raising poses to fully verify the robustness of this invention under civilian mobile devices, non-standardized shooting environments, and complex human postures. This system suppresses feature aliasing at the source through a three-way independent feature space architecture combined with orthogonal decoupling constraints; it adapts to difficult postures such as side-facing, arm-raising, and occlusion through precise semantic region matching and dense deformation fields; it solves the fusion defects of blurred boundaries in home settings through dynamic adaptive fusion masks; and it ensures the final image is realistic and natural through five consistency constraints. This system has low deployment costs, wide terminal compatibility, and high image quality. It can not only be used for commercial fitting mirrors in offline stores but also be widely adapted to diverse civilian scenarios such as mobile online shopping fitting, home outfit comparison, and clothing e-commerce material generation, possessing extremely high engineering promotion value and market application prospects.

[0101] The system and module embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for synthesizing virtual fitting images, characterized in that, include: Acquire user body images and target clothing images; Human key point detection and human semantic segmentation are performed on the user's human body image to extract human pose features and construct a human pose feature space. The target clothing image is segmented into clothing subjects and its semantics are parsed to extract clothing texture features and construct a clothing texture feature space. Background environment features are separated and extracted from the user's human body image to construct a background environment feature space; The human posture feature space, clothing texture feature space, and background environment feature space are independent of each other. The information coupling between the human posture features, the clothing texture features, and the background environment features is reduced by feature decoupling constraints; Establish a correspondence between the first clothing semantic region and the first human body semantic region, and generate a dense deformation field based on the correspondence; The first clothing semantic region is extracted from the clothing texture feature space, and the first human body semantic region is extracted from the human body posture feature space. Based on the dense deformation field, the clothing texture features are spatially transformed to obtain candidate clothing features aligned to the human body changing area; An adaptive fusion weight mask is generated based on local boundary features, and the clothing candidate features, human skin features and background environment features are adaptively fused. Image reconstruction is performed under the first consistency constraint, and the reconstructed virtual fitting image is output.

2. The method according to claim 1, characterized in that, The reduction of information coupling between the human posture features, the clothing texture features, and the background environment features through feature decoupling constraints includes: Human posture features, clothing texture features, and background environment features are each input into an independent encoding branch for feature encoding; Apply structure-preserving constraints to human posture features to represent human skeleton, contour, and posture information; Appearance-preserving constraints are applied to the texture features of clothing to represent the color, pattern, material, and pattern information of the target clothing; Apply scene-preserving constraints to background environment features, representing background lighting, shadows, color temperature, and texture information; By using orthogonal feature constraints or mutual information to reduce constraints or contrastive learning constraints, the information overlap between the human posture features, clothing texture features, and background environment features can be reduced.

3. The method according to claim 2, characterized in that, The orthogonal constraint uses an orthogonal regularization term to constrain the feature space basis vectors. The orthogonal constraint formula is as follows: in, For human posture feature matrix, For clothing texture feature matrix, The background environment feature matrix, The Frobenius norm is used; the inner product norm of any two feature matrices is constrained to be no greater than 0.05, achieving a pairwise weakly correlated orthogonal distribution of the three feature spaces.

4. The method according to claim 1, characterized in that, The step of establishing the correspondence between the first clothing semantic region and the first human body semantic region, and generating a dense deformation field based on the correspondence, includes: Align the neckline area of ​​the garment with the neck and upper chest area of ​​the human body; Align the shoulder line area of ​​the garment with the line connecting the left and right shoulders of the human body; Align the cuff area of ​​the garment with the wrist or forearm area of ​​the human body; Align the waistline area of ​​the garment with the waist area of ​​the human body; Align the hem area of ​​the garment with the lower boundary area of ​​the human torso; Based on the above correspondence, the location of key points on the human body, the boundary of the human body region, and the boundary of the clothing region, a dense deformation field is generated from the clothing texture features to the target area of ​​the human body.

5. The method according to claim 1, characterized in that, The step of generating an adaptive fusion weight mask based on local boundary features, and adaptively fusing the clothing candidate features, human skin features, and background environment features, includes: Define the collar boundary, cuff boundary, hem boundary, and outer contour boundary of the garment as the composite boundary; Calculate the color difference, texture gradient, and boundary curvature within local windows on both sides of the boundary. Based on color differences, texture gradients, and boundary curvature, a pixel-wise fusion weight mask with values ​​ranging from 0 to 1 is generated. The candidate features of clothing, human skin, and background environment are fused using an adaptive weight mask.

6. The method according to claim 5, characterized in that, The step of fusing clothing candidate features, human skin features, and background environment features according to adaptive weighted masks includes: Using the weight values ​​of corresponding pixels in the adaptive weight mask as fusion coefficients, the clothing candidate features, human skin features, and background environment features are weighted and fused at corresponding pixel positions; For garment details such as buttons, pockets, and prints, reduce the smooth blending weight to preserve the texture details of the garment; For regions where the boundary curvature is greater than a preset curvature threshold, increase the smoothing blending weight to eliminate edge jaggedness; For boundary areas where the color difference exceeds a preset color difference threshold, the boundary transition width is increased to achieve a natural color transition.

7. The method according to claim 6, characterized in that, The adaptive fusion weight mask is output by the dynamic edge-aware network, and the mask weights satisfy the following: Where σ is the Sigmoid function, γ∈0.3~0.7, δ∈0.2~0.5; |Ig−Is| is the boundary color difference, and ||∇I(x,y)|| is the texture gradient, which makes the width of the boundary transition region adaptively range from 3 to 12 pixels.

8. The method according to claim 1, characterized in that, The first consistency constraint includes: Human structure consistency constraints ensure that the proportions of the human skeleton, shoulder width, and arm length in the generated image are consistent with the human body image of the input user. Clothing texture consistency constraint ensures that the color, pattern, and material texture of the clothing in the generated image are consistent with those of the target clothing image; Semantic region consistency constraints ensure that the first clothing semantic region is aligned with the first human body semantic region. Boundary smoothing and consistency constraints eliminate seams and breaks between clothing and skin, and between the background; Lighting consistency constraints adjust the brightness, shadow direction, and color temperature of the clothing area according to the characteristics of the background environment.

9. The method according to claim 1, characterized in that, The step of performing spatial transformation on the clothing texture features based on the dense deformation field to obtain candidate clothing features aligned to the human body changing area includes: In the first pose, affine transformation or thin plate spline transformation is used to complete the overall alignment of the clothing with the candidate features of the clothing in the human body changing area; In the second pose, including side-facing and / or raising hands and / or occlusion, dense deformation field is used to perform pixel-level resampling of clothing texture features to achieve local accurate fitting and complete the candidate features of clothing from clothing to the human body dressing area.

10. A virtual fitting image synthesis system, characterized in that, include: The image input module is configured to acquire images of the user's human body and the target clothing. The multidimensional feature encoding module is configured to perform human keypoint detection and human semantic segmentation on the user's human body image to extract human pose features and construct a human pose feature space; to perform clothing subject segmentation and clothing semantic parsing on the target clothing image to extract clothing texture features and construct a clothing texture feature space; and to separate and extract background environment features from the user's human body image to construct a background environment feature space; wherein the human pose feature space, clothing texture feature space and background environment feature space are independent of each other; The feature decoupling constraint module is configured to reduce the information coupling between the human posture features, the clothing texture features and the background environment features through feature decoupling constraints. The spatial alignment module is configured to establish a correspondence between a first clothing semantic region and a first human semantic region and generate a dense deformation field based on the correspondence, wherein the first clothing semantic region is extracted from the clothing texture feature space and the first human semantic region is extracted from the human posture feature space; and to perform spatial transformation on the clothing texture features according to the dense deformation field to obtain clothing candidate features aligned to the human dressing area. The boundary adaptive fusion module is configured to generate an adaptive fusion weight mask based on local boundary features and adaptively fuse the clothing candidate features, human skin features and background environment features. The image reconstruction module is configured to perform image reconstruction under a first consistency constraint and output a virtual fitting image.

Images