A method, system, and medium for converting model images into stereoscopic tiled images based on physical perception.

By using a physical perception-based method, high-fidelity garment flat lay images are generated using deformation fields and self-attention mechanisms. This solves the problems of overlapping feature contamination and pattern distortion in the process of converting model images to flat lay images, and achieves efficient and lossless garment flat lay image generation and cross-modal zero-sample extraction.

CN122368256BActive Publication Date: 2026-11-17QINGDAO TRISTATE BIT TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610508066.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-04-17
Publication Date
2026-11-17
Estimated Expiration
2046-04-17

AI Technical Summary

Technical Problem

Existing technologies suffer from problems such as overlapping feature contamination, pattern distortion, high latency, and lack of category adaptation capabilities during the conversion of model images to flat lay images. In particular, when processing high-resolution clothing images, it is difficult to generate high-quality three-dimensional clothing flat lay images.

Method used

A physical perception-based approach is adopted. By acquiring images of clothing models and text instructions, a prediction network is used to obtain the deformation field of geometric mapping relationship, extract clothing-related content features and suppress interference from non-clothing areas. Spatial reconstruction is performed by combining 2.5D normalized deformation field and self-attention mechanism to achieve parallel decoding and generate high-fidelity clothing flat lay images.

Benefits of technology

It completely solves the problems of feature cross-contamination and pattern distortion, improves the generation speed, achieves lossless preservation of high-definition fabric details and cross-modal instruction-driven zero-sample category adaptive extraction, and reduces the cost of manual intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122368256B_ABST
    Figure CN122368256B_ABST
Patent Text Reader

Abstract

This invention relates to the field of artificial intelligence, specifically disclosing a method, system, and medium for converting model images into 3D flat lay images based on physical perception. The method acquires a model image and text instructions, uses a prediction network to obtain a semantic mask and a 2.5D normalized deformation field, then extracts pure clothing content features based on the semantic mask, and uses the deformation field to reconstruct the spatial relationships of the content features, eliminating pose deformation in the feature space to obtain flattened clothing features. Finally, using the text instructions and flattened features as conditions, a high-fidelity flat lay image is generated through parallel decoding. This invention innovatively introduces a 2.5D physical perception mechanism and self-supervised training, effectively solving the problems of feature cross-contamination, pattern distortion, and generation speed bottlenecks, achieving high-speed, high-definition automatic generation of clothing flat lay images end-to-end without manual annotation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, system, and medium for converting model images into stereoscopic tiled images based on physical perception. Background Technology

[0002] In e-commerce, fashion design, and virtual try-on, high-quality flat lay images of clothing are crucial digital assets. However, due to limitations in shooting costs and product management, much e-commerce data consists only of images of models wearing clothing (i.e., model photos), lacking standardized, accurate flat lay images, especially those with a three-dimensional, openwork effect. Therefore, how to automatically extract and generate high-quality, three-dimensional flat lay images of clothing from model photos using computer vision technology has become a pressing issue in this field.

[0003] Existing techniques for converting model images to tiled images (or clothing cutout and redrawing) suffer from the following significant drawbacks. First, to preserve fabric details, models typically require high-resolution input. Existing visual encoders (such as CNNs with strides or ViT with Patch Merging) rely solely on the Euclidean distance of the two-dimensional image grid when performing feature downsampling and compression. When the model's limbs physically overlap due to complex poses (e.g., arms hanging down and obscuring the front of clothing), conventional encoders forcibly fuse arm skin color features with clothing fabric features within the same receptive field. This results in thoroughly contaminated features in subsequent generation, ultimately leading to severe color crosstalk and limb ghosting in the image, causing overlapping feature contamination issues caused by human poses.

[0004] Then, existing image generation models have difficulty distinguishing between the "non-rigid distortion" of clothing (caused by human movement) and the "inherent pattern". Traditional methods often rely on complex 3D mesh rendering to unfold the texture, which not only consumes a lot of computing power, but also has difficulty in handling the soft wrinkles of clothing and the overlap of the front and back pieces. This often results in the generated tiled image having a skewed neckline, stuck sleeves, or double patterns on the front and back pieces, causing a lack of physical space decoupling and resulting in pattern distortion problems.

[0005] Next, on the image generation side, existing autoregressive (AR) models heavily rely on vector quantization (VQ) vocabularies. Limited by the memory bottleneck of the classification head, the small vocabulary cannot cover complex fabric micro-textures (such as lace and fine stitching), resulting in blurry and distorted generated results. Furthermore, traditional autoregressive models must decode in a serial "token-by-token" manner. When generating high-resolution tiled images at the megapixel level, the inference time can reach tens of seconds or even minutes, completely failing to meet the high-concurrency, low-latency real-time processing requirements of actual SaaS businesses, thus creating a fidelity and speed bottleneck in traditional autoregressive generation architectures.

[0006] Finally, existing algorithms often require users to manually provide accurate clothing masks for image cutout, and a single network is difficult to flexibly respond to intentions such as "extract only the top" or "extract only the bottom," lacking zero-shot automated extraction capabilities guided by cross-modal instructions, resulting in high manual intervention costs and a lack of category-adaptive capabilities.

[0007] Therefore, there is an urgent need for a new end-to-end generation method and architecture that can completely eliminate the interference of model poses, retain high-definition fabric details without loss, and achieve ultra-fast parallel generation driven by text instructions, without the need for manual annotation and complex 3D rendering. Summary of the Invention

[0008] To address at least one of the problems existing in the prior art, such as overlapping feature contamination, pattern distortion, high latency, and lack of category adaptability, this invention proposes a method, system, and medium for converting model images into 3D tiled images based on physical perception.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A method for converting model images into 3D tiled images based on physical perception: The method includes the following steps; Information acquisition steps: Obtain clothing model images and associated text instructions; Geometric mapping step: Input the clothing model image into the prediction network to obtain the semantic mask and the deformation field representing the geometric mapping relationship of the clothing from the model pose space to the standard tiling space; Feature extraction steps: Based on the semantic mask, extract content features related to clothing from the clothing model image and suppress interference from non-clothing areas; Spatial reconstruction steps: Based on the deformation field, the spatial relationship of the content features is reconstructed to eliminate the clothing deformation caused by the model's posture in the feature space, and obtain the flattened clothing features. Image output steps: Based on the text instructions and the flattened clothing features, a high-fidelity clothing flat lay image is generated through parallel decoding.

[0010] Furthermore, the deformation field is a 2.5D normalized deformation field, which includes the two-dimensional continuous coordinates (u, v) of each image position in the clothing model image in the standard tiling space and a discrete topological level (w). The spatial reconstruction step also includes: Based on the coordinates (u, v) and the level (w) contained in the 2.5D normalized deformation field, a corresponding rotation position encoding matrix is ​​generated for the content feature sequence; Using the rotation position encoding matrix, the query vector and key vector of the content feature sequence in the self-attention calculation are transformed respectively; The attention weight is calculated based on the transformed query vector and key vector, wherein the attention weight is determined by the difference between the coordinates (u, v) of the image positions corresponding to the query vector and the key vector and the difference between the levels (w). The value vectors in the self-attention calculation are weighted and aggregated based on the attention weights to generate the flattened clothing features.

[0011] Furthermore, the prediction network in the geometric mapping step is trained through the following steps: Obtain weakly matched model images and flat lay images of the same garment; Features of the clothing model image and the clothing flat lay image are extracted using a visual basic model, and the initial dense matching relationship between the clothing model image and the clothing flat lay image is calculated; The human topological structure prior of the clothing model image is obtained by using a human pose estimation model; Based on the prior knowledge of the human body topology, the initial dense matching relationship is verified and spatially guided to generate a corrected matching relationship; By integrating the corrected matching relationship with the prior knowledge of the human body topology, the semantic mask ground truth and deformation field ground truth used to train the prediction network are automatically generated.

[0012] Furthermore, the deformation field truth value is a 2.5D normalized deformation field truth value. The 2.5D normalized deformation field truth value is characterized by the continuous position mapping of each image position in the clothing model image in the standard tiling space using two-dimensional continuous coordinates (u, v), and by the discrete topological hierarchy (w) to characterize the physical spatial hierarchy caused by occlusion and / or the relationship between the front and back pieces of clothing.

[0013] Furthermore, the discrete topological level (w) in the true value of the deformation field is determined in the following way: Obtain the human body part indexes output by the human pose estimation model; based on the human body part indexes and according to the mapping rules predefined for constructing the 2.5D normalized deformation field, map different human body part index values ​​to different discrete integers to form the discrete topological layer (w) used to characterize the occlusion layer in physical space.

[0014] Furthermore, the geometric mapping step and the feature extraction step are implemented through a two-stream decoupled encoder, including: Semantic and Deformation Prediction Steps: Using a semantic and deformation flow network, predict the semantic mask and the deformation field based on the clothing model image; Content extraction and compression steps: Perform the following operations using a content feature stream network: Extract the initial content features of the clothing model image, and use the semantic mask to suppress features of non-clothing areas in the initial content features; Using the deformation field, the physical topological distance between image locations involved in the compression of the initial content features is calculated; and the weights of the convolution operation are dynamically modulated based on the physical topological distance to securely compress the initial content features, thereby obtaining the content features.

[0015] Furthermore, the physical topological distance ( Based on the deformation field calculation, the expression includes: the two-dimensional coordinate difference in the tiling space defined by the deformation field, and the penalty term that is not triggered simultaneously by the discrete topological level (w) in the deformation field; The weights of the dynamically modulated convolution operation based on the physical topological distance are calculated by... The modulation weights (G) are generated using a negative exponential function of the physical topology distance ( ), where the modulation weights (G) are the physical topology distance ( ). It is a monotonically decreasing function.

[0016] Furthermore, in the image output step, a high-fidelity clothing flat lay image is generated through parallel decoding, including: The features of the flattened clothing are combined with the features of the text instruction to construct a joint condition; Autoregressive generation is performed on a feature block basis. For each feature block to be generated, the following operations are performed: Initialize the noise tensor, and guided by the joint conditions and the generated historical features, denoise the noise tensor in a small number of iteration steps using a lightweight diffusion model; The denoised tensor is binarized to generate all high-entropy binary discrete feature representations within the feature block in parallel; The discrete feature representation sequence generated from all feature blocks is decoded into the high-fidelity clothing flat lay image.

[0017] On the other hand, the present invention also provides a system for converting model images into stereoscopic tiled images based on physical perception, comprising: The information acquisition module retrieves images of clothing models and associated text instructions. The geometric mapping module inputs the clothing model image into the prediction network to obtain a semantic mask and a deformation field that represents the geometric mapping relationship of the clothing from the model's pose space to the standard tiling space. The feature extraction module extracts content features related to clothing from the clothing model image based on the semantic mask, and suppresses interference from non-clothing areas. The spatial reconstruction module reconstructs the spatial relationships of the content features based on the deformation field, so as to eliminate the clothing deformation caused by the model's posture in the feature space and obtain the flattened clothing features. The image output module generates a high-fidelity flat lay image of the garment through parallel decoding, based on the text instructions and the flattened garment features.

[0018] On the other hand, the present invention also provides a computer-readable storage medium having stored thereon computer program code, which, when loaded into and executed on a computer system, causes the computer system to perform all the steps of the method according to any one of claims 1-8.

[0019] Compared with the prior art, the present invention has the following beneficial effects: First, this invention completely solves the problems of feature cross-contamination and ghosting in high-ratio compression. It innovatively proposes a region-level hierarchical compression encoder based on dual-stream decoupling. Addressing the shortcoming of existing CNN downsampling that easily forces the mixing of limbs and clothing, this invention utilizes a 2.5D normalized deformation field predicted by "semantic and deformation flow" to calculate the physical topological distance and dynamically generate deformation modulation weights. Even if the model's arm severely occludes clothing in the 2D image, the physical topological distance is still calculated because the two are at the topological level (…). (axis) or tiled coordinates ( The differences on the axis are huge, and the feature fusion weights will be forced to decay to zero. This design achieves absolute feature isolation at the mathematical and physical level, ensuring that even with aggressive compression at high resolutions of up to 16 times, it can still output pure fabric features that are completely free from skin color and background contamination.

[0020] Secondly, it improves the implicit virtual flattening and anti-ghosting capabilities under zero 3D rendering overhead. Addressing the limitations of existing technologies in handling non-rigid deformation and front-to-back adhesion in clothing, this invention proposes a 2.5D Rotation Position Encoding (2.5D RoPE) mechanism. This mechanism abandons the bulky 3D mesh reconstruction pipeline and directly encodes the discrete topology hierarchy (…). The deformation field of the axis is transformed into an orthogonal phase rotation matrix. By injecting this matrix into the Query and Key of the self-attention mechanism, the weights of network feature interactions depend entirely on the physical distance on the standard tiling unfolded image. This extremely lightweight mathematical operation not only automatically "irons out" the posture distortion of the garment in the latent space, but also achieves physical insulation between the front and back pieces through orthogonal phase shifting, fundamentally eliminating pattern distortion and pattern ghosting when generating the tiling image.

[0021] Next, a high-fidelity method for pixel-level detail restoration that breaks through the resolution limit is proposed. Addressing the problem of fabric detail loss (such as blurred lace and stitching) caused by traditional autoregressive models relying on a limited VQ vocabulary, this invention introduces a lookup-free quantization high-entropy binary representation mechanism. Through an independent symbolic activation function, a single visual token is given a high entropy... It possesses the ability to express extreme states. This extremely high-dimensional discrete state space, from a mathematical upper limit perspective, guarantees the lossless inverse mapping and restoration of extremely complex materials and high-frequency textures.

[0022] Subsequently, a revolutionary autoregressive inference acceleration and zero-shot category adaptation scheme was designed. Addressing the bottleneck of the extremely slow "word-by-word generation" speed of traditional autoregressive models, this invention pioneered an instruction-driven block-level joint diffusion decoding architecture. At the Transformer output, a lightweight continuous spatial diffusion model replaces the massive Softmax classifier. By executing a small number of ordinary differential equations and Euler integrals, dozens of high-dimensional discrete tokens within a feature block can be output in parallel with a single forward propagation. This not only reduces the inference time for generating megapixel-level high-definition images by several orders of magnitude, but also, combined with text instruction conditional prefixes, enables the system to accurately achieve cross-modal zero-shot addressing extraction ("generate only the top" or "generate only the bottom") without any manual masking intervention, greatly improving user experience and processing concurrency in commercial applications.

[0023] Finally, a high-precision, self-supervised closed-loop system for ground truth was achieved with zero manual cost. Addressing the severe scarcity of pixel-level aligned data, this invention proposes a bidirectional consistency fusion pipeline based on dense matching of a visual foundation model and human topology proxy. This method fully utilizes existing, inexpensive, weakly paired e-commerce images to automatically "distill" extremely accurate fine-grained semantic masks and 2.5D normalized deformation field ground truth, thereby breaking through the data bottleneck of supervised training and significantly reducing the R&D and deployment costs of large-scale industrial-grade vertical models.

[0024] This invention, through an innovative method of converting model images into 3D tiled images, achieves for the first time, without relying on manual annotation or expensive 3D graphics rendering engines, the precise addressing and physical stripping of target clothing from high-resolution model images with arbitrarily complex poses in a single forward pass, based on natural language commands. This method successfully breaks through the industry's technical barriers that traditional autoregressive raw image models struggle to balance "high-precision fabric lossless reconstruction" and "high-speed inference of large-size images." It provides a low-computing-cost, high-fidelity, end-to-end industrial-grade AIGC solution for fields such as e-commerce digital asset reconstruction, virtual try-on, and automated clothing design. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the method for converting model images into 3D tiled images according to the present invention. Figure 2 This is a diagram of the algorithm structure of the present invention; Figure 3 This is a flowchart of the global interaction and 2.5D RoPE algorithm of the present invention; Figure 4 This is a flowchart of the instruction-driven block-level joint diffusion decoding algorithm of the present invention; Figure 5 This is a display image showing the result of converting a model image into a 3D tiled image according to the present invention. Detailed Implementation

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0027] like Figure 1-5 As shown, this invention proposes a method for converting model images into stereoscopic tiled images based on physical perception.

[0028] In a general embodiment of the present invention, a method for converting a model image into a 3D tiled image based on physical perception is provided. The method includes the following steps; Information acquisition steps: Obtain clothing model images and associated text instructions; Geometric mapping step: Input the clothing model image into the prediction network to obtain the semantic mask and the deformation field representing the geometric mapping relationship of the clothing from the model pose space to the standard tiling space; Feature extraction steps: Based on the semantic mask, extract content features related to clothing from the clothing model image and suppress interference from non-clothing areas; Spatial reconstruction steps: Based on the deformation field, the spatial relationship of the content features is reconstructed to eliminate the clothing deformation caused by the model's posture in the feature space, and obtain the flattened clothing features. Image output steps: Based on the text instructions and the flattened clothing features, a high-fidelity clothing flat lay image is generated through parallel decoding.

[0029] Specifically, the following steps are included: S1: Retrieve flat lay images of the same garment and corresponding model images from the database, and obtain their garment type tags (top / bottom / full-body outfit). Combine text instructions with images to form a set of paired image triplets. Then, a pre-trained Vision Transformer (ViT) model is used to extract features from the image and calculate a dense matching matrix. Simultaneously, a Human Pose Network (DWPose) is introduced to calculate the UV coordinates and body part indices of the human body as a topological prior of the human physical structure. Finally, based on a bidirectional consistency check, the feature matching results and the human topological prior are fused to automatically generate a fine-grained semantic mask corresponding to the model image. True values ​​of the 2.5D normalized deformation field ; Specifically, the first step is to retrieve paired images of clothing models and flat lay images of clothing from the database. Simultaneously, obtain the category labels of the garment flat lay image and convert them into text instructions. Data that form paired triplet pairs Then, the deep semantic features of both are extracted using the frozen visual foundation model, outputting a high-dimensional semantic feature map: in, , where is the feature space resolution after downsampling. Next, the extracted feature vectors are processed... After normalization, the dense cosine similarity matrix (DenseCost Volume) between the features of the model image and the features of the tiled image is calculated. For each spatial location on the model's feature map. Calculate its position relative to all locations on the tiled feature map. Similarity: Subsequently, based on the similarity matrix Application carrying temperature hyperparameter The Softmax operation calculates the spatial probability expectation and obtains the points on the model image. Initial two-dimensional absolute physical coordinates mapped onto the tiled map : in, Position on the tiled feature map The absolute coordinates of the two-dimensional grid.

[0030] Due to issues such as arms obscuring the image and clothing wrinkles in the model photo, the initial mapping... There may be incorrect matching points. To address this, this invention introduces a bidirectional mapping error check. This method calculates the inverse mapping matrix from the tiled image back to the model image using the same principle. Specifically, regarding the points on the model's picture Calculate the reprojection error after "forward-backward" cyclic mapping. Its mathematical definition is: And by constructing a Boolean validity mask Only retain pixels with an error less than a preset pixel threshold. High-confidence matching points are used to eliminate invalid mapping noise caused by occlusion and background.

[0031] Secondly, because pure visual matching lacks physical structure recognition, The data is input into a pre-trained dense pose estimation network to extract two key macroscopic priors from the model's body surface and normalize the human UV map. Human Body Parts Index These represent the 2D continuously unfolded coordinates of the human body surface and record discrete integer labels (e.g., 0 represents the background, 1 represents the torso, 2 represents the left arm, and 3 represents the right arm).

[0032] Next, the filtered microscopic high-frequency features are matched with coordinates. Macro-human topology proxy Perform channel concatenation and complete the reasonable coordinates of regions removed due to occlusion by performing smooth spatial interpolation based on a bilateral filter between valid matching points.

[0033] Finally, the system automatically outputs two absolute ground truth labels to support subsequent end-to-end network training: fine-grained semantic mask ground truth. With the true value of the 2.5D normalized deformation field ,in, Combination The part index and the attention map boundary of the visual model are used to accurately classify each image block into the category label (one-hot format) of "top", "bottom", "skin" or "background". In After fusion and smoothing Provides the true continuous physical coordinates of the image block on the model image on the flat, unposed layout of the garment; This represents a discrete topological occlusion level, using human body part indexing. Using a pre-defined topology mapping dictionary (e.g., mapping the "arm" region) Set it to 2, and set the "torso" area to 2. Derived from (setting the background to 0), this dimension fundamentally removes the overlapping front and back pieces of clothing or limb occlusion in 2D images through a three-dimensional physical space hierarchy. Thus, the generated high-precision pseudo-tags... and This will be seamlessly transferred to subsequent training steps for use as The absolute objective of the loss function, and the decoupled learning capability of strongly supervised networks.

[0034] More specifically, in a general embodiment of the present invention, the prediction network in the geometric mapping step is trained through the following steps: Obtain weakly matched model images and flat lay images of the same garment; Features of the clothing model image and the clothing flat lay image are extracted using a visual basic model, and the initial dense matching relationship between the clothing model image and the clothing flat lay image is calculated; The human topological structure prior of the clothing model image is obtained by using a human pose estimation model; Based on the prior knowledge of the human body topology, the initial dense matching relationship is verified and spatially guided to generate a corrected matching relationship; By integrating the corrected matching relationship with the prior knowledge of the human body topology, the semantic mask ground truth and deformation field ground truth used to train the prediction network are automatically generated.

[0035] The specific steps for constructing the paired image set data and data preprocessing in step S1 are as follows: S11: Filter clothing image data from the existing clothing database that are tagged with the terms "tops," "bottoms," and "full-body outfits," and use product identifiers (IDs) to identify video model images of the same clothing style. Flat lay image of clothing Perform matching and convert category labels into text instructions. Thus, a set of image triples is constructed { }; S12: Extract the matched data using the pre-trained and frozen visual base model (Vision Transformer, ViT). Deep semantic feature maps of both and Calculate pixel-level dot products on the feature maps to generate a dense cosine similarity matrix across the images. ; S13: Regarding the similarity matrix described in S12 By applying the Softmax operation and calculating the spatial expectation, the points on the model diagram are obtained. The initial corresponding coordinate mapping truth value on the tile plot In order to eliminate incorrect matching caused by occlusion (such as clothing parts covered by arms) or background noise, a two-way consistency check is performed on the mapping results. That is, only valid matching points are retained when the coordinate reprojection error is less than a preset threshold when mapping from the model image to the tiled image and then from the tiled image back to the model image. S14: Extract model images using a pre-trained Human Pose Network (DWPose). 2D Normalized UV Coordinate Distribution Map of Human Body Surface and corresponding human body part index diagram The two are combined as a macroscopic topological prior characterizing the relationship between human physical structure and occlusion. In a general embodiment of the present invention, the deformation field truth value is a 2.5D normalized deformation field truth value, which represents the continuous position mapping by the two-dimensional continuous coordinates (u, v) of each image position in the clothing model image in the standard tiling space, and represents the physical space hierarchy caused by occlusion and / or the relationship between the front and back pieces of clothing by the discrete topological hierarchy (w).

[0036] The discrete topological level (w) in the true value of the deformation field is determined in the following way: Obtain the human body part indexes output by the human pose estimation model; based on the human body part indexes and according to the mapping rules predefined for constructing the 2.5D normalized deformation field, map different human body part index values ​​to different discrete integers to form the discrete topological layer (w) used to characterize the occlusion layer in physical space.

[0037] Specifically: S15: Match the microscopic features extracted in S13 to the coordinates. Compared with the macroscopic human topological prior extracted in S14 Channel stitching is performed, and spatial interpolation is applied through smoothing filtering to automatically generate two core ground truth labels for subsequent network training without any manual annotation or 3D simulation. (Semantics and the truth value of the occlusion mask) and (Normalized deformation field truth values), which accurately label and distinguish clothing, skin, and background areas, and include continuous areas. Tiled mapping coordinates and discrete topological hierarchy derived from human body part indexes (Replacing the vague 3D physical depth, it is clearly defined as the physical spatial barrier relationship such as the discrete topological hierarchy of "front piece / back piece / left sleeve / right sleeve").

[0038] In a general embodiment of the present invention, the geometric mapping step and the feature extraction step are implemented by a two-stream decoupled encoder, including: Semantic and Deformation Prediction Steps: Using a semantic and deformation flow network, predict the semantic mask and the deformation field based on the clothing model image; Content extraction and compression steps: Perform the following operations using a content feature stream network: Extract the initial content features of the clothing model image, and use the semantic mask to suppress features of non-clothing areas in the initial content features; Using the deformation field, the physical topological distance between image locations involved in the compression of the initial content features is calculated; and the weights of the convolution operation are dynamically modulated based on the physical topological distance to securely compress the initial content features, thereby obtaining the content features.

[0039] Specifically: S2: Design a region-level hierarchical compression encoder based on two-stream decoupling and deformation awareness. The model image is input into the two-stream decoupling encoder, where the "semantic and deformation streams" predict the current semantic mask. and 2.5 D normalized deformation field "Content Feature Stream" first utilizes By masking the background and human skin features, pure local microscopic content features are extracted; subsequently, using... Guided by physical spatial distance, deformation-aware high-rate convolutional compression is performed to avoid feature cross-contamination at overlapping regions; the final output is a high-compressed content feature sequence after secure downsampling. and the characteristics of synchronously aligned deformation fields ; This invention innovatively proposes a region-level hierarchical compressed visual encoder based on dual-stream decoupling and deformation-aware technology. The dual-stream encoder architecture consists of parallel "Semantic & Warp Streams". "and Content Stream" ",in, Responsible for macroscopic cognition, predicting the semantic mask of the current image in real time. and 2.5 D normalized deformation field ; Responsible for microscopic extraction and compression, dynamically modulating convolutional weights to safely compress high-resolution sequences to [a specific value]. .

[0040] Specifically, high-resolution model images are automatically obtained from the constructed training data. Divide it into sizes of (For example: A sequence of non-overlapping image patches, spatially resized to (in The image patch sequence is then synchronously fed into the semantic and deformation streams of the dual-stream decoupled encoder. and content feature stream middle.

[0041] In semantics and deformation flow In this process, a lightweight convolutional network is used to capture global low-frequency structures. Internally, it employs a cascaded prediction mechanism "from semantics to geometry," sequentially outputting two key prior tensors (which will be subjected to the absolute truth in subsequent steps). and (Regression supervision). Then, fine-grained semantic masks are computed. To predict whether each image patch belongs to a preset category set. Probability distribution of elements such as background, skin, top, and bottom: Then, define a 2.5D normalized deformation field. (Cascaded condition prediction) will Concatenated with intermediate features from the network, this serves as a hard-constraint deformation prediction head, outputting the absolute coordinate mapping of each image patch on the "poseless clothing tiling unfolded map," including 2D continuous physical coordinates. With discrete topology occlusion levels : Next, in the content feature stream In the middle section, the front end is responsible for extracting high-frequency fabric details. To avoid high-resolution memory exhaustion, a window-based attention mechanism is used. To ensure the absolute purity of the extracted fabric features, a new feature is introduced. Predicted semantic mask This is used to extract channels identified as "background" or "human skin" and construct an inverse Boolean mask. For the first division A subset of content features within a local window Its attention interaction formula is defined as: in, Given the feature projection matrix, if the patch corresponding to the query vector (Q) belongs to the non-clothing region, The corresponding item is set to (Blocking feature weights) to output a clean microscopic feature map stripped of background noise. .

[0042] Secondly, in the content feature stream The latter part is for secure compression of high-resolution features. To address the issue of downsampling (e.g., 16x), this invention introduces a deformation-aware convolutional compression module. This module is used to compress the center point of the stride convolutional kernel. and its neighboring points within the sensing field Instead of blindly fusing 2D spatial features, it utilizes... Predicted 2.5D normalized deformation field To perform physical topology isolation, firstly, calculate the center point. With neighboring points Physical topological distance in the “2.5D tiling specification space” : in, Calculate the continuous distance on a flat fabric using an indicator function. This indicates that if two points are in the topological hierarchy... Differences (e.g., when hands and clothing obscure each other) are assigned an extremely large penalty constant. .

[0043] Subsequently, based on this physical topological distance, the network dynamically generates deformation modulation weights. Its mathematical definition is: in, The temperature parameter is learnable. Finally, a controlled stride convolution (Stride=2) is performed for computation: in, For standard convolution kernel parameters, For the activation function. With this design, if (For example: skin on the arm) covering (For example: the front of clothing at the waist) due to ,lead to Maximum, modulation weight The feature decays instantaneously to zero, and the convolutional kernel refuses to fuse its features. Thus, the system safely completes high-rate downsampling and outputs highly compressed two-dimensional content features. .

[0044] To ensure the characteristics of the compressed content It can still maintain an absolute spatial correspondence with the deformation field, for Output 2.5D normalized deformation field The pooling kernel size and stride are both executed synchronously. (For example, average pooling operation with a step size of 4): Finally, the two-dimensional compressed content features are flattened into a one-dimensional sequence. ,in .Will With the aligned deformation field Pack it up to prepare for the next steps.

[0045] More specifically, in a general embodiment of the invention, the physical topological distance ( Based on the deformation field calculation, the expression includes: the two-dimensional coordinate difference in the tiling space defined by the deformation field, and the penalty term that is not triggered simultaneously by the discrete topological level (w) in the deformation field; The weights of the dynamically modulated convolution operation based on the physical topological distance are calculated by... The modulation weights (G) are generated using a negative exponential function of the physical topology distance ( ), where the modulation weights (G) are the physical topology distance ( ). It is a monotonically decreasing function.

[0046] The method of the region-layer compression encoder based on dual-stream decoupling and deformation sensing in step S2 is as follows: S21: Upgrade high-resolution model images Divided into sizes The non-overlapping image patch sequences are respectively input into the parallel "semantic and deformable flow networks" in the two-stream decoupled encoder. and "Content Feature Stream Network" ; S22: In Semantic and Deformational Flow Networks In this process, by employing lightweight convolutional networks or shallow Transformer architectures, global structural and pose information of images is captured, and two-level prior-guided tensor fine-grained semantic masks are output sequentially. and 2.5 D normalized deformation field ,in, Predict the probability distribution map of each image patch belonging to a preset semantic category (such as background, human skin, upper clothing, lower clothing, etc.); Based on the aforementioned semantic mask features, cascaded conditional prediction is performed, outputting the absolute coordinate mapping of each image patch on the poseless clothing tiling unfolded image, including: 2D continuous physical coordinates. Discrete topological hierarchy representing the occlusion relationship between front and rear pieces ; S23: In content feature flow networks In the middle stage, the front end uses a pure local window-based attention mechanism to extract high-frequency fabric details to avoid memory overflow at high resolutions; at this stage, the receiver... fine-grained semantic mask of output Extract the channels identified as "background" or "human skin" to construct an inverse Boolean mask. Injecting attention into the attention calculation within a local window Force the attention weight of non-clothing area tiles to be reset to a minimum value (e.g.: This cuts off the inflow of background and skin tone features, outputting a pure local microscopic fabric feature map. ; S24: In order to radically compress the length of high-resolution sequences Times (e.g., 16 times) A deformation-aware convolutional compression module is introduced in the middle stage to receive... Output 2.5D normalized deformation field For the center point of the convolution kernel and its neighboring points within the sensing field Calculate them in Physical topological distance in coordinate system ; and based on the physical topology distance Network dynamically generates deformation modulation weights This is then used as a multiplication factor in the stride convolution operation. Through this design, even if the model's limbs severely overlap and occlude in the 2D image, the compression process only fuses continuous clothing features from the real 3D physical space, outputting a highly compressed feature sequence free from overlap contamination. ; S25: To ensure the characteristics of the compressed content It can still maintain an absolute spatial correspondence with the deformation field, for Output 2.5D normalized deformation field Both the core size and step size are synchronized. An average pooling downsampling operation (e.g., with a step size of 4) yields an aligned compressed deformation field. Finally, the highly compressed content features, stripped of pose and background noise, are extracted. Flattened into a one-dimensional sequence , and the aligned deformation field The data is then synchronously transmitted to the subsequent global interaction layer (i.e., step S3) for implicit virtual flattening.

[0047] In a general embodiment of the present invention, the deformation field is a 2.5D normalized deformation field, which includes two-dimensional continuous coordinates (u, v) of each image position in the clothing model image in a standard tiling space and a discrete topological level (w). The spatial reconstruction step also includes: Based on the coordinates (u, v) and the level (w) contained in the 2.5D normalized deformation field, a corresponding rotation position encoding matrix is ​​generated for the content feature sequence; Using the rotation position encoding matrix, the query vector and key vector of the content feature sequence in the self-attention calculation are transformed respectively; The attention weight is calculated based on the transformed query vector and key vector, wherein the attention weight is determined by the difference between the coordinates (u, v) of the image positions corresponding to the query vector and the key vector and the difference between the levels (w). The value vectors in the self-attention calculation are weighted and aggregated based on the attention weights to generate the flattened clothing features.

[0048] Specifically: S3: Output of S2 and The data is fed into the global interaction layer and extracted using a 2.5D Rotary Position Embedding (2.5D RoPE) mechanism based on topological semantic deformation fields. 2.5D geometric coordinates prior for each token ,in Represents continuous tiled UV coordinates. Representing discrete topological occlusion levels, this is transformed into an orthogonal 2.5D Rotated Position Encoding (RoPE) matrix and injected into the Query and Key of the attention mechanism. The feature interaction weights are forced to be determined by the true physical distance in the standard tiling space, implicitly eliminating clothing distortion caused by the model's pose in the latent space, outputting perfectly decoupled and upright clean clothing features. ; For the highly compressed content feature sequence output in the process ,in, The total number of tokens. For the channel dimension, and the 2.5D normalized deformation field aligned with synchronous average pooling. Simultaneously, it is input into the final dense global attention layer. Specifically, firstly, Linear projection mapping to query vector matrix Key vector matrix Sum value vector matrix For the first in the sequence Each image block (Token) is extracted from its... The corresponding geometric prior coordinate vector ,in, Represented as continuous two-dimensional tiled UV coordinates. is a discrete topological level integer, representing the physical depth obstruction caused by limb occlusion or the front and back pieces of clothing.

[0049] In order for the self-attention mechanism to simultaneously perceive the geometric priors of the above three dimensions, for the third... Query vector of each token and key vector In channel dimension Divide the above equally, and Divide into three sub-vector spaces, each with an independent response. Physical information of the shaft: Each subvector has a dimension of 1. Key vector The same dimensional splitting is performed.

[0050] Then, for continuous planar coordinates... and Construct a conventional two-dimensional continuous rotation position matrix. Using the coordinate axes... For example, define a rotating fundamental frequency sequence that decreases proportionally. ,in, ( Preferred For subvectors The first in For adjacent elements Based on the extracted corresponding physical coordinate values Calculate two-dimensional rotational transformations: Then all of them Rotation blocks are combined using block diagonal matrices to obtain specific results for each rotation block. shaft and Continuous rotation matrix of axes and .

[0051] because The axis represents a discrete occlusion relationship (e.g., 0 for the front piece and 1 for the back piece). If the continuous rotation described above is used, the model cannot completely sever the physical spatial break in the relationship, resulting in ghosting of the patterns in the front and back pieces during generation. This invention addresses... Axis introduces a topological orthogonal phase transformation with a scaling factor: setting a maximal phase scaling constant. (Preferred) This constant forces different topological levels (e.g.) and The difference in rotation angle between the two points is instantly stretched to an orthogonal state. Its rotation matrix... The calculation formula is configured as follows: The generated three single-axis rotation matrices Perform diagonal merging of blocks to obtain the result for the first... The complete union of the tokens in a 2.5D rotated position encoding matrix : Therefore, the joint rotation matrix is ​​applied to the original query vectors of all tiles respectively. and key vector : Subsequently, in the global self-attention calculation, based on the orthogonality property of the rotation matrix, the query vector after 2.5D RoPE is injected. With key vector The self-attention inner product is transformed into: Through the rigorous mathematical transformations described above, the inner product weights of the attention interaction are entirely determined by the difference in the actual physical coordinates of the two image patches on the "standard tiled unfolded map". The dominant factor is its position on the 2D distorted model image, and it has nothing to do with its position on the image. That is, if two tokens are adjacent on the 2D image (like a hand and waist in a pose), but... If the coordinates are far apart on the tiled coordinates, then Its maximum value causes its inner product to automatically decay, achieving "topological automatic bounce" in the latent space. If it encounters tokens with the same UV coordinates but belonging to the front / back sheet or arm occlusion (i.e.,...), it will automatically bounce back. ),because The scaling effect produces a 90-degree orthogonal phase shift, and the inner product of this axis dimension is forced to be equal to 0, achieving "absolute physical isolation" between levels.

[0052] Ultimately, through Aggregate the value vector matrix of uninjected positional encoding It outputs pure clothing features that are perfectly stripped of distorted postures, completely decoupled, and with a well-proportioned shape. The input instructions drive the diffusion autoregressive decoder to generate high-fidelity images at extremely high speed.

[0053] More specifically, in a general embodiment of the present invention, the method for 2.5D rotational position encoding based on topological semantic deformation field in step S3 is as follows: S31: Receive the highly compressed content feature sequence output in step S2. And a 2.5D normalized deformation field aligned by synchronous pooling. In the final Dense GlobalAttention layer, firstly... Linear projection mapping to query vector matrix Key vector matrix Sum value vector matrix Meanwhile, for the first in the sequence Each image block (Token) is extracted from its... The corresponding geometric prior coordinate vector ,in For continuous two-dimensional tiled UV coordinates, A discrete topological layer integer representing the physical depth occlusion relationship; S32: In order for the attention mechanism to perceive multi-dimensional geometric priors, for the first... Query vector of image patches and key vector In channel dimension Divide the above equally; Divide into three sub-vector spaces, each with an independent response. Physical information of the three coordinate axes Each subvector has a dimension of 1. For key vectors Similarly, the corresponding dimension segmentation is performed to achieve three-axis equal projection of the feature dimension; S33: For continuous planar flattened coordinates and Construct a continuous rotation position matrix with coordinate axes For example, define a rotating fundamental frequency sequence that decreases proportionally. ,in ( Usually taken For subvectors The first in For adjacent elements Based on the corresponding physical coordinate values Calculate two-dimensional rotational transformations: All of Rotation blocks are combined using block diagonal matrices to obtain specific results for each rotation block. shaft and Continuous rotation matrix of axes and ; S34: Discrete topological hierarchy for characterizing the occlusion relationship between front and back sheets or inner and outer layers. By introducing a topological orthogonal phase transform with a scaling factor, a maximal phase scaling constant is defined. (Preferred) ), making different topological levels (e.g.: and The rotation angles between them are extremely different, ensuring physical barrier properties, and their rotation matrix... The calculation formula is configured as follows: ; S35: Combine the three single-axis rotation matrices generated in steps S33 and S34. Perform diagonal merging of blocks to obtain the complete target for the first... Joint 2.5D rotation position encoding matrix of image patches : Apply the joint rotation matrix to the original query vectors of all tiles respectively. and key vector : ; S36: Based on the orthogonality property of the rotation matrix, calculate the self-attention inner product of the query vector and the key vector after injecting 2.5D RoPE: At this point, the weights of the attention interaction are entirely determined by the difference in the actual physical distance between the two image patches on the "standard tiled unfolded map". The decision is based on the grid distance of the original 2D image, rather than the distance between the two images, when they encounter images with the same UV coordinates but belonging to different front and back layers. When dealing with the tile, because The scaling effect produces an orthogonal phase shift, forcing the inner product to approach 0, achieving absolute physical isolation between layers. Ultimately, through... Aggregate uninjected position-encoded value vectors In the implicit space, the distortion and deformation of clothing caused by the model's posture are implicitly offset, resulting in a pure garment with perfectly decoupled and well-proportioned features. .

[0054] In a general embodiment of the present invention, the image output step, which generates a high-fidelity clothing flat lay image through parallel decoding, includes: The features of the flattened clothing are combined with the features of the text instruction to construct a joint condition; Autoregressive generation is performed on a feature block basis. For each feature block to be generated, the following operations are performed: Initialize the noise tensor, and guided by the joint conditions and the generated historical features, denoise the noise tensor in a small number of iteration steps using a lightweight diffusion model; The denoised tensor is binarized to generate all high-entropy binary discrete feature representations within the feature block in parallel; The discrete feature representation sequence generated from all feature blocks is decoded into the high-fidelity clothing flat lay image.

[0055] Specifically: S4: In the core of the autoregressive graph model, this invention designs an instruction-driven block-level joint diffusion decoding architecture. This architecture first abandons the traditional restricted VQ vocabulary and introduces a high-entropy binary representation based on a lookup-free mechanism for the target tiled graph. Binarization activation is performed to construct a high-entropy binary discrete target sequence with an extremely high state space. Simultaneously, an instruction-driven block-level joint diffusion decoder is constructed to process text instructions. Features and The concatenation is used as a conditional prefix. Finally, the decoder performs autoregression on a feature patch basis and fits the joint probability distribution of dozens of high-dimensional discrete tokens within the feature patch at the prediction output through a continuous space "Diffusion Head". In the training phase, to construct a highly accurate generated target, the first step is to obtain a real clothing flat lay image prepared during the data preprocessing phase. (Ground Truth) The input is fed into the encoder of a pre-trained pure convolutional visual autoencoder to extract its feature tensor in the continuous latent space. ,in The total length of the target visual token sequence. For channel dimensions (e.g.: Meanwhile, in order to overcome the capacity bottleneck of the classification head under the traditional VQ mechanism, this invention abandons explicit codebook clustering search; and directly targets the latent features. Each channel applies an independent sign activation function: This operation will Mapped to a state with limit-state representation capability (single-token state space up to...) High-entropy binary discrete feature truth sequence The upper limits of mathematics and physics guarantee the pixel-level lossless reproduction of extremely complex fabrics.

[0056] Next, obtain the text instructions representing the user's intent. (For example, "extracting the top tile from a model image"), and extracting its cross-modal text features using a pre-trained large text encoder (e.g., CLIP Text Encoder). Receives the upright clothing features output from the preceding steps, which have already been virtually flattened in the latent space. and will and Concatenation is performed along the sequence dimension to construct a joint conditional prefix to guide the generation of autoregression. : Then, to overcome the performance bottleneck of traditional autoregressive models that generate tokens sequentially, this invention adopts a patch-level parallel generation paradigm, which generates the high-entropy target sequence to be generated. Divided into Each feature block is a non-overlapping spatial feature block. Include A high-entropy binary token (e.g.: Each block contains 16 tokens, i.e. Meanwhile, the decoder adopts a Transformer architecture and performs conditional probability autoregressive modeling on a block-by-block basis: in, These are historical feature blocks that have already been generated. Inside the decoder, the cross-attention mechanism is based on... Precise addressing The corresponding category features (e.g., response weights for quickly filtering out lower-end features) are used to guide the generation of strong zero-shot instructions and output the hidden state context of the current block. ( (For hidden layer dimensions).

[0057] Because each feature block to be predicted Include Each independent binary bit, if used in a traditional classification head for prediction, would cause an astronomical explosion of parameters (e.g., requiring output...). To address this issue, this invention abandons the classifier at the Transformer output and introduces a lightweight Continuous Diffusion Model to fit the joint probability distribution of this set of high-dimensional discrete tokens. During the training phase, for the _th_ ... A real discrete block In time step Introducing standard Gaussian noise Then, based on the Rectified Flow framework, a linear interpolated noisy state is constructed that points from pure noise to the real discrete vertices. : Therefore, the target velocity field for seamless transition from this state to the real data is defined as: Subsequently, a lightweight diffusion prediction network was trained. By minimizing the diffusion velocity matching loss: This loss function forces the network to operate within a given autoregressive context. At the same time, learn how to precisely guide a mass of high-dimensional continuous Gaussian noise to discrete feature sets at the vertices of a high-dimensional hypercube. .

[0058] During the online inference phase, for the currently unknown feature blocks to be generated... By initializing a mass of pure Gaussian noise tensor It also utilizes the Euler Solver for ordinary differential equations to predict the velocity field based on the trained network. Under the conditions Under precise guidance, execute a small number of Step (preferred) Discrete iterative integral: Among them, step size The points reached Obtain continuous denoised tensors Then, hard binary constraint projection is applied, mathematically defined as follows: This operation precisely maps continuously denoised tensors back to the vertices of discrete hypercubes. The system can acquire dozens (e.g., 16) high-entropy discrete tokens within the patch in a single forward propagation. .cycle After the entire image prediction is completed, it is fed into the decoding end of the pre-trained visual autoencoder, which losslessly maps it back to the RGB pixel space and outputs a high-resolution, highly detailed tiled image. .

[0059] More specifically, in a general embodiment of the present invention, the instruction-driven block-level joint diffusion decoding architecture method in step S4 is as follows: S41: During the training phase, in order to construct a lossless and faithful generation target, a realistic garment flat lay image is obtained. (Ground Truth); will The input is fed into the encoder of a pre-trained pure convolutional visual autoencoder to extract its feature tensor in the continuous latent space. ,in The total length of the target visual token sequence. For channel dimension (e.g.) ); S42: To overcome the vocabulary capacity bottleneck caused by the limited memory of the classification head in traditional Vector Quantization (VQ) (e.g., small vocabulary easily leads to loss of fabric details), this invention abandons explicit codebook storage and lookup operations; it directly applies the continuous latent features... Each channel dimension is binarized using an independent sign activation function: Through this binarization operation, Mapped to a state with limit-state representation capability (single-token state space up to...) High-entropy binary discrete feature sequences The upper limits of mathematics and physics guarantee the pixel-level lossless restoration of extremely complex fabrics; S43: Obtain text commands input by the user. (For example, "extracting the top tile from a model image"), and extracting its cross-modal text features using a pre-trained large text encoder (e.g., CLIP Text Encoder). Simultaneously, it receives the clean clothing features output from step S3, which have already undergone implicit virtual flattening and pose decoupling in the latent space. and will and Feature concatenation is performed along the sequence dimension to construct a joint conditional prefix to guide autoregressive generation. : ; S44: To overcome the bottleneck of the slow, sequential generation of traditional autoregressive models using a "token-by-token" approach, this invention employs a patch-level parallel generation paradigm, which generates the high-entropy target sequence to be generated. Divided in spatial dimension Each feature block is a non-overlapping feature block. Include High-entropy binary tokens (e.g.) Each block contains 16 tokens, i.e. The decoder uses a Transformer architecture and performs probabilistic conditional autoregressive modeling on a block-by-block basis. in, For the already generated historical feature blocks; during the decoding process, the cross-attention mechanism is based on... Precise addressing Based on the corresponding category features, strong zero-shot instruction-guided generation is achieved; S45: Due to each feature block to be predicted Include Each individual binary bit, if predicted using a traditional Softmax classification head, would cause an astronomical explosion of parameters (e.g., ...). (There are multiple possible classifications). To address this, at the output of the Transformer decoder, this invention abandons the bulky classifier and innovatively introduces a lightweight Continuous Diffusion Model to fit the joint probability distribution of this set of high-dimensional discrete tokens. S46: During training, for the first... A real discrete feature block In time step Introducing standard Gaussian noise Simultaneously, based on the Rectified Flow framework, a linear interpolated noisy state is constructed that points from pure noise to the real discrete vertices. : And define the target velocity field for seamless transition from this state to real data as: Let the hidden state output by the autoregressive Transformer in the current step be... Train a lightweight prediction network Then, by minimizing the diffusion velocity matching loss, Forcing the network to operate within a given autoregressive context At the same time, learn how to accurately guide pure Gaussian noise to discrete feature sets at the vertices of a high-dimensional hypercube. ; S47: In actual reasoning generation, the first When initializing a feature block, a pure Gaussian noise tensor is used. It also utilizes the Euler Solver for ordinary differential equations to predict the velocity field based on the trained network. Under the conditions Under precise guidance, execute a small number of Step (preferred) Discrete iterative integration: Where the step size is When the score reaches Obtain continuous tensors Then, perform hard binarization constraint projection: This operation precisely "locks" the continuous tensor back to the vertices of the discrete hypercube, allowing dozens of high-entropy discrete tokens within the patch to be output in parallel with a single forward propagation. The loop completes the whole Figure 2 After the hexadecimal sequence is predicted, it is fed into the decoding end of the visual autoencoder and losslessly restored into the final high-resolution tiled image.

[0060] S5: Using the paired data generated in S1, perform joint parameter optimization on the end-to-end network composed of S2-S4, and calculate the predicted output of "semantic and deformation flow" and the true value of the pseudo-label in S2. , The regression loss is calculated, and Gaussian noise is input into the diffusion matching head of S4, conditioned on the current hidden state of the decoder, to calculate the path from pure noise to the high-entropy discrete target vertex. The diffusion rate is matched with the loss, and the parameters of the encoder and decoder are simultaneously optimized through gradient backpropagation; Specifically, in a general embodiment of the present invention, the method for joint parameter optimization in step S5 is as follows: S51: At the beginning of each training iteration, start from the set of training triples constructed in step S1. The training data consists of a batch of paired triplets, and the absolute ground truth labels pre-generated by S1 are extracted from this batch of data; that is, the fine-grained semantic mask ground truth. True values ​​of the 2.5D normalized deformation field At the same time, flat lay images of the actual clothing in this batch were also provided. The input is fed into a pre-trained visual autoencoder and binarized using a sign activation function to pre-construct a high-entropy binary discrete feature target sequence for the decoder to learn. ; S52: Extract high-resolution model images from the sampling batch. The input is fed into the dual-stream decoupled encoder in step S2; in "Semantic and Deformation Streams"... During the forward propagation process, the semantic mask predicted by the network in real time is obtained. and 2.5 D normalized deformation field The two prior fields are strongly supervised using pre-generated truth labels, and the prior regression loss is calculated. ; S53: In the "Content Feature Stream" In the middle, using the above predictions Generate an inverse Boolean mask to remove background features, and utilize... Calculate the physical topological distance to guide the deformable high-magnification convolution module to perform safe feature downsampling and output a highly compressed content feature sequence. ; then, Deformation field aligned with pooling The data is fed into the global interaction layer in step S3. By constructing a 2.5D rotation position encoding (2.5D RoPE) matrix and injecting an attention mechanism, implicit virtual flattening of features is completed in the latent space, outputting pure and clean clothing-specific features. ; S54: Text command Transform into features Features after flattening Concatenate along the sequence dimension to construct a joint conditional prefix. The high-entropy binary target sequence pre-constructed in step S51 according to Size division into A continuous feature block; condition prefix and the historical true feature block sequence offset by one bit (TeacherForcing strategy) The input is fed into the block-level joint autoregressive Transformer in step S4 for parallel forward propagation computation, and the output is the decoder hidden state context corresponding to each feature block position. ( ); S55: At the output of the autoregressive Transformer, for each real target feature block... In time step ( From uniform distribution Under random sampling, standard Gaussian noise is introduced. Constructing linear interpolation noisy states in continuous space and define the target velocity field Noisy state Current time step and the corresponding hidden state context output in step S54. As a joint input, it is fed into a lightweight diffusion prediction network. In the process, the velocity field predicted by the computational network is calculated. With the target velocity field The mean square error between them yields the diffusion velocity matching loss. ; S56: Calculate the prior regression loss in step S52. Matching loss with the diffusion rate calculated in step S55 We perform weighted combinations to construct the total loss of multi-task joint optimization for the entire network. Using gradient descent optimization algorithms (such as the AdamW optimizer), based on the total loss Calculate the gradients of all learnable parameters in the network, perform end-to-end backpropagation, and synchronously update the two-stream decoupled encoder. ), global interaction layer, autoregressive Transformer decoder, and block-level joint diffusion matching head ( The weight parameters are adjusted until the model converges on the validation set or reaches the preset maximum number of training iterations.

[0061] S6: In practical applications, users can upload any clothing model image. and text instructions (For example: extracting a flat lay image of a model's top), the image is input into a trained autoregressive network, passed through networks S2 and S3 for forward propagation, and features are extracted from the input image using a two-stream encoder, resulting in a rapidly outputting compressed and decoupled clothing features. ; generator with and For joint conditions, autoregressive generation is enabled on a per-patch basis. For each unknown feature patch, pure noise is initialized and a small amount of Euler integration is performed using the trained diffusion head for denoising. Under precise guidance, irrelevant lower-level and background features are quickly filtered out. Dozens of high-entropy discrete tokens are output in parallel during a single forward propagation, and the process is repeated to complete the entire process. Figure 2 number system sequence The predictions are then fed into a pre-trained visual autoencoder (Decoder) to losslessly reconstruct a high-resolution stereoscopic tiled image. And return it to the user; During the reasoning process, the system first receives the original model image uploaded by the user without any background removal processing. and text instructions used to specify the extraction intent. (For example: "Extract a flat lay image of the top worn by the model"). Scale uniformly to the target high resolution and divide into portions of size [size missing]. (like The sequence of non-overlapping image patches is used. Simultaneously, a large text encoder aligned with the training phase (e.g., CLIP Text Encoder) is invoked to process natural language instructions. Encoded as a fixed-length cross-modal text feature vector Load it into video memory for later use.

[0062] Then, the image sequence after being divided into blocks The input is directly fed into the dual-stream decoupled encoder that has already completed joint optimization, entering the semantic and deformation streams. It instantly outputs the specific image through a single feedforward network computation. Two core prior tensors: fine-grained semantic masks and 2.5 D normalized deformation field Then enter the content feature stream. In the first level of local window attention, utilizing Automatically construct inverse Boolean mask Instantly resetting the focus of attention away from "background" and "human skin color" Obtain pure microscopic features after removing the background. ,and When performing a 16x compression ratio, utilize Calculate the center point within the receptive field of the convolution. With neighboring points Physical topological distance Simultaneously, the network automatically generates spatial modulation weights. When encountering a model's arm covering clothing (i.e., depth layer) In the case of mutation, The decay rate reaches zero, automatically isolating feature contamination. Ultimately, Secure downsampling outputs highly compressed content feature sequences ,at the same time Extremely fast average pooling, output aligned downsampled deformation field .

[0063] Next, and Synchronously sent to the global interaction layer, utilizing Geometric coordinate prediction values ​​for each image patch The system constructs a 2.5D rotation position encoding (2.5D RoPE) matrix of block diagonals online in real time. This is then injected into the query and key vectors of the global self-attention mechanism. At this point, the weights of feature interactions are entirely determined by the coordinate differences in the "standard tiled physical space". The decision is made that the network automatically calculates the flattened topology within the latent space, completely offsetting the previous structure. The non-rigid distortion and adhesion between the front and back pieces of clothing, resulting from the model's posture, contribute to the perfectly flattened and pure appearance of all types of garments. .

[0064] Subsequently, the system extracts text features With flattening features Concatenate along the sequence dimension to construct a joint conditional prefix: Then In the block-level joint autoregressive transformer (Autoregressive Transformer), in the cross-attention layer of the decoder, the network... The semantics of "upper garment" automatically focus on The image features that match the upper garment are used, and the response weights of the "lower garment" and redundant background are attenuated to 0, so as to achieve zero-sample accurate category addressing without manual masking.

[0065] Secondly, based on the high-resolution of the tiled image, the system presets the total number of feature blocks to be generated as follows: Each block contains (For example, 16) high-entropy binary feature bits, initialize the sequence to generate pointers. And targeting the first An unknown feature block, the decoder in joint condition and the generated history blocks Under guidance, output the hidden state context of the current block. Simultaneously, a pure Gaussian noise tensor is directly initialized in a continuous space. Call the trained continuous spatial diffusion matching network ,exist With precise guidance, very few steps are required (e.g. The discrete denoising iteration of ) is mathematically defined as follows: Among them, step size ,when When the integration stops, the continuously denoised tensor is obtained. Perform an absolute sign function truncation on it: A single loop can acquire 16 high-fidelity binary tokens within this patch in parallel. This completely breaks through the computing power bottleneck of traditional autoregressive character-by-character generation, and updates... Until the entire binary feature sequence of the whole image. Prediction complete.

[0066] Then, the rapidly generated full high-entropy binary discrete sequence The decoder, fed into a pre-trained visual autoencoder that is strictly aligned with the training phase, benefits from the Lookup-Free Quantization (LFQ) mechanism, due to the sequence... Each token inherently contains With its ability to express extreme states, the Decoder does not need to perform any vocabulary comparison. It directly and losslessly reverse maps the high-dimensional discrete feature space back to the RGB pixel space through the transposed convolution network.

[0067] Finally, the system renders and outputs a high-resolution, extremely detailed (such as the finest textures of clothing and perfectly preserved stitching) flat lay image of the top, with a well-proportioned silhouette that precisely matches the user's intended design. Then, return to the system front end for display.

[0068] More specifically, in a general embodiment of the present invention, The method for reasoning using a trained autoregressive network based on the user-input clothing model image and text instructions in step S6 is as follows: S61: In practical application scenarios, the system receives any real-life photos of clothing models uploaded by the user. and natural language text instructions used to specify the target category to be extracted. (For example: "Extract the flat lay image of the model's top"), Perform resolution alignment and cropping, dividing the data into portions of size [size missing]. The non-overlapping image patch sequence is used as online input to the test set. Simultaneously, the large text encoder pre-trained in step S4 is used to... Extracted as cross-modal text conditional features ; S62: Process the image patch sequence The data is directly input into the dual-stream decoupled encoder that has undergone joint optimization in step S5, without any manual masking. Simultaneously, it integrates semantic and deformation flows. Rapid forward prediction of the semantic mask of the current model image With 2.5 D normalized deformation field "Content Feature Stream" take over Automatically masks background and human skin tone, extracting local microscopic fabric features. Subsequently, using This guides high-ratio deformable convolutional compression operations, ensuring complete isolation of overlapping features caused by model poses, and outputs a high-concentration content feature sequence with safe downsampling (e.g., 16x compression) in a single forward pass. and the deformation field of synchronous pooling alignment ; S63: Will and Send it to the global interaction layer, based on the extracted 2.5D geometric coordinates prior for each image patch The system constructs a 2.5D rotation position encoding (2.5D RoPE) matrix of block diagonals online. This matrix is ​​injected into the Query and Key vectors of the self-attention mechanism, where it is implicitly canceled out in the latent space. The non-rigid distortion and deformation of clothing caused by the model's posture outputs a clean garment feature that is perfectly decoupled and physically flattened. ; S64: Extract the text conditional features from step S61. The clothing features output in step S63 Feature concatenation is performed along the sequence dimension to construct a joint conditional prefix specific to the current generation task. Combined condition prefix Input is fed into the block-level joint diffusion decoding architecture (i.e., autoregressive Transformer) to initialize the pointers to the generated sequences. The total number of blocks to be generated is preset to be [number]. (Each block contains) (One unknown binary discrete feature bit), prepare to start parallel autoregressive decoding in block (Patch) order; S65: For the currently pending generation of the first Given an unknown feature block, the autoregressive Transformer uses a joint conditional prefix. With the generated historical feature blocks Guided by (initially empty) conditions, precise targeting is achieved through a cross-attention mechanism. Chinese and text instructions The matched feature regions (e.g., quickly filtering out lower garment and background features) are output, along with the hidden state context of the current step. At the Transformer output, the system initializes a mass of pure Gaussian noise tensors. And utilize the trained continuous spatial diffusion matching network ,by As a condition, perform a small number of Euler steps of ordinary differential equations (preferred). (intra-step iteration) Points reach the finish line Obtain continuous denoised tensors Then, apply hard binarization constraints. This operation can output dozens (e.g., 16) high-entropy binary discrete tokens in parallel with a single forward propagation. This significantly overcomes the performance bottleneck of traditional autoregressive token generation and pointer updating. Repeat step S65 until the entire process is complete. Figure 2 number system sequence The prediction; S66: Generate the full high-entropy binary discrete sequence The input is fed into the decoder of the visual autoencoder corresponding to step S41 of the training phase, due to the sequence Inherited from the Lookup-Free Queuing (LFQ) mechanism, Gundam With its ability to express extreme states, the Decoder can losslessly map extremely high-dimensional, high-entropy states back to the RGB pixel space, ultimately outputting a high-resolution image that is rich in detail (such as perfectly replicating stitching and fabric texture), with a well-proportioned shape and precise conformity to the text instructions. Expected 3D clothing flat lay image This is then used as the final output of the system and returned to the user for display.

[0069] This invention also provides a system for converting model images into stereoscopic tiled images based on physical perception, comprising: The information acquisition module retrieves images of clothing models and associated text instructions. The geometric mapping module inputs the clothing model image into the prediction network to obtain a semantic mask and a deformation field that represents the geometric mapping relationship of the clothing from the model's pose space to the standard tiling space. The feature extraction module extracts content features related to clothing from the clothing model image based on the semantic mask, and suppresses interference from non-clothing areas. The spatial reconstruction module reconstructs the spatial relationships of the content features based on the deformation field, so as to eliminate the clothing deformation caused by the model's posture in the feature space and obtain the flattened clothing features. The image output module generates a high-fidelity flat lay image of the garment through parallel decoding, based on the text instructions and the flattened garment features.

[0070] The present invention also provides a computer-readable storage medium having stored thereon computer program code, which, when loaded into and executed on a computer system, causes the computer system to perform all the steps of the method according to any one of claims 1-8.

[0071] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, system, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0073] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0074] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0075] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0076] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0077] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0078] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0079] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for converting a model image into a 3D tiled image based on physical perception, characterized in that: The method includes the following steps: Information acquisition steps: Obtain clothing model images and associated text instructions; Geometric mapping step: Input the clothing model image into the prediction network to obtain the semantic mask and the deformation field representing the geometric mapping relationship of the clothing from the model pose space to the standard tiling space; wherein, the deformation field is a 2.5D normalized deformation field, which includes the two-dimensional continuous coordinates (u,v) of each image position in the clothing model image in the standard tiling space and a discrete topological level (w). Feature extraction steps: Based on the semantic mask, extract content features related to clothing from the clothing model image and suppress interference from non-clothing areas; Spatial reconstruction steps: Based on the deformation field, the spatial relationship of the content features is reconstructed to eliminate the clothing deformation caused by the model's posture in the feature space and obtain the flattened clothing features. Image output steps: Based on the text instructions and the flattened clothing features, generate a high-fidelity clothing flat lay image through parallel decoding; The geometric mapping step and the feature extraction step are implemented through a dual-stream decoupled encoder, which includes a semantic and deformation stream network and a content feature stream network. In the geometric mapping step, the semantic mask and the deformation field are predicted based on the clothing model image through the semantic and deformation flow network. In the feature extraction step, the initial content features of the clothing model image are extracted through the content feature flow network, and the features of non-clothing areas in the initial content features are suppressed using the semantic mask; the physical topological distance between the image positions involved in the compression of the initial content features is calculated using the deformation field, and the weights of the convolution operation are dynamically modulated based on the physical topological distance to securely compress the initial content features, thereby obtaining the content features; Wherein, the physical topological distance ( Based on the deformation field, the expression includes the two-dimensional coordinate difference in the tiling space defined by the deformation field, and the penalty term for different discrete topological levels (w) in the deformation field being triggered simultaneously; by calculating the physical topological distance ( The negative exponential function of ) generates modulation weights (G), and the modulation weights (G) are used to dynamically modulate the weights of the convolution operation, wherein the modulation weights (G) are the physical topological distance ( A monotonically decreasing function; In the spatial reconstruction step, a corresponding rotational position encoding matrix is ​​generated for the content feature sequence based on the coordinates (u,v) and level (w) contained in the 2.5D normalized deformation field. Using the rotational position encoding matrix, the query vector and key vector of the content feature sequence in the self-attention calculation are transformed respectively. Based on the transformed query vector and key vector, attention weights are calculated, wherein the attention weights are determined by the difference between the coordinates (u,v) and level (w) of the image positions corresponding to the query vector and key vector. Based on the attention weights, the value vectors in the self-attention calculation are weighted and aggregated to generate the flattened clothing features.

2. The method for converting a model image into a stereoscopic tiled image based on physical perception according to claim 1, characterized in that: The prediction network in the geometric mapping step is trained through the following steps: Obtain weakly matched clothing model images and clothing flat lay images for the same garment; Features of the clothing model image and the clothing flat lay image are extracted using a visual basic model, and the initial dense matching relationship between the clothing model image and the clothing flat lay image is calculated; The human topological structure prior of the clothing model image is obtained by using a human pose estimation model; Based on the prior knowledge of the human body topology, the initial dense matching relationship is verified and spatially guided to generate a corrected matching relationship; By integrating the corrected matching relationship with the prior knowledge of the human body topology, the semantic mask ground truth and deformation field ground truth used to train the prediction network are automatically generated.

3. The method for converting a model image into a stereoscopic tiled image based on physical perception according to claim 2, characterized in that: The true value of the deformation field is the 2.5D normalized deformation field true value. The 2.5D normalized deformation field true value is characterized by the continuous position mapping of each image position in the clothing model image in the standard tiling space using two-dimensional continuous coordinates (u,v), and by the discrete topological hierarchy (w) to characterize the physical spatial hierarchy caused by occlusion and / or the relationship between the front and back pieces of clothing.

4. The method for converting a model image into a stereoscopic tiled image based on physical perception according to claim 3, characterized in that: The discrete topological level (w) in the true value of the deformation field is determined in the following way: Obtain the human body part indexes output by the human pose estimation model; Based on the human body part index and according to the mapping rules predefined for constructing the 2.5D normalized deformation field, different human body part index values ​​are mapped to different discrete integers to form the discrete topological layer (w) for representing the occlusion layer in physical space.

5. The method for converting a model image into a stereoscopic tiled image based on physical perception according to claim 1, characterized in that: In the image output step, a high-fidelity flat lay image of the clothing is generated through parallel decoding, including: The features of the flattened clothing are combined with the features of the text instruction to construct a joint condition; Autoregressive generation is performed on a feature block basis. For each feature block to be generated, the following operations are performed: Initialize the noise tensor, and guided by the joint conditions and the generated historical features, denoise the noise tensor in a small number of iteration steps using a lightweight diffusion model; The denoised tensor is binarized to generate all high-entropy binary discrete feature representations within the feature block in parallel; The discrete feature representation sequence generated from all feature blocks is decoded into the high-fidelity clothing flat lay image.

6. A system for converting model images into stereoscopic tiled images based on physical perception, characterized in that, include: The information acquisition module is used to acquire images of clothing models and associated text instructions; The geometric mapping module is used to input the clothing model image into the prediction network to obtain a semantic mask and a deformation field that represents the geometric mapping relationship of the clothing from the model pose space to the standard tiling space; wherein, the deformation field is a 2.5D normalized deformation field, which includes the two-dimensional continuous coordinates (u, v) of each image position in the clothing model image in the standard tiling space and a discrete topological level (w). The feature extraction module is used to extract content features related to clothing from the clothing model image based on the semantic mask, and suppress interference from non-clothing areas; The spatial reconstruction module is used to reconstruct the spatial relationship of the content features based on the deformation field, so as to eliminate the clothing deformation caused by the model's posture in the feature space and obtain the flattened clothing features. The image output module is used to generate a high-fidelity flat lay image of the garment by parallel decoding, based on the text instructions and the features of the flattened garment. The geometric mapping module and the feature extraction module are implemented through a dual-stream decoupled encoder, which includes a semantic and deformation stream network and a content feature stream network. The geometric mapping module predicts the semantic mask and the deformation field based on the clothing model image through the semantic and deformation flow network. The feature extraction module extracts the initial content features of the clothing model image through the content feature flow network, and uses the semantic mask to suppress the features of non-clothing areas in the initial content features; using the deformation field, it calculates the physical topological distance between the image positions involved in the compression of the initial content features, and dynamically modulates the weights of the convolution operation based on the physical topological distance to perform secure compression of the initial content features, thereby obtaining the content features; The physical topological distance is calculated based on the deformation field, and its expression includes the two-dimensional coordinate difference in the tiling space defined by the deformation field, and the penalty term for the discrete topological levels (w) in the deformation field not being triggered simultaneously; modulation weights (G) are generated by calculating the negative exponential function of the physical topological distance, and the weights of the convolution operation are dynamically modulated using the modulation weights (G), wherein the modulation weights (G) are a monotonically decreasing function of the physical topological distance; The spatial reconstruction module generates a corresponding rotational position encoding matrix for the content feature sequence based on the coordinates (u, v) and level (w) contained in the 2.5D normalized deformation field. Using the rotational position encoding matrix, the query vector and key vector of the content feature sequence in the self-attention calculation are transformed respectively. Based on the transformed query vector and key vector, attention weights are calculated, whereby the attention weights are determined by the difference between the coordinates (u, v) and the level (w) of the image positions corresponding to the query vector and key vector. Based on the attention weights, the value vectors in the self-attention calculation are weighted and aggregated to generate the flattened clothing features.

7. A computer-readable storage medium, characterized in that: It stores computer program code thereon, which, when loaded into and executed on a computer system, causes the computer system to perform all the steps of the method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Method and system for generating 3D (three-dimensional) human motion under text driving by using 2D (two-dimensional) video

    CN120876683A

  • Training method and system of image pre-training model and storage medium

    CN120913029A