AI model chart generation method, system and device and medium

Abstract

The invention relates to an AI model chart generation method, system and device and a medium, and belongs to the technical field of image processing, multi-modal data of target clothes is collected, and the multi-modal data comprises a mannequin chart, a tiled static chart and a material close-up chart; selecting a reference image of which the attitude accords with expectation from a target attitude library, and extracting an attitude skeleton image corresponding to the reference image as a target attitude skeleton image; inputting the tiled static graph and the material close-up graph into a visual language model to generate text description; inputting the target attitude skeleton diagram, the mannequin diagram and the text description into a pre-trained LoRA diffusion model to generate a preliminary dressing image; an input layer of the LoRA diffusion model is additionally provided with an independent attitude condition injection channel, and the target attitude skeleton diagram is input into the LoRA diffusion model through the attitude condition injection channel. The method has the beneficial technical effects that the stiff and unnatural posture of the AI model is avoided, and the aesthetic feeling of the generated image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an AI model picture generation method, system, device and medium. BACKGROUND

[0002] In recent years, with the development of deep learning and generative artificial intelligence (AI), using diffusion models to generate high-quality images has become a popular technology. In the field of clothing e-commerce, using AI to generate virtual model pictures can greatly reduce the cost of shooting and shorten the new product cycle.

[0003] Among them, a mainstream technical path is to use a low-rank adaptation (Low-Rank Adaptation) or LoRA fine-tuning method. This method injects a small number of trainable parameters into a pre-trained diffusion model to customize learning for specific concepts (such as a set of clothes). Specifically, developers will take a set of photos (for example, 10-20 photos) of a mannequin (fake model) wearing a set of clothes, and use these photos to train a dedicated clothing LoRA model.

[0004] Because the pose of the mannequin is very fixed and stiff, it can only be rotated simply and cannot be posed as rich, natural and graceful as a real model. And because of the singleness of the training data pose, the LoRA model inevitably forces the association (coupling) of the clothing with the stiff pose of the mannequin while learning the features of the clothing. This causes the model to generate AI model poses that are still highly similar to the mannequin poses during training when generating images, appearing stiff and unnatural, which seriously affects the aesthetic of the generated images. SUMMARY

[0005] In order to avoid the stiffness and unnaturalness of the AI model pose and improve the aesthetic of the generated images, the present application provides an AI model picture generation method, system, device and medium.

[0006] In a first aspect, the present application provides an AI model picture generation method, which adopts the following technical scheme: An AI model picture generation method, comprising: Collecting multi-modal data of a target clothing, the multi-modal data comprising a mannequin picture, a flat static picture, and a material close-up picture; Selecting a reference picture with a desired pose from a target pose library, and extracting a pose skeleton picture corresponding to the reference picture as a target pose skeleton picture; Inputting the flat static picture and the material close-up picture into a visual language model to generate a text description; The target pose skeleton diagram, mannequin image, and text description are input into the pre-trained LoRA diffusion model to generate a preliminary dressed image. An independent pose condition injection channel is added to the input layer of the LoRA diffusion model, and the target pose skeleton diagram is input into the LoRA diffusion model through the pose condition injection channel.

[0007] By adopting the above technical solution, a reference image with the desired posture is obtained and the target posture skeleton image is extracted. An independent posture condition injection channel is added to the input layer of the LoRA diffusion model to input the target posture skeleton image into the model, thereby decoupling the posture and clothing. This allows the generated preliminary clothing image to present the desired posture, avoiding the stiffness and unnaturalness of the AI ​​model's posture and improving the aesthetics of the generated image.

[0008] Optionally, the training method for the LoRA diffusion model includes: Collect multi-angle real-life photos of mannequins wearing the relevant clothing; Each captured mannequin image is processed to extract a skeleton diagram containing pose information; For each mannequin image, a text description is matched and paired with the corresponding skeleton image to form training samples; A pre-trained LoRA model is used as the base model. An independent attitude condition injection channel is added to the input layer to form a LoRA diffusion model. The parameters of the base model and the attitude encoder weights are frozen. The training samples are input into the LoRA diffusion model to train the parameters of the LoRA adaptation layer and the projection layer output by the attitude encoder. Calculate the difference between the noise predicted by the LoRA diffusion model and the actual noise, and update the LoRA parameters based on this difference through backpropagation.

[0009] By employing the aforementioned technical solution, the LoRA diffusion model is explicitly informed that the pose information in an image is defined by the input "pose skeleton map." This allows the attention layer of the diffusion model to attribute the learning of pose features to skeleton map conditions, while enabling the LoRA network to focus more on learning visual features beyond pose, such as core details like clothing style, color, texture, and folds. Because pose is stripped away as a known prior, the LoRA learning task becomes purer and more efficient, potentially improving convergence speed.

[0010] Optionally, the steps for extracting the skeleton map containing pose information include: For each captured mannequin image, perform 2D human pose estimation to obtain the coordinates of key human points, and connect them to generate the corresponding initial pose skeleton diagram. The initial posture skeleton diagram is input into the posture repair network, which optimizes and completes the key points based on human kinematic priors to generate the repaired skeleton diagram.

[0011] By adopting the above technical solution, the prior knowledge of human kinematics includes knowledge such as the reasonable range of human joint movement and interrelationships. Using this knowledge, unreasonable key point positions that may exist in the initial posture skeleton diagram can be corrected, so that the generated repaired skeleton diagram is more in line with the actual movement and posture of the human body, and the accuracy of posture information can be improved.

[0012] Optionally, the steps for building the target pose library include: Collect diverse real-life pose reference images from the internet, fashion image libraries, and professional photography materials; A consistent pose estimation algorithm, used in the same phase as the training phase, is used to extract skeleton maps corresponding to real-person pose reference images. These skeleton maps are then labeled and categorized according to action type to establish a searchable target pose database.

[0013] By adopting the above technical solutions, a rich and diverse target pose library, which is classified and organized, provides more different types of pose data for model training.

[0014] Optionally, the steps following the generation of the initial dressed image include: The mannequin image is semantically segmented using a clothing analysis model to generate a pixel-aligned clothing region segmentation mask. Based on the clothing region segmentation mask and the text description, the physical attribute semantic tags of the clothing are labeled to form a comprehensive clothing representation that includes appearance, structure and physical characteristics. The integrated clothing representation, the target pose skeleton diagram, and the preliminary clothing image are input into the pre-trained Clothing Deformation Network (GDN) to generate a deformation field. Based on the deformation field, pixel-level deformation distortion is performed on the clothing area in the preliminary clothing image to output the final clothing display image; The physical attribute semantic tags are embedded as prior knowledge into the Garment Deformation Network (GDN) to guide the GDN in generating differentiated deformation patterns based on different material responses. The physical attribute semantic tags are automatically labeled through visual understanding of the large model, including material semantic tags and pattern semantic tags; Among them, the material semantic tag is used to describe the optical and mechanical properties of the fabric; the pattern semantic tag is used to describe the overall silhouette and cutting features of the garment.

[0015] By adopting the above technical solutions, different fabrics and patterns will exhibit different deformation behaviors when subjected to external forces. This method can simulate more realistic clothing deformation effects, making the final clothing display image more lifelike. It allows the clothing to deform naturally according to the target posture, better conforming to the human body, improving the realism and aesthetics of the clothing display. The generated final clothing display image can more realistically reflect the appearance and physical characteristics of the clothing in different postures, providing users with a more intuitive and accurate clothing display.

[0016] Optionally, the training method for the garment deformation network (GDN) includes: Collect images of the test mannequin wearing the relevant clothing; The test mannequin image is segmented into regions, and a clothing segmentation mask is defined and the physical properties of the clothing are labeled to form a comprehensive test clothing representation. Extract the skeleton image of the test mannequin in the current training pose; The skeleton diagrams under the standard posture, the skeleton diagrams under the current training posture, and the test integrated clothing representation are input into the GDN to predict the deformation field from the standard posture to the current posture. The deformation field is applied to the test mannequin image, and a reconstructed image is obtained through a differentiable image deformation operation. The deformation operation is performed only on the clothing area defined by the clothing segmentation mask. Calculate the pixel-level difference loss between the reconstructed image and the original real image, and update the GDN parameters based on the difference loss through backpropagation.

[0017] By adopting the above technical solution, the accurately trained GDN can generate a high-quality deformation field. In the process of generating AI model images, applying such a deformation field to the clothing area can make the clothing fit the model's posture more naturally, avoiding mismatches and unnaturalness between clothing and posture, thereby improving the quality and aesthetics of the final generated AI model images and providing users with a better visual experience.

[0018] Optionally, the AI ​​model image generation method includes: Extract ambient lighting information from the original reference image to construct a lighting map; The target pose skeleton map and the clothing region segmentation mask are input into a pre-trained shadow generation model to predict the shadow distribution of the clothing in the target pose. The lighting map and shadow distribution are fused into the final clothing display image to generate a lighting rendering image.

[0019] By employing the aforementioned technical solution and adding lighting maps, the generated model images can present lighting effects that more closely resemble real-world scenes, enhancing the realism and immersiveness of the images. Furthermore, the appropriate presentation of light and shadow helps to better showcase the material, color, and style of the clothing.

[0020] Secondly, this application provides an AI model image generation system, which adopts the following technical solution: An AI model image generation system includes: The data acquisition module is used to collect multimodal data of the target garment, including mannequin images, flat static images, and close-up images of the material. The skeleton extraction module is used to extract the corresponding posture skeleton map as the target posture skeleton map after selecting a reference map whose posture matches the expectation from the target posture library. The multimodal semantic parsing module is used to input the tiled static image and the material close-up image into the visual language model to generate a text description; The model processing module is used to input the target pose skeleton diagram, mannequin diagram and text description into the pre-trained LoRA diffusion model to generate a preliminary dressed image; the input layer of the LoRA diffusion model is equipped with an independent pose condition injection channel, and the target pose skeleton diagram is input into the LoRA diffusion model through the pose condition injection channel.

[0021] Thirdly, this application provides a computer device that adopts the following technical solution: A computer device includes a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the AI ​​model image generation method as described in the first aspect.

[0022] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing the AI ​​model image generation method as described in the first aspect.

[0023] In summary, this application includes at least one of the following beneficial technical effects: 1. It achieves effective decoupling of clothing and pose, fundamentally solving the problem of binding clothing features with mannequin pose; allowing the LoRA model to focus on the clothing itself, while pose becomes an independent control variable that can be freely replaced.

[0024] 2. Greatly enhances the richness and aesthetics of poses: Users can provide reference images of any pose to control the poses of the generated model. Whether it is an elegant standing posture, a dynamic walking posture or a complex dance movement, it can be easily achieved, completely getting rid of the stiffness of the mannequin.

[0025] 3. Ensures high fidelity in clothing reproduction: Because the LoRA model has a more focused learning objective, the style, pattern, and material details of the clothing are fully learned and preserved.

[0026] 4. By introducing material and pattern semantic tags as priors, the Garment Deformation Network (GDN) is guided to generate differentiated deformations that conform to material characteristics, significantly improving the realism and artistic expression of the generated results. Attached Figure Description

[0027] Figure 1 This is a first flowchart of an embodiment of the method of this application; Figure 2 This is a second flowchart of an embodiment of the method of this application; Figure 3 This is a third flowchart of an embodiment of the method of this application; Figure 4 This is the fourth flowchart of an embodiment of the method of this application; Figure 5 This is the fifth flowchart of an embodiment of the method of this application. Detailed Implementation

[0028] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1-5 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0029] The first embodiment of this application discloses an AI model image generation method. (Refer to...) Figure 1 The AI ​​model image generation method may include steps S110-S140: S110 collects multimodal data of the target garment, including mannequin images, flat static images, and close-up images of the material. S120: Select a reference image whose posture matches the desired posture from the target posture library, and extract the corresponding posture skeleton image as the target posture skeleton image. S130: Input the tiled static image and material close-up image into the visual language model to generate a text description; S140: Input the target pose skeleton map, mannequin image, and text description into the pre-trained LoRA diffusion model to generate a preliminary dressed image; The input layer of the LoRA diffusion model adds an independent pose condition injection channel, and the target pose skeleton map is input into the LoRA diffusion model through the pose condition injection channel.

[0030] Specifically, in the S110 stage of acquiring multimodal data of the target garment, it is first necessary to systematically acquire mannequin images, flat lay static images, and material close-up images of the target garment. Mannequin images are taken in a professional photography studio to ensure uniform lighting and a clean background. A 360-degree rotating shooting stand can be used to obtain mannequin images from multiple angles to fully showcase the garment's three-dimensional structure. Flat lay static images require the garment to be laid flat and photographed from directly above using a high-definition camera to ensure accurate proportions of all parts of the garment and to prevent wrinkles from interfering with key design details. Material close-up images utilize a macro lens to focus on the texture, weave, luster, and detailed decorations of the fabric (such as buttons, zippers, embroidery, etc.). During shooting, the lighting angle can be adjusted appropriately to highlight the differences in material texture. All images must have a uniform resolution of at least 2048×2732 pixels to ensure the accuracy of subsequent processing.

[0031] In S120, the steps for constructing the target pose library include S121-S122: S121 collects diverse real-life pose reference images from the Internet, fashion image libraries, and professional photography materials; S122, using the same pose estimation algorithm as in the training phase, extract the skeleton map corresponding to the real person pose reference image, and label and classify it according to the action category to establish a searchable target pose database.

[0032] Specifically, firstly, diverse real-life pose reference images were collected from fashion blogger accounts (such as Pinterest and Instagram), fashion trend platforms like WGSN, and professional stock image websites like Shutterstock using web crawling technology. This ensured coverage of different action categories, including standing, sitting, walking, and dynamic displays, while also taking into account the pose diversity of models of different body types and ages. High-definition, unobstructed, and high-quality images were selected while avoiding copyright risks. Subsequently, the collected real-life images were processed using the OpenPose pose estimation algorithm, which was completely consistent with the subsequent training phase. The two-dimensional coordinates of 18 key nodes (including head, neck, and limb joints) were extracted. The Bresenham algorithm was used to connect the key points to generate a standardized skeleton diagram. The skeleton lines were marked with solid red lines, and the key points were marked with solid green circles. Then, the images were manually labeled according to action categories (such as "casual standing pose," "runway walk," and "business sitting pose"). Cluster analysis was performed based on the key point coordinate features of the skeleton diagram to optimize the classification system. Finally, a MySQL database was used to store image paths, skeleton diagram data, and action labels, constructing a target pose database that supports keyword retrieval and pose similarity matching.

[0033] When generating text descriptions, the S130 inputs a tiled static image and a close-up image of the material into the CLIP visual language model. First, the images are preprocessed (including normalization and resizing to 224×224 pixels). The model's visual encoder extracts image features, which are then matched cross-modally with the lexical features in the text encoder to generate a preliminary description. Subsequently, the description is manually optimized by adding key information such as clothing style (e.g., A-line skirt, suit jacket), collar type (e.g., lapel, round neck), sleeve type (e.g., bell sleeve, three-quarter sleeve), color (e.g., "smog blue", "burgundy"), and pattern (e.g., "stripes", "polka dots", "checkered") to form a structured text description that includes appearance, structure, and material. For example, "Dark blue denim jacket, lapel design, double pockets on the front, metal buttons, long sleeves, 100% cotton fabric with medium thickness and slight elasticity."

[0034] When generating the initial dressed image using S140, the LoRA diffusion model input needs to integrate the target pose skeleton map, mannequin image, and text description. An independent pose condition injection channel is added to the input layer of the diffusion model. A VAE model is used to compress the skeleton map features to the same dimension as the image features, and then fuse them with the image features through a spatial attention mechanism. The target pose skeleton map is preprocessed (converted to a grayscale image of the same size as the input image, with the skeleton line pixel value set to 255 and the background set to 0) and then injected into the model through this channel. The text description is converted into a 768-dimensional feature vector by the CLIP text encoder and then fused with the pose features and image features through cross-attention at different levels of the LoRA diffusion model. During model inference, the sampling steps are set to 50, the DDIM sampler is used, and the CFG value is set to 7.5, generating an initial dressed image with a resolution of 1024×1365. This image already possesses the basic fusion effect of the target pose and clothing.

[0035] Pose estimation algorithms should be of a fixed version (e.g., OpenPose v1.7.0) and run on the same hardware environment (e.g., NVIDIA Tesla V100 GPU) to avoid skeleton map extraction bias caused by algorithm differences. Classification and labeling can be combined with a crowdsourcing platform for manual verification to improve label accuracy.

[0036] Reference Figure 2 The training methods for the LoRA diffusion model include S210-S260: S210, collect multi-angle real-shot images of mannequins for the relevant clothing; S220 processes each captured mannequin image to extract a skeleton diagram containing posture information; S230: Match text descriptions for each mannequin image and pair them with the corresponding skeleton diagram to form training samples; S240 uses a pre-trained LoRA model as the base model, adds an independent attitude condition injection channel to the input layer to form a LoRA diffusion model, and freezes the base model parameters and attitude encoder weights. S250, input the training samples into the LoRA diffusion model to train the parameters of the LoRA adaptation layer and the projection layer output by the attitude encoder; S260, calculate the difference between the noise predicted by the LoRA diffusion model and the actual noise, and update the LoRA parameters based on the difference through backpropagation.

[0037] In S220, the steps for extracting the skeleton map containing pose information include S221-S222: S221, Perform 2D human pose estimation on each acquired mannequin image to obtain the coordinates of key human points and connect them to generate the corresponding initial pose skeleton diagram. S222, the initial pose skeleton map is input into the pose repair network. The pose repair network optimizes and completes the key points based on human kinematics priors to generate the repaired skeleton map.

[0038] Specifically, the training phase of the LoRA diffusion model requires strict control over data quality and training parameters. In S210, when collecting multi-angle real-world images of the mannequin, several to dozens of images of each garment are selected, ensuring consistency in lighting and background from multiple angles. In S220, the same pose estimation algorithm as S122 is used to extract key points and generate an initial pose skeleton map. A pose restoration network based on the diffusion model is constructed, with the input being a grayscale image of the skeleton map (size adjusted to 256x256). The network design includes an encoder-decoder structure, where the encoder uses ResNet-18 to extract features, and the decoder combines human kinematic priors (such as joint angle constraints and limb length ratios) with a custom loss function (e.g., adding a kinematic regularization term to penalize unreasonable angle deviations) to optimize key point positions. The model is implemented in PyTorch, with pre-trained weights from the MPII Human Pose dataset. The output is a restored skeleton map (e.g., filling in occluded points or correcting distorted poses), which is saved as a high-resolution image to ensure skeleton continuity.

[0039] For each mannequin image, a detailed text description is matched and associated with a repaired skeleton diagram. The text description covers information such as clothing category (e.g., dress, blazer), style features (e.g., A-line skirt, lapel design), color (e.g., bright red, navy blue), material (e.g., silk, denim), and details (e.g., metal buttons, embroidery patterns). A structured description template (e.g., "A [color][material][category] item with [style features] and [details]") is used to ensure a high degree of matching between the text and image content. Subsequently, the mannequin image, corresponding skeleton diagram, and text description are packaged into training sample pairs and divided into training and validation sets in an 8:2 ratio.

[0040] The LoRA model architecture is modified by adding an independent pose condition injection channel to the input layer. This channel uses a convolutional neural network (CNN) to encode the pose skeleton map into a conditional vector with the same dimension as the image features, and then fuses it with the low-level visual features of the diffusion model through a spatial attention mechanism. Simultaneously, a pre-trained LoRA parameter module is loaded, the basic model network parameters are frozen, and only the parameters related to the LoRA adaptation layer and the newly added pose condition injection channel are retained as trainable.

[0041] During training, training set samples are input into the model in batches. The skeleton map is fed in through an independent pose condition injection channel, the mannequin image serves as the primary visual input, and the text description is converted into text embedding vectors by the CLIP text encoder. These three inputs together guide the model to learn the generative mapping from the mannequin image to the clothing image. Training employs the AdamW optimizer with an initial learning rate of 2e-4, dynamically adjusted using a cosine annealing strategy. The batch size is set to 8-16 based on the GPU memory. The total number of training epochs is determined based on the sample size (typically 100-300 epochs). Every 10 epochs, the pose consistency and clothing detail retention of the generated images are evaluated on the validation set. The mean squared error (MSE) between the model's predicted noise and the actual Gaussian noise is used as the primary loss function. Perceptual loss is also introduced to improve the visual quality of the generated images. The parameters of the LoRA adaptation layer and the projection layer of the pose condition injection channel are updated based on the loss value through backpropagation until the images generated by the model on the validation set achieve a balance between pose accuracy and clothing reproduction.

[0042] Reference Figure 3 The steps following the generation of the initial clothing image include S310-S340: S310 uses a clothing analysis model to perform semantic segmentation on the mannequin image and generates a pixel-aligned clothing region segmentation mask. S320, based on the clothing region segmentation mask and text description, marks the physical attribute semantic tags of the clothing to form a comprehensive clothing representation that includes appearance, structure and physical characteristics; S330 inputs the integrated clothing representation, preliminary dressing image and target pose skeleton map into the pre-trained clothing deformation network GDN to generate the deformation field; S340, perform pixel-level deformation distortion on the clothing area in the preliminary clothing image according to the deformation field, and output the final clothing display image.

[0043] Specifically, when performing semantic segmentation on mannequin images using the clothing parsing model, the S310 employs a clothing region segmentation model based on an improved Mask R-CNN. This model, pre-trained on the COCO dataset and fine-tuned using the FashionAI dataset, focuses on optimizing the boundary discrimination ability between clothing and human body regions. It supports pixel-level segmentation of at least 20 semantic categories of clothing, including the body, sleeves, collar, pockets, and buttons. After inputting the mannequin image into the model, multi-scale features are extracted through a Feature Pyramid Network (FPN), and candidate boxes are generated by combining it with a Region Proposal Network (RPN). After RoIAlign operation, a pixel-level clothing region segmentation mask is output. In the mask, the pixel value of the clothing region is set to 1, while the background and mannequin regions are set to 0, ensuring that the mask is aligned with the original image at the sub-pixel level. After segmentation, morphological closing operations are used to eliminate holes in the mask, and edge detection algorithms are used to extract the clothing outline, further optimizing the mask accuracy and providing accurate region localization for subsequent annotation of physical attribute semantic labels.

[0044] The physical property semantic labels in S320 are automatically annotated using a large visual understanding model, including material semantic labels and pattern semantic labels. Material semantic labels describe the optical and mechanical properties of the fabric, while pattern semantic labels describe the overall silhouette and tailoring features of the garment. These physical property semantic labels are embedded as prior knowledge into the Garment Deformation Network (GDN), guiding the GDN to generate differentiated deformation patterns based on different material responses.

[0045] Specifically, when labeling the semantic tags of clothing physical attributes, the BLIP-2 visual understanding big data model is used for automatic labeling based on the segmentation mask generated by S310 and the text description of S130.

[0046] First, close-up images of the materials and segmentation masks are input into the model. The model learns by comparison to identify the fabric's optical properties (e.g., gloss "matte / glossy", transparency "opaque / semi-transparent") and mechanical properties (e.g., drape "soft / crisp", elasticity "high elasticity / non-elasticity", wrinkle resistance "wrinkle-prone / wrinkle-resistant"), generating material semantic labels. Simultaneously, by combining the contour features of flat lay images and mannequin images, the model analyzes the overall garment silhouette (e.g., "X-shape", "H-shape", "O-shape"), cutting methods (e.g., "cinched waist", "loose fit", "asymmetrical cut"), and detailed structures (e.g., "princess seam" "yoke design"), generating pattern semantic labels. These labels are converted into vectors and embedded as prior knowledge into the input layer of the Garment Deformation Network (GDN). An attention mechanism guides the network to generate differentiated deformation patterns for different materials (e.g., the drape deformation of silk, the stiff deformation of denim).

[0047] The processing flow of the Garment Deformation Network (GDN) in S330 is to take the comprehensive garment representation (including segmentation mask, physical attribute semantic label vector), the preliminary dressing image, and the target pose skeleton diagram as input.

[0048] GDN employs a U-Net-based deformation prediction architecture. The input layer simultaneously receives visual features from a preliminary dressed image and a target pose skeleton map, along with semantic features from a comprehensive clothing representation. The encoder extracts multi-scale image and semantic features. In the decoder, deconvolution and skip connections generate a 2D displacement field with the same resolution as the image. The vector value of each pixel in the displacement field represents the offset of that location in the horizontal and vertical directions. GDN utilizes embedded physical attribute semantic labels to generate differentiated deformations for clothing regions of different materials and styles: for example, for a silk dress with excellent drape, a larger, natural drape deformation is generated at the hem; for a structured suit jacket, a slight, ergonomic fit deformation is generated at the shoulders and waist while preserving the fabric's crispness.

[0049] After generating the deformation field, the S340 uses a bilinear interpolation algorithm to perform pixel-level deformation distortion on the clothing areas marked by the segmentation mask in the initial clothing image, while the non-clothing areas remain unchanged. This allows the clothing to produce deformation effects that conform to physical laws according to the movement trends of the human body in the target posture (such as the stretching of sleeves when the arms are raised and the wrinkling of trouser legs when the legs are bent). Finally, a 2048×2732 pixel final clothing display image is output, with clear clothing details, natural posture, and realistic fabric characteristics.

[0050] Reference Figure 4 The training methods for the Garment Deformation Network (GDN) include S410-S460: S410, acquires images of the test mannequin for the relevant clothing; S420 performs region segmentation on the test mannequin image, defines the clothing segmentation mask and labels the physical attributes of the clothing to form a comprehensive test clothing representation; S430, extract the skeleton map of the test mannequin image under the current training posture; S440 inputs the skeleton map in the standard pose, the skeleton map in the current training pose, and the test integrated clothing representation into the clothing deformation network GDN to predict the deformation field from the standard pose to the current pose. S450, apply the deformation field to the test mannequin image, and obtain the reconstructed image through a differentiable image deformation operation, performing the deformation operation only on the clothing area defined by the clothing segmentation mask; S460 calculates the pixel-level difference loss between the reconstructed image and the original real image, and updates the GDN parameters based on this difference loss through backpropagation.

[0051] Specifically, the training of the Garment Deformation Network (GDN) (S310-S360) requires the construction of a dedicated training dataset. S410 collects mannequin images of 500 garments, each containing standard poses (e.g., standing at attention) and 5 non-standard poses (e.g., side bend, arm raise). S420 generates garment segmentation masks through a combination of manual annotation and automatic segmentation, with physical attribute labels provided by professional fashion designers. S430 extracts skeleton images under different poses as the basis for pose changes. During training, the standard pose skeleton image, the current training pose skeleton image, and the test garment representation are input into the GDN. After predicting the deformation field, deformation is applied to the standard pose mannequin image. The pixel-level L1 loss between the reconstructed image and the real non-standard pose mannequin image is calculated, while a deformation field smoothing loss is added (to avoid drastic local deformation). The SGD optimizer is used with a learning rate of 1e-3, momentum of 0.9, and training for 200 epochs, enabling the network to generate realistic deformation fields based on pose changes and physical attributes.

[0052] Reference Figure 5 Furthermore, the AI ​​model image generation methods include S510-S530: S510: Extract ambient lighting information from the original reference image and construct a lighting map; S520: Input the target pose skeleton map and clothing region segmentation mask into the pre-trained shadow generation model to predict the shadow distribution of the clothing in the target pose. The S530 integrates the light map and shadow distribution into the final garment display image to generate a light-rendered image.

[0053] Specifically, in the lighting and shadow optimization stages S510-S530, S510 extracts ambient lighting information from the original reference image (such as the real-person reference image corresponding to the target pose selected in S120), and uses a deep learning-based lighting estimation model (such as Neural Illumination Estimation) to analyze the brightness distribution, highlight position, and shadow direction in the image, constructing a lighting map containing the light source direction, intensity, and color temperature. The lighting map is stored in the form of an environment map, supporting global lighting adjustments to the generated image. S520 inputs the target pose skeleton map and clothing region segmentation mask into a pre-trained shadow generation model. This model predicts the shadow distribution of clothing under the target pose based on the relative positional relationship between the human body pose and the clothing region, combined with a physical lighting model (such as the Phong lighting model), including the attached shadows of the clothing in contact with the human body (such as underarms and waist), the projected shadows formed by the clothing's own folds, and the projection relationship with the ground / background. The intensity and softness of the shadows are dynamically adjusted according to the light source distance and surface roughness. The S530 integrates the ambient lighting parameters of the lighting map and the predicted shadow distribution into the final clothing display image using image fusion technology (such as alpha blending). By adjusting the light intensity coefficient (0.3-0.7) and shadow transparency (0.4-0.6), the generated image has a realistic sense of light and shadow hierarchy, improving the overall visual realism.

[0054] Based on the above method embodiments, the second embodiment of this application discloses an AI model image generation system. The AI ​​model image generation system of this application embodiment can implement any of the above-described AI model image generation methods, and the specific working process of each module in the AI ​​model image generation system can be referred to the corresponding process in the above method embodiments.

[0055] For ease of understanding, an example is as follows: An AI model image generation system includes: The data acquisition module is used to collect multimodal data of the target garment, including mannequin images, flat lay static images, and material close-up images. The skeleton extraction module is used to extract the corresponding posture skeleton map as the target posture skeleton map after selecting a reference map whose posture matches the expectation from the target posture library. The multimodal semantic parsing module is used to input tiled static images and material close-up images into the visual language model to generate text descriptions; The model processing module is used to input the target pose skeleton map, mannequin image and text description into the pre-trained LoRA diffusion model to generate a preliminary dressed image; the input layer of the LoRA diffusion model adds an independent pose condition injection channel, and the target pose skeleton map is input into the LoRA diffusion model through the pose condition injection channel.

[0056] The third embodiment of this application provides a computer device, which may include a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement an AI model image generation method.

[0057] The memory can communicate with the processor via a communication bus, which can be an address bus, a data bus, a control bus, etc.

[0058] Additionally, the memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device.

[0059] Furthermore, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0060] The fourth embodiment of this application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as an AI model image generation method.

[0061] The computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device; the program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0062] It should be noted that the computer device and storage medium in the embodiments of this application are respectively electronic devices and storage media that apply the above-described AI model image generation method. Therefore, all embodiments of the above-described AI model image generation method are applicable to the computer device and storage medium, and can achieve the same or similar beneficial effects. For the computer device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple; relevant details can be found in the descriptions of the method embodiments.

[0063] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce a good effect.

[0064] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. An AI model picture generation method, characterized by, The method comprises the following steps: Collecting multi-modal data of target clothes, the multi-modal data comprising a mannequin graph, a flat static graph, and a material close-up graph; Selecting a reference graph with a desired pose from a target pose library, and extracting a pose skeleton graph corresponding to the reference graph as a target pose skeleton graph; Inputting the flat static graph and the material close-up graph into a visual language model to generate a text description; Inputting the target pose skeleton graph, the mannequin graph, and the text description into a pre-trained LoRA diffusion model to generate a preliminary dressed image; an independent pose condition injection channel is added to the input layer of the LoRA diffusion model, and the target pose skeleton graph is input into the LoRA diffusion model through the pose condition injection channel.

2. The AI model picture generation method of claim 1, wherein, The training method of the LoRA diffusion model comprises the following steps: Collecting mannequin multi-angle real photos of related clothes; Processing each collected mannequin photo to extract a skeleton graph containing pose information; Matching a text description for each mannequin photo, and pairing and associating the corresponding skeleton graph to form a training sample; Using a pre-trained LoRA model as a base model, adding an independent pose condition injection channel to the input layer to form a LoRA diffusion model, and freezing the base model parameters and the pose encoder weights; Inputting the training sample into the LoRA diffusion model to train the LoRA adaptation layer and the projection layer parameters output by the pose encoder; Calculating the difference between the noise predicted by the LoRA diffusion model and the real noise, and updating the LoRA parameters in the reverse direction according to the difference.

3. The AI model picture generating method of claim 2, wherein, The step of extracting a skeleton graph containing pose information comprises the following steps: Performing 2D human pose estimation on each collected mannequin photo to obtain human key point coordinates, and connecting to generate a corresponding initial pose skeleton graph; Inputting the initial pose skeleton graph into a pose repair network, which optimizes and completes the key points according to human kinematics prior to generate a repaired skeleton graph.

4. The AI model picture generating method of claim 1, wherein, The step of constructing a target pose library comprises the following steps: Collecting diversified real human pose reference images from the Internet, fashion image libraries, and professional photography materials; Uniformly extracting skeleton graphs corresponding to the real human pose reference images using a consistent pose estimation algorithm as in the training stage, and marking and classifying them according to action categories to establish a searchable target pose database.

5. The AI model picture generating method of claim 1, wherein, The steps after generating a preliminary dressed image comprise the following steps: Performing semantic segmentation on the mannequin graph using a clothes analysis model to generate a pixel-level aligned clothes region segmentation mask; According to the clothes region segmentation mask and the text description, labeling the physical attribute semantic labels of the clothes to form a comprehensive clothes representation containing appearance, structure, and physical properties; Inputting the comprehensive clothes representation, the target pose skeleton graph, and the preliminary dressed image into a pre-trained clothes deformation network GDN to generate a deformation field; Performing pixel-level deformation and distortion on the clothes region in the preliminary dressed image according to the deformation field to output a final clothes display image; The physical attribute semantic labels are embedded into the clothes deformation network GDN as prior knowledge to guide the clothes deformation network GDN to generate differential deformation patterns according to different materials; The physical attribute semantic label is automatically labeled by a visual understanding large model, and includes a material semantic label and a pattern semantic label. The material semantic label is used for describing optical and mechanical properties of the fabric, and the pattern semantic label is used for describing overall contour and cutting features of the garment.

6. The AI model picture generating method of claim 5, wherein, The training method of the garment deformation network GDN includes the following steps: Collecting test mannequin images of related garments; Performing region segmentation on the test mannequin images, defining garment segmentation masks, and labeling physical attributes of the garments to form test comprehensive garment representations; Extracting a skeleton graph corresponding to the test mannequin images in a current training posture; Inputting the skeleton graph in a standard posture, the skeleton graph in the current training posture, and the test comprehensive garment representation into the GDN to predict a deformation field from the standard posture to the current posture; Applying the deformation field to the test mannequin images to obtain a reconstructed image through a differentiable image deformation operation, and performing a deformation operation only on a garment region defined by the garment segmentation mask; Calculating a pixel-level difference loss between the reconstructed image and an original real image, and updating GDN parameters according to the difference loss.

7. The AI model picture generating method of claim 5, wherein, The AI model image generation method further includes the following steps: Extracting ambient light information from the original reference image to construct a light map; Inputting the target posture skeleton graph and the garment region segmentation mask into a pre-trained shadow generation model to predict a shadow distribution of the garment in the target posture; Fusing the light map and the shadow distribution into the final garment display image to generate a light rendering image.

8. An AI model picture generation system characterized by comprising: The AI model image generation method according to any one of claims 1 to 7 includes the following steps: A data acquisition module is configured to acquire multi-modal data of a target garment, and the modal data includes a mannequin image, a flat static image, and a material close-up image; A skeleton extraction module is configured to extract a posture skeleton graph corresponding to a reference image selected from a target posture library as a target posture skeleton graph after the reference image is selected in which the posture meets an expectation; A multi-modal semantic analysis module is configured to input the flat static image and the material close-up image into a visual language model to generate a text description; A model processing module is configured to input the target posture skeleton graph, the mannequin image, and the text description into a pre-trained LoRA diffusion model to generate a preliminary dressed image; an input layer of the LoRA diffusion model is additionally provided with an independent posture condition injection channel, and the target posture skeleton graph is input into the LoRA diffusion model through the posture condition injection channel.

9. A computer device, comprising: A computer program is stored in a memory and executable on a processor, and the processor implements the AI model image generation method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that, A computer program is stored in a memory and executable on a processor, and the processor implements the AI model image generation method according to any one of claims 1 to 7 when executing the program.

Images