Multi-modal clue-guided customized clothing image generation model and processing method thereof

Abstract

The invention discloses a multi-modal clue-guided customized clothing image generation model and a processing method thereof. The model comprises a multi-modal encoder array, a multi-modal feature analysis module, a diffusion model and a multi-modal data set. The multi-modal encoder array is used for extracting multi-modal conditions including semantic and structural features of a pattern image, global texture representation of a texture image, style semantic embedding of text description and figure posture features to obtain multi-modal embedded information; the multi-modal feature analysis module is used for performing space compression, dimension alignment and fusion operation on the multi-modal embedded information to obtain a multi-modal condition sequence; the diffusion model is used for performing iterative denoising reconstruction on the initial Gaussian noise map and outputting a clothing image; the multi-modal data set is used for training and evaluating a pattern-driven diffusion model. According to the method, complex garment making can be rapidly processed, and the method has the advantages of being wide in universality, high in applicability, high in accuracy of generated garment patterns and textures and high in image quality.

Description

Technical Field

[0001] This invention relates to a custom clothing image generation method, and more particularly to a multimodal cue-guided custom clothing image generation model and processing method. Background Technology

[0002] In recent years, driven by AIGC, the generation of realistic clothing has made significant progress in clothing design and manufacturing. Modern automated frameworks have significantly reduced the need for skilled designers to provide labor-intensive and time-consuming design details, improving the efficiency and aesthetic quality of clothing creation.

[0003] Currently, the focus is generally on generating multi-modal images through methods such as text, shape, texture, and human pose. For example, Fashion Gen used T2I StackGAN to explore the impact of text on clothing design, while DeepFashionMultiModal studied the interaction between human pose and detailed textual descriptions (such as shape, fabric, and color attributes), representing pioneering efforts in text-driven clothing design. To enrich semantic guidance, FashionTex, Textfit, and MFIENet further expand the diversity and granularity of input methods, combining text, texture, human pose, and sketches, thus ushering in a new era of text-guided clothing design.

[0004] While the aforementioned image generation schemes offer versatility and flexibility, existing multimodal models primarily rely on "fuzzy text" as the main design instruction. This reliance often leads to coarse and misleading output, especially when creating clothing with complex patterns and textures. For example, inputting the text description "a yellow dress with a black outline of the Eiffel Tower" into a multimodal model will result in an error because existing models are only sensitive to the preceding color words and ignore the following ones. The generated result might simply be a black dress, failing to produce the intended image of the garment.

[0005] Therefore, existing image generation solutions are unable to accurately formulate specific clothing patterns and textures based on text descriptions, failing to meet the needs of complex and rapid clothing engineering production, and affecting the timeliness and accuracy of automated clothing design. Summary of the Invention

[0006] The purpose of this invention is to provide a multimodal cue-guided model for generating customized clothing images and its processing method. This invention can quickly process complex clothing production processes and features wide versatility, strong applicability, high accuracy in generating clothing patterns and textures, and high image quality.

[0007] The technical solution of this invention is a multimodal cue-guided customized clothing image generation model, comprising a multimodal encoder array, a multimodal feature parsing module, a diffusion model, and a multimodal dataset. The multimodal encoder array extracts multimodal conditions, including semantic and structural features of pattern images, global texture representation of texture images, style semantic embedding of text descriptions, and pose features, providing semantically consistent multimodal embedding information to the multimodal feature parsing module. The multimodal feature parsing module performs spatial compression, dimensional alignment, and fusion operations on the multimodal embedding information, providing a unified and standardized multimodal condition sequence to the diffusion model. The diffusion model iteratively denoises and reconstructs an initial Gaussian noise map under multimodal condition constraints based on the multimodal condition sequence, outputting a clothing image conforming to the user's desired style. The multimodal dataset is used to train and evaluate the pattern-driven diffusion model, containing precisely paired pattern images, texture images, concise style text, and corresponding clothing images.

[0008] In the aforementioned multimodal cue-guided customized clothing image generation model, the multimodal encoder array includes an image encoder, a texture encoder, and a text encoder. The image encoder is used to extract the semantic and structural features of the pattern image. The texture encoder is used to extract the global texture representation of the texture image, including color information and texture information. The text encoder is used to encode the text input into a semantic embedding, sharing the embedding space with the image encoder to facilitate modal alignment.

[0009] In the aforementioned multimodal cue-guided customized clothing image generation model, the multimodal feature parsing module includes a spatial feature compression unit, a modal projector, and a modal fusion unit. The spatial feature compression unit is used to perform multi-scale dimensionality reduction compression on pattern images, texture images, and text descriptions, outputting low-dimensional tensors, which are then flattened and stitched together to obtain a structural embedding vector. The modal projector is used to unify the linear projection and dimension of the structural embedding vector, achieving dimension alignment. The modal fusion unit is used to stitch and fuse the dimensionality-reduced and dimension-aligned multimodal conditions into a unified multimodal condition sequence for the diffusion model to receive.

[0010] In the aforementioned multimodal cue-guided customized clothing image generation model, the spatial feature compression unit includes a multi-scale pooling layer, a flattening layer, and a connection layer. The multi-scale pooling layer is used to perform average pooling operations on multimodal conditions at different scales to reduce dimensionality at multiple scales and generate multiple low-dimensional tensors. The flattening layer is used to flatten each low-dimensional tensor into a one-dimensional vector. The connection layer is used to concatenate all one-dimensional vectors into a unified structural embedding vector. The modal projector includes a pattern structure projector, a texture structure projector, and a text structure projector; the pattern structure projector is used to map a pattern image to a standard feature space; the texture structure projector is used to map a texture image to a standard feature space; and the text structure projector is used to map text to a standard feature space.

[0011] In the aforementioned multimodal cue-guided customized clothing image generation model, the diffusion model includes a backbone network, a hybrid attention mechanism, a pose preservation module, and a multimodal conditional injection mechanism. The backbone network is used to progressively generate clothing images from random noise images. The hybrid attention mechanism is used to jointly model three modalities: pattern, texture, and text. The pose preservation module is used to maintain the human pose structure through residual connections. The multimodal conditional injection mechanism is used to input multimodal conditional sequences into the generation process of the backbone network.

[0012] In the aforementioned multimodal cue-guided customized clothing image generation model, the backbone network includes an encoder and decoder module, a pose encoder module, and a time encoding module; the encoder and decoder module is used to perform image feature extraction and reconstruction; the pose encoder module is used to maintain the person's pose information; and the time encoding module is used to receive the time step embedding of the diffusion process. The hybrid attention mechanism includes a trimodal conditional embedding input module, three sets of attention calculation modules, and a modality fusion module. The trimodal conditional embedding input module receives multimodal conditional sequences from three modalities: pattern, texture, and text. The three sets of attention calculation modules use each modal's multimodal conditional sequence as an independent carrier key and information key, and the model's current generation state as a query key, interacting with the three modal conditions to jointly model and obtain multimodal information data. The modality fusion module weights or merges the multimodal information data with the backbone network to generate image-guided features centered on the pattern. The pose preservation module includes a pose map extractor, a pose encoder, and a residual injection module. The pose map extractor extracts pose information from the original human image to form a pose map. The pose encoder converts the pose map into a pose feature tensor and outputs a feature dimension that matches the intermediate layers of the U-Net backbone network, allowing it to be used in subsequent backbone networks. The residual injection module injects the pose feature tensor as a hard constraint residual guiding signal into the diffusion network, thereby ensuring that the human pose in the generated image is consistent with the human pose in the original input image and that no structural errors occur.

[0013] In the aforementioned multimodal cue-guided customized clothing image generation model, the multimodal dataset includes multiple sample sets consisting of pattern images, texture images, style text descriptions, and target clothing images. Each sample constitutes a clearly paired input-output pair of the three modalities, which is used to train the pattern-driven diffusion model.

[0014] The above-mentioned processing method for the customized clothing image generation model includes the following steps: S1. Input pattern images, texture images, text descriptions, images of clothing coverings, and the body posture structure of the target person into the multimodal encoder array; S2. The multimodal encoder array adjusts the pattern image and texture image to a fixed resolution, normalizes the image pixel values, segments and tokenizes the text description, and converts it into a suitable format. S3. The multimodal encoder array extracts multimodal conditions, including semantic and structural features corresponding to pattern images, texture images and text descriptions, and generates multimodal embedding information; S4. The multimodal feature parsing module performs multi-scale dimensionality reduction compression, dimension alignment, and splicing fusion on the multimodal embedding information to obtain the multimodal conditional sequence. S5. Input the multimodal conditional sequence into the diffusion model through a multimodal conditional injection mechanism to obtain image guidance features; S6. Pose preservation and image decoding are performed using a diffusion model; S7. Output virtual clothing images of characters that meet the style requirements of patterns, textures, and text descriptions.

[0015] In the aforementioned method, step S3 specifically includes: S31. Input the pattern image, the image with clothes covered, and the human pose image into the image encoder, convert them into a structural feature map in a low-dimensional space, and extract the geometric shape, edges, local texture layout, and human pose features of the pattern image and the image with clothes covered. S32. Input the pattern image and texture image into the image encoder and texture encoder to extract the semantic features of the pattern image and texture image; S33. Input the text description into the text encoder and extract the semantic features of the text description; S34. Finally, a set of three modal representations of pattern-texture-text is obtained, including the structural features of the pattern image, the semantic features of the pattern image, the semantic features of the text description, and the semantic features of the texture image, generating multimodal embedding information.

[0016] In the aforementioned method, step S4 specifically includes: S41. Select different pooling window sizes for the structural features of the pattern image, the semantic features of the pattern image, the semantic features of the texture image, and the semantic features of the text description, respectively, perform average pooling operation to achieve multi-scale dimensionality reduction, output low-dimensional tensors, flatten each low-dimensional tensor into a one-dimensional vector, and concatenate them sequentially to obtain the structure embedding vector. S42. Perform linear projection on multiple structure embedding vectors to obtain pattern structure identifier, pattern semantic identifier, texture semantic identifier and text semantic identifier, and align them in semantic space; S43. Use a hybrid attention mechanism to weightedly combine the pattern structure identifier and the pattern semantic identifier to obtain a comprehensive pattern identifier; S44. Combine the integrated pattern identifier, texture semantic identifier, and text semantic identifier in a preset order to form a unified multimodal condition sequence.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention designs a customized clothing image generation model. Through a diffusion model centered on multimodal conditions, it generates color clothing images based on more different modalities. By integrating multimodal conditions such as pattern structure, pattern semantics, texture semantics, and text description, it can generate high-quality images with clear structure, rich texture, and consistent semantics, significantly improving the realism and diversity of the generated images.

[0018] The multimodal feature parsing module aligns and fuses multimodal conditions, effectively integrating them and enhancing the model's ability to understand and process complex text, patterns, and textures in clothing images. A hybrid attention mechanism, combining self-attention and cross-attention, guides the model to focus on multimodal condition information (including text descriptions, pattern structures, pattern semantics, color textures, etc.) during the generation process, improving the efficiency and stability of clothing image generation while reducing training time and resource consumption.

[0019] The method of this invention is applicable to a variety of application scenarios, such as pattern design, texture synthesis, and image editing in the fashion and apparel industry, and has broad application prospects and commercial value. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall operation steps of the present invention.

[0021] Figure 2 This is a flowchart illustrating the operation of the multimodal encoder array in this invention.

[0022] Figure 3 This is a flowchart of the multimodal feature parsing module in this invention.

[0023] Figure 4 This is an example of the generated result diagram of the present invention.

[0024] Figure 5 This is another example of the generated result diagram of the present invention. Detailed Implementation

[0025] The present invention will be further described below with reference to embodiments, but these embodiments are not intended to limit the scope of the invention.

[0026] Example: like Figure 1 As shown, the multimodal cue-guided customized clothing image generation model includes a multimodal encoder array, a multimodal feature parsing module, a modality-centric diffusion model, and a multimodal dataset. The multimodal encoder array extracts multimodal conditions from pattern images, texture images, text descriptions, images with clothing obscured, and the body pose structure of the target person. These conditions include semantic and structural features of the pattern image, global texture representation of the texture image, style semantic embedding of the text description, and pose features of the person. This provides semantically consistent multimodal embedding information for the multimodal fusion and conditional sequence generation of the multimodal feature parsing module. The multimodal feature parsing module performs spatial compression, dimensional alignment, and fusion operations on the multimodal embedding information generated by the multimodal encoder array, aiming to provide a unified and standardized multimodal embedding model for the diffusion model. The diffusion model is a controlled diffusion network structure used to iteratively denoise and reconstruct an initial Gaussian noise map under multimodal conditional constraints based on a multimodal conditional sequence that integrates pattern features, texture features, and style text embeddings, outputting a clothing image that matches the user's desired style. The multimodal dataset is used to train and evaluate the pattern-driven diffusion model, containing precisely paired pattern images, texture images, simple style text, and corresponding clothing images. It can support the diffusion model's image generation learning under pattern and texture control, improving generation quality and controllability, and also serves as a benchmark for performance comparison experiments.

[0027] The multimodal encoder array includes an image encoder, a texture encoder, and a text encoder. The image encoder is used to extract the semantic and structural features of the pattern image. The texture encoder is used to extract the global texture representation of the texture image, including color information and texture information. The text encoder is used to encode the text input into a semantic embedding, sharing the embedding space with the image encoder to facilitate modal alignment.

[0028] The multimodal feature parsing module includes a spatial feature compression unit, a modal projector, and a modal fusion unit. The spatial feature compression unit performs multi-scale dimensionality reduction compression on pattern images, texture images, and text descriptions, outputting low-dimensional tensors, which are then sequentially flattened and stitched together to obtain structural embedding vectors. These structural embedding vectors describe the multi-scale geometric structures present in the image and provide a unified, fixed-size representation for subsequent multimodal condition fusion. The modal projector unifies the linear projection and dimension of the structural embedding vectors of the multimodal conditions, achieving dimension alignment. The modal fusion unit stitches and fuses the dimensionality-reduced and dimension-aligned multimodal conditions into a unified multimodal condition sequence for the diffusion model to receive. Through the spatial feature compression unit, modal projector, and modal fusion unit, the multimodal feature parsing module achieves dimensionality reduction compression, dimension alignment, and stitching fusion of different modal features, improving the accuracy and control of each modal condition generation.

[0029] The spatial feature compression unit includes a multi-scale pooling layer, a flattening layer, and a connection layer. The multi-scale pooling layer is used to perform average pooling operations on multimodal conditions at different scales to reduce dimensionality at multiple scales and generate multiple low-dimensional tensors. The flattening layer is used to flatten the low-dimensional tensors output by each pooling layer into one-dimensional vectors, each one-dimensional vector representing a compressed representation of the pattern structure at a certain scale. The connection layer is used to concatenate all one-dimensional vectors into a unified structure embedding vector to describe the multi-scale geometric structure existing in the pattern, providing a unified and fixed-size representation for subsequent multimodal condition fusion, enhancing the diffusion model's ability to understand the pattern geometry, and effectively mitigating the structural degradation problem caused by image scaling.

[0030] The modal projector includes a pattern structure projector, a texture structure projector, and a text structure projector; the pattern structure projector is used to map a pattern image to a standard feature space; the texture structure projector is used to map a texture image to a standard feature space; and the text structure projector is used to map text to a standard feature space.

[0031] The diffusion model includes a backbone network, a hybrid attention mechanism, a pose preservation module, and a multimodal conditional injection mechanism. The backbone network is used to progressively generate clothing images from random noise images, undertaking the image generation task. The hybrid attention mechanism is used to jointly model three modalities: pattern, texture, and text. The pose preservation module is used to maintain the human posture structure through residual connections. The multimodal conditional injection mechanism is used to input the multimodal conditional sequence processed by the multimodal feature parsing module into the generation process of the backbone network. By introducing a spatial feature compression unit and integrating the hybrid attention mechanism, the diffusion model achieves high-precision control and image synthesis of pattern style, position, and style.

[0032] The backbone network includes an encoder and decoder module, a pose encoder module, and a time encoder module. The encoder and decoder module performs image feature extraction and reconstruction. The pose encoder module maintains the person's pose information. The time encoder module receives the time step embedding of the diffusion process. In the backbone network, during each iteration, a related noise is predicted based on multimodal conditions. Using the Markov chain principle, this noise is removed from the random noise image, realizing the process of progressively generating clothing images.

[0033] The hybrid attention mechanism includes a trimodal conditional embedding input module, three sets of attention calculation modules, and a modality fusion module. The trimodal conditional embedding input module receives multimodal conditional sequences from three modalities: pattern, texture, and text. The three sets of attention calculation modules use each set of multimodal conditional sequences as independent carrier keys and information keys, and the current generation state of the model as a query key, to interact with the three modal conditions and jointly model to obtain multimodal information data. The modality fusion module weights or merges the multimodal information data with the backbone network to generate image guidance features centered on the pattern. This guides the backbone network to generate clothing images that match the three modal conditions in each iteration, and the output serves as the image representation for that iteration before entering the next step of the backbone network.

[0034] The pose preservation module includes a pose map extractor, a pose encoder, and a residual injection module. The pose map extractor extracts pose information from the original human image to form a pose map. The pose encoder converts the pose map into a pose feature tensor and outputs a feature dimension that matches the intermediate layers of the U-Net backbone network, allowing it to be used in subsequent backbone networks. The residual injection module injects the pose feature tensor as a "hard constraint" residual guiding signal into the diffusion network, thereby ensuring that the human pose in the generated image is consistent with the human pose in the original input image and that no structural errors occur.

[0035] The multimodal dataset includes multiple sample sets consisting of pattern images, texture images, text descriptions, and target clothing images. Each sample constitutes a clearly paired input-output pair of the three modalities, which is used to train a pattern-driven diffusion model.

[0036] The pattern images mentioned above are used to represent the visual patterns of clothing, the texture images are used to represent the material or texture style, the text descriptions are used to represent the semantics of the style, and the target images are used to supervise the output quality of the diffusion model.

[0037] The processing method of the above-mentioned customized clothing image generation model, such as Figure 1 As shown, it includes the following steps: S1. Input pattern images, texture images, text descriptions, images of clothing coverings, and the body posture structure of the target person into the multimodal encoder array; S2. The multimodal encoder array adjusts the pattern image and texture image to a fixed resolution, normalizes the image pixel values, segments and tokenizes the text description, and converts it into a suitable CLIP format. S3. The multimodal encoder array extracts multimodal conditions, including semantic and structural features corresponding to pattern images, texture images and text descriptions, and generates multimodal embedding information; like Figure 2 As shown, step S3 specifically involves: S31. Input the pattern image, the image with clothes covered, and the human pose image into the VAE encoder, convert them into a structural feature map in a low-dimensional space, and extract the geometric shape, edge and local texture layout and other structural features of the pattern image and the image with clothes covered, as well as the human pose features. S32. Input the pattern image and texture image into the CLIP image encoder to extract the semantic features of the pattern image and texture image, including color information and texture information; S33. Input the text description into the CLIP text encoder to convert it into a semantic token sequence and extract semantic features such as clothing style and design from the text description. S34. Finally, a set of three modal representations of pattern-texture-text is obtained, including the structural features of the pattern image, the semantic features of the pattern image, the semantic features of the text description, and the semantic features of the texture image, generating multimodal embedding information.

[0038] S4. The multimodal feature parsing module performs multi-scale dimensionality reduction compression, dimension alignment, and splicing fusion on the multimodal embedding information to obtain the multimodal conditional sequence. like Figure 3 As shown, step S4 specifically involves: S41. Select different pooling window sizes for the structural features of the pattern image, the semantic features of the pattern image, the semantic features of the texture image, and the semantic features of the text description, respectively, perform average pooling operation to achieve multi-scale dimensionality reduction, output low-dimensional tensors, flatten each low-dimensional tensor into a one-dimensional vector, and concatenate them sequentially to obtain the structure embedding vector. S42. Perform linear projection on the structure embedding vector of the multimodal condition to obtain the pattern structure identifier, pattern semantic identifier, texture semantic identifier and text semantic identifier, and align them in the semantic space. S43. The pattern structure identifier and the pattern semantic identifier are weighted and combined to obtain the comprehensive pattern identifier; S44. The integrated pattern identifier, texture semantic identifier, and text semantic identifier are concatenated in a preset order to form a unified multimodal conditional sequence. The multimodal conditional sequence has a fixed length and a unified dimension, which makes it convenient to directly use it as a conditional input to the diffusion model to guide the hybrid attention mechanism of each layer of the diffusion model when generating images.

[0039] S5. Input the multimodal conditional sequence into the diffusion model through the multimodal conditional injection mechanism, and generate (K,V) pairs of pattern, texture and text respectively through the hybrid attention mechanism, where K represents the carrier key and V represents the information key, to obtain image guidance features; Step S5 is as follows: S51. The multimodal conditional injection mechanism inputs the multimodal conditional sequence into the diffusion model, and the hybrid attention mechanism generates multimodal information data ((K,V) pairs) of patterns, textures and text from the multimodal conditional sequence. S52. The intermediate layer of the backbone network matches and weights the multimodal information data ((K,V) pairs) through the modality fusion module. The intermediate layer of the backbone network first multiplies the data with the information key matrix, then applies a signod activation function to the obtained matrix, and then multiplies it with the carrier key matrix to obtain image guidance features centered on the pattern, which can accurately guide image generation.

[0040] S6. Pose preservation and image decoding are performed using a diffusion model.

[0041] Step S6 is as follows: S61. Under a random initial value of the U-Net backbone network, the pose map extracted from the original human image is introduced as structural constraint information through the pose preservation module to ensure that the human pose in the generated image is consistent with the human body pose structure in the original image. The S62 and U-Net backbone networks use Markov chains for denoising diffusion, and in each iteration, they gradually generate feature maps that match the three modal conditions by using image-guided features. S63. The feature map is deconvolved and decoded by the image decoder to obtain a clothing image with a complete appearance.

[0042] S7. Output a virtual clothing image of a character that meets the requirements of pattern, texture, and text description.

[0043] like Figure 4 and Figure 5 As shown, the input is an original image of a person's clothing. The required multimodal conditions (pattern, texture, text) are obtained through a multimodal encoder array. Based on the multimodal conditions, a virtual image of a person's clothing that meets the expected conditions is generated using a multimodal feature parsing module, a diffusion model, and a multimodal dataset.

[0044] It should be understood that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Those skilled in the art can modify the technical solutions described in the above embodiments, or make equivalent substitutions for some of the technical features; and all such modifications and substitutions should fall within the protection scope of the appended claims of the present invention.

Claims

1. A multimodal cue-guided custom clothing image generation model, characterized in that: The system includes a multimodal encoder array, a multimodal feature parsing module, a diffusion model, and a multimodal dataset. The multimodal encoder array is used to extract multimodal conditions, including semantic and structural features of pattern images, global texture representation of texture images, style semantic embedding of text descriptions, and human pose features, providing multimodal embedding information. The multimodal feature parsing module is used to perform spatial compression, dimensional alignment, and fusion operations on the multimodal embedding information, providing a multimodal condition sequence. The diffusion model is used to iteratively denoise and reconstruct an initial Gaussian noise map under multimodal condition constraints based on the multimodal condition sequence, outputting a clothing image that conforms to the user's desired style. The multimodal dataset is used to train and evaluate the pattern-driven diffusion model.

2. The multimodal cue-guided customized clothing image generation model according to claim 1, characterized in that: The multimodal encoder array includes an image encoder, a texture encoder, and a text encoder. The image encoder is used to extract the semantic and structural features of the pattern image. The texture encoder is used to extract the global texture representation of the texture image, including color information and texture information. The text encoder is used to encode the text input into a semantic embedding, sharing the embedding space with the image encoder to facilitate modal alignment.

3. The multimodal cue-guided customized clothing image generation model according to claim 1, characterized in that: The multimodal feature parsing module includes a spatial feature compression unit, a modal projector, and a modal fusion unit. The spatial feature compression unit performs multi-scale dimensionality reduction compression on pattern images, texture images, and text descriptions, outputting low-dimensional tensors, which are then flattened and stitched together to obtain a structure embedding vector. The modal projector unifies the linear projection and dimension of the structure embedding vector, achieving dimension alignment. The modal fusion unit stitches and fuses the dimensionality-reduced and dimension-aligned multimodal conditions into a unified multimodal condition sequence for the diffusion model to receive.

4. The multimodal cue-guided customized clothing image generation model according to claim 3, characterized in that: The spatial feature compression unit includes a multi-scale pooling layer, a flattening layer, and a connection layer. The multi-scale pooling layer is used to perform average pooling operations on multi-modal conditions at different scales to reduce dimensionality at multiple scales and generate multiple low-dimensional tensors. The flattening layer is used to flatten each low-dimensional tensor into a one-dimensional vector. The connection layer is used to concatenate all one-dimensional vectors into a unified structure embedding vector. The modal projector includes a pattern structure projector, a texture structure projector, and a text structure projector; the pattern structure projector is used to map a pattern image to a standard feature space; the texture structure projector is used to map a texture image to a standard feature space. The text structure projector is used to map text to a standard feature space.

5. The multimodal cue-guided customized clothing image generation model according to claim 1, characterized in that: The diffusion model includes a backbone network, a hybrid attention mechanism, a pose preservation module, and a multimodal conditional injection mechanism. The backbone network is used to progressively generate clothing images from random noise images. The hybrid attention mechanism is used to jointly model three modalities: pattern, texture, and text. The pose preservation module is used to preserve the human pose structure through residual connections. The multimodal conditional injection mechanism is used to input multimodal conditional sequences into the generation process of the backbone network.

6. The multimodal cue-guided customized clothing image generation model according to claim 5, characterized in that: The backbone network includes an encoder and decoder module, an attitude encoder module, and a time encoder module; the encoder and decoder module is used to perform image feature extraction and reconstruction. The posture encoder module is used to maintain the person's posture information; the time encoding module is used to receive the time step embedding of the diffusion process. The hybrid attention mechanism includes a trimodal conditional embedding input module, three sets of attention calculation modules, and a modality fusion module. The trimodal conditional embedding input module receives multimodal conditional sequences from three modalities: pattern, texture, and text. The three sets of attention calculation modules use each modal's multimodal conditional sequence as an independent carrier key and information key, and the model's current generation state as a query key, interacting with the three modal conditions to jointly model and obtain multimodal information data. The modality fusion module weights or merges the multimodal information data with the backbone network to generate image-guided features centered on the pattern. The pose preservation module includes a pose map extractor, a pose encoder, and a residual injection module. The pose map extractor extracts pose information from the original human image to form a pose map. The pose encoder converts the pose map into a pose feature tensor and outputs a feature dimension that matches the intermediate layers of the U-Net backbone network, allowing it to be used in subsequent backbone networks. The residual injection module injects the pose feature tensor as a hard constraint residual guiding signal into the diffusion network, thereby ensuring that the human pose in the generated image is consistent with the human pose in the original input image and that no structural errors occur.

7. The multimodal cue-guided customized clothing image generation model according to claim 1, characterized in that: The multimodal dataset includes multiple sample sets consisting of pattern images, texture images, style text descriptions, and target clothing images. Each sample constitutes a clearly paired input-output pair of the three modalities, which is used to train a pattern-driven diffusion model.

8. The processing method for the customized clothing image generation model according to any one of claims 1-7, characterized in that: Includes the following steps: S1. Input pattern images, texture images, text descriptions, images of clothing coverings, and the body posture structure of the target person into the multimodal encoder array; S2. The multimodal encoder array adjusts the pattern image and texture image to a fixed resolution, normalizes the image pixel values, segments and tokenizes the text description, and converts it into a suitable format. S3. The multimodal encoder array extracts multimodal conditions, including semantic and structural features corresponding to pattern images, texture images and text descriptions, and generates multimodal embedding information; S4. The multimodal feature parsing module performs multi-scale dimensionality reduction compression, dimension alignment, and splicing fusion on the multimodal embedding information to obtain the multimodal conditional sequence. S5. Input the multimodal conditional sequence into the diffusion model through a multimodal conditional injection mechanism to obtain image guidance features; S6. Pose preservation and image decoding are performed using a diffusion model; S7. Output virtual clothing images of characters that meet the style requirements of patterns, textures, and text descriptions.

9. The processing method according to claim 8, characterized in that: Step S3 is as follows: S31. Input the pattern image, the image with clothes covered, and the human pose image into the image encoder, convert them into a structural feature map in a low-dimensional space, and extract the geometric shape, edges, local texture layout, and human pose features of the pattern image and the image with clothes covered. S32. Input the pattern image and texture image into the image encoder and texture encoder to extract the semantic features of the pattern image and texture image; S33. Input the text description into the text encoder and extract the semantic features of the text description; S34. Finally, a set of three modal representations of pattern-texture-text is obtained, including the structural features of the pattern image, the semantic features of the pattern image, the semantic features of the text description, and the semantic features of the texture image, generating multimodal embedding information.

10. The processing method according to claim 8, characterized in that: Step S4 is as follows: S41. Select different pooling window sizes for the structural features of the pattern image, the semantic features of the pattern image, the semantic features of the texture image, and the semantic features of the text description, respectively, perform average pooling operation to achieve multi-scale dimensionality reduction, output low-dimensional tensors, flatten each low-dimensional tensor into a one-dimensional vector, and concatenate them sequentially to obtain the structure embedding vector. S42. Perform linear projection on multiple structure embedding vectors to obtain pattern structure identifier, pattern semantic identifier, texture semantic identifier and text semantic identifier, and align them in semantic space; S43. Use a hybrid attention mechanism to weightedly combine the pattern structural identifier and the pattern semantic identifier to obtain a comprehensive pattern identifier; S44. Combine the integrated pattern identifier, texture semantic identifier, and text semantic identifier in a preset order to form a unified multimodal condition sequence.

Images