DiT architecture-based costume design increment generation method and system

Abstract

The invention discloses a DiT architecture-based costume design increment generation method and system, and relates to the technical field of fashion design. According to the method, the powerful generation capability of the DiT architecture is utilized, local details can be gradually adjusted through multi-stage increment generation and refinement on the basis of initial design, and design continuity and local refinement adjustment are achieved. Through combination of a condition control technology and a multipath diffusion model, a designer can flexibly modify a specific area in a design process and dynamically optimize an overall design scheme according to feedback. According to the method, the precision and controllability of costume design are remarkably improved, the problem of insufficient flexibility in a traditional design process is effectively solved, and a more intelligent and efficient design tool is provided for designers. According to the method, gradual refinement and adjustment of the design can be realized through combination of staged increment generation, condition control and multi-path diffusion, so that the defects of a traditional design tool are remarkably overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of fashion design, and in particular to a method and system for incrementally generating fashion design based on a DiT architecture. Background Art

[0002] In the field of intelligent design and computer-aided design, especially in fashion design, existing generative AI (GenerativeAI) tools such as StyleGAN and Variational Autoencoder (VAE) have begun to be tried and applied. These tools have greatly promoted design innovation by learning a large number of design samples and generating novel design drawings. However, although these technologies have outstanding performance in generating novel designs, they still have some significant defects, which are analyzed as follows:

[0003] Insufficient design continuity: Traditional generative AI tools usually adopt a full-scale generation method, and each design update must start from scratch. This method lacks continuity between design versions, and designers often lose creative coherence when adjusting the original design, resulting in low creative efficiency. When faced with multiple modifications, designers are unable to retain the completed design elements and find it difficult to form a complete system design.

[0004] Accuracy issues: In traditional design processes, designers often rely on manual editing when adjusting local details, while AI-based generation tools have limited control over details. This is because the generative model often has difficulty understanding the designer's specific intentions when generating local features, resulting in a significant gap between the generated results and the designer's expectations. This lack of accuracy is mainly due to the model's lack of understanding of complex design concepts, which limits its effective application in professional design.

[0005] Local adjustability limitations: Existing generative AI tools often lack flexible local adjustment mechanisms. Most tools only support global parameter adjustments and cannot make detailed modifications to specific parts. This limitation prevents designers from getting immediate feedback and optimization when dealing with specific design details, which affects the quality and creative expression of the final design. The structure of traditional tools is usually based on a statically generated framework and cannot support complex interactive design processes.

[0006] Insufficient feedback and iteration capabilities: Existing design tools often lack the ability to interact with designers in real time and cannot respond to designers' feedback in a timely manner. The lack of flexibility in the design process requires designers to spend a lot of time on subsequent optimization when making modifications, and this delayed feedback leads to a decrease in overall design efficiency. Structurally, these tools mostly use a single generation path and lack a mechanism for dynamic adjustment based on feedback information. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for incremental generation of clothing design based on the DiT architecture, which can realize the gradual refinement and adjustment of the design through the combination of phased incremental generation, conditional control and multi-path diffusion, thereby significantly improving the shortcomings of traditional design tools.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A clothing design incremental generation method based on DiT architecture, comprising:

[0010] Selecting target elements in a database to determine initial design requirements; the database includes a design style model library, a user preference library, and a material library;

[0011] Based on the initial design requirements, the AI ​​prompt words of the design drawing are automatically generated using the dialog box module of the design prompt word builder; the AI ​​prompt words reflect the original intention, background, design style, user preference and clothing material of the design;

[0012] Based on the AI ​​prompt word, a global image generator is used to perform text word meaning analysis and encoding, and output semantic encoding information with a global context; the global image generator has a built-in Transformer model; the Transformer model processes the input text through word segmentation, word embedding, and encoding based on word meaning context;

[0013] According to the designer's needs, it is determined whether to generate a global design image or perform local optimization to determine the final output design image; when it is decided to generate a global design image, the first processing flow is executed and the diffusion model is used for optimization and adjustment; when it is decided to perform local optimization, the second processing flow is executed and the DiT model is used for optimization and adjustment.

[0014] Optionally, the style model library stores classifications, characteristics, examples and related attribute information of different styles; the user preference library stores historical design data and preference information corresponding to user types, and supports personalized recommendations and dynamic updates; the material library stores the characteristics, applicable scenarios and supplier information of various materials involved in clothing design.

[0015] Optionally, the generating a corresponding clothing design drawing according to the semantic coding information using a diffusion model specifically includes:

[0016] The semantic encoding information is used as the conditional input of the diffusion model. Then, the model uses the initial Gaussian noise image as the starting point, and combines the vector of the design requirement with the initial noise through conditional mapping, so that the potential features in the noise are consistent with the design requirement, thereby guiding the image generation to develop in a specific direction and completing the conditional injection;

[0017] After the injection conditions, conditional control and image constraints are performed on each stage of image generation, and denoising is performed in each diffusion step to obtain the corresponding clothing design image.

[0018] Optionally, the first processing flow specifically includes:

[0019] Generate a corresponding clothing design drawing according to the semantic coding information using a diffusion model;

[0020] The clothing design drawing is provided to the designer in high definition through the image display module of GEI, and a feedback interface is provided. When the designer is not satisfied with the feedback, the clothing design drawing is adjusted and optimized to determine the final output design image; the adjustment and optimization process includes editing the modification area, generating the modification area Mask and local iterative design.

[0021] Optionally, the process of editing and modifying the area includes:

[0022] Designers use the "Free Selection Tool" in the image editing interactive module to draw irregular polygons or rectangles on the image to select the parts that need to be modified. It supports the drawing of single or multiple discontinuous selections to facilitate local optimization. During operation, designers use the mouse or stylus to draw on any part of the image. The selected area can be a pattern, material texture, color block or cropping detail. The system will automatically identify and generate the boundary line of the selection to ensure accuracy, and allow designers to adjust the shape and size of the selection by dragging the edge to achieve precise control. At the same time, it supports the undo and redraw functions to facilitate and quickly correct unsatisfactory selections.

[0023] The process of generating the modified area Mask includes:

[0024] After the designer selects and adjusts the area to be modified, the system will automatically generate a mask with the selected area in white and the overall image in black with a white background, which is used to accurately mark the modified area and assist in subsequent optimization processing; Mask is a mask layer covering the image, ensuring that the modification operation only acts on the selected area without affecting other parts; when the designer completes the selection of the local area, the system will automatically generate a mask with the same shape according to the selection, which is used to isolate and mark the area; the generated mask is separated from the background image, and the designer operates it separately to ensure that the modification is limited to the selected area;

[0025] The generated Mask image and the original design drawing will be saved together in the continuous design timing library for subsequent multiple continuous and precise design work;

[0026] The local iterative design process includes:

[0027] After the mask image and the original design drawing are saved in the continuous design timing library, the designer will repeat the previous steps of "determining global design image generation or local optimization according to designer needs to determine the final output design image".

[0028] Optionally, the second processing flow specifically includes:

[0029] Based on the semantic coding information, the DiT model is used to generate an image for the designed local area and seamlessly merge it with the original image; the processing process of the DiT model includes: adding noise, image merging, local generation condition injection, local generation condition control, local image feature sampling, local image feature recovery and output image.

[0030] Optionally, the process of adding noise includes:

[0031] In the process of generating a local optimized image, the system first adds Gaussian noise to the original image and the local optimized area Mask respectively. This step provides the initial state for the subsequent image generator, and gradually builds an image that meets the designer's requirements from the noise. In order to ensure that the generated image retains global information while focusing on local optimization, the system independently adds noise to the original image and the local area covered by the Mask. The intensity of the noise injection is controlled by preset parameters to ensure that the clarity of the unmodified area is not disturbed, while providing sufficient freedom for the generation of local images to adjust and optimize.

[0032] The image merging process includes:

[0033] After adding noise to the original image and the local mask area, the system merges the encoding information of the two to generate a complete input image encoding;

[0034] The process of local generation condition injection includes:

[0035] Based on the design requirement semantics encoded by Transformer, the system first extracts the designer's local optimization requirements through the design requirement encoding, and combines the requirements with the local Mask region encoding to generate a conditional local generation vector;

[0036] The process of local generation condition control includes:

[0037] Through a multi-level condition control mechanism, the system continuously re-injects and adjusts the design conditions at each step of diffusion generation to ensure that the generation of the local area is consistent with expectations;

[0038] The process of local image feature sampling includes:

[0039] Through sampling technology, the system extracts the color, texture, light and shadow and other detailed features of the local area, and compares them with the global information of the original image to ensure that the local modification not only meets the designer's needs, but also maintains the same visual style as the unmodified part;

[0040] The process of restoring local image features includes:

[0041] By adopting the inverse steps of the diffusion model and performing boundary smoothing transition processing, the system avoids visual discontinuity or conflict between local modifications and surrounding areas, and gradually restores clear image details from the initial noise.

[0042] The present invention also provides a clothing design incremental generation system based on the DiT architecture, which is applied to the clothing design incremental generation method as described above, and includes: a prompt word constructor, a GEI image displayer, a GEI image editor, a global image generator, and a DiT local image generator;

[0043] Among them, the prompt word constructor is respectively connected to the design style model library, the user preference library, the material library and the global image generator; the global image generator is also respectively connected to the GEI image displayer and the DiT local image generator; the DiT local image generator is also respectively connected to the GEI image displayer and the GEI image editor.

[0044] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] The invention discloses a method and system for incremental generation of clothing design based on DiT architecture, the method comprising: selecting a target element in a database to determine the initial design requirements; based on the initial design requirements, automatically generating AI prompt words of the design drawing using a dialog box module of a design prompt word constructor; based on the AI ​​prompt words, using a global image generator to perform text semantic analysis and encoding, and outputting semantic encoding information with global context; the global image generator is built with a Transformer model; the Transformer model processes the input text through word segmentation, word embedding and encoding based on word meaning context; determining to perform global design image generation or local optimization according to the designer's needs, and determining the final output design image; when deciding to perform global design image generation, executing a first processing flow, using a diffusion model for optimization adjustment, and when deciding to perform local optimization, executing a second processing flow, using a DiT model for optimization adjustment. The present invention can realize the gradual refinement and adjustment of the design through the combination of phased incremental generation, conditional control and multi-path diffusion, thereby significantly improving the shortcomings of traditional design tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 It is the overall implementation logic diagram of the present invention;

[0048] Figure 2 Schematic diagram of the system implementation mechanism structure in this embodiment. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The purpose of the present invention is to provide a method and system for incremental generation of clothing design based on the DiT architecture, which can realize the gradual refinement and adjustment of the design through the combination of phased incremental generation, conditional control and multi-path diffusion, thereby significantly improving the shortcomings of traditional design tools.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1-Figure 2 As shown, the present invention provides a clothing design incremental generation method based on the DiT architecture, and the specific processing process of each step is as follows:

[0053] Step 1. Choose a design style

[0054] Before AI generates complex design drawings, fashion designers can first select elements such as the clothing style, user preferences, and materials to be designed in the design material model library to facilitate the rapid generation of design requirements. Designers interact with the system through the design prompt word constructor. First, they select a style from the design style classification provided by the system, such as modern style, retro style, street style, or high-end fashion style. They can also quickly search for specific styles through keywords. The system automatically recommends style options that match the designer's preferences based on the designer's historical design data, and recommends relevant design elements such as color, version, and tailoring based on the user preference model. Next, designers can select clothing materials such as cotton, silk, wool, and leather from the material library, and filter and select according to material properties such as breathability and elasticity. After selecting the design style, user preferences, and materials, the system combines this information into structured design prompt words to guide the subsequent AI design generation process.

[0055] (1) Design style model library. Stores the classification, characteristics, examples and related attribute information of different styles to help designers quickly find the right style for design. The style_model data structure is as follows:

[0056] style_id: A unique identifier for a style, used to uniquely identify each design style.

[0057] style_name: The name of the style, such as "modern style", "retro style", etc.

[0058] style_category: The category to which the style belongs, such as "street fashion", "haute couture", "minimalism", etc.

[0059] description: A brief description of the style, providing text information to help designers understand the style characteristics.

[0060] examples: Contains sample images and brief descriptions of the style for designers to refer to.

[0061] Attributes: Key characteristics of a design style, including color, shape, pattern, and target audience (e.g., men, women, children, etc.).

[0062] related_styles: Style IDs related to this style, used to provide recommendations for similar styles.

[0063] (2) User preference library. Stores historical design data and preference information corresponding to user types, supporting personalized recommendations and dynamic updates. The preferred_model data structure is as follows:

[0064] user_type: A unique identifier for each user type, used to associate preference and user type information.

[0065] preferred_styles: A list of styles that the user prefers, including the style ID and the designer's preference for the style.

[0066] preferred_colors: The user's preferred colors and their corresponding preference levels, expressed in hexadecimal code.

[0067] preferred_materials: The materials that the user prefers and their degree of preference.

[0068] preferred_shapes: The user's preferred clothing cut, fit, or shape and their preference.

[0069] historical_designs: past purchase records of user types, recording elements such as the style and materials used in the design.

[0070] recent_interactions: The user's recent interactions with the system, including styles and materials viewed, liked, or used.

[0071] (3) Material library. Stores the characteristics, applicable scenarios and supplier information of various materials involved in clothing design to help designers choose materials that meet their needs. The material_model data structure is as follows:

[0072] material_id: A unique identifier for the material.

[0073] material_name: The name of the material, such as "cotton", "silk", "wool", etc.

[0074] material_category: Material classification, such as "natural materials", "synthetic materials", etc.

[0075] description: A brief description of the material, outlining its characteristics and scope of application.

[0076] properties: The physical properties of materials, including breathability, elasticity, durability, texture, gloss and other information, help designers choose suitable materials.

[0077] usage_scenarios: Scenarios or types of clothing the material is suitable for, e.g. "Summer wear", "Formal wear", "Sports wear", etc.

[0078] environmental_impact: The environmental attributes of the material, including sustainability and whether it is recyclable.

[0079] color_options: The color options available for this material, using hexadecimal color codes.

[0080] cost_per_meter: The cost of material per meter, usually a decimal number for easy cost calculation.

[0081] supplier_info: Relevant information of material suppliers, including supplier name and contact information, to facilitate subsequent material procurement.

[0082] Step 2. Generate design requirements

[0083] The designer enters the initial design requirements in natural language in the chatbox module of the design prompt builder. Combined with the design style selected in step 1, the AI ​​prompt for the design drawing is automatically generated. The prompt will reflect the design's original intention, background, design style, user preferences, clothing materials and other detailed design requirements. The structured field structure of the design requirement prompt design_prompt is as follows:

[0084] (1) style

[0085] style_type: Specific clothing design style type, such as "minimalism", "retro style", "street fashion", etc.

[0086] style_elements: A list of specific elements involved in this style, such as "geometric pattern", "symmetrical design", "futuristic sense", etc.

[0087] (2)garment_type (garment type)

[0088] garment_name (garment name): the type of garment to be designed, such as "dress", "jacket", "shirt", etc.

[0089] occasion: The occasion for which the garment is suitable, such as "dinner party", "daily leisure", "formal occasion", etc.

[0090] (3) material_texture (material and texture)

[0091] material_type: The type of material used for the garment, such as "cotton", "silk", "wool", "leather", etc.

[0092] material_properties: A list of material properties, such as "breathability", "elasticity", "gloss", "lightness", etc.

[0093] (4)color_tone (color and tone)

[0094] primary_color: The main color of the clothing, such as "blue", "red", etc.

[0095] secondary_color: The secondary auxiliary color, which may be a matching color or an accent color.

[0096] color_mood: The overall mood or atmosphere brought by the color tone, such as "soft", "bright", "cool", "warm", etc.

[0097] (5) pattern_decoration (pattern and decoration)

[0098] pattern_type: The type of pattern used on the garment, such as "stripes", "checkered", "floral", etc.

[0099] decoration_elements: Decorative details on clothing, such as "embroidery", "tassels", "sequins", "buttons", etc.

[0100] (6) cut_silhouette (cutting and silhouette)

[0101] fit_type: The style or fit of the garment, such as "slim", "loose", "A-line", etc.

[0102] silhouette_features: The overall silhouette features of the garment, such as "high waist", "drape", "layering", etc.

[0103] structural_details: Structural details in clothing design, such as "V-neck", "slit", "bat sleeves", etc.

[0104] (7) functional_requirements

[0105] Seasonality: The season for which the clothing is suitable, such as "spring", "summer", "autumn and winter", etc.

[0106] functional_properties: functional requirements of clothing, such as "warmth", "waterproof", "breathable", "lightweight", etc.

[0107] target_audience: The target audience of the clothing design, such as "male", "female", "children", "neutral", etc.

[0108] (8) overall_style_mood (overall style and mood)

[0109] mood_inclination: The overall emotion or atmosphere conveyed by the clothing, such as "romantic", "elegant", "cool", "casual", etc.

[0110] theme_concept: The theme concept of clothing design, such as "natural style", "technological future", "retro fashion", etc.

[0111] (9)special_requirements (special requirements)

[0112] Sustainability: Environmental or sustainable requirements for clothing, such as "environmentally friendly materials", "renewable resources", "non-toxic dyes", etc.

[0113] production_technique: The special production technique of the garment, such as "handmade", "laser cutting", "3D printing", etc.

[0114] (10) brand_culture (Brand and culture)

[0115] brand_positioning: The positioning of the brand of clothing, such as "high-end", "luxury", "affordable fashion", etc.

[0116] cultural_background: The cultural background of the design reference, such as "Oriental elements", "European and American classics", "African tribal style", etc.

[0117] The content of each field is described in natural language, but the overall design requirement prompt word design_prompt can be structured. The design requirement prompt word will be transmitted to the global image generator through the API in JSON format to control the generation of design renderings.

[0118] Input: design style model style_m (JSON), user preference model preferred_m (JSON), clothing material model material_m (JSON).

[0119] Output: design requirement prompt word design_prompt (JSON).

[0120] Step 3. Transformer encoding

[0121] After the design requirements (structured prompt words organized in JSON format) submitted by the designer are received by the global image generator, the system first needs to perform semantic analysis and encoding on these texts. In order to ensure that the design requirements can be correctly interpreted and used to generate images that meet the expectations, the Transformer model processes the input text through word segmentation, word embedding, and encoding based on word meaning context. This process will use the global shared vector space design_global_vector, that is, the vector space pre-trained by a deep learning model (such as BERT or CLIP) to ensure that each word or phrase has a unified semantic representation in the image generation task. The specific sub-steps include:

[0122] (1) Word segmentation. The input design requirements are structured data in JSON format, which contains the designer's requirements (such as design style, material, color, tailoring, decoration, etc.). Word segmentation is to split the design requirement text into basic words or subword units (tokens), which will serve as input for subsequent word embedding and encoding. The Transformer model used in the present invention adopts subword tokenization (Subword Tokenization), such as BPE (Byte-Pair Encoding) or WordPiece algorithm, to decompose words into smaller units, especially for uncommon words.

[0123] Input: Design requirement prompt word design_prompt (JSON).

[0124] Output: design requirement tokens design_tokens (string array).

[0125] (2) Word embedding. The tokens after word segmentation will be mapped to the shared vector space of clothing design images, which is established in advance through pre-trained models (such as BERT, GPT, CLIP, etc.). It is a multimodal vector space trained through a large-scale pre-trained corpus (such as image-text pairs), so that images and texts can be matched in the same semantic space. Each word, phrase or concept can be represented by a vector, and similar semantics will be mapped to adjacent positions.

[0126] The present invention adopts the CLIP framework (Contrastive Language-Image Pre-training) based on the Transformer model to perform word embedding. It generates alignment between text and image, and can map the design requirement text into a vector space related to image generation.

[0127] Input: design requirement tokens design_tokens (string array).

[0128] Output: design requirement word embedding design_embeddings (32-bit floating point matrix).

[0129] (3) Context encoding. In order to better understand the exact semantics of each design word in the design requirement description, the present invention uses a multi-head self-attention mechanism to calculate the relationship between each design word and other words in the sequence. The context of each design word will be used to adjust its embedding vector so that it contains relevant information about other words in the sentence. For example, when "geometric patterns" and "modern minimalism" are put together, the overall design intention may emphasize the style of "simple, technological" rather than other possible interpretations. The system integrates the context-encoded word embedding vectors to generate a global 512-dimensional feature vector to represent the semantics of the entire design requirement.

[0130] For example, each part of the design requirements (style, material, color, etc.) will be converted into feature vectors with global context, which will be used in subsequent image generation tasks.

[0131] Input: design requirement word embedding design_embeddings (32-bit floating point matrix); clothing image design shared vector space design_global_vector (32-bit floating point matrix).

[0132] Output: self-attention output attention_output (32-bit floating point matrix); global representation design_representation (512-bit vector).

[0133] Step 4. Diffustion image generation

[0134] At this point, we need to logically determine whether to generate a global design drawing. If so, execute this step. If not, jump to step 9. In this step, we will use the diffusion model (DiffusionModel) to generate the clothing design drawings required by the designer. The basic principle of this model is to gradually generate high-quality images that meet the design requirements from noise through the reverse diffusion process. Specifically, we will combine the semantic encoding information (such as style, material, color, etc.) in the third step and the design image shared vector space design_global_vector to guide the diffusion model to gradually generate images through a conditional control mechanism.

[0135] The key sub-steps involved in this step include: conditional injection, conditional control, reverse diffusion, output image, etc. The following is a detailed description.

[0136] (1) Conditional injection. The generation process of the diffusion model requires the input of initial noise, but in order to make the generated image meet the specific needs of the designer, conditional injection is required. First, the semantic vector design_representation (containing elements such as design style, clothing type, color, material, etc.) generated by Transformer encoding in step 3 is used as the conditional input of the diffusion model; then, the model uses the initial Gaussian noise image as the starting point, and combines the vector of design requirements with the initial noise through conditional mapping, so that the potential features in the noise are consistent with the design requirements, thereby guiding the image generation in a specific direction. For example: If the designer's needs are "street style, black jacket with geometric patterns", then these conditions will be injected into the initial noise of the model to ensure that the final generated image meets these characteristics.

[0137] Input: global semantic vector design_representation, the 512-dimensional feature vector from step 3, which contains the semantic vector space expression of all the designer’s design requirements.

[0138] Output: conditional vector condition_vector (512-dimensional feature vector) is used to control each step of the generation process; initial noise image initial_noise (32-bit floating point matrix), Gaussian noise image of the input diffusion model.

[0139] (2) Condition control. In order to ensure that the diffusion process generates images that meet the design requirements, condition control is required at each stage. This is achieved through multi-level condition control. In the process of gradually generating images, the diffusion model will reintroduce design conditions at each step (T steps) to ensure that the generated images meet the requirements, such as "street style" or "black jacket". At the same time, the model uses the self-attention mechanism to capture the dependencies between design requirements and coordinate the associations between elements such as clothing color and material. In addition, the model will impose image constraints to ensure that the generated images meet the functional requirements such as material texture and color matching while satisfying visual aesthetics. For example: When the diffusion model generates an image in the middle stage (i.e., the middle stage of the T step), the system will check again whether the generated part of the image has a geometric pattern and ensure that the material texture meets the "leather" texture selected by the designer.

[0140] Input: conditional vector condition_vector (512-dimensional feature vector), initial noise image initial_noise.

[0141] Output: Diffusion step diffusion_t (integer), progress information of each step in the diffusion process (from T to 0).

[0142] (3) Reverse diffusion. The core mechanism of the diffusion model is reverse diffusion, which is to generate a clear image by gradually removing noise. First, the model starts with a completely random Gaussian noise image, and in each diffusion step (from T to 0), it gradually removes noise according to conditional control and design requirements, making the image clearer and clearer. As the denoising progresses, the model continues to integrate design elements: the early stage generates basic outlines, such as the overall shape and style of the clothes; the mid-stage refines visual details such as color and material texture; the late stage adds details such as patterns, decorations, and tailoring. Ultimately, reverse diffusion generates a high-quality image that contains all design requirements, including style, material, color, pattern, and decoration.

[0143] Input: Diffusion step diffusion_t.

[0144] Output: generated image generated_image, the final generated clothing design image.

[0145] (4) Output image. After reverse diffusion, the system will output the final design image, which will be saved and presented to the designer for further adjustment or confirmation. The image has the following characteristics: First, it meets the design requirements entered by the designer in step 3, including elements such as style, color and material; second, the refinement of the diffusion model makes the image have a resolution of 300DPI or higher, which is suitable for subsequent design decisions or presentations; finally, the generated image file will be transmitted to the GEI image displayer through the API. The designer can further adjust the generated results or request the system to generate a new design drawing to expand the design inspiration.

[0146] Input: generated image generated_image.

[0147] Output: preliminary design rendering file initial_design_file, which can be in standard image format such as PNG, JPEG, etc., with a resolution of 300DPI or higher.

[0148] Step 5. Display the design

[0149] The clothing design file initial_design_file generated in step 4 will provide high-definition images to designers through GEI's image display module (GV). At the same time, the system provides a feedback interface where designers can give instant feedback on the generated images (such as "like" or "dislike" with additional comments).

[0150] At this point, if the designer is satisfied with the design, this design process ends. If not, especially if the designer is not satisfied with the design details, then go to step 6 to further adjust and optimize the design. Of course, if the designer is not satisfied with the overall design, then there is no need to execute step 6, but to re-execute step 1 to generate a new clothing design.

[0151] Input: preliminary design rendering file initial_design_file or final_design_file.

[0152] Output: interface interaction.

[0153] Step 6. Edit the modification area

[0154] Designers use the "Free Selection Tool" in the Image Editing Interaction Module (GEI) to draw irregular polygons or rectangles on the image, accurately select the parts that need to be modified, and support the drawing of single or multiple discontinuous selections for local optimization. During operation, designers can use the mouse or stylus to draw on any part of the image. The selected area can be a pattern, material texture, color block, or visual elements such as cropping details. The system will automatically identify and generate the boundary line of the selection to ensure accuracy, and allow designers to adjust the shape and size of the selection by dragging the edge for precise control. At the same time, it supports undo and redraw functions, which is convenient and quick to correct unsatisfactory selections.

[0155] Input: Preliminary design rendering file initial_design_file.

[0156] Output: interface interaction.

[0157] Step 7. Generate the modified area Mask

[0158] After the designer selects and adjusts the area that needs to be modified, the system will automatically generate a Mask with the selected area in white and the overall image in black with a white background, which is used to accurately mark the modified area and assist in subsequent optimization processing. Mask is a mask layer covering the image to ensure that the modification operation only acts on the selected area without affecting other parts. When the designer completes the selection of the local area, the system will automatically generate a Mask with the same shape as the selection to isolate and mark the area. The generated Mask is separated from the background image, and the designer can operate it separately to ensure that the modification is limited to the selected area. The entire Mask generation process supports high-precision adjustment to ensure the accuracy of complex details (such as irregular edges or gradient transitions).

[0159] The generated Mask image and the original design drawing will be saved together in the continuous design timing library for subsequent multiple continuous and precise design work.

[0160] Input: Preliminary design rendering file initial_design_file.

[0161] Output: mask image file mask_design_file of black background and white area.

[0162] Step 8. Local iterative design

[0163] After the mask image and the original design are saved in the continuous design timing library, the designer will repeat the above design steps 1, 2 and 3. Generate a new design requirement prompt word design_prompt and a global vector design_representation for the selected area of ​​the local optimization of the design.

[0164] Step 9. DiT local image generation

[0165] After the designer's needs are encoded through the Transformer (step 3), the system will decide whether to perform local optimization (step 9) or global design image generation (step 4) based on the needs. If it is a local optimization requirement, the system will enter step 9 and use the DiT (Diffusion Transformer) model to generate an image for the local area of ​​the design and seamlessly merge it with the original image. The specific sub-steps include: adding noise, image merging, local generation condition injection, local generation condition control, local image feature sampling, local image feature recovery and output image.

[0166] (1) Adding noise

[0167] In the process of generating a locally optimized image, the system first adds Gaussian noise to the original image and the local optimized area (the same as the Mask generation process). This step provides an initial state for the subsequent image generator, enabling it to gradually build an image that meets the designer's requirements from the noise. In order to ensure that the generated image retains global information while focusing on local optimization, the system independently adds noise to the original image and the local area covered by the Mask. The intensity of the noise injection is controlled by preset parameters to ensure that the clarity of the unmodified area is not disturbed, while providing sufficient freedom for the generation of local images to adjust and optimize.

[0168] Input: initial design image initial_design_file, Mask image of local optimization area mask_design_file.

[0169] Output: the noisy original image noised_initial_image, the noisy Mask area image noised_mask_image.

[0170] (2) Image merging

[0171] After adding noise to the original image and the local mask area, the system merges their encoding information to generate a complete input image encoding. The merged encoding retains the overall feature information of the global image and also contains the specific features of the local optimized area. By combining the global image encoding with the local mask area encoding, the system creates a comprehensive image representation (merged_image_encoding), which ensures that in the subsequent generation process, the consistency of the overall image can be maintained while the local area can be refined and optimized.

[0172] Input: the noisy original image noised_initial_image, the noisy Mask area image noised_mask_image.

[0173] Output: merged_image_encoding, a comprehensive image representation containing global image information and local optimized area information - a 32-bit floating point matrix.

[0174] (3) Local generation condition injection

[0175] In the process of local image generation, the system injects the designer's specific modification requirements into the local generation conditions based on the design requirement semantics encoded by the Transformer to ensure that the details of the generated image can meet expectations. This process involves fine-grained control of elements such as style, color, material, and cropping in the local area. The system first extracts the designer's local optimization requirements (such as color adjustment, material change, or decorative addition) through the design requirement encoding, and combines these requirements with the local Mask area encoding to generate a conditional local generation vector. This conditional vector will guide the image generator to gradually implement the modifications specified by the designer in the subsequent generation steps to ensure that the final image can reflect the key elements in the requirements and be consistent with the overall design style.

[0176] Input: Local design semantic vector design_representation, which comes from the 512-dimensional feature vector in step 3. It contains the semantic vector space expression of the designer's current local design requirements. It includes the designer's specific modification requirements, such as style, color, material, and cutting.

[0177] Output: Conditional local generation vector (local_condition_vector): A local generation condition vector containing design requirements - a 512-bit vector that combines designer requirements and local area features to guide the local image generation process.

[0178] (4) Local generation condition control

[0179] At each stage of image generation, the system continuously monitors and adjusts the conditions of local generation to ensure that the generation results are consistent with the designer's needs. Through a multi-level condition control mechanism, the system is able to accurately manage local details at each step of the generation process. Specifically, the system continuously re-injects and adjusts the design conditions at each step of the diffusion generation (the reverse diffusion process from initial noise to gradually generate a clear image) to ensure that the generation of local areas is consistent with expectations.

[0180] For example, if the designer requests to add geometric patterns or change material textures in a local area, the system will gradually introduce these features through various generation stages to ensure that they are accurately reflected in the final image. This step-by-step control mechanism not only ensures the accuracy of local optimization, but also maintains the consistency of the overall design, allowing local modifications to blend naturally with unmodified parts.

[0181] Input: Condition vector local_condition_vector (512-dimensional feature vector).

[0182] Output: Diffusion step diffusion_t (integer), progress information of each step in the diffusion process (from T to 0).

[0183] (5) Local image feature sampling

[0184] During the local image generation process, the system will perform feature sampling on the feature coding of the original image and the feature coding of the local Mask area to ensure that the generated local image is consistent with the overall style and forms a natural transition with the unmodified area. Through sampling technology, the system extracts the color, texture, light and shadow and other detailed features of the local area, and compares them with the global information of the original image to ensure that the local modification not only meets the designer's needs, but also maintains a consistent visual style with the unmodified part.

[0185] This sampling process is crucial, as it ensures that the local area retains the designer's modification intention during the generation process and blends with the overall style of the entire image, without the local area being separated from the global image. The local feature sampling vector finally generated will be used to guide the natural transition and consistency of the local image generation stage.

[0186] Input: Original image feature encoding (initial_design_features): global feature information extracted from the original design image, including visual elements such as color, texture, light and shadow; local mask area feature encoding (mask_design_features): feature encoding of the local optimized area, including the color, material, light and shadow information of the area.

[0187] Output: local feature sampling vector local_feature_vector: The sampling result combining local area features and global image style is used to guide style consistency and natural transition during local generation.

[0188] (6) Local image feature recovery

[0189] In the process of restoring local image features, the system gradually removes noise to restore the local area to a high-quality image that meets the designer's requirements. This process uses the reverse step of the diffusion model to gradually restore clear image details from the initial noise. In this process, the system not only restores the color, texture, light and shadow characteristics of the local image, but also ensures that the local area is seamlessly integrated with the boundary of the original image. Through boundary smoothing transition processing, the system avoids visual discontinuity or conflict between local modifications and surrounding areas. This process still requires the training data support of the design image shared vector space design_global_vector to ensure the authenticity of image feature restoration.

[0190] Ultimately, the generated denoised local image is naturally connected to the global image, and the system outputs a fused image to ensure that the locally optimized area is consistent with the overall design style and meets the designer’s expectations.

[0191] Input: Local feature sampling vector local_feature_vector: A feature vector that combines local and global information generated from local image feature sampling.

[0192] Original noise image noised_initial_image, Mask local noise image noised_mask_image.

[0193] Local generation condition vector local_condition_vector: Designer's modification requirements and local generation conditions, used to guide the noise removal process.

[0194] Output: denoised_local_image, a high-quality local image generated after gradual denoising, which meets the design requirements and blends seamlessly with the original image. Blended image final_blended_image, the final output complete image, with a smooth transition between the local modified area and the global image, achieving visual harmony and consistency.

[0195] (7) Output image

[0196] When the local optimized image is generated, the system will output the final design fusion image final_blended_image, which includes the designer's modifications to the local area. The system will combine the generated local optimized image with the original image and generate the final image file. The output image will meet the designer's local optimization needs, with high resolution and complete details, suitable for subsequent display, feedback or further modification.

[0197] The final clothing design will be output to the GEI Image Viewer (GV) through the API for image presentation. Designers can repeat step 5 to continuously and accurately design the design effect until they are satisfied with it.

[0198] Input: fused image final_blended_image.

[0199] Output: Final design rendering file final_design_file. The format can be PNG, JPEG and other standard image formats, with a resolution of 300DPI or higher.

[0200] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0201] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A clothing design incremental generation method based on DiT architecture, characterized in that: include: Selecting target elements in a database to determine initial design requirements; the database includes a design style model library, a user preference library, and a material library; Based on the initial design requirements, the AI ​​prompt words of the design drawing are automatically generated using the dialog box module of the design prompt word builder; the AI ​​prompt words reflect the original intention, background, design style, user preference and clothing material of the design; Based on the AI ​​prompt word, a global image generator is used to perform text word meaning analysis and encoding, and output semantic encoding information with a global context; the global image generator has a built-in Transformer model; the Transformer model processes the input text through word segmentation, word embedding, and encoding based on word meaning context; According to the designer's needs, it is determined whether to generate a global design image or perform local optimization to determine the final output design image; when it is decided to generate a global design image, the first processing flow is executed and the diffusion model is used for optimization and adjustment; when it is decided to perform local optimization, the second processing flow is executed and the DiT model is used for optimization and adjustment.

2. The incremental generation method of clothing design based on DiT architecture according to claim 1 is characterized in that: The style model library stores classifications, characteristics, examples and related attribute information of different styles; The user preference library stores historical design data and preference information corresponding to user types, and supports personalized recommendations and dynamic updates; The material library stores the characteristics, applicable scenarios and supplier information of various materials involved in clothing design.

3. The incremental generation method of clothing design based on DiT architecture according to claim 1 is characterized in that: The step of using the diffusion model to generate a corresponding clothing design drawing according to the semantic coding information specifically includes: The semantic encoding information is used as the conditional input of the diffusion model. Then, the model uses the initial Gaussian noise image as the starting point, and combines the vector of the design requirement with the initial noise through conditional mapping, so that the potential features in the noise are consistent with the design requirement, thereby guiding the image generation to develop in a specific direction and completing the conditional injection; After the injection conditions, conditional control and image constraints are performed on each stage of image generation, and denoising is performed in each diffusion step to obtain the corresponding clothing design image.

4. The incremental generation method of clothing design based on DiT architecture according to claim 1 is characterized in that: The first processing flow specifically includes: Generate a corresponding clothing design drawing according to the semantic coding information using a diffusion model; The clothing design drawing is provided to the designer in high definition through the image display module of GEI, and a feedback interface is provided. When the designer is not satisfied with the feedback, the clothing design drawing is adjusted and optimized to determine the final output design image; the adjustment and optimization process includes editing the modification area, generating the modification area Mask and local iterative design.

5. The incremental generation method of clothing design based on DiT architecture according to claim 4 is characterized in that: The process of editing and modifying a region includes: Designers use the "Free Selection Tool" in the image editing interactive module to draw irregular polygons or rectangles on the image to select the parts that need to be modified. It supports drawing single or multiple discontinuous selections to facilitate local optimization. During operation, designers use the mouse or stylus to draw on any part of the image. The selected area can be a pattern, material texture, color block or cropping detail. The system will automatically identify and generate the boundary line of the selection to ensure accuracy, and allow designers to adjust the shape and size of the selection by dragging the edge to achieve precise control. At the same time, it supports undo and redraw functions to facilitate and quickly correct unsatisfactory selections. The process of generating the modified area Mask includes: After the designer selects and adjusts the area to be modified, the system will automatically generate a mask with the selected area in white and the overall image in black with a white background, which is used to accurately mark the modified area and assist in subsequent optimization processing; Mask is a mask layer covering the image, ensuring that the modification operation only acts on the selected area without affecting other parts; when the designer completes the selection of the local area, the system will automatically generate a mask with the same shape according to the selection, which is used to isolate and mark the area; the generated mask is separated from the background image, and the designer operates it separately to ensure that the modification is limited to the selected area; The generated Mask image and the original design drawing will be saved together in the continuous design timing library for subsequent multiple continuous and precise design work; The local iterative design process includes: After the mask image and the original design drawing are saved in the continuous design timing library, the designer will repeat the steps before "determine the global design image generation or local optimization according to the designer's needs to determine the final output design image".

6. The incremental generation method of clothing design based on DiT architecture according to claim 1 is characterized in that: The second processing flow specifically includes: Based on the semantic coding information, the DiT model is used to generate an image for the designed local area and seamlessly merge it with the original image; the processing process of the DiT model includes: adding noise, image merging, local generation condition injection, local generation condition control, local image feature sampling, local image feature recovery and output image.

7. The incremental generation method of clothing design based on DiT architecture according to claim 6 is characterized in that: The process of adding noise includes: In the process of generating a local optimized image, the system first adds Gaussian noise to the original image and the local optimized area Mask respectively. This step provides the initial state for the subsequent image generator, and gradually builds an image that meets the designer's requirements from the noise. In order to ensure that the generated image retains global information while focusing on local optimization, the system independently adds noise to the original image and the local area covered by the Mask. The intensity of the noise injection is controlled by preset parameters to ensure that the clarity of the unmodified area is not disturbed, while providing sufficient freedom for the generation of local images to adjust and optimize. The image merging process includes: After adding noise to the original image and the local mask area, the system merges the encoding information of the two to generate a complete input image encoding; The process of local generation condition injection includes: Based on the design requirement semantics encoded by Transformer, the system first extracts the designer's local optimization requirements through the design requirement encoding, and combines the requirements with the local Mask region encoding to generate a conditional local generation vector; The process of local generation condition control includes: Through a multi-level condition control mechanism, the system continuously re-injects and adjusts the design conditions at each step of diffusion generation to ensure that the generation of the local area is consistent with expectations; The process of local image feature sampling includes: Through sampling technology, the system extracts the color, texture, light and shadow and other detailed features of the local area, and compares them with the global information of the original image to ensure that the local modification not only meets the designer's needs, but also maintains the same visual style as the unmodified part; The process of restoring local image features includes: By adopting the inverse steps of the diffusion model and performing boundary smoothing transition processing, the system avoids visual discontinuity or conflict between local modifications and surrounding areas, and gradually restores clear image details from the initial noise.

8. A clothing design incremental generation system based on DiT architecture, applied to the clothing design incremental generation method according to any one of claims 1 to 7, characterized in that: include: Prompt word constructor, GEI image displayer, GEI image editor, global image generator and DiT local image generator; Among them, the prompt word constructor is respectively connected to the design style model library, the user preference library, the material library and the global image generator; the global image generator is also respectively connected to the GEI image displayer and the DiT local image generator; the DiT local image generator is also respectively connected to the GEI image displayer and the GEI image editor.

Images