Building effect picture generation method based on multi-condition weighted fusion and block control

Abstract

The application discloses a kind of based on multi-condition weighted fusion and block control's building rendering generation method, 1, customized model training;2, preliminary generation rough rendering A;3, global prompt extraction and calibration, obtain global prompt P1;4, style condition modeling: using the Redux model of Flux obtains style condition vector C3;5, image enlargement obtains high-resolution image B;6, image block and block prompt acquisition, obtain block prompt set P2;7, condition fusion;8, geometric control preprocessing, for constraint geometry in the process of block generation;9, block refinement generation, realize the high-resolution block diagram of detail rich, geometric accuracy;10, block merging and consistency repair, obtain final building rendering.The present application solves the problems of existing generation resolution limitation, building geometric structure distortion, global and local prompt inconsistency, style drift, obvious block splicing trace and lack of engineering parameter interval.

Description

Technical Field

[0001] This invention relates to computer vision and image generation technology, specifically to a method for generating architectural renderings based on multi-condition weighted fusion and block control, applicable to the generation and optimization of architectural bird's-eye views, planning and design renderings, and other large-size architectural renderings. Background Technology

[0002] In recent years, text-to-image and image-to-image generation based on diffusion models has been widely used in the field of design visualization. Latent space diffusion models, represented by Stable Diffusion XL (SDXL), achieve better generation quality and controllability for high-resolution images (native 1024×1024) through a larger U-Net backbone, dual text encoders, and various conditional mechanisms, outperforming earlier versions in complex scene synthesis. Building on this, the industry commonly uses lightweight fine-tuning techniques such as LoRA (Low-Rank Adaptation) to perform domain transfer on general diffusion models, learning specific styles or categories (such as building facades, bird's-eye views, etc.) with lower computational power increments. The core idea is to insert a low-rank trainable matrix while freezing the underlying model, significantly reducing the number of trainable parameters and memory usage.

[0003] To improve structural controllability, methods like ControlNet introduce spatial priors such as edge, depth, and semantic segmentation as additional conditions without compromising the underlying model's capabilities. This constrains geometric contours, hierarchical relationships, and layout stability during the generation process. These methods, by coupling conditional branches with the backbone network through structures like "zero convolution," have become mainstream control techniques in engineering practice. On the other hand, regarding the controllability of prompt words, research and practice have also explored the approach of "inferring words from the graph" (such as inversion / backward inference paradigms like Textual Inversion) to extract reusable semantic / style descriptions from existing images for subsequent regeneration and editing.

[0004] In addition to the general generation and control techniques mentioned above, image variants and re-stylization tools, such as the FLUX series (e.g., FLUX.1 Redux), have emerged in recent years to address "style-consistent re-creation / refinement." These tools support style-preserving redrawing and fine-tuning based on a given image, providing an engineering means to maintain style continuity across stages.

[0005] Currently, existing methods for generating architectural renderings can be summarized as follows:

[0006] A general diffusion model (such as SDXL) is used in conjunction with LoRA fine-tuning to obtain the ability to generate architectural scenes; edge / depth conditions of ControlNet are used to improve the controllability of structure and hierarchy; in large-size output scenes with limited video memory, a "block generation / enlargement + block" strategy is used to generate locally first and then stitch together; consistency and detail quality are improved through cue word engineering and a small number of inversion techniques; when necessary, image variants / re-stylization tools are introduced to perform style correction on the stage products.

[0007] The aforementioned existing technologies still have the following common problems and shortcomings when it comes to the engineering output of "architectural bird's-eye views / large-size architectural renderings":

[0008] (a) The disconnect between global and local prompts leads to insufficient consistency:

[0009] Existing workflows are typically driven by a single global cue word or use cue words automatically derived from chunks. When using a single global cue word, a slightly higher noise reduction intensity (>0.5) can cause unwanted objects to appear within a chunk; a lower intensity fails to correct erroneous images. When each chunk uses its own cue word, the lack of a systematic connection and constraint between global and chunk-level cue words can easily lead to inconsistencies in style, material, or functional elements across chunks (such as disordered changes in road continuity, vegetation density, and roof material).

[0010] (ii) Style drifting between multiple stages:

[0011] The typical process is "coarse generation → enlargement → refinement / repair". Each stage may be executed by different models or different parameters. If the style of the early stage is not explicitly extracted and carried through to the subsequent stages, the common phenomenon is the gradual drift of tone, contrast and material details, making it difficult to maintain a unified style tone in the final image.

[0012] (iii) Segmented splicing easily produces seam marks and discontinuous edges:

[0013] The conventional approach is to increase overlapping areas and feathering, but when each block is affected by different prompts or different randomness, the edge direction and texture cycle are difficult to continue seamlessly, resulting in visible seams, repeated textures or local stretching; there is a lack of a final consistency repair step at the "full map level" (such as low-intensity global map generation (img2img)) to unify the style and texture phase.

[0014] (iv) The parameter range lacks engineering calibration and self-consistency:

[0015] For example, the denoising intensity (0–1) of image-to-image (img2img) often depends on experience in how to determine its value at each stage; if the denoising intensity is too strong (close to redrawing) in the block segmentation stage, it will destroy the global layout, while if it is too weak, the details will be insufficient and the distorted lines cannot be corrected; existing public solutions rarely provide quantitative calibration and reproducible practice guidelines for these intervals (papers such as SDXL / ControlNet focus on the method ontology and only give principle descriptions of engineering parameters).

[0016] (v) Lack of "unified expression and weighted fusion of conditions":

[0017] In current practice, global hints, block hints, and style constraints are often used in a scattered manner, lacking an explicit mechanism of "condition vectorization → weight setting → joint guidance", which leads to non-repeatability and difficulty in batching and automation.

[0018] (vi) The professional expressions in architectural scenes are difficult to fully capture using general prompts:

[0019] Relying solely on automatic reverse engineering (image captions / inversion) can easily lead to missing professional terms or semantic ambiguity (such as "terraced roof", "elevated floor", "atrium lighting"). It is necessary to introduce human-computer collaboration of "automatic reverse engineering + manual calibration" to ensure the industry correctness and executability of the prompts.

[0020] (vii) Limited generation resolution results in severe line distortion:

[0021] Because the base model of a diffusion model is typically limited to a specific resolution during the training phase (such as the native 1024×1024 pixels of SDXL), coupled with the physical limitations of video memory capacity, existing technologies struggle to directly generate ultra-large pixel images (greater than 1024×1024 pixels) that meet the needs of architectural representation. As a result, the pixel density of the initially generated image is insufficient, and the lines, edges, and details of the building often appear distorted, stretched, or jagged. Once such defects are formed, even subsequent compensation through super-resolution or model enlargement often only enhances pixel clarity, failing to truly restore the original geometric regularity and structural logic. This leads to an overall distorted appearance in the architectural renderings, making it difficult to meet the requirements of design presentation and review. Summary of the Invention

[0022] In view of this, the present invention provides a method for generating architectural renderings based on multi-condition weighted fusion and block control, which aims to solve the problems of limited generation resolution, distortion of architectural geometry, inconsistency between global and local prompts, style drift, obvious block splicing marks, and lack of engineering parameter range in existing methods.

[0023] To achieve the above objectives, the present invention adopts the following technical solution:

[0024] The architectural rendering generation method based on multi-condition weighted fusion and block control described in this invention includes the following steps:

[0025] Step 1, Customized Model Training: Using SDXL and Flux diffusion models as the basic framework, and training a special model for architectural renderings through LoRA technology to enhance the ability to express architectural scene features (such as building facades, building volumes, street layouts, and road textures).

[0026] Step 2, Initial Generation: Based on the input architectural line drawing, a rough rendering A is generated by combining the SDXL model; this stage only provides a rough spatial layout and building volume distribution.

[0027] Step 3, Global Prompt Extraction and Calibration: Prompts are extracted from the rough effect diagram A and manually calibrated to obtain the global prompt P1, which is used for subsequent global consistency guidance.

[0028] Step 4, Style Condition Modeling: Use Flux's Redux model to perform style analysis on the rough rendering A to obtain the style condition vector C3, which is used to ensure the stability and consistency of color, material and lighting in the generated results at different stages.

[0029] Step 5, Image Enlargement: The coarse image A is processed using a super-resolution enlargement model to obtain a high-resolution image B, laying the foundation for refined segmentation.

[0030] Step 6, Image segmentation and segmentation prompt word acquisition: The image B is segmented, and prompt words are derived for each segment and manually calibrated to obtain a segmentation prompt word set P2, ensuring the accuracy of local detail description.

[0031] Step 7, Conditional Fusion: Generate a global conditional vector C1 based on the global prompt word P1, generate a block conditional vector set C2 based on the block prompt word set P2, and combine it with the style conditional vector C3 to establish a conditional fusion model; obtain the fusion conditional vector C0 for each block through weighted averaging, and realize the unified expression of global and local semantics.

[0032] Step 8, Geometric Control Preprocessing: Image B is preprocessed to extract the line drawing and depth map, which are used as ControlNet control signals to constrain the geometric structure during the block generation process.

[0033] Step 9, Block Refinement Generation: Guided by the Flux model, each block of image B is subjected to image-to-image denoising (denoising coefficient 0.5–0.8), and the fusion condition vector C0, line drawing control and depth control are introduced at the same time to achieve a high-resolution block image with rich details and geometric accuracy.

[0034] Step 10, Block Merging and Consistency Repair: Merge the block results into a complete image C, and use global graph denoising (denoising coefficient ≤ 0.3) to repair minor stitching marks and inconsistencies at the splicing edges, guided by the global prompt word P1, to obtain the final architectural rendering.

[0035] Furthermore, step 2 also includes the following steps:

[0036] Step 2.1, Obtain the architectural line drawing:

[0037] When exporting line drawings from 3D design software (such as SketchUp, Revit, etc.), it is preferable to output high-precision rendering results with a resolution of ≥4K to ensure clear building geometric boundaries.

[0038] Step 2.2, High-resolution line art preprocessing:

[0039] This invention addresses the specific requirements of line drawing input in the generation of architectural renderings by proposing a two-stage line drawing preprocessing method.

[0040] The first stage involves high-resolution line art extraction, which specifically includes:

[0041] Line detection and enhancement: Extract the main outline, dividing lines and key component boundaries of buildings through gradient-based edge operators or deep learning line detection networks;

[0042] Noise and pseudo-line suppression: For interfering lines that may be generated by texture materials (such as roof textures, curtain wall reflections, etc.) in the input, texture filtering and region consistency judgment are performed.

[0043] Binarization and standardization: The extracted effective lines are converted into black and white binary images, and the lines are thinned to ensure uniform line width, which meets the requirements of ControlNet as a geometric control input.

[0044] The second phase involves high-quality scaling and resolution adaptation.

[0045] Scaling mode selection: Prioritize the proportional scaling strategy to ensure that the aspect ratio of the image remains unchanged and avoid incorrect geometric deformation of the building structure due to stretching or compression.

[0046] Boundary preservation mechanism: During scaling, the system automatically identifies whether the line drawing is a binary boundary image. If it is a one-pixel wide line, the system uses non-maximum suppression and thinning algorithms to prevent the lines from disappearing or breaking.

[0047] Noise reduction and background processing: Mask synchronization adjustment is performed on the scaled image, and a color filling strategy based on edge pixel statistics is adopted to avoid background artifacts or cropping marks.

[0048] Step 2.3, Write the prompt words:

[0049] Based on the architectural image content and design requirements, write a prompt description that includes the following elements:

[0050] Building types (residential, office buildings, commercial complexes, etc.);

[0051] Scene elements (roads, greenery, water bodies, etc.);

[0052] Style preferences (modern, European, Chinese, postmodern, etc.).

[0053] Step 2.4, generating structural constraints based on ControlNet:

[0054] The preprocessed line drawing is used as a structural constraint input to the ControlNet module. Combined with the prompts mentioned in step 2.3 and the LoRA model fine-tuning, the diffusion model is used to initially generate a rough effect diagram.

[0055] Step 2.5, Manual screening and repair:

[0056] Through the above steps, combined with manual screening and post-processing (partial redrawing) to fix minor issues, a rough rendering A that conforms to the architectural semantics is obtained, improving the iterative efficiency of architectural design.

[0057] Furthermore, in step 3, the present invention first uses the prompt word back-inference module to perform semantic recognition on the overall content of the rough rendering A to obtain an initial semantic description; then, a Large Language Model (LLM) is introduced, and based on a predefined prompt word template (including dimensions such as camera perspective, building category, scene elements, style features and environmental conditions), the semantic description is structurally reorganized to generate a global prompt word P1; finally, the generated result is reviewed and corrected through manual calibration to ensure that the prompt word expression is consistent with the professional terminology and design intent in the field of architecture.

[0058] Furthermore, step 4, which involves obtaining the style condition vector C3, includes:

[0059] Input phase:

[0060] The rough rendering A is input into the style analysis module, which can simultaneously receive:

[0061] Image input: serving as the primary source of style information;

[0062] Optional text input: Include only words related to the output quality (such as "HD", "rich in detail", "best quality") to enhance the clarity and detail of the output.

[0063] Step 4.2, Feature Encoding Stage:

[0064] The style analysis module uses a visual semantic encoder to process the input image and extract the following style information:

[0065] Color scheme (such as warm tones, cool tones, contrast characteristics);

[0066] Material characteristics (such as glass, metal, concrete, etc.);

[0067] Light and shadow characteristics (such as soft light, backlight, and night scene lighting effects).

[0068] Step 4.3, Style Condition Generation Stage:

[0069] Combining the image encoding results with optional text conditions, the style analysis module outputs a style condition vector C3, which serves as the generation condition input for the diffusion model.

[0070] Furthermore, in step 5, after the rough effect diagram A is generated, the resolution is first improved by using a super-resolution model (such as ESRGAN or Latent SR), and then gradually refined by combining block generation, thereby reducing memory overhead and increasing the overall resolution limit.

[0071] Furthermore, in step 8, the extracted line art is preprocessed, including:

[0072] Phase 1: High-resolution line art extraction:

[0073] The original architectural line drawing is converted into a clearly defined black and white line drawing. Specifically:

[0074] Line detection and enhancement: Extract the main outline, dividing lines and key component boundaries of buildings through gradient-based edge operators or deep learning line detection networks;

[0075] Noise and pseudo-line suppression: To address interfering lines caused by texture materials (such as roof textures, curtain wall reflections, etc.) that may exist in the input, this invention introduces texture filtering and region consistency judgment to avoid the extraction of unstructured lines;

[0076] Binarization and standardization: The extracted effective lines are converted into black and white binary images, and the lines are thinned to ensure uniform line width, which meets the requirements of ControlNet as a geometric control input.

[0077] The second phase involves high-quality scaling and resolution adaptation.

[0078] Scaling mode selection: Prioritize the proportional scaling strategy to ensure that the aspect ratio of the image remains unchanged and avoid incorrect geometric deformation of the building structure due to stretching or compression.

[0079] Boundary preservation mechanism: During scaling, the system automatically identifies whether the line drawing is a binary boundary image. If it is a pixel-wide line, non-maximum suppression and thinning algorithms are used to prevent the lines from disappearing or breaking.

[0080] Noise reduction and background processing: Mask synchronization adjustment is performed on the scaled image, and a color filling strategy based on edge pixel statistics is adopted to avoid background artifacts or cropping marks.

[0081] Furthermore, step 9 also includes the following steps:

[0082] Step 9.1, Input and Initialization:

[0083] The image blocks obtained by the image segmentation module are input into the thinning generation module; each block contains its corresponding cropping region, mask information and local feature description.

[0084] Step 9.2, Conditional Loading:

[0085] During the generation process, the fusion condition vector C0 is used as the main guiding signal, while geometric control signals (line drawing and depth map) are injected simultaneously.

[0086] Step 9.3, noise reduction of the segmented image:

[0087] Step 9.3.1: Perform image-to-image denoising based on a diffusion model on each image block;

[0088] Step 9.3.2: Encode the original block image into the latent space;

[0089] Step 9.3.3: Inject moderate noise into the latent space and perform iterative sampling with a high noise reduction intensity (0.4–0.8);

[0090] Step 9.3.4: During the sampling process, a higher noise reduction intensity is used, and the fusion condition vector C0 and geometric control signal are combined for guidance, thereby increasing local details and material texture representation while maintaining geometric accuracy.

[0091] Repeat steps 9.3.1-9.3.4 until all blocks are generated.

[0092] Furthermore, step 10 also includes the following steps:

[0093] Step 10.1.1: Within the overlapping region, extract the generation results Ri, Rj of adjacent blocks;

[0094] Step 10.1.2: Construct a weight mask 𝑀, whose value range is [0,1]; its weight is larger at the center of the block and gradually decreases to 0 towards the boundary; the decay function can be implemented by linear function, cosine function or Gaussian function, etc., to ensure that the transition between blocks is smooth and natural;

[0095] Step 10.1.3: Perform a pixel-level weighted average on the overlapping regions:

[0096]

[0097] Where Ri and Rj represent the image results from two adjacent blocks, (x,y) represents the pixel coordinates in the overlapping area, R(x,y) is the final fusion result, and 𝑀 is the weight mask function.

[0098] Through the pixel-level fusion processing described above, obvious boundary lines, misalignments, and style abrupt changes generated during the block stitching process can be effectively eliminated, so that the final generated image maintains a high degree of continuity in geometric structure and detail representation.

[0099] Step 10.2, Global Consistency Repair:

[0100] Despite employing segmented refinement and overlapping region fusion techniques, the final merged high-resolution image may still exhibit issues such as local color differences and subtle stylistic variations. Therefore, this invention further proposes a global consistency restoration method:

[0101] Step 10.2.1: Divide the merged high-resolution image into multiple small blocks and perform latent space coding using a variational autoencoder (VAE);

[0102] Step 10.2.2: In the latent space, the global cue word P1 is used as the main semantic guide, supplemented by the geometric control signals (line drawing and depth map) provided by ControlNet.

[0103] Step 10.2.3: Perform image restoration sampling in the diffusion model with a low noise reduction intensity (noise figure < 0.3) to obtain the architectural rendering.

[0104] The inventiveness of this invention is reflected in the following aspects:

[0105] 1. Improved generation resolution and geometric accuracy: Breaking through the limitations of training resolution and video memory of the base model, and relying solely on the existing mode of single small-size output, this paper proposes a multi-stage process of "initial generation → enlargement → block refinement" to generate architectural renderings with ultra-large pixels (far exceeding 1024×1024 pixels), while ensuring the straightness of architectural lines and the geometric regularity of the structure. This avoids the serious distortion and difficult-to-repair problems caused by low resolution and significantly improves the straightness and geometric regularity of architectural lines.

[0106] 2. Ensuring Global and Local Consistency: Addressing the lack of systematic correlation between segmented prompts and global prompts in existing technologies, this invention employs a two-tiered mechanism of "global prompt back-calculation and manual calibration + segmented prompt extraction," along with a unified conditional fusion method. This ensures that the generated local details maintain consistency with the global semantics, guaranteeing the continuity of elements such as roads, vegetation, and rooftops in overall style and spatial logic, effectively guaranteeing consistency in both overall layout and local details of the image.

[0107] 3. Solve the problem of style drift across stages: Introduce the extraction and reuse of style conditions in different stages such as coarse generation, magnification and refinement, and guide the unified process through weighted fusion to ensure the stability and consistency of the image in terms of tone, material and lighting style, thereby avoiding the style inconsistency caused by multi-stage generation in existing methods.

[0108] 4. Enhanced geometric control and hierarchical stability: A dual control strategy that simultaneously incorporates line art and depth information is proposed, using ControlNet for coupling constraints. This allows for a balance between edge sharpness and spatial hierarchy during the block refinement stage, making the generated results more compliant with architectural standards in terms of outline, spatial hierarchy, and structural logic. This solves the problem that existing single control modes cannot achieve both.

[0109] 5. Reduce stitching artifacts: A pixel-level fusion algorithm for overlapping areas was designed and applied, which is fast and effective. Then, a low-intensity global image generator (noise reduction coefficient ≤ 0.3) is used to restore the consistency of the merged images, achieving seamless boundary transitions and material continuity. This overcomes the seams, deviations and color differences that traditional overlapping / feathering methods still cannot completely eliminate, resulting in a coherent final image.

[0110] 6. Provide engineering calibration of parameter ranges: Clearly define the noise reduction coefficient range (e.g., 0.5–0.8 and ≤0.3) in the three stages of coarse generation, block refinement and final repair, to provide designers with reproducible and quantitative engineering operation standards, reduce the randomness of generation results relying on experience debugging, and improve productivity.

[0111] 7. Establish a unified expression and weighted fusion mechanism for conditions: By modeling global prompt word conditions, block prompt word conditions and style conditions separately, and performing weighted averaging in the condition space, a unified guiding signal is formed, which solves the problems of existing conditions being scattered, difficult to batch and automate, and facilitates large-scale application.

[0112] 8. Improve the professionalism and accuracy of architectural semantic expression: By combining prompt word back-reasoning and manual calibration mechanisms, the generation process can correctly cover architecturally unique professional elements (such as terraces, elevated floors, atriums, etc.), solving the problem of insufficient semantic expression of general models in architectural context, thereby improving the usability and industry recognition of architectural representation.

[0113] Through the above improvements, this invention achieves several advantages in terms of production efficiency: it can generate higher resolution and more consistent architectural renderings under the same computing power conditions, reducing the time and cost of manual post-processing; in terms of cost control, it utilizes lightweight LoRA and a phased strategy to avoid directly training ultra-large models or using ultra-high-end graphics card clusters, thereby reducing the investment in computing resources; and in terms of performance quality, it significantly improves the geometric accuracy, style consistency, and seamless splicing of architectural renderings, making the generated results closer to the needs of professional design and presentation. Attached Figure Description

[0114] Figure 1 This is an overall flowchart of the method of the present invention.

[0115] Figure 2 This is a flowchart of the customized model training process described in this invention.

[0116] Figure 3 This is a flowchart illustrating the initial generation of a rough effect diagram as described in this invention.

[0117] Figure 4 This is a flowchart of the global prompt word reverse deduction and manual calibration described in this invention.

[0118] Figure 5 This is a flowchart of the conditional fusion described in this invention.

[0119] Figure 6 This is a schematic diagram of the rough effect diagram A described in this invention.

[0120] Figure 7 This is a schematic diagram of the high-resolution image B after being divided into blocks according to the present invention. Detailed Implementation

[0121] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. These embodiments are implemented based on the technical solution of the present invention, and detailed implementation methods and specific operation processes are given. However, the scope of protection of the present invention is not limited to the following embodiments.

[0122] like Figure 1 As shown, the architectural rendering generation method based on multi-condition weighted fusion and block control of the present invention includes the following steps:

[0123] Step 1, customized model training, such as Figure 2 As shown:

[0124] The purpose of LoRA technology in training architectural rendering models is to enable diffusion models to better adapt to the rendering representation of specific types of buildings (such as residential communities and industrial parks). Specifically:

[0125] Step 1.1, Training Data Collection:

[0126] Collect a large amount of architectural rendering data from architectural design institutes, public image libraries, or existing project cases; to ensure the homogeneity of the training data, the following requirements must be met:

[0127] Image resolution ≥1K, content is clear;

[0128] The building types should remain consistent, for example, all residential or all industrial parks;

[0129] The styles should be similar to avoid mixing a large number of different styles, which would make the model difficult to converge.

[0130] The view types should be consistent, for example, all should be bird's-eye views, to avoid mixing in elevation views, section views, and other perspectives.

[0131] Data size ≥ 200 images, ideally 500 or more.

[0132] Step 1.2, Image preprocessing:

[0133] The collected images undergo preprocessing, which mainly includes:

[0134] Cropping images with white borders or frames; removing watermarks, logos, text, or noise to reduce interference with the model;

[0135] High-definition processing is performed using super-resolution algorithms (such as Real-ESRGAN);

[0136] Colors are normalized to ensure consistent style in training sessions;

[0137] After automated image processing, combined with manual correction, images with poor quality or resolution below 1K are removed.

[0138] Step 1.3, Label the image:

[0139] The training images are analyzed using a large visual model to automatically generate corresponding text descriptions (prompts). The generated prompts are then manually selected and corrected to ensure consistency and accuracy with the image semantics. For some scenes with many elements and complex structures (such as architectural bird's-eye views), a hierarchical description strategy can be further adopted to extract and label the global and local features of the image, and then fuse them to enhance the model's ability to learn and express architectural semantics.

[0140] Tag files should be saved as txt files, with the naming rule being: original image name.txt, that is, keep the txt tag file name the same as the image name; for example, if a certain rendering image is named "Industry(182).png", then the tag file should be named "Industry(182).txt".

[0141] Step 1.4 Data Augmentation:

[0142] Data augmentation is performed on the original images, including using super-resolution algorithms to enlarge them to generate multi-scale versions (e.g., ×1.5, ×2, ×3) to form multi-resolution training samples;

[0143] It should be noted that the magnification should preferably not exceed 4x to avoid introducing false textures; the maximum size should not exceed 4K, otherwise it may lead to training difficulties.

[0144] When saving an enlarged image, it is recommended to name it as: original image name_resolution.file extension; the corresponding image tag should also be copied and renamed as: original image name_resolution.txt; keep the image and tag file names consistent, only the file types (file extensions) are different.

[0145] Step 1.5, Image cropping:

[0146] The enhanced image is cropped so that its width and height are both integer multiples of 64; generally speaking, the traditional center-cropping algorithm can ensure that the main body of the building is not truncated due to cropping.

[0147] Step 1.6, LoRA model training:

[0148] On the base diffusion model (such as SDXL, Flux), the low-rank adaptation (LoRA) method is used for training; this method trains only a small number of low-rank weight matrices while freezing the parameters of the base model, thereby:

[0149] Significantly reduces training computation costs and GPU memory usage;

[0150] To ensure convergence stability with a limited number of samples;

[0151] Avoid excessive forgetting of the capabilities of the basic model.

[0152] During training, different LoRA rank and alpha values ​​can be set according to the experiment to balance learning ability and generalization ability.

[0153] It is recommended to save the training state at each step during training so that training can continue from a certain state in conjunction with the resume parameter; during training, the model is automatically saved once every 1 to 2 epochs.

[0154] Step 1.7, Result Evaluation and Selection:

[0155] After training, multiple selected LoRA models will be tested with fixed prompts and seeds, and the results will be used to test text-to-image representation.

[0156] The CLIP Score evaluation method can be used to assess the consistency between the generated LoRA results and the semantics of the prompt words; the higher the score, the better.

[0157] The generated renderings are manually scored by architectural experts, who mainly examine indicators such as architectural composition, line integrity, and stylistic consistency. The LoRA model with the best overall performance is ultimately selected as the customized architectural rendering generation model.

[0158] Step 2: Generate a preliminary rough rendering, such as... Figure 3 As shown:

[0159] This step will quickly generate a rough rendering that meets the spatial layout and style requirements based on the input architectural line drawing, providing a base map for subsequent precise rendering; specifically:

[0160] Step 2.1, Obtain the architectural line drawing:

[0161] When exporting line drawings from 3D design software (such as SketchUp, Revit, etc.), it is preferable to output high-precision rendering results with a resolution of ≥4K to ensure clear building geometric boundaries.

[0162] To avoid interference from texture noise, complex textures should be avoided as much as possible; however, textures with regular geometric patterns that help the model understand (such as the grid texture of roof solar panels) can be retained.

[0163] In terms of perspective selection, priority should be given to viewing angles that are approximately orthogonal to the main facade of the building to reduce the risk of distortion caused by perspective distortion.

[0164] Step 2.2, High-resolution line art preprocessing:

[0165] This invention addresses the specific requirements of line drawing input in architectural rendering generation by proposing a two-stage line drawing preprocessing method. Its purpose is to generate standardized black and white line drawings that can be used by diffusion models and ControlNet while ensuring the accuracy of architectural geometric boundaries, and to adapt to the model input resolution through high-quality scaling.

[0166] Phase 1: High-resolution line art extraction:

[0167] For the input original architectural line drawing (typically at 4K resolution), this invention uses a combination of edge detection and noise suppression to convert it into a clear black-and-white line drawing, specifically including:

[0168] Line detection and enhancement: Extract the main outline, dividing lines and key component boundaries of buildings through gradient-based edge operators or deep learning line detection networks;

[0169] Noise and pseudo-line suppression: To address interfering lines caused by texture materials (such as roof textures, curtain wall reflections, etc.) that may exist in the input, this invention introduces texture filtering and region consistency judgment to avoid the extraction of unstructured lines;

[0170] Binarization and standardization: The extracted effective lines are converted into black and white binary images, and the lines are thinned to ensure uniform line width, which meets the requirements of ControlNet as a geometric control input.

[0171] The second phase involves high-quality scaling and resolution adaptation.

[0172] Since the diffusion model has a fixed requirement for input resolution (usually 1K–2K), high-resolution line art needs to be scaled down to the target range. Ordinary image scaling methods are prone to problems such as blurred lines, distortion, and edge disappearance when scaling down from high resolution. This invention adopts an edge-preserving scaling method to avoid edge disappearance and blurring and distortion, and ensure that the geometric boundaries are not distorted during the scaling down process.

[0173] Scaling mode selection: Prioritize the proportional scaling strategy to ensure that the aspect ratio of the image remains unchanged and avoid incorrect geometric deformation of the building structure due to stretching or compression.

[0174] Boundary preservation mechanism: During scaling, the system automatically identifies whether the line drawing is a binary boundary image. If it is a one-pixel wide line, the system uses non-maximum suppression and thinning algorithms to prevent the lines from disappearing or breaking.

[0175] Noise reduction and background processing: Mask synchronization adjustment is performed on the scaled image, and a color filling strategy based on edge pixel statistics is adopted to avoid background artifacts or cropping marks.

[0176] Through the above two-stage processing, the present invention can convert the original high-resolution line drawing into a standardized black and white line drawing usable by ControlNet, and maintain the clarity of building boundaries and the stability of geometric structure after scaling, providing a reliable structural control input for subsequent diffusion model-based image generation.

[0177] Step 2.3, Write the prompt words:

[0178] Based on the architectural image content and design requirements, write a prompt description that includes the following elements:

[0179] Building types (residential, office buildings, commercial complexes, etc.);

[0180] Scene elements (roads, greenery, water bodies, etc.);

[0181] Style preferences (modern, European, Chinese, postmodern, etc.).

[0182] The prompts can be written manually or automatically generated by the visual model and then manually corrected to ensure semantic consistency with architectural features. These prompts serve as semantic guides for the diffusion model, ensuring that the generated images conform to the design intent.

[0183] Step 2.4, generating structural constraints based on ControlNet:

[0184] The preprocessed line drawing is used as a structural constraint input to the ControlNet module. Combined with the prompts mentioned in step 2.3 and the LoRA model fine-tuning, the diffusion model is used to generate a rough effect diagram.

[0185] The generated results can express the spatial layout, volume distribution and main shape features of the building; scene elements such as roads and greenery can also be reasonably displayed.

[0186] Step 2.5, Manual screening and repair:

[0187] By following the steps above, multiple images can be generated in a short time. Combined with manual selection and post-processing (partial redrawing) to correct minor issues, a rough rendering A that conforms to the architectural meaning can be obtained, such as... Figure 6 As shown, this improves the iteration efficiency of architectural design schemes.

[0188] Step 3, global prompt word reverse analysis and manual calibration, such as Figure 4 As shown:

[0189] This step provides a global prompt generation method that combines image back-inference, prompt word templates with large language model processing, and manual correction. This method is used to further extract and optimize semantic descriptions after the rough image is generated, thereby providing stable semantic conditional input for subsequent image refinement generation.

[0190] Step 3.1, Input a rough rendering:

[0191] The rough result image A obtained in step 2 is input into the prompt word reverse deduction module; this module can perform content parsing on the input image based on a multimodal model (such as the text matching model CLIP, the image caption generation model, etc.).

[0192] Step 3.2, Initial global prompt word generation:

[0193] Using the aforementioned prompt word reverse deduction module, semantic recognition is performed on the overall content of the rough effect diagram A, and corresponding initial global prompt words are automatically generated. What is obtained here is scattered natural language, which is broad in description, lacks hierarchy and structure, and is difficult for humans to understand.

[0194] Step 3.3, prompt word template processing:

[0195] To improve the consistency and completeness of the prompts, this step introduces a prompt template mechanism; specifically, the initial global prompts obtained through reverse engineering are structurally filled according to a predefined semantic template, which includes at least:

[0196] Camera perspective (such as bird's-eye view, one-point perspective, axonometric view, etc.);

[0197] Building types (such as residential buildings, office buildings, industrial plants, etc.);

[0198] Scene elements (such as roads, greenery, water bodies, parking lots, etc.);

[0199] Style characteristics (such as modern, European, minimalist, industrial, etc.);

[0200] Environmental conditions (such as daytime, nighttime, cloudy days, etc.).

[0201] Through this template-based processing, prompts can cover the core dimensions of architectural semantics and be expressed in a structured form.

[0202] Step 3.4, Large Language Model Processing:

[0203] The initial global prompt words and prompt word templates are input into the Large Language Model (LLM), which performs semantic expansion and language optimization to obtain the final prompt words that conform to the generation habits of the diffusion model.

[0204] Step 3.5, Manual Calibration and Supplementation:

[0205] Based on the prompts output by the LLM, designers can manually filter and modify them, including:

[0206] Remove redundant descriptions that are irrelevant to the building objectives;

[0207] Supplement missing but important semantic elements to the design objectives (such as building facade materials and construction, roof structure, etc.).

[0208] Adjust the description style to align with project requirements.

[0209] The manually optimized prompt word becomes the global prompt word P1, which serves as the unified semantic condition input for subsequent generation stages.

[0210] This method introduces back-inference, prompt word templates, and LLM processing, resulting in more comprehensive prompt word coverage and semantic expressions that better conform to the habits of generative models. Manual calibration ensures the matching degree between semantic description and design goals, improves the controllability of generation results, and reduces the deviation caused by a single automated method.

[0211] Step 4, Style Condition Extraction:

[0212] After generating the rough rendering A and calibrating the global cue word P1, this step provides an image-driven style feature extraction method to further extract the style feature information of the architectural image. The core idea is to use the input image and optional quality descriptors as input, and through visual semantic encoding and style modeling, generate a style condition vector that can be used by the diffusion model. The process is as follows:

[0213] Step 4.1, Input Stage:

[0214] Input the rough rendering A into the style analysis module, which can simultaneously receive:

[0215] Image input: serving as the primary source of style information;

[0216] Optional text input: Include only words related to the output quality (such as "HD", "rich in detail", "best quality") to enhance the clarity and detail of the output.

[0217] Step 4.2, Feature Encoding Stage:

[0218] The style analysis module uses a visual semantic encoder to process the input image and extract the following style information:

[0219] Color scheme (such as warm tones, cool tones, contrast characteristics);

[0220] Material characteristics (such as glass, metal, concrete, etc.);

[0221] Light and shadow characteristics (such as soft light, backlight, and night scene lighting effects).

[0222] Step 4.3, Style Condition Generation Stage:

[0223] Combining the image encoding results with optional text conditions, the style analysis module outputs a style condition vector C3, which serves as the generation condition input for the diffusion model.

[0224] The features and effects are as follows:

[0225] (1) Image as prompt:

[0226] Even without inputting detailed prompts, the style analysis module can still generate conditional vectors that closely resemble the original style based on the image itself; the image not only serves as a reference material but can also directly serve as the primary source of prompt information, thereby reducing reliance on manually written prompts.

[0227] (2) Consistency and controllability:

[0228] The style condition vector C3 is combined with semantic conditions such as the global cue word P1 and the block cue word set P2 in subsequent generation, which can ensure the consistency of the results in color, material and lighting, reduce manual intervention and improve the stability and efficiency of generation.

[0229] Step 5, Image zoom in:

[0230] The low-resolution coarse rendering A generated by the overdiffusion model and conditional control network only provides the overall structure and preliminary style features. Its resolution and level of detail are insufficient to support the subsequent block-based fine generation process, so it needs to be enlarged.

[0231] This step provides an image upscaling method based on a super-resolution model, which specifically includes the following process:

[0232] First, the coarse image A is enlarged using a super-resolution reconstruction model to obtain a high-resolution image B. The super-resolution model can be an existing mature deep learning super-resolution method, such as Real-ESRGAN, SwinIR, or EDSR. Those skilled in the art should understand that the model is not limited to a specific implementation, and any super-resolution method that can achieve mapping from low resolution to high resolution is applicable.

[0233] This magnification step yields a high-resolution image that surpasses the original rough image in terms of resolution, detail, and edge sharpness. This high-resolution image is then used as the input benchmark for subsequent block generation and local refinement, ensuring consistent sharpness and detail levels in each block generation. This avoids problems such as blurring, distortion, and discontinuous stitching caused by directly dividing the image into blocks on a low-resolution image.

[0234] This step improves the overall image quality and resolution, playing a crucial role in connecting "coarse generation" and "block refinement" in the entire generation process. It is an essential step to ensure that the generated results have high-resolution consistency.

[0235] Step 6, Image segmentation and segmentation prompt word acquisition:

[0236] Due to limitations in current diffusion model capabilities and graphics card performance, it is difficult to directly refine high-resolution images. In this invention, in order to refine high-resolution images, it is necessary to divide the image into multiple processable tiles and then obtain the prompt words for each tile.

[0237] Step 6.1, the specific method for image segmentation is as follows:

[0238] Step 6.1.1, Input and parameter settings:

[0239] The high-resolution image B generated in the previous step is used as input, and parameters such as block size (bbox_size), overlap coefficient (min_overlap), and crop factor (crop_factor) are set. The block size determines the size of each locally generated region; the overlap coefficient is used to ensure the continuity of the boundary between adjacent blocks and avoid stitching gaps during local generation; the crop factor is used to leave extra pixels around the blocks to avoid loss of edge information, making it easier to obtain the true content of the block image when deducing prompt words later.

[0240] Step 6.1.2, calculate the number and location of blocks:

[0241] Based on the image size, block size, and overlap coefficient, the number of blocks in the horizontal and vertical directions is calculated, and the coordinate range of each block in the image is determined. For edge regions, the start and end points of the blocks are automatically adjusted to ensure that the entire image is covered and that the block size is consistent with the image boundary.

[0242] Step 6.1.3, generate the segmented cropping region:

[0243] For each block, calculate the corresponding clipping region in the original image and apply the clipping expansion factor to generate an enlarged block region for subsequent local generation and processing.

[0244] Step 6.1.4, generate the block mask:

[0245] When regional control of an image is required, this invention can generate a binary mask for each block to mark the effective pixel range. This mask can be combined with user-defined global constraint regions (such as excluding regions that do not need to be processed, or limiting the processing to specific regions), or it can automatically adapt to the block boundaries. In this way, only the effective regions are processed during local generation, thereby improving generation accuracy and efficiency.

[0246] If the user does not set area control requirements, or needs to perform unified repair and generation of the entire image, the mask can be degenerated into a fully valid mask, meaning that all pixels participate in the generation.

[0247] Step 6.1.5, output block information:

[0248] The coordinate information, cropping region, and mask of each block are packaged into a block object set, which serves as input for subsequent local image generation, image thinning, or style condition application.

[0249] Through the above steps, this invention achieves automatic block segmentation of high-resolution images, such as... Figure 7 As shown, this ensures that each block can completely cover the image, while taking into account the needs of edge continuity and local detail processing, providing basic data and structural guarantees for subsequent block generation and fine rendering.

[0250] Step 6.2, the method for obtaining block-based prompts is as follows:

[0251] After the segmentation is completed, in order to further improve the semantic accuracy and detail consistency of the local generation, a segmentation prompt word acquisition method is provided. This method obtains a segmentation prompt word set by performing semantic back-inference and manual calibration on the local regions after high-resolution image segmentation, which is used to guide subsequent local fine-grained generation.

[0252] Step 6.2.1, Input blocks:

[0253] After the high-resolution image B is segmented, it is input into the image segmentation module to generate segments. Unlike the common method of generating sliding window image segments based on the overlap coefficient, this invention uses a cropping expansion factor to control the segmentation. Specifically, during segmentation, not only is the core content of the target region extracted, but also a certain range of contextual information is retained around it according to the set expansion factor, thereby forming segments with context awareness.

[0254] This method has the following advantages:

[0255] The pruning expansion factor forms a larger block range, which can extract more information and improve the accuracy of prompt word inference; while in actual noise reduction, the overlap coefficient is used to form the block, which is usually smaller than the pruning expansion factor block, which helps to reduce the consumption of computing resources.

[0256] Step 6.2.2, reverse the clue word analysis:

[0257] After obtaining the blocks, the image back-inference module is used to perform semantic parsing on each block to generate corresponding natural language prompts. This back-inference step ensures that the prompts can faithfully reflect the objective characteristics of the blocks and avoids subjective bias caused by relying solely on manually written prompts.

[0258] Step 6.2.3, Manual calibration:

[0259] Considering that automated prompt generation may result in redundancy, ambiguity, or omissions, this invention further introduces a manual calibration step. Specifically, operators screen, supplement, and simplify the candidate prompts for each block to ensure that the prompts accurately reflect the semantic content and stylistic features of the local target. For example, in architectural image scenarios, the descriptions of elements such as "window material," "roof type," and "facade style" can be corrected.

[0260] After manual calibration, the resulting set of block prompts P2 can serve as an important input condition for subsequent block refinement generation, ensuring the accuracy and consistency of local detail descriptions.

[0261] Step 7, condition fusion, such as Figure 5 As shown:

[0262] When applying conditional suggestions, there are two main existing methods: relying solely on global suggestions or relying solely on block suggestions; rigorous testing has revealed serious flaws in both methods:

[0263] When relying solely on global prompts, the encoding length (token limitation) is limited, making it impossible to cover the local details of each block. The lack of correct prompts leads to severe missing details in local buildings and uncorrectable structural ambiguity. At the same time, the global prompts may contain too many semantic descriptions that should not exist in the local area. The model "completes" these descriptions based on the prompts, resulting in the generation of redundant buildings or elements that do not belong to the design of this block.

[0264] When relying solely on segmented prompts, the details of each segment can be guaranteed, but the style and lighting conditions between different segments show obvious inconsistencies as the noise reduction intensity increases; after splicing, obvious seams often appear, especially the lines of the building facade cannot be aligned, and there are abrupt changes in the color and texture of adjacent segments, which are difficult to repair later.

[0265] To address the above problems, this invention proposes a dynamic conditional fusion mechanism. This mechanism dynamically weights the global conditional vector C1, the block-based conditional vector set C2, and the style conditional vector C3 to obtain a unified fusion conditional vector C0. This method demonstrates significant advantages in experiments. The specific steps are as follows:

[0266] Step 7.1, Source of conditions:

[0267] Global conditional vector P1: Generated by the global cue word P1 through a semantic coding model, it contains the overall building type, scene elements and environmental features;

[0268] Block condition vector set 𝐶2: obtained by encoding the block cue word set 𝑃2, containing local detail information of the block region;

[0269] Style condition vector F3: Generated by the style extraction module, it includes color distribution, material features, and lighting style.

[0270] Step 7.2, Embedding Extraction:

[0271] a. Semantic embedding extraction:

[0272] Input the image 𝐵 into a multimodal semantic coding model (such as CLIP, BLIP2) to obtain the global semantic embedding 𝐸0;

[0273] The block image 𝐵𝑖 is input into the same model to obtain the block semantic embedding 𝐸𝑖;

[0274] After L2 normalization of E0 and 𝐸𝑖, semantic similarity is calculated:

[0275]

[0276] b. Style embedding extraction:

[0277] Input the image 𝐵 into a style encoder (such as a Redux style model or a CNN style extraction network) to obtain the style embedding 𝑆0;

[0278] The block image 𝐵𝑖 is input into the same model to obtain the style embedding 𝑆𝑖;

[0279] Style similarity is calculated after L2 normalization of S0 and Si:

[0280]

[0281] Step 7.3, Dynamic Weight Allocation:

[0282] Based on the similarity calculation results, conditional weights are dynamically assigned:

[0283] Global weights:

[0284]

[0285] Style weight:

[0286]

[0287] Local weights:

[0288]

[0289] Where f(⋅) is a normalization or linear mapping function, ensuring that αi, βi, γi ∈ [0, 1] and αi + βi + γi = 1; βmin is a user-defined lower limit, ensuring that local conditions are always involved.

[0290] Step 7.4, Condition Fusion:

[0291] The final unified fusion condition vector C0 is obtained by weighted fusion of three types of conditions:

[0292]

[0293] The technical effects of the dynamic condition fusion mechanism of the present invention are:

[0294] Global consistency: By calculating the semantic similarity between the entire graph and its segments, we ensure that the global building types and layouts do not deviate.

[0295] Local accuracy: By using a residual allocation strategy, we ensure that local detail conditions always exist, avoiding excessive coverage of global or style information;

[0296] Style consistency: Through style similarity calculation and weight allocation, we ensure that all blocks are consistent in color, material and lighting.

[0297] Dynamic adaptability: Weight allocation is automatically adjusted based on block and global / style similarity, rather than a fixed ratio, to adapt to different types of architectural images and generated scenes.

[0298] Step 8, Geometric control preprocessing:

[0299] In engineering applications, directly using prompts to control noise reduction in segmented images results in several issues: when the noise reduction intensity is less than 0.3, it cannot effectively correct distorted building lines; when the noise reduction intensity is greater than 0.4, it easily generates compositions that do not meet expectations or are unreasonable. To ensure the accuracy and consistency of building geometry during segmentation generation, this solution proposes a geometric control preprocessing method.

[0300] Step 8.1, Input image:

[0301] The high-resolution image B obtained by super-resolution magnification is used as input; although the image has a high resolution and has problems such as blurred local geometric boundaries and distorted lines, it can be used as a skeleton to guide the subsequent fine-grained generation.

[0302] Step 8.2, Geometric Feature Extraction:

[0303] Geometric feature preprocessing is performed on high-resolution image B to extract the following control signals:

[0304] Line art information: The building outline and main body boundary lines are obtained through edge detection and line extraction methods to ensure that the building shape is clear in the generated result;

[0305] Depth information: The spatial hierarchy between buildings and scenes is obtained through depth estimation methods to maintain the consistency of perspective and volume during block generation.

[0306] It should be understood that the above-mentioned line drawing extraction and depth estimation can be implemented by existing mature algorithms, and this invention is not limited to a specific implementation method.

[0307] Step 8.3, control signal generation:

[0308] In the subsequent segmented and refined generation process, this invention uses the line drawing and depth map as control signal inputs to ControlNet, and works in conjunction with the fusion condition vector C0 obtained in the previous step; ControlNet uses geometric control signals to constrain the generation results during the sampling process of the diffusion model, thereby ensuring that the geometric structure of the image is accurate and stable.

[0309] Specifically, during the block-based fine-grained generation, the diffusion model uses existing block images as input and employs a relatively high noise reduction intensity (preferred range: 0.4–0.8). To avoid distortion caused by excessive constraints, the control intensity of ControlNet should not be set too high, typically not exceeding 0.6. In terms of time step control, the start time is set to 0.0–0.2, and the end time is set not to exceed 0.6. The start time of the line drawing condition is generally earlier than that of the depth condition. In this way, the overall geometric structure of the building can be stabilized in the initial stage to avoid shape deformation, while retaining sufficient degrees of freedom for generation in subsequent stages, allowing the model to fine-tune and supplement the image details.

[0310] Through the above control strategies, the line drawing signal ensures the clarity of the building boundaries and component shapes, significantly improving geometric accuracy; the depth signal ensures the consistency of different blocks in spatial perspective and volumetric relationships, thereby improving the overall spatial harmony.

[0311] Step 9, refine and generate blocks:

[0312] After completing conditional fusion and geometric control preprocessing, this step proposes a block-based image refinement generation method to improve the quality of local details in architectural renderings while ensuring overall structural consistency. The specific steps are as follows:

[0313] Step 9.1, Input and Initialization:

[0314] The image blocks obtained by the image segmentation module are input into the thinning generation module; each block contains its corresponding cropping region, mask information and local feature description.

[0315] Step 9.2, Conditional Loading:

[0316] During the generation process, the fusion condition vector C0 is used as the main guiding signal, while geometric control signals (line drawing and depth map) are injected simultaneously.

[0317] Step 9.3, noise reduction of the segmented image:

[0318] Step 9.3.1, the image-to-image denoising process based on the diffusion model is performed on each image block as follows:

[0319] Step 9.3.2: Encode the original block image into the latent space;

[0320] Step 9.3.3: Inject moderate noise into the latent space and perform iterative sampling with a high noise reduction intensity (0.4–0.8);

[0321] Step 9.3.4: During the sampling process, a higher noise reduction intensity is used, and the fusion condition vector C0 and geometric control signal are combined for guidance, thereby increasing local details and material texture representation while maintaining geometric accuracy.

[0322] Repeat steps 9.3.1-9.3.4 until all blocks are generated.

[0323] Step 10, Block merging and global consistency repair:

[0324] Step 10.1, merging blocks and blending overlapping regions:

[0325] After completing the detailed generation of all blocks, the present invention first determines the position coordinates of each block in the whole image based on the clipping area information output by the block module; then, based on the block boundaries and overlap settings, the overlapping area of ​​adjacent blocks is calculated uniformly.

[0326] In the stitching stage, this invention further proposes a pixel-level fusion algorithm, the steps of which are as follows:

[0327] Step 10.1.1: Within the overlapping region, extract the generation results Ri, Rj of adjacent blocks;

[0328] Step 10.1.2: Construct a weight mask 𝑀, whose value range is [0,1]. Its weight is larger at the center of the block and gradually decreases to 0 towards the boundary. The decay function can be implemented by linear function, cosine function or Gaussian function, etc., to ensure that the transition between blocks is smooth and natural.

[0329] Step 10.1.3: Perform a pixel-level weighted average on the overlapping regions:

[0330]

[0331] Where Ri and Rj represent the image results from two adjacent blocks, (x,y) represents the pixel coordinates in the overlapping area, R(x,y) is the final fusion result, and 𝑀 is the weight mask function.

[0332] Through the pixel-level fusion processing described above, obvious boundary lines, misalignments, and style abrupt changes generated during the block stitching process can be effectively eliminated, so that the final generated image maintains a high degree of continuity in geometric structure and detail representation.

[0333] Step 10.2, Global Consistency Repair:

[0334] Despite employing segmented refinement and overlapping region fusion techniques, the final merged high-resolution image may still exhibit issues such as local color differences and subtle stylistic variations. Therefore, this invention further proposes a global consistency restoration method:

[0335] Step 10.2.1: Divide the merged high-resolution image into multiple small blocks and perform latent space coding using a variational autoencoder (VAE);

[0336] Step 10.2.2: In the latent space, the global cue word P1 is used as the main semantic guide, supplemented by the geometric control signals (line drawing and depth map) provided by ControlNet.

[0337] Step 10.2.3: Perform image restoration sampling in the diffusion model with a low noise reduction intensity (noise figure < 0.3) to obtain the architectural rendering.

[0338] This method can reunify the semantics and style of subtle differences in the image while maintaining the stability of the overall structure, thereby ensuring global style consistency and color harmony. The final output architectural renderings are highly consistent in geometric structure, local details and overall style, avoiding the abrupt seams and style inconsistencies common in traditional block generation methods.

[0339] Finally, it should be emphasized that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still make modifications to the technical solutions described in the foregoing embodiments without creative effort, or make equivalent substitutions for some of the technical features. Therefore, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0340] The terminology explained in this application is as follows:

[0341] 1. Architectural Renderings: Architectural renderings are visual images generated based on architectural design schemes. They are used to intuitively showcase the building's appearance, spatial layout, and relationship with its environment before construction. Their content typically includes the building's volume, form, materials, colors, lighting effects, and surrounding scene elements (such as roads, greenery, and water features). Architectural renderings can be drawn manually or automatically generated using computer-aided design (CAD / BIM), rendering engines, or generative model-based algorithms.

[0342] In this invention, architectural renderings mainly refer to high-resolution architectural images generated through generative artificial intelligence technologies such as diffusion models, emphasizing the realism and consistency in architectural geometry, material texture, and overall style.

[0343] 2. Diffusion Model: A generative artificial intelligence model that gradually recovers the image from the noise by progressively adding noise to the image and learning the inverse process, thus achieving high-quality image generation.

[0344] Meaning in this invention: The basic generative model used to generate architectural renderings, including but not limited to diffusion model frameworks such as StableDiffusion XL and Flux.

[0345] 3. Sub-model: A specific functional model trained or fine-tuned within the diffusion model framework, such as the LoRA model specifically for generating architectural bird's-eye views, or the Redux model specifically for extracting styles.

[0346] In this invention, the LoRA model is used to learn architectural scene features; the Redux model is used to extract style conditions; and other sub-models such as ControlNet are used to control the structure.

[0347] 4. LoRA (Low-Rank Adaptation): A lightweight model fine-tuning method that quickly learns new task features by inserting low-rank adaptation layers into a large model weight matrix.

[0348] In this invention, the meaning is: to specifically train architectural renderings to enhance features such as architectural geometry and material representation, so that the generated results are more in line with the architectural scene.

[0349] 5. Prompt: A natural language description input to the diffusion model, used to control the subject, style, and details of the images generated by the model.

[0350] In this invention, it means: including a global prompt word P1 and a block-specific prompt word set P2, which are used to control the entire... Figure 1 Consistency and local details.

[0351] 6. Prompt Inversion / Backtracking: This involves deriving prompts that describe an image from an existing image. This can be achieved through image-text matching models or image description models.

[0352] Meaning in this invention: used to extract latent semantic descriptions from the initially generated rough effect diagram A or block image B, and then manually calibrate them to form high-quality prompt words P1 and P2.

[0353] 7. Style Condition: Vectorized feature information used to describe the overall style characteristics of an image (such as tone, lighting, and texture).

[0354] Meaning in this invention: Extracted from the rough rendering A by the Redux model to ensure style consistency in subsequent generated images.

[0355] 8. Condition Fusion: Combines multiple conditions from different sources (such as global, block, and style) according to their weights to obtain a comprehensive condition vector.

[0356] 9. ControlNet: A neural network structure that introduces additional conditions (such as edges, depth, semantic segmentation, etc.) during the generation of diffusion models.

[0357] In this invention, it means: to guide the generated image to maintain consistency with the input line drawing and depth, ensuring clear building structure and reasonable landscape layout.

[0358] 10. Denoising Strength: The degree of noise injection during image-to-image (img2img) generation by the diffusion model. It is usually in the range of 0–1. The higher the value, the closer it is to regeneration, and the lower the value, the closer it is to the original image.

[0359] In this invention, the value is: 0.4–0.8 for block refinement to ensure detailed generation; and ≤0.3 for global consistency repair to ensure overall stability and correct only minor gaps.

[0360] 11. Tiled Generation: Divide a high-resolution image into several small blocks, generate or repair each block individually, and then stitch them together to form a complete image.

[0361] Meaning in this invention: Used to generate high-resolution architectural renderings under limited video memory conditions, while allowing each block to independently use blending conditions to control local details.

[0362] 12. Seamless Refinement: After stitching together segmented images, an additional global image generation step is used to eliminate gaps and style differences between blocks.

[0363] Meaning in this invention: Perform low-intensity global image generation (img2img) noise reduction on the generated image to achieve consistency in style and detail.

[0364] 13. VAE (Variational Autoencoder): A VAE is a neural network structure commonly used in diffusion models. It can map high-dimensional images to a low-dimensional latent space for compressed representation and decode the latent space data into an image after generation.

[0365] In this invention, the application of VAE includes the following two aspects:

[0366] In the block processing stage: during the image-to-image thinning process of each image block, the block is mapped to the latent space through VAE encoding, and noise is injected into the latent space to guide denoising sampling under certain conditions. After generation, VAE decoding is used to restore the high-resolution image block. This process effectively reduces the video memory overhead of block image processing and improves generation efficiency.

[0367] Global Restoration Phase: After all segments are generated and merged into a complete image, this invention further utilizes the VAE's segmented encoding and decoding mechanism to uniformly restore the entire image. During this process, combining global cue words and low-intensity noise reduction effectively eliminates gaps in segment stitching and stylistic inconsistencies, improving the overall consistency and visual quality of the image.

[0368] Through the above dual application, VAE not only plays a role in latent space compression and generation during local block processing, but also provides cross-block consistency optimization capabilities during the global repair phase.

[0369] 14. Manual calibration: The automatically generated prompts or conditions are manually modified and filtered to ensure their accuracy and professionalism.

[0370] 15. Image-to-Image (img2img): Image-to-image refers to a method of regenerating an existing input image using a diffusion or generative model. Its core principle is to inject noise of a certain intensity into the latent space representation of the input image and combine this with text prompts or conditional controls for iterative noise reduction sampling, thereby obtaining a new image that is geometrically related to the input image but controllable in terms of detail and stylistic features. In this invention, image-to-image technology is used to refine and repair architectural line drawings or preliminary renderings, maintaining the original spatial layout and structural form while enhancing details such as materials, lighting, and style.

[0371] 16. Text-to-Image (text2img): Text-to-image refers to a method that uses text descriptions as the main input and directly generates images through a diffusion or generative model. Its principle is to encode the input text prompts into semantic vectors, guiding the diffusion model to iteratively sample in the latent space of random noise, thereby generating an image that matches the semantics of the text. In this invention, text-to-image is mainly used to test the training effect of the LoRA model and select a suitable fine-tuning model.

[0372] 17. Local Inpainting: Local inpainting refers to a method of generating or repairing an image by focusing on a specified area based on the input image. Its principle is to resample the image within a user-defined mask area: injecting noise into the mask area and iteratively generating the image using prompts or conditional controls, while areas not covered by the mask remain unchanged.

[0373] In this invention, local redrawing is used to modify or enhance specific areas in architectural renderings, such as replacing roof materials, adding greenery, or repairing local structures. This method ensures that the generated result receives detailed updates in local areas while maintaining consistency with the unmodified areas in terms of geometry and style, thus achieving more controllable image editing.

Claims

1. A method for generating architectural renderings based on multi-condition weighted fusion and block control, characterized in that: Includes the following steps: Step 1, Customized Model Training: Using SDXL and Flux diffusion models as the basic framework, and training a special model for architectural renderings through LoRA technology to enhance the ability to express the characteristics of architectural scenes; Step 2, Initial Generation: Based on the input architectural line drawing, a rough rendering A is generated by combining the SDXL model; Step 3, Global Prompt Extraction and Calibration: Prompts are back-derived from the rough effect diagram A and manually calibrated to obtain global prompt P1, which is used for subsequent global consistency guidance; Step 4, Style Condition Modeling: Use Flux's Redux model to perform style analysis on the rough rendering A to obtain the style condition vector C3; Step 5, Image magnification: The rough image A is processed using a super-resolution magnification model to obtain a high-resolution image B; Step 6, Image segmentation and segmentation prompt word acquisition: The image B is segmented, and prompt words are derived for each segment and manually calibrated to obtain the segmentation prompt word set P2; Step 7, Conditional Fusion: Generate a global conditional vector C1 based on the global prompt word P1, generate a block conditional vector set C2 based on the block prompt word set P2, and combine it with the style conditional vector C3 to establish a conditional fusion model; obtain a unified fusion conditional vector C0 for each block through weighted averaging. Step 8, Geometric Control Preprocessing: Preprocess image B to extract line art and depth map as ControlNet control signals, which are used to constrain the geometric structure during the block generation process; Step 9, Block Refinement Generation: Guided by the Flux model, each block of image B is subjected to image-to-image noise reduction, and the fusion condition vector C0, line drawing control and depth control are introduced at the same time to achieve a high-resolution block image with rich details and geometric accuracy. Step 10, Block Merging and Consistency Repair: Merge the block results into a complete image C, and perform global image noise reduction. Using the global prompt word P1 as a guide, repair the slight stitching marks and inconsistencies at the splicing edges to obtain the final architectural rendering.

2. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: Step 2 also includes the following steps: Step 2.1, Obtain the architectural line drawing: When exporting line drawings from 3D design software, it is preferable to output high-precision rendering results with a resolution of ≥4K. Step 2.2, High-resolution line art preprocessing: The first stage involves high-resolution line art extraction, which specifically includes: Line detection and enhancement: Extract the main outline, dividing lines and key component boundaries of buildings through gradient-based edge operators or deep learning line detection networks; Noise and pseudo-line suppression: For interfering lines caused by texture materials in the input, texture filtering and region consistency judgment are performed; Binarization and standardization: The extracted effective lines are converted into black and white binary images, and the lines are thinned to ensure uniform line width and meet the requirements of ControlNet as a geometric control input; The second phase involves high-quality scaling and resolution adaptation. Scaling mode selection: Uses a proportional scaling strategy to ensure that the aspect ratio of the image remains unchanged; Boundary preservation mechanism: During scaling, the system automatically identifies whether the line drawing is a binary boundary image. If it is a pixel-wide line, non-maximum suppression and thinning algorithms are used to prevent the lines from disappearing or breaking. Noise reduction and background processing: Mask synchronization adjustment is performed on the scaled image, and a color filling strategy based on edge pixel statistics is adopted to avoid background artifacts or cropping marks; Step 2.3, Write the prompt words: Based on the architectural image content and design requirements, write a prompt description that includes the following elements: Architectural type, scene elements, and style preferences; Step 2.4, generating structural constraints based on ControlNet: The preprocessed line drawing is used as a structural constraint input to the ControlNet module. Combined with the prompt words mentioned in step 2.3 and the fine-tuning of the LoRA model, the diffusion model is used to initially generate a rough effect diagram. Step 2.5, manual screening and repair: combining manual screening and partial redrawing, a rough rendering A that conforms to the architectural semantics is obtained.

3. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: In step 3, the semantic information of the initial generated result is automatically parsed using a large language model to generate the global prompt word P1.

4. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: Step 4, the steps to obtain the style condition vector C3, include: Step 4.1, Input Stage: The rough rendering A is input into the style analysis module, which simultaneously receives image input and optional text input; Step 4.2, Feature Encoding Stage: The style analysis module uses a visual semantic encoder to process the input image and extract the following style information: color tone, material features, and lighting features; Step 4.3, Style Condition Generation Stage: Combining the image encoding results with optional text conditions, the style analysis module outputs a style condition vector C3, which serves as the generation condition input for the diffusion model.

5. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: In step 5, after the rough effect diagram A is generated, the resolution is first improved by super-resolution model, and then gradually refined by block generation.

6. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: Step 8 involves preprocessing the extracted line art, including: Phase 1: High-resolution line art extraction: The original architectural line drawing is converted into a clearly defined black and white line drawing. Specifically: Line detection and enhancement: Extract the main outline, dividing lines and key component boundaries of buildings through gradient-based edge operators or deep learning line detection networks; Noise and pseudo-line suppression: For interfering lines caused by texture materials in the input, texture filtering and region consistency judgment are performed to avoid the extraction of unstructured lines; Binarization and standardization: The extracted effective lines are converted into black and white binary images, and the lines are thinned to ensure uniform line width and meet the requirements of ControlNet as a geometric control input; The second phase involves high-quality scaling and resolution adaptation. Scaling mode selection: Uses a proportional scaling strategy to ensure that the aspect ratio of the image remains unchanged; Boundary preservation mechanism: During scaling, the system automatically identifies whether the line drawing is a binary boundary image. If it is a pixel-wide line, non-maximum suppression and thinning algorithms are used to prevent the lines from disappearing or breaking. Noise reduction and background processing: Mask synchronization adjustment is performed on the scaled image, and a color filling strategy based on edge pixel statistics is adopted to avoid background artifacts or cropping marks.

7. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: Step 9 also includes the following steps: Step 9.1, Input and Initialization: The image blocks obtained by the image segmentation module are input into the thinning generation module; each block contains its corresponding cropping region, mask information and local feature description; Step 9.2, Conditional Loading: During the generation process, the fusion condition vector C0 is used as the main guiding signal, and a geometric control signal is injected at the same time; Step 9.3, noise reduction of the segmented image: Step 9.3.1: Perform image-to-image denoising based on a diffusion model on each image block; Step 9.3.2: Encode the original block image into the latent space; Step 9.3.3: Inject moderate noise into the latent space and perform iterative sampling with a noise reduction intensity of 0.4–0.8; Step 9.3.4: During the sampling process, a noise reduction intensity of 0.4–0.8 is used, and the fusion condition vector C0 and the geometric control signal are combined for guidance. Repeat steps 9.3.1-9.3.4 until all blocks are generated.

8. The architectural rendering generation method based on multi-condition weighted fusion and block control according to claim 1, characterized in that: Step 10 also includes the following steps: Step 10.1.1: Within the overlapping region, extract the generation results Ri, Rj of adjacent blocks; Step 10.1.2: Construct a weight mask 𝑀, whose value range is [0,1]; its weight is larger at the center of the block and gradually decreases to 0 towards the boundary; the decay function is a linear function, a cosine function or a Gaussian function; Step 10.1.3: Perform a pixel-level weighted average on the overlapping regions: Where Ri and Rj represent the image results from two adjacent blocks, (x,y) represent the pixel coordinates in the overlapping area, R(x,y) is the final fusion result, and 𝑀 is the weight mask function; Step 10.2, Global Consistency Repair: Step 10.2.1: Divide the merged high-resolution image into multiple small blocks and perform latent space coding using a variational autoencoder; Step 10.2.2: In the latent space, the global cue word P1 is used as the main semantic guide, supplemented by the geometric control signals provided by ControlNet; Step 10.2.3: Perform image restoration sampling in the diffusion model with a noise reduction intensity of < 0.3 to obtain the architectural rendering.

Images