Architectural design drawing generation method and system based on stable diffusion model

Abstract

The invention discloses a building design drawing generation method and system based on a stable diffusion model, and the method comprises the steps: obtaining image materials of different building facades, and carrying out the screening and marking of the image materials, and obtaining a data set; a stable diffusion model based on Control-Net guidance is adopted to train and finely adjust the data set, and a building image generation model is obtained; inputting the sketch and the cue word into a building image generation model, and outputting a building design drawing; and rendering and enhancing the architectural design drawing by adopting an image super-resolution algorithm based on the generative adversarial network. According to the method, building facade image materials are acquired, a data set is constructed, stable diffusion model training guided by Control-Net is combined, so that rapid generation from a sketch to a building design drawing is realized, meanwhile, a generative adversarial network is introduced to render and enhance the generated building design drawing, the definition and detail performance of the image are remarkably improved, and the image quality is improved. And a high-fidelity visual effect picture is quickly output, and the design efficiency and quality are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of architectural image processing, and in particular to a method and system for generating architectural design drawings based on a stable diffusion model. Background Art

[0002] In the traditional architectural design process, designers usually rely on hand-drawn sketches, physical models, or computer-aided design (CAD) software to present design concepts. However, these methods have the following problems: Low efficiency: Hand-drawing and physical model production are time-consuming and labor-intensive, with high modification costs, making it difficult to quickly iterate design solutions.

[0003] Limited expressiveness: Although traditional rendering technologies (such as 3D modeling + ray tracing) can produce realistic effects, they consume large amounts of computing resources and require high technical skills from designers.

[0004] Limited creative expression: Designers’ inspiration is often limited by the functionality of existing tools, making it difficult to quickly explore multiple styles or scene changes.

[0005] In recent years, generative artificial intelligence (such as GAN, VAE, and Diffusion Models) has made significant progress in the field of image generation. However, its application in architectural design still faces the following challenges: Poor adaptability of general models: Existing generative models such as stable diffusion are mainly optimized for natural images. When directly applied to the architectural field, the generated architectural structures may not meet professional standards (such as distorted proportions and missing details).

[0006] Insufficient control capabilities: Ordinary text-to-image models have difficulty accurately understanding architectural terminology (such as "cantilever structure" and "Gothic arch"), resulting in significant deviations between generated results and design intent.

[0007] Unstable rendering quality: Traditional super-resolution algorithms (such as SRGAN) are prone to artifacts or blurring when enhancing architectural details (such as textures, materials, and light and shadow), affecting visual effects. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method and system for generating architectural design drawings based on a stable diffusion model, which can generate architectural design drawings according to customized sketches and prompt word descriptions and perform high-resolution automatic rendering, thereby improving the speed and diversity of architectural design implementation.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for generating architectural design drawings based on a stable diffusion model comprises the following steps: S1. Obtain image materials of different building facades, and screen and label the image materials to obtain a data set; S2. Using a Control-Net-guided stable diffusion model to train and fine-tune the dataset to obtain a building image generation model; S3, inputting the sketch and prompt words into the building image generation model to output the building design drawing; S4. Rendering and enhancing the architectural design drawing using an image super-resolution algorithm based on a generative adversarial network.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is: A system for generating architectural design drawings based on a stable diffusion model, comprising: A sample construction module is used to obtain image materials of different building facades, and to screen and label the image materials to obtain a data set; A model building and training module is used to train and fine-tune the dataset using a stable diffusion model guided by Control-Net to obtain a building image generation model; An image generation module, configured to input a sketch and prompt words into the building image generation model and output a building design drawing; The model rendering module is used to render and enhance the architectural design drawing by using an image super-resolution algorithm based on a generative adversarial network.

[0011] The beneficial effects of the present invention are: providing a method and system for generating architectural design drawings based on a stable diffusion model, which aims to solve the problem that traditional architectural effects mainly rely on hand-drawing and model making, which is not only labor-intensive but also has great limitations in the expression of details and the presentation of complex scenes. By obtaining architectural facade image materials and constructing a data set, combined with the Control-Net-guided stable diffusion model training, a building image generation model with stable prediction results and applicable to the architectural field is obtained, so as to realize the rapid generation from sketches to architectural design drawings. At the same time, a feature-enhanced generative adversarial network is introduced to render and enhance the generated architectural design drawings, which significantly improves the clarity and detail expression of the image, can quickly output high-fidelity visualization effect drawings, and improves design efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Picture 1 This is a main flow chart of a method for generating architectural design drawings based on a stable diffusion model according to an embodiment of the present invention; Picture 2 This is a specific flow chart of a method for generating architectural design drawings based on a stable diffusion model according to an embodiment of the present invention; Picture 3The present invention is a structural diagram of a system for generating architectural design drawings based on a stable diffusion model.

[0013] Description of labels: 201. Sample construction module; 202. Model construction and training module; 203. Image generation module; 204. Model rendering module. DETAILED DESCRIPTION

[0014] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0015] Please refer to Picture 1 and Picture 2 , a method for generating architectural design drawings based on a stable diffusion model, comprising the steps of: S1. Obtain image materials of different building facades, and screen and label the image materials to obtain a data set; S2. Using a Control-Net-guided stable diffusion model to train and fine-tune the dataset to obtain a building image generation model; S3, inputting the sketch and prompt words into the building image generation model to output the building design drawing; S4. Rendering and enhancing the architectural design drawing using an image super-resolution algorithm based on a generative adversarial network.

[0016] From the above description, it can be seen that the beneficial effects of the present invention are: providing a method for generating architectural design drawings based on a stable diffusion model, which aims to solve the problem that traditional architectural effects mainly rely on hand-drawing and model making, which is not only labor-intensive but also has great limitations in the expression of details and the presentation of complex scenes. By obtaining building facade image materials and constructing a data set, combined with the Control-Net-guided stable diffusion model training, a building image generation model with stable prediction results and applicable to the architectural field is obtained, so as to realize the rapid generation from sketches to architectural design drawings. At the same time, a feature-enhanced generative adversarial network is introduced to render and enhance the generated architectural design drawings, which significantly improves the clarity and detail expression of the image, and can quickly output high-fidelity visualization effect drawings, thereby improving design efficiency and quality.

[0017] Furthermore, the step S1 is specifically as follows: S11, acquiring image materials of different building facades, scaling the image materials to within a preset range, and deleting foreground materials in the scaled image materials that obstruct the main building and whose obstruction area exceeds a preset area; S12, cropping the non-building background image material from the image material from which the foreground material has been deleted; S13, screening multiple materials whose similarity reaches a first threshold among the cropped image materials and randomly retaining one material; S14, using an edge extraction operator algorithm to extract edges from the filtered image material to generate an edge detail image of the filtered image material; S15. Designing a labeling logic database based on architectural terminology, labeling the edge detail image, generating prompt phrases related to the filtered image material, and constructing a data set including the edge detail image and the corresponding prompt phrases.

[0018] As can be seen from the above description, by scaling, cropping, filtering, and edge extraction of image materials, the diversity and quality of the dataset are ensured, and the waste of computing resources caused by redundant data is avoided. At the same time, the prompt phrases generated based on the annotation logic of architectural terminology enable the model to more accurately understand the design intent, thereby generating design drawings that better meet actual needs.

[0019] Furthermore, the step S2 is specifically as follows: S21, extracting text prompt words from each prompt phrase in the data set, converting each text prompt word into a word, and encoding each word and converting it into a vector; S22. Encode each edge detail image in the data set using a variational autoencoder, and convert the encoded edge detail image from image data in pixel space to tensor data in latent space; S23, restoring the tensor data by multiple downsampling and amplification in the hidden layer space to obtain a U-shaped network image generation model; S24. Use LoRA fine-tuning technology to adjust the weights of the U-shaped network image generation model during training to obtain a building image generation model.

[0020] As can be seen from the above description, through the vectorization of text prompt words and the latent space transformation of edge detail images, combined with the learning ability of the U-shaped network, the model can efficiently capture the characteristics of architectural design. At the same time, the use of LoRA fine-tuning technology to optimize weights significantly improves the training efficiency and adaptability of the model, making it more stable and accurate when generating high-quality images in the architectural field.

[0021] Furthermore, the step S24 is specifically as follows: Given a pre-trained matrix , R represents the set of real numbers, m and n Respectively The number of rows and columns; For the pre-training matrix Define a low-rank r,in d is the original dimension of the matrix and r << d ; The pre-trained matrix Updated to The low-rank decomposition representation of , , express BA The multiplication result of For the pre-training matrix Perform singular value decomposition to obtain m The low-rank matrix of singular values is decomposed into: ; in, U For one m × m The orthogonal matrix, Σ For one m × n The diagonal matrix of For one n × n The transpose of the orthogonal matrix; definition For the pre-training matrix The singular value matrix of r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix , the main matrix Together with the residual matrix, it forms the initialization matrix of the fine-tuning weight; Initialize two low-rank matrices according to the weights of the U-shaped network image generation model A and B : ; ; in and Respectively represent the selected r The singular values corresponding to U Submatrix and submatrix, Therefore r A diagonal matrix with singular values on the diagonal, A and B Is the weight containing fine-tuning, indicating the difference to be added to the original weight, the fine-tuned weight is combined with the original pre-trained weight, then for the input of the U-network image generation model xThe modified forward pass is the final building image generation model, which is as follows: ; in is the residual matrix obtained by selecting a smaller singular value, gamma is the adjustment parameter and gamma The value is 0-0.1.

[0022] From the above description, we can see that through singular value decomposition and low-rank matrix initialization, LoRA fine-tuning technology can quickly adapt to new task requirements while retaining the knowledge of the pre-trained model; at the same time, the combination of the main matrix and the residual matrix enhances the expressive power of the model, enabling it to better handle the complex features of architectural design and generate more detailed and realistic images.

[0023] Furthermore, the step S4 is specifically as follows: S41, inputting the architectural design drawing into a shallow feature extraction network, and outputting shallow features of the architectural design drawing, wherein the shallow feature extraction network includes a convolutional layer and a linear rectification function; S42, inputting the shallow features into a deep feature extraction network, introducing a feature enhancement module, and using multiple feature-enhanced residual dense blocks to extract deep features of the architectural design drawing; S43. The extracted deep features are input into the reconstruction module, and a rendered and enhanced architectural design image is generated through convolution and upsampling operations.

[0024] As can be seen from the above description, by combining shallow and deep feature extraction networks, introducing feature enhancement modules and multi-scale feature fusion, the super-resolution reconstruction capability of images has been significantly improved, ensuring that the generated high-resolution architectural design drawings are clearer and more realistic in terms of textures, materials and complex patterns, meeting the needs of high-quality visualization.

[0025] Please refer to Picture 3 , a building design drawing generation system based on a stable diffusion model, comprising: A sample construction module is used to obtain image materials of different building facades, and to screen and label the image materials to obtain a data set; A model building and training module is used to train and fine-tune the dataset using a stable diffusion model guided by Control-Net to obtain a building image generation model; An image generation module, configured to input a sketch and prompt words into the building image generation model and output a building design drawing; The model rendering module is used to render and enhance the architectural design drawing by using an image super-resolution algorithm based on a generative adversarial network.

[0026] From the above description, it can be seen that the beneficial effects of the present invention are: based on the same technical concept, in conjunction with the above-mentioned architectural design drawing generation method based on a stable diffusion model, a system for generating architectural design drawings based on a stable diffusion model is provided. In view of the problem that traditional architectural effects mainly rely on hand-drawing and model making, which is not only labor-intensive but also has great limitations in the expression of details and the presentation of complex scenes, by obtaining building facade image materials and constructing a data set, combined with the Control-Net guided stable diffusion model training, an architectural image generation model with stable prediction results and suitable for the architectural field is obtained, so as to realize the rapid generation from sketches to architectural design drawings, and at the same time introduce a feature-enhanced generative adversarial network to render and enhance the generated architectural design drawings, significantly improving the clarity and detail expression of the image, so that designers can quickly obtain high-fidelity visualization renderings, thereby improving design efficiency and quality.

[0027] Furthermore, the sample construction module is specifically used to: Obtain image materials of different building facades, scale the image materials to within a preset range, and delete foreground materials in the scaled image materials that obstruct the main building and whose obstruction area exceeds a preset area; Crop the non-building background footage from the image footage after deleting the foreground footage; Filter multiple materials whose similarity reaches a first threshold among the cropped image materials and randomly retain one material; An edge extraction operator algorithm is used to extract edges of the filtered image material to generate an edge detail image of the filtered image material; Based on a design annotation logic database of architectural terminology, the edge detail images are annotated, prompt phrases related to the filtered image materials are generated, and a data set including the edge detail images and the corresponding prompt phrases is constructed.

[0028] As can be seen from the above description, by scaling, cropping, filtering, and edge extraction of image materials, the diversity and quality of the dataset are ensured, and the waste of computing resources caused by redundant data is avoided. At the same time, the prompt phrases generated based on the annotation logic of architectural terminology enable the model to more accurately understand the design intent, thereby generating design drawings that better meet actual needs.

[0029] Furthermore, the model building and training module is specifically used to: Extracting text prompt words from each prompt phrase in the data set, converting each text prompt word into a word, and encoding each word and converting it into a vector; Encoding each edge detail image in the data set through a variational autoencoder, and converting the encoded edge detail image from image data in a pixel space to tensor data in a latent space; The tensor data is downsampled and amplified multiple times to restore in the hidden layer space to obtain a U-shaped network image generation model; The LoRA fine-tuning technology is used to adjust the weights in the training process of the U-shaped network image generation model to obtain the building image generation model.

[0030] As can be seen from the above description, through the vectorization of text prompt words and the latent space transformation of edge detail images, combined with the learning ability of the U-shaped network, the model can efficiently capture the characteristics of architectural design. At the same time, the use of LoRA fine-tuning technology to optimize weights significantly improves the training efficiency and adaptability of the model, making it more stable and accurate when generating high-quality images in the architectural field.

[0031] Furthermore, the LoRA fine-tuning technology is used to adjust the weights in the U-shaped network training process to obtain the final building image generation model, specifically: Given a pre-trained matrix , R represents the set of real numbers, m and n Respectively The number of rows and columns; For the pre-training matrix Define a low-rank r ,in d is the original dimension of the matrix and r << d ; The pre-trained matrix Updated to The low-rank decomposition representation of , , express BA The multiplication result of For the pre-training matrix Perform singular value decomposition to obtain m The low-rank matrix of singular values is decomposed into: ; in, U For one m × m The orthogonal matrix, Σ For one m × n The diagonal matrix of For one n × n The transpose of the orthogonal matrix; definition For the pre-training matrix The singular value matrix ofr The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix , the main matrix Together with the residual matrix, it forms the initialization matrix of the fine-tuning weight; Initialize two low-rank matrices according to the weights of the U-shaped network image generation model A and B : ; ; in and Respectively represent the selected r The singular values corresponding to U Submatrix and submatrix, Therefore r A diagonal matrix with singular values on the diagonal, A and B Is the weight containing fine-tuning, indicating the difference to be added to the original weight, the fine-tuned weight is combined with the original pre-trained weight, then for the input of the U-network image generation model x The modified forward pass is the final building image generation model, which is as follows: ; in is the residual matrix obtained by selecting a smaller singular value, gamma is the adjustment parameter and gamma The value is 0-0.1.

[0032] From the above description, we can see that through singular value decomposition and low-rank matrix initialization, LoRA fine-tuning technology can quickly adapt to new task requirements while retaining the knowledge of the pre-trained model; at the same time, the combination of the main matrix and the residual matrix enhances the expressive power of the model, enabling it to better handle the complex features of architectural design and generate more detailed and realistic images.

[0033] Furthermore, the model rendering module is specifically used to: Inputting the architectural design drawing into a shallow feature extraction network and outputting shallow features of the architectural design drawing, wherein the shallow feature extraction network includes a convolutional layer and a linear rectification function; Inputting the shallow features into a deep feature extraction network, introducing a feature enhancement module, and using multiple feature-enhanced residual dense blocks to extract deep features of the architectural design drawing; The extracted deep features are input into the reconstruction module, and after convolution and upsampling operations, rendered and enhanced architectural design images are generated.

[0034] As can be seen from the above description, by combining shallow and deep feature extraction networks, introducing feature enhancement modules and multi-scale feature fusion, the super-resolution reconstruction capability of images has been significantly improved, ensuring that the generated high-resolution architectural design drawings are clearer and more realistic in terms of textures, materials and complex patterns, meeting the needs of high-quality visualization.

[0035] The present invention provides a method and system for generating architectural design drawings based on a stable diffusion model, which is mainly used in the field of architecture for rapid and high-quality design of architectural design drawings. The following is a detailed description of the method and system with reference to specific embodiments: Please refer to Picture 1 , embodiment 1 of the present invention is: A method for generating architectural design drawings based on a stable diffusion model, such as Picture 1 As shown, the steps include: S1. Obtain image materials of different building facades, filter and label the image materials, and build a dataset for subsequent model training.

[0036] S2. The dataset is trained and fine-tuned using a stable diffusion model guided by Control-Net to obtain an architectural image generation model suitable for the architectural field.

[0037] Control-Net is a tool for controlling and monitoring large-scale computer networks. It helps administrators monitor network health and ensure efficient management of network resources. Therefore, in this embodiment, Control-Net is used to assist the stable diffusion model in generating images. By controlling Control-Net, the stable diffusion model can be better adjusted to achieve the desired results.

[0038] S3. Input the sketch and prompt words into the architectural image generation model to generate a real scene image and output the architectural design drawing, thereby realizing the rapid conversion of the design sketch into the architectural design drawing.

[0039] S4. Use an image super-resolution algorithm based on a generative adversarial network to render and enhance architectural design drawings, resulting in clearer and more detailed visual effects, displaying design images of textures, materials, and complex patterns with high fidelity.

[0040] The key idea behind the generative adversarial network-based image super-resolution algorithm, ESRGAN (Enhanced Super-Resolution Generative Adversarial Network), is to improve image resolution by learning the mapping between low-resolution (LR) images and their high-resolution (HR) counterparts. ESRGAN combines a U-net architecture with a residual structure, enhancing the super-resolution performance of the generator through a confrontation between a generator network and a discriminator network. The generator is responsible for feature extraction and reconstruction, while the discriminator assesses the authenticity of the generated image.

[0041] That is, in this embodiment, in order to address the problem that traditional architectural effects mainly rely on hand-drawing and model making, which is not only labor-intensive but also has great limitations in the expression of details and the presentation of complex scenes, by obtaining building facade image materials and constructing a data set, combined with the stable diffusion model training guided by Control-Net, an architectural image generation model with stable prediction results and applicable to the architectural field is obtained, so as to achieve rapid generation from sketches to architectural design drawings. At the same time, a feature-enhanced generative adversarial network is introduced to render and enhance the generated architectural design drawings, which significantly improves the clarity and detail expression of the image, can quickly output high-fidelity visualization renderings, and improves design efficiency and quality.

[0042] Please refer to Picture 2 , the second embodiment of the present invention is: A method for generating architectural design drawings based on a stable diffusion model, based on the above-mentioned embodiment 1, in this embodiment, step S1 is specifically as follows: S11. Obtain image materials of different building facades, scale the image materials to within a preset range, and delete foreground materials in the scaled image materials that obstruct the main building and whose obstruction area exceeds a preset area.

[0043] S12. Crop the non-building background material from the image material where the foreground material has been deleted, and retain the main part of the building.

[0044] S13: Filter multiple materials whose similarity reaches a first threshold in the cropped image materials and randomly retain one material to avoid waste of computing resources caused by data redundancy.

[0045] S14. Use edge extraction operator algorithms such as Canny to extract edges from the filtered image material to generate edge detail images of the filtered image material.

[0046] S15. Design a labeling logic database based on architectural terminology, label the edge detail images, generate prompt phrases related to the filtered image materials, and construct a data set including edge detail images and corresponding prompt phrases.

[0047] That is, in this embodiment, by scaling, cropping, filtering and edge extraction of image materials, the diversity and quality of the data set are ensured, and the waste of computing resources due to redundant data is avoided; at the same time, the prompt phrases generated based on the annotation logic of architectural terminology enable the model to understand the design intent more accurately, thereby generating design drawings that better meet actual needs.

[0048] In this embodiment, it's worth noting that the core concept of the stable diffusion model is this: since each image follows a certain regular distribution, this distribution information contained in the text is used as a guide to gradually denoise a pure noise image and generate an image that matches the text information. First, the text encoder converts the input text data into a series of semantic vectors containing the text information. Then, the image information generator uses the semantic vectors as control conditions to generate a low-dimensional image. Finally, the image decoder takes the low-dimensional spatial vectors of the image information generator as input and, through dimensionality increase and amplification, produces a complete image.

[0049] The diffusion process occurs in the image information generator. The initial pure noise latent variable is input into the U-net network and then combined with the semantic control vector. Repeated iterations are performed to continuously remove noise from the pure noise latent variable and continuously inject semantic information into the latent vector, resulting in a latent space vector rich in semantic information. The sampler is responsible for coordinating the entire denoising process, dynamically adjusting the U-net denoising strength at different denoising stages according to the design pattern. Therefore, the specific steps for model training in step S2 are as follows: S21. Extract text prompt words from each prompt phrase in the data set, and convert each text prompt word into a word, that is, into a token, and use a text encoder to encode each word and then convert it into a vector. In this embodiment, CLIP (Contrastive Language–Image Pretraining) can be used as a text encoder. Since the embedding vector dimension of the final output of CLIP's text branch is fixed to 768, and the hidden layer of CLIP's Transformer architecture is also usually 768, CLIP aligns image and text features through comparative learning, and the final output needs to match the encoder's 768-dimensional vector. Therefore, in this embodiment, when quickly generating architectural design drawings through the trained model, you can input text prompt words and have the CLIP text encoder generate the corresponding 768 feature vector.

[0050] S22. Encode each edge detail image in the dataset through a variational auto encoder (VAE), and convert the encoded edge detail image from image data in pixel space to tensor data in latent space.

[0051] S23. The tensor data is repeatedly downsampled and amplified in the latent space to obtain a U-shaped network image generation model. The latent space is a noise iteration process. During training, normally distributed noise is continuously added to the original image to obtain the final noise matrix.

[0052] S24. Use LoRA fine-tuning technology to adjust the weights in the U-shaped network image generation model training process to obtain the final building image generation model.

[0053] That is, through the vectorization of text prompt words and the latent space transformation of edge detail images, combined with the learning ability of the U-shaped network, the model can efficiently capture the characteristics of architectural design; at the same time, the LoRA fine-tuning technology is used to optimize the weights, which significantly improves the training efficiency and adaptability of the model, making it more stable and accurate when generating high-quality images in the architectural field.

[0054] The specific steps of using LoRA fine-tuning technology to adjust the weights in the U-type network training process in step S24 are as follows: First, given a pre-training matrix , R represents the set of real numbers, m and n Respectively The number of rows and columns.

[0055] Then the pre-training matrix Define a suitable low rank r ,in d is the original dimension of the matrix and r << d .

[0056] Then the pre-trained matrix Updated to The low-rank decomposition representation of A and B are two low-rank matrices, and , , express BA The multiplication result is the changing term.

[0057] Then the pre-training matrix Perform singular value decomposition to obtain mThe low-rank matrix of singular values is decomposed into: ; in, U For one m × m The orthogonal matrix, Σ For one m × n A diagonal matrix of , indicating that the elements on the diagonal are singular values and the other elements are 0, For one n × n The transpose of the orthogonal matrix .

[0058] Then define is the pre-trained matrix The singular value matrix of r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix , the main matrix Together with the residual matrix, it forms the initialization matrix for fine-tuning weights.

[0059] In this embodiment, the singular values at the front of the matrix are larger, corresponding to the main part, and the singular values at the back are smaller, corresponding to the details. Adjusting only the principal components with large singular values may not necessarily achieve the best results. It is also necessary to pay attention to those features that were not emphasized in pre-training but are important in the new task. Therefore, two adjustments at different positions are used. r singular values, that is, select the r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix .

[0060] Then initialize two low-rank matrices according to the weights of the U-shaped network image generation model A and B , compared with traditional methods A Use Gaussian initialization to initialize randomly and B Use zero initialization, which has higher performance in initialization tasks, faster convergence during fine-tuning, and better retention of knowledge in the original pre-trained model. A and B The initialization formula is as follows: ; ; in and Respectively represent the selectedr The singular values corresponding to U Submatrix and submatrix, Therefore r A diagonal matrix with singular values on the diagonal.

[0061] During training, is frozen and does not receive gradient updates, while A and B Is the weight containing fine-tuning, which represents the difference to be added to the original weight. Finally, during inference, the fine-tuned weight is combined with the original pre-trained weight. For the input of the U-network image generation model x The modified forward pass is the final building image generation model, and the final building image generation model is as follows: ; in is the residual matrix obtained by selecting a smaller singular value, gamma is the adjustment parameter and gamma The value is 0-0.1.

[0062] That is, through singular value decomposition and low-rank matrix initialization, LoRA fine-tuning technology can quickly adapt to new task requirements while retaining the knowledge of the pre-trained model; at the same time, the combination of the main matrix and the residual matrix enhances the expressive ability of the model, enabling it to better handle the complex features of architectural design and generate more detailed and realistic images.

[0063] Meanwhile, in this embodiment, step S4 is specifically as follows: S41. Input the low-resolution (LR) architectural design drawing into a shallow feature extraction network, which outputs shallow features of the architectural design drawing. The shallow feature extraction network includes a convolutional layer and a linear rectifier function. The shallow feature extraction network is primarily used to extract edge information and complete the mapping from spatial dimensions to feature dimensions. The activation function uses a Leaky Relu function to avoid losing data less than 0.

[0064] S42. The output shallow features are input into the deep feature extraction network, and the feature information is passed into multiple basic units. The residual blocks and dense blocks are used for selection and design. Here, in order to better extract feature information, a feature enhancement module is introduced, and multiple feature-enhanced residual dense blocks (RRDB) are used to extract the deep features of the architectural design drawings.

[0065] S43. The extracted deep features are input into the reconstruction module, and a rendered and enhanced high-resolution (SR) architectural design image is generated through convolution and upsampling operations.

[0066] That is, by combining shallow and deep feature extraction networks, introducing feature enhancement modules and multi-scale feature fusion, the super-resolution reconstruction capability of images is significantly improved, ensuring that the generated high-resolution architectural design drawings are clearer and more realistic in terms of texture, materials and complex patterns, meeting the needs of high-quality visual display.

[0067] In addition, in this embodiment, the specific steps of using the introduced feature enhancement module are as follows: For the original stable diffusion model, information will be lost as the number of network layers increases. To solve this problem, a feature fusion module is introduced. Starting from the second layer of the network, before performing convolution operations on each layer of the network, the feature map of the previous layer is added to the feature map of this layer, thereby partially retaining the information of the previous layer and reducing information loss.

[0068] Then, multi-scale feature extraction and fusion are introduced into the residual dense block of the stable diffusion model, which enables the network to extract feature information of different receptive fields and perform residual calculation, thereby improving its reconstruction performance of high-frequency image information.

[0069] Then, the feature fusion structure with different receptive fields is obtained, which consists of three parallel branches. First, the feature map output from the previous layer is processed with 1×1, 3×3 and 5×5 convolution kernels respectively, and then a 3×3 void convolution is performed with expansion rates of 1, 3 and 5 respectively, so that different receptive fields can be obtained, and multiple branches represent features of different scales.

[0070] Finally, all feature maps of different scales are connected through addition operations, compressed by a 1×1 convolution kernel, and added to the feature maps from the previous layer, and the fused and enhanced features are used as the output of this layer.

[0071] Please refer to Picture 3 , the third embodiment of the present invention is: A building design drawing generation system based on a stable diffusion model, such as Picture 3 Shown, including: The sample construction module is used to obtain image materials of different building facades, filter and label the image materials, and construct a dataset for subsequent model training.

[0072] The model building and training module is used to train and fine-tune the dataset using a stable diffusion model guided by Control-Net to obtain an architectural image generation model suitable for the architectural field.

[0073] Control-Net is a tool for controlling and monitoring large-scale computer networks. It helps administrators monitor network health and ensure efficient management of network resources. Therefore, in this embodiment, Control-Net is used to assist the stable diffusion model in generating images. By controlling Control-Net, the stable diffusion model can be better adjusted to achieve the desired results.

[0074] The image generation module is used to input the sketch and prompt words into the architectural image generation model, generate a real scene image, and output the architectural design drawing, thereby realizing the rapid conversion of the design sketch into the architectural design drawing.

[0075] The model rendering module is used to render and enhance architectural design drawings using an image super-resolution algorithm based on a generative adversarial network, thereby producing clearer and more detailed visual effects and displaying design images of textures, materials, and complex patterns with high fidelity.

[0076] The key idea behind the generative adversarial network-based image super-resolution algorithm, ESRGAN (Enhanced Super-Resolution Generative Adversarial Network), is to improve image resolution by learning the mapping between low-resolution (LR) images and their high-resolution (HR) counterparts. ESRGAN combines a U-net architecture with a residual structure, enhancing the super-resolution performance of the generator through a confrontation between a generator network and a discriminator network. The generator is responsible for feature extraction and reconstruction, while the discriminator assesses the authenticity of the generated image.

[0077] That is, in this embodiment, in order to address the problem that traditional architectural effects mainly rely on hand-drawing and model making, which is not only labor-intensive but also has great limitations in the expression of details and the presentation of complex scenes, by obtaining building facade image materials and constructing a data set, combined with the stable diffusion model training guided by Control-Net, an architectural image generation model with stable prediction results and applicable to the architectural field is obtained, so as to achieve rapid generation from sketches to architectural design drawings. At the same time, a feature-enhanced generative adversarial network is introduced to render and enhance the generated architectural design drawings, which significantly improves the clarity and detail expression of the image, can quickly output high-fidelity visualization renderings, and improves design efficiency and quality.

[0078] The fourth embodiment of the present invention is: A system for generating architectural design drawings based on a stable diffusion model. Based on the above third embodiment, in this embodiment, the sample construction module is specifically used to: Obtain image materials of different building facades, scale the image materials to within a preset range, and delete foreground materials in the scaled image materials that obstruct the main building and whose obstruction area exceeds the preset area.

[0079] Crop the non-building background footage from the image material with the foreground material deleted, and retain the main part of the building.

[0080] Filter multiple materials whose similarity reaches a first threshold among the cropped image materials and randomly retain one material to avoid the waste of computing resources caused by data redundancy.

[0081] The Canny and other edge extraction operator algorithms are used to extract the edges of the filtered image materials and generate edge detail images of the filtered image materials.

[0082] Based on the design of the annotation logic database of architectural terminology, the edge detail images are annotated, prompt phrases related to the filtered image materials are generated, and a dataset including edge detail images and corresponding prompt phrases is constructed.

[0083] That is, in this embodiment, by scaling, cropping, filtering and edge extraction of image materials, the diversity and quality of the data set are ensured, and the waste of computing resources due to redundant data is avoided; at the same time, the prompt phrases generated based on the annotation logic of architectural terminology enable the model to understand the design intent more accurately, thereby generating design drawings that better meet actual needs.

[0084] In this embodiment, it's worth noting that the core concept of the stable diffusion model is this: since each image follows a certain regular distribution, this distribution information contained in the text is used as a guide to gradually denoise a pure noise image and generate an image that matches the text information. First, the text encoder converts the input text data into a series of semantic vectors containing the text information. Then, the image information generator uses the semantic vectors as control conditions to generate a low-dimensional image. Finally, the image decoder takes the low-dimensional spatial vectors of the image information generator as input and, through dimensionality increase and amplification, produces a complete image.

[0085] The diffusion process occurs in the image information generator. After the initial pure noise latent variable is input into the U-net network and combined with the semantic control vector, repeated iterations are performed to continuously remove noise from the pure noise latent variable and continuously inject semantic information into the latent vector, resulting in a latent space vector rich in semantic information. The sampler is responsible for coordinating the entire denoising process, dynamically adjusting the U-net denoising strength at different denoising stages according to the design pattern. Therefore, the model construction and training module is specifically used to: Text prompt words are extracted for each prompt phrase in the dataset, and each text prompt word is converted into a word, that is, into a token. A text encoder is used to encode each word and then convert it into a vector. In this embodiment, CLIP (Contrastive Language–Image Pretraining) can be used as a text encoder. Since the embedding vector dimension of the final output of CLIP's text branch is fixed to 768, and the hidden layer of CLIP's Transformer architecture is also usually 768, CLIP aligns image and text features through comparative learning, and the final output needs to match the encoder's 768-dimensional vector. Therefore, in this embodiment, when quickly generating architectural design drawings through the trained model, the CLIP text encoder can generate the corresponding 768 feature vector by inputting text prompt words.

[0086] Each edge detail image in the dataset is encoded by a variational autoencoder (VAE), which converts the encoded edge detail image from image data in pixel space to tensor data in latent space.

[0087] The U-shaped network image generation model is constructed by repeatedly downsampling and upscaling the tensor data in the latent space. The latent space is the process of noise iteration. During training, normally distributed noise is continuously added to the original image to obtain the final noise matrix.

[0088] The LoRA fine-tuning technology is used to adjust the weights in the training process of the U-shaped network image generation model to obtain the final building image generation model.

[0089] That is, through the vectorization of text prompt words and the latent space transformation of edge detail images, combined with the learning ability of the U-shaped network, the model can efficiently capture the characteristics of architectural design; at the same time, the LoRA fine-tuning technology is used to optimize the weights, which significantly improves the training efficiency and adaptability of the model, making it more stable and accurate when generating high-quality images in the architectural field.

[0090] Among them, LoRA fine-tuning technology is used to adjust the weights in the U-shaped network training process to obtain the final building image generation model, specifically: First, given a pre-training matrix , R represents the set of real numbers, m and n Respectively The number of rows and columns.

[0091] Then the pre-training matrix Define a suitable low rankr ,in d is the original dimension of the matrix and r << d .

[0092] Then the pre-trained matrix Updated to The low-rank decomposition representation of A and B are two low-rank matrices, and , , express BA The multiplication result is the changing term.

[0093] Then the pre-training matrix Perform singular value decomposition to obtain m The low-rank matrix of singular values is decomposed into: ; in, U For one m × m The orthogonal matrix, Σ For one m × n A diagonal matrix of , indicating that the elements on the diagonal are singular values and the other elements are 0, For one n × n The transpose of the orthogonal matrix .

[0094] Then define is the pre-trained matrix The singular value matrix of r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix , the main matrix Together with the residual matrix, it forms the initialization matrix for fine-tuning weights.

[0095] In this embodiment, the singular values at the front of the matrix are larger, corresponding to the main part, and the singular values at the back are smaller, corresponding to the details. Adjusting only the principal components with large singular values may not necessarily achieve the best results. It is also necessary to pay attention to those features that were not emphasized in pre-training but are important in the new task. Therefore, two adjustments at different positions are used. r singular values, that is, select the r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix .

[0096] Then initialize two low-rank matrices according to the weights of the U-shaped network U-shaped network image generation model A and B , compared with traditional methods A Use Gaussian initialization to initialize randomly and B Use zero initialization, which has higher performance in initialization tasks, faster convergence during fine-tuning, and better retention of knowledge in the original pre-trained model. A and B The initialization formula is as follows: ; ; in and Respectively represent the selected r The singular values corresponding to U Submatrix and submatrix, Therefore r A diagonal matrix with singular values on the diagonal.

[0097] During training, is frozen and does not receive gradient updates, while A and B Is the weight containing fine-tuning, which represents the difference to be added to the original weight. Finally, during inference, the fine-tuned weight is combined with the original pre-trained weight. Then for the input of the U-network U-network image generation model x The modified forward pass is the final building image generation model, and the final building image generation model is as follows: ; in is the residual matrix obtained by selecting a smaller singular value, gamma is the adjustment parameter and gamma The value is 0-0.1.

[0098] That is, through singular value decomposition and low-rank matrix initialization, LoRA fine-tuning technology can quickly adapt to new task requirements while retaining the knowledge of the pre-trained model; at the same time, the combination of the main matrix and the residual matrix enhances the expressive ability of the model, enabling it to better handle the complex features of architectural design and generate more detailed and realistic images.

[0099] At the same time, in this embodiment, the model rendering module is specifically used to: The low-resolution (LR) architectural design image is fed into a shallow feature extraction network, which outputs shallow features of the architectural design image. The shallow feature extraction network consists of convolutional layers and linear rectification functions. The shallow feature extraction network primarily extracts edge information and maps spatial dimensions to feature dimensions. The activation function uses the Leaky Relu (Leaky ReLU) linear rectification function to avoid missing data less than 0.

[0100] The output shallow features are input into the deep feature extraction network, and the feature information is passed into multiple basic units. Residual blocks and dense blocks are used for selection and design. In order to better extract feature information, a feature enhancement module is introduced here, and multiple feature-enhanced residual dense blocks (RRDB) are used to extract deep features of architectural design drawings.

[0101] The extracted deep features are input into the reconstruction module, and after convolution and upsampling operations, a rendered and enhanced high-resolution (SR) architectural design image is generated.

[0102] That is, by combining shallow and deep feature extraction networks, introducing feature enhancement modules and multi-scale feature fusion, the super-resolution reconstruction capability of images is significantly improved, ensuring that the generated high-resolution architectural design drawings are clearer and more realistic in terms of texture, materials and complex patterns, meeting the needs of high-quality visual display.

[0103] In addition, in this embodiment, the specific steps of using the introduced feature enhancement module are as follows: For the original stable diffusion model, information will be lost as the number of network layers increases. To solve this problem, a feature fusion module is introduced. Starting from the second layer of the network, before performing convolution operations on each layer of the network, the feature map of the previous layer is added to the feature map of this layer, thereby partially retaining the information of the previous layer and reducing information loss.

[0104] Then, multi-scale feature extraction and fusion are introduced into the residual dense block of the stable diffusion model, which enables the network to extract feature information of different receptive fields and perform residual calculation, thereby improving its reconstruction performance of high-frequency image information.

[0105] Then, the feature fusion structure with different receptive fields is obtained, which consists of three parallel branches. First, the feature map output from the previous layer is processed with 1×1, 3×3 and 5×5 convolution kernels respectively, and then a 3×3 void convolution is performed with expansion rates of 1, 3 and 5 respectively, so that different receptive fields can be obtained, and multiple branches represent features of different scales.

[0106] Finally, all feature maps of different scales are connected through addition operations, compressed by a 1×1 convolution kernel, and added to the feature maps from the previous layer, and the fused and enhanced features are used as the output of this layer.

[0107] In summary, the present invention provides a method and system for generating architectural design drawings based on a stable diffusion model, which has the following beneficial effects: 1. Realize the rapid generation from sketch to architectural design drawing.

[0108] 2. Significantly improve image clarity and detail.

[0109] 3. Improved architectural design efficiency and quality.

[0110] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for generating architectural design drawings based on a stable diffusion model, characterized in that: Including steps: S1. Obtain image materials of different building facades, and screen and label the image materials to obtain a data set; S2. Using a Control-Net-guided stable diffusion model to train and fine-tune the dataset to obtain a building image generation model; S3, inputting the sketch and prompt words into the building image generation model to output the building design drawing; S4. Rendering and enhancing the architectural design drawing using an image super-resolution algorithm based on a generative adversarial network.

2. The method for generating architectural design drawings based on a stable diffusion model according to claim 1, characterized in that: The step S1 is specifically as follows: S11, acquiring image materials of different building facades, scaling the image materials to within a preset range, and deleting foreground materials in the scaled image materials that obstruct the main building and whose obstruction area exceeds a preset area; S12, cropping the non-building background image material from the image material from which the foreground material has been deleted; S13, screening multiple materials whose similarity reaches a first threshold among the cropped image materials and randomly retaining one material; S14, using an edge extraction operator algorithm to extract edges from the filtered image material to generate an edge detail image of the filtered image material; S15. Designing a labeling logic database based on architectural terminology, labeling the edge detail image, generating prompt phrases related to the filtered image material, and constructing a data set including the edge detail image and the corresponding prompt phrases.

3. The method for generating architectural design drawings based on a stable diffusion model according to claim 2, characterized in that: The step S2 is specifically as follows: S21, extracting text prompt words from each prompt phrase in the data set, converting each text prompt word into a word, and encoding each word and converting it into a vector; S22. Encode each edge detail image in the data set using a variational autoencoder, and convert the encoded edge detail image from image data in pixel space to tensor data in latent space; S23, restoring the tensor data by multiple downsampling and amplification in the hidden layer space to obtain a U-shaped network image generation model; S24. Use LoRA fine-tuning technology to adjust the weights of the U-shaped network image generation model during training to obtain a building image generation model.

4. The method for generating architectural design drawings based on a stable diffusion model according to claim 3, characterized in that: The step S24 is specifically as follows: Given a pre-trained matrix , R represents the set of real numbers, m and n Respectively The number of rows and columns; For the pre-training matrix Define a low-rank r ,in d is the original dimension of the matrix and r << d ; The pre-training matrix Updated to The low-rank decomposition representation of , , express BA The multiplication result of For the pre-training matrix Perform singular value decomposition to obtain m The low-rank matrix of singular values is decomposed into: ； in, U For one m × m An orthogonal matrix, Σ is a m × n The diagonal matrix of For one n × n The transpose of the orthogonal matrix; definition For the pre-training matrix The singular value matrix of r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix , the main matrix Together with the residual matrix, it forms the initialization matrix of the fine-tuning weight; Initialize two low-rank matrices according to the weights of the U-shaped network image generation model A and B : ； ； in and Respectively represent the selected r The singular values corresponding to U Submatrix and submatrix, Therefore r A diagonal matrix with singular values on the diagonal, A and B Is the weight containing fine-tuning, indicating the difference to be added to the original weight, the fine-tuned weight is combined with the original pre-trained weight, then for the input of the U-network image generation model x The modified forward pass is the final building image generation model, which is as follows: ； in is the residual matrix obtained by selecting a smaller singular value, γ is the adjustment parameter and γ The value is 0-0.

1.

5. The method for generating architectural design drawings based on a stable diffusion model according to claim 1, characterized in that: The step S4 is specifically as follows: S41, inputting the architectural design drawing into a shallow feature extraction network, and outputting shallow features of the architectural design drawing, wherein the shallow feature extraction network includes a convolutional layer and a linear rectification function; S42, inputting the shallow features into a deep feature extraction network, introducing a feature enhancement module, and using multiple feature-enhanced residual dense blocks to extract deep features of the architectural design drawing; S43. The extracted deep features are input into the reconstruction module, and a rendered and enhanced architectural design image is generated through convolution and upsampling operations.

6. A system for generating architectural design drawings based on a stable diffusion model, characterized in that: include: A sample construction module is used to obtain image materials of different building facades, and to screen and label the image materials to obtain a data set; A model building and training module is used to train and fine-tune the dataset using a stable diffusion model guided by Control-Net to obtain a building image generation model; An image generation module, configured to input a sketch and prompt words into the building image generation model and output a building design drawing; The model rendering module is used to render and enhance the architectural design drawing by using an image super-resolution algorithm based on a generative adversarial network.

7. The architectural design drawing generation system based on the stable diffusion model according to claim 6 is characterized in that: The sample construction module is specifically used for: Obtain image materials of different building facades, scale the image materials to within a preset range, and delete foreground materials in the scaled image materials that obstruct the main building and whose obstruction area exceeds a preset area; Crop the non-building background footage from the image footage after deleting the foreground footage; Filter multiple materials whose similarity reaches a first threshold among the cropped image materials and randomly retain one material; An edge extraction operator algorithm is used to extract edges of the filtered image material to generate an edge detail image of the filtered image material; Based on a design annotation logic database of architectural terminology, the edge detail images are annotated, prompt phrases related to the filtered image materials are generated, and a data set including the edge detail images and the corresponding prompt phrases is constructed.

8. The architectural design drawing generation system based on the stable diffusion model according to claim 7 is characterized in that: The model building and training module is specifically used to: Extracting text prompt words from each prompt phrase in the data set, converting each text prompt word into a word, and encoding each word and converting it into a vector; Encoding each edge detail image in the data set through a variational autoencoder, and converting the encoded edge detail image from image data in a pixel space to tensor data in a latent space; The tensor data is downsampled and amplified multiple times to restore in the hidden layer space to obtain a U-shaped network image generation model; The LoRA fine-tuning technology is used to adjust the weights in the training process of the U-shaped network image generation model to obtain the building image generation model.

9. The architectural design drawing generation system based on the stable diffusion model according to claim 8 is characterized in that: The LoRA fine-tuning technology is used to adjust the weights in the U-shaped network training process to obtain the final building image generation model, specifically: Given a pre-trained matrix , R represents the set of real numbers, m and n Respectively The number of rows and columns; For the pre-training matrix Define a low-rank r ,in d is the original dimension of the matrix and r << d ; The pre-training matrix Updated to The low-rank decomposition representation of , , express BA The multiplication result of The pre-trained matrix Perform singular value decomposition to obtain m The low-rank matrix of singular values is decomposed into: ； in, U For one m × m The orthogonal matrix, Σ For one m × n The diagonal matrix of For one n × n The transpose of the orthogonal matrix; definition For the pre-training matrix The singular value matrix of r The matrix corresponding to the singular values is used as the main matrix , and then randomly select from the remaining singular values r The matrix corresponding to the singular values is used as the residual matrix , the main matrix Together with the residual matrix, it forms the initialization matrix of the fine-tuning weight; Initialize two low-rank matrices according to the weights of the U-shaped network image generation model A and B : ； ； in and Respectively represent the selected r The singular values corresponding to U Submatrix and submatrix, Therefore r A diagonal matrix with singular values on the diagonal, A and B Is the weight containing fine-tuning, indicating the difference to be added to the original weight, the fine-tuned weight is combined with the original pre-trained weight, then for the input of the U-network image generation model x The modified forward pass is the final building image generation model, which is as follows: ； in is the residual matrix obtained by selecting a smaller singular value, γ is the adjustment parameter and γ The value is 0-0.

1.

10. The architectural design drawing generation system based on the stable diffusion model according to claim 6, characterized in that: The model rendering module is specifically used for: Inputting the architectural design drawing into a shallow feature extraction network and outputting shallow features of the architectural design drawing, wherein the shallow feature extraction network includes a convolutional layer and a linear rectification function; Inputting the shallow features into a deep feature extraction network, introducing a feature enhancement module, and using multiple feature-enhanced residual dense blocks to extract deep features of the architectural design drawing; The extracted deep features are input into the reconstruction module, and after convolution and upsampling operations, rendered and enhanced architectural design images are generated.

Images