A building structure effect picture generation method and system for architectural design

Abstract

The application relates to the technical field of image data processing, in particular to a building structure effect picture generation method and system for building design, which comprises the following steps: obtaining a historical effect picture and a current initial effect picture of a building structure, determining a reference effect picture consistent with the current initial effect picture in the historical effect picture; determining a target prompt picture in the reference effect picture; determining the possibility of the same prompt word between the connected domains of any two target prompt pictures by using the image features corresponding to the connected domains of the target prompt pictures; clustering the connected domains of the target prompt picture to obtain a connected domain cluster; and determining a picture generation area block corresponding to the target prompt word; inputting the current initial effect picture and the corresponding current prompt word into a retrained model to obtain a current final effect picture. The technical scheme can effectively improve the control precision and accuracy of the building structure effect picture generation.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, specifically to a method and system for generating architectural structural renderings for architectural design. Background Technology

[0002] In the construction industry, after completing the structural design, designers often generate a rendering to assess the rationality of the structural system. These renderings visually demonstrate the form, component relationships, stress logic, and even the overall harmony of the building design. In recent years, with the development of AI technology, generating these structural renderings has become increasingly simple and rapid. Various large-scale models, learning from massive amounts of architectural design and rendering data, and relying on the advancements in machine learning, now offer platforms for rapidly generating structural renderings using technologies such as stable diffusion (SD).

[0003] Existing architectural design tools (SDs) typically generate basic architectural structural renderings using keywords or positive / negative prompts provided by the designer, along with a base model. They then use LoRa and ControlNet models to fine-tune and optimize these renderings. ControlNet first divides the rendering into blocks for control, while the LoRa model, controlled by prompts, manages the style and details. However, existing ControlNet models rely solely on superficial local image features for block division, neglecting the designer's overall layout considerations. This results in image block divisions that fail to align with the designer's intended design, leading to a final architectural rendering that deviates from the designer's vision. Summary of the Invention

[0004] To address the technical problem that ControlNet's segmentation control of basic architectural structure renderings does not conform to the designer's original intentions, resulting in inaccurate and unsatisfactory final architectural renderings, the present invention aims to provide a method and system for generating architectural structure renderings for architectural design. The specific technical solution adopted is as follows:

[0005] This invention provides a method for generating architectural structural renderings for architectural design, the method being applied to a stable diffusion model, comprising:

[0006] Obtain historical renderings and current initial renderings of the building structure generated based on user input, compare the settings of the two renderings, and determine a reference rendering in the historical renderings that has the same style as the current initial rendering.

[0007] Identify target cue images that all contain target cue words in the reference effect images, and use the image features corresponding to the connected components of any two target cue images to determine the probability of the same cue word between their respective connected components.

[0008] Clustering of connected components in the target prompt graph based on the probability of the same prompt word yields clusters of connected components representing similar building structures. These clusters are then used to determine the regions of the raw image corresponding to the target prompt word.

[0009] By dividing each raw image region into blocks, a retrained ControlNet model is obtained. The current initial image and its corresponding current prompt word are then input into the ControlNet model to obtain the current final image.

[0010] Furthermore, the step of comparing the settings of the two raw images to determine a reference image in the historical rendering that has the same style as the current initial rendering includes:

[0011] By utilizing the similarity between historical renderings and the current initial renderings at any two constraint points during generation, the style similarity between historical renderings and the current initial renderings can be determined.

[0012] Historical renderings with a style similarity greater than a preset style similarity threshold are used as reference renderings that are consistent with the style of the current initial rendering.

[0013] The limiting points include prompt words and working nodes in the stable diffusion model.

[0014] Furthermore, determining the style similarity between the historical rendering and the current initial rendering by utilizing the similarity between any two constraint points during generation includes:

[0015] Determine the reference constraint point with the highest cosine similarity to the target constraint point in the historical rendering and the current initial rendering, and obtain the cosine similarity between the target constraint point and the reference constraint point;

[0016] By utilizing the cosine similarity between the target constraint point and the reference constraint point, the style similarity between the historical rendering and the current initial rendering is determined.

[0017] Furthermore, the step of determining the probability of identical prompt words between connected components of any two target prompt graphs using image features corresponding to their respective connected components includes:

[0018] Determine the shape similarity between the shape features corresponding to any connected component of any two target cue graphs, and use the shape similarity to determine the probability of the same cue word between the corresponding connected components.

[0019] Furthermore, the step of determining the probability of identical prompt words between connected components of any two target prompt graphs using image features corresponding to their respective connected components includes:

[0020] Determine the texture similarity between texture features corresponding to any connected component of any two target cue graphs;

[0021] The probability of identical prompts between corresponding connected components is determined by using shape similarity and texture feature similarity.

[0022] Furthermore, the step of clustering connected components of the target cue graph using the probability of the same cue word to obtain clusters of connected components representing similar building structures includes:

[0023] Determine the style similarity between any two target cue images, and combine the probability of the same cue word with its corresponding style similarity to determine the distance information between the corresponding connected components;

[0024] Clustering of connected components using distance information between them yields clusters of connected components that represent similar building structures.

[0025] Furthermore, the step of determining the raw image region segmentation corresponding to the target prompt word using connected component clustering includes:

[0026] By using the ratio between the number of target cue graphs corresponding to any connected component cluster and the total number of target cue graphs, the probability that the image features of the corresponding connected component cluster are target cue word image features can be obtained.

[0027] The cluster of connected components corresponding to the highest probability of the prompt feature is taken as the cluster of raw image regions corresponding to the target prompt word, and the connected components in the raw image region cluster are taken as blocks of raw image regions corresponding to the target prompt word.

[0028] Furthermore, the step of using the connected components in the cluster of raw image regions as blocks of raw image regions corresponding to the target prompt words includes:

[0029] By utilizing the number of target hint graphs corresponding to each of any two connected component clusters, we can determine the common kinship differences between the two connected component clusters and identify the common kinship clusters of any connected component cluster.

[0030] The raw image region clusters and the connected components in their respective clusters are used as raw image region blocks corresponding to the target prompt words.

[0031] Furthermore, the step of obtaining a retrained ControlNet model by dividing each raw image region into blocks includes:

[0032] Each raw image region is divided into blocks and labeled according to its corresponding prompt words to obtain a semantic segmentation mask image;

[0033] The semantic segmentation mask image is used as training data for the ControlNet model to obtain a retrained ControlNet model.

[0034] This invention also provides a system for generating architectural structural renderings for architectural design, the system being used to implement the method for generating architectural structural renderings for architectural design as described in any of the preceding claims; the system includes:

[0035] The image filtering module is used to obtain historical renderings and current initial renderings of the building structure generated based on user input, compare the settings of the two images, and determine a reference rendering in the historical renderings that has the same style as the current initial rendering.

[0036] The image segmentation module is used to determine target prompt images that contain target prompt words in the reference effect images. It uses the image features corresponding to the connected components of any two target prompt images to determine the probability of the same prompt word between their respective connected components. It uses the probability of the same prompt word to cluster the connected components of the target prompt images to obtain connected component clusters that represent similar building structures. It uses the connected component clusters to determine the raw image region segment corresponding to the target prompt word. It uses each raw image region segment to obtain a retrained ControlNet model. It inputs the current initial effect image and its corresponding current prompt word into the ControlNet model to obtain the current final effect image.

[0037] The present invention has the following beneficial effects:

[0038] Existing image generation block strategies often simply divide the rendered image into connected component blocks using edge detection, ignoring the overall integrity of the architectural structure description. This can easily lead to over-adjustment and deviation from the intended adjustment during fine-tuning, making it difficult to correct according to the designer's description. As a result, the generated final architectural structure rendering often deviates from the designer's original design concept, meaning that the final rendering is neither accurate nor ideal.

[0039] This invention relies on the SD model (Stable Diffusion Model, or platform) to obtain generation information related to architectural design, involving current and historical architectural designs. Based on the designer's (user's) current prompt, it generates a rough initial rendering of the current architectural design. Combining this with historical renderings generated by the designer and their corresponding prompts, it compares the stylistic similarity between the current initial rendering and historical renderings. Then, it filters and categorizes historical renderings using any prompt (target prompt). Finally, it constructs the corresponding image regions for each prompt under different prompts using connected component clustering. Finally, it segments and labels the image regions corresponding to each prompt. Note that the ControlNet model is optimized and trained. The retrained ControlNet model, combined with the current initial rendering and corresponding prompts, generates the current corresponding area blocks. Since these area blocks are accurately correlated with the prompts, they fully consider and relate to the overall layout features. Based on these more accurate area blocks that align with the designer's intentions, the designer can further refine and adjust them to obtain the final architectural rendering. This effectively improves the precision and accuracy of the control over the generation of architectural renderings, as well as the overall control over the generated renderings, thus better aligning with the designer's actual design concept and generating the ideal rendering. Attached Figure Description

[0040] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating the steps of a method for generating architectural structural renderings for architectural design, as provided in an embodiment of the present invention.

[0042] Figure 2 This is a detailed flowchart of step S1 in a method for generating architectural structural renderings for architectural design, provided in an embodiment of the present invention.

[0043] Figure 3 This is a detailed flowchart of step S2 in a method for generating architectural structural renderings for architectural design, provided in an embodiment of the present invention.

[0044] Figure 4 This is a detailed flowchart of step S3 in a method for generating architectural structural renderings for architectural design, provided in an embodiment of the present invention.

[0045] Figure 5A detailed flowchart of step S3 in a method for generating architectural structural renderings for architectural design, provided in another embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the hardware operating environment of the architectural structural rendering generation device for architectural design involved in the embodiments of the present invention;

[0047] Figure 7 This is a schematic diagram of the framework structure of the architectural structure rendering generation system for architectural design involved in the embodiments of the present invention. Detailed Implementation

[0048] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method for generating architectural structural renderings for architectural design based on the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0050] To facilitate understanding of the following embodiments of the present invention, it is necessary to provide a related explanation of the overall technical concept and principles of the present invention:

[0051] When designers use SD to generate architectural structural renderings, they often first summarize the keywords and stylistic features of their designs (collectively referred to as design feature parameters). Then, based on these design feature parameters, such as positive characteristic words (positive prompts) and stylistic features, they select appropriate base models and other auxiliary raw model files. Simultaneously, they need to use ControlNet to finely control various details. Each designer has their own habits in designing architectural structures, resulting in their designs always bearing their own stylistic characteristics. This means that the fine control, style, and detail descriptions they employ when conceiving the entire rendering can be seen in their previous designs. Therefore, by studying the designer's historical portfolio, their design habits can be learned, thus assisting in the adaptive segmentation of the design renderings.

[0052] The following describes in detail, with reference to the accompanying drawings, a specific scheme for generating architectural structural renderings for architectural design provided by the present invention.

[0053] Example 1:

[0054] For the method for generating architectural structural renderings for architectural design provided by this invention, please refer to [link / reference]. Figure 1 The diagram illustrates a flowchart of a method for generating architectural structural renderings for architectural design, provided by an embodiment of the present invention.

[0055] The method for generating architectural structural renderings for architectural design is applied to a stable diffusion model, including:

[0056] Step S1: Obtain the historical renderings and the current initial renderings of the building structure generated based on user input, and compare the settings of the two to determine the reference rendering in the historical renderings that has the same style as the current initial renderings;

[0057] First, collect relevant data from designers (users) when they generated architectural renderings (hereinafter referred to as "architectural renderings" or "renderings," which correspond to historical renderings in the past): such as the detailed text descriptions of architectural renderings entered by designers and the corresponding architectural renderings, as well as relevant data when they generated architectural renderings using SD (Stable Diffusion);

[0058] The SD-related data includes corresponding prompts and data for each node, such as ControlNet control data (edge ​​map, depth map, normal map and other auxiliary control information), LoRA (Low-Rank Adaptation) model training data (specialized training datasets for specific architectural styles or structural types), and other related data.

[0059] At the same time, collect various professional drawings corresponding to each architectural rendering: such as architectural design, interior design, landscape design, architectural planning design and other professional design drawings, as well as possible BIM (Building Information Modeling) model data;

[0060] In addition, data related to the generation of architectural renderings from other designers can be collected. Thus, a number of architectural rendering generation data have been collected.

[0061] When designing a building structure, designers often first create a rough draft, and then describe the design based on this draft, producing corresponding descriptive statements. These descriptive statements provide relatively accurate descriptions of various parts of the building structure. Therefore, SD (Structured Rendering) can generate a preliminary rendering using these descriptions and related settings. Furthermore:

[0062] You can first use a Large Language Model (LLM) to extract keywords from the description input by the designer and use them as positive and negative prompts for the SD model's generated images.

[0063] After SD obtains the positive and negative prompts from the designer, the base model will generate an initial rendering (called the current initial rendering) based on the corresponding variable parameters and these prompts.

[0064] Alternatively, the designer can provide a sketch as a base image, and SD can use this base image and the designer's positive and negative prompts to generate a new image, thereby obtaining the current initial rendering.

[0065] In addition, the historical renderings can also be obtained by first obtaining the initial renderings in the above manner, and then by fine-tuning and optimizing them using the ControlNet model and LoRa model to obtain the final renderings.

[0066] Specifically, please refer to Figure 2 Step S1, comparing the settings of the two raw images to determine a reference image in the historical rendering that has the same style as the current initial rendering, includes:

[0067] Step S11: Determine the style similarity between the historical rendering and the current initial rendering by using the similarity between any two constraint points during generation.

[0068] More specifically, step S11 includes:

[0069] Determine the reference constraint point with the highest cosine similarity to the target constraint point in the historical rendering and the current initial rendering, and obtain the cosine similarity between the target constraint point and the reference constraint point;

[0070] By utilizing the cosine similarity between the target constraint point and the reference constraint point, the style similarity between the historical rendering and the current initial rendering is determined.

[0071] Step S12: Use historical effect images with a style similarity greater than a preset style similarity threshold as reference effect images with a style consistent with the current initial effect image; wherein, the limiting points include prompt words and working nodes in the stable diffusion model.

[0072] In this embodiment, when generating architectural renderings, SD first generates a rough image (initial rendering) based on the base model selected by the designer and the corresponding positive and negative prompts. If the designer does not adjust the corresponding raw image settings (referring to the constraints, which are explained below), the currently generated image will be similar in overall style and general direction to previous images with the same prompts. Based on this, the basic similarity between the designer's past architectural designs and the current design can be filtered out.

[0073] First, the bag-of-words model is used to process each prompt word when the designer uses SD to generate architectural structural renderings, and the prompt word vector corresponding to each prompt word is obtained;

[0074] Then, using one parameter configuration in the working node when SD uses the bottom model to generate the image, we construct the configuration vector corresponding to that node as a parameter dimension;

[0075] Then, the normalized value of cosine similarity is used to evaluate the similarity between each prompt word or node in the two image generation processes, also known as single-constraint consistency.

[0076] Then, the similarity (which could be cosine similarity) between the two generated renders at each constraint point (each node or each prompt word) is used to derive the two renders (e.g., the current initial render). Compared with historical renderings Similarity in design style :

[0077]

[0078] In the formula, Indicates the current architectural design The total number of points (nodes or prompts) is limited when the rendering (current initial rendering) is generated; Indicates the current architectural design When generating the rendering, the first Description vector (cue vector or configuration vector) corresponding to the constraint point (target constraint point, referring to any node or prompt word); This indicates the previous architectural design. When generating the rendering (historical rendering), the first The description vector corresponding to the constraint point; Indicates the current architectural design When generating the rendering, the first Limitations and previous architectural design When generating the rendering, the first Consistency of a single constraint point (cosine similarity); This indicates that the previous architectural design was generated. Selecting from numerous constraints and current architectural design When generating the rendering, the first The cosine similarity is determined by the constraint point (reference constraint point) that has the most consistent constraint point configuration. In other words, the reference constraint point with the highest cosine similarity to the target constraint point in the historical rendering is found, and the cosine similarity between the target constraint point and the reference constraint point is determined.

[0079] It should be noted that the generated effect image is in The first in When the constraint point is a node, the corresponding The Middle The constraint points are also nodes, and the generated effect diagram is in The first in When the restriction point is a cue word, the corresponding The Middle The restriction points are also prompt words, thus enabling the calculation of cosine similarity of the same type.

[0080] A similar operation can be performed to calculate the similarity between the current architectural design and many past designs in the settings used when generating architectural renderings, that is, the style similarity between the two architectural renderings.

[0081] If the current and historical building structures have similar settings when generating renderings, their styles and structures will be largely similar. In other words, if the style similarity between the two building structures is high, then if the style similarity between the two building structure designs is greater than or equal to the preset style similarity threshold (e.g., 0.6), then the historical building structure design is recorded as a style-consistent design with the current building structure design. That is, the corresponding historical rendering and the current initial rendering are style-consistent reference renderings.

[0082] Step S2: Determine the target prompt image that contains the target prompt word in the reference effect image, and use the image features corresponding to the connected components of any two target prompt images to determine the probability of the same prompt word between their respective connected components;

[0083] Because the design styles of the two architectural renderings were similar when they were generated, their overall structure and general direction are consistent. Therefore, these two architectural renderings will also have certain similarities in various modules (each prompt word causes the rendering engine to include the corresponding feature module when generating the rendering). Thus, by analyzing numerous final renderings of many past architectural designs consistent with the current design style (refer to the renderings), we can derive the style characteristics corresponding to each prompt word. Furthermore:

[0084] First, each reference rendering designed in a style consistent with the current initial rendering is... Edge detection; morphological closing operations are performed on the detected edges to close the contour, followed by flood fill to transform the closed contour into a solid region, and finally connected component labeling is performed on the solid region; connected components corresponding to each part in each reference rendering can be labeled using a two-pass scanning algorithm.

[0085] Simultaneously, opening and closing operations are used to remove small objects and tiny structures such as small holes in the reference renderings; ultimately, numerous connected components and their corresponding edge lines are obtained from each reference rendering.

[0086] Similarly, we can obtain the connected components and edge lines corresponding to each connected component in the current initial rendering of the current building.

[0087] Specifically, step S2, which uses image features corresponding to the connected components of any two target cue graphs to determine the probability of identical cue words between their respective connected components, includes:

[0088] Determine the shape similarity between the shape features corresponding to any connected component of any two target cue graphs, and use the shape similarity to determine the probability of the same cue word between the corresponding connected components.

[0089] More specifically, please refer to Figure 3 Step S2, which uses image features corresponding to the connected components of any two target cue graphs to determine the probability of the same cue word between their respective connected components, includes:

[0090] Step S21: Determine the texture similarity between the texture features corresponding to any connected component of any two target hint graphs;

[0091] Step S22: Use shape similarity and texture feature similarity to determine the probability of the same prompt word between corresponding connected components.

[0092] In the stable diffusion model (SD) for image generation, the principle is that each cue word has a set of parameter weights. When generating an image containing that cue word, these parameter weights are used to perform calculations on the base image to obtain the image containing that cue word. This results in the generated initial image and numerous reference images containing various image regions corresponding to various cue words. Based on this, we can first analyze each connected component to obtain the image features of the parameter weights corresponding to various cue words in the generated image. Taking any connected component as an example:

[0093] First, the connected component is processed by the gray-level co-occurrence matrix to obtain the texture features corresponding to the connected component, which are used to evaluate the material information of the building structure corresponding to the connected component, and are denoted as the texture feature vector.

[0094] Next, the width, height, and rotation angle (tilt angle) of the connected region are obtained by rotating the bounding box. The aspect ratio of the connected region is approximated by the height-to-width ratio, and its roundness is calculated. The basic shape of the corresponding building structure is estimated by using the aspect ratio and roundness. The geometric center of the connected region is calculated using the centroid calculation method. The integrity of the building structure corresponding to the connected region is evaluated by the infill rate, and the slenderness of the building structure corresponding to the connected region is evaluated by the elongation rate. In addition, 7 (adjustable) nodes are constructed for the connected region. Invariant moments;

[0095] Then, using each of the above parameters (shape parameters) as a dimension, and the positive correlation normalization (such as maximum and minimum value normalization) value of each parameter as the corresponding dimension value, the shape feature vector corresponding to the connected component is constructed.

[0096] Similar operations can be used to calculate and obtain the texture feature vectors and shape feature vectors corresponding to many other connected components.

[0097] Because the reference image uses the same base model as the initial image during generation, the same generation parameter weights are applied to the same prompt word. Therefore, for the same prompt word, the final generated image portion has the same image features. Based on this, the image features corresponding to each prompt word are constructed (taking one prompt word as an example, denoted as the target prompt word):

[0098] First, from numerous reference images, extract those that all contain the target prompt word during generation, denoted as the target prompt image; then, calculate the similarity of any two connected components between target prompt images across various features:

[0099] Because SD can generate architectural renderings through subsequent... Model and The model is adjusted, and due to differences in the base image, the generated structural materials and other features differ somewhat. However, this does not significantly affect the overall architectural structure. Therefore, if the shape features of two connected components in two architectural renderings are identical, it indicates that the structures of these two parts are likely the same, and they may have been generated by the same prompt word. Of course, if the texture features of the two connected components are also similar, the likelihood of structural similarity is even higher. Based on this, the current initial rendering is constructed. The Middle Connected components and target hint graph The Middle Possibility of co-prompts between connected components :

[0100]

[0101] In the formula, This represents the current initial rendering. The Middle The shape feature vector corresponding to the connected component; Target hint image The Middle The shape feature vector corresponding to the connected component; Used to calculate the cosine similarity between two vectors, including shape similarity and texture similarity; Used for positive correlation normalization (such as maximum and minimum value normalization, with a value range of [0,1]); This represents the current initial rendering. The Middle Texture feature vectors corresponding to connected components; Target hint image The Middle Texture feature vectors corresponding to connected components.

[0102] Similar operations can be used to calculate the probability of identical words between any two connected components in the current initial rendering and numerous other target cue graphs. Likewise, the probability of identical words between connected components between any two target cue graphs can be obtained.

[0103] Step S3: Cluster the connected components of the target prompt image using the probability of the same prompt word to obtain clusters of connected components that represent similar building structures; use the connected component clusters to determine the raw image region blocks corresponding to the target prompt word.

[0104] Furthermore, since the image portions generated from the same prompt word exhibit a certain degree of structural similarity, the numerous target prompt images generated from that prompt word should also share a corresponding structure. For complex prompt words, their corresponding structures may consist of multiple connected components. Therefore, if connected components with this characteristic are commonly found in numerous target prompt images for a single prompt word, it indicates that the features corresponding to these connected components all correspond to that prompt word. Based on this, the raw image features corresponding to the target prompt word can be obtained.

[0105] Specifically, in one embodiment, please refer to Figure 4 Step S3, which involves clustering connected components of the target cue graph based on the probability of the same cue word to obtain clusters of connected components representing similar building structures, includes:

[0106] Step S31: Determine the style similarity between any two target cue images, and combine the probability of the same cue word with its corresponding style similarity to determine the distance information between the corresponding connected components;

[0107] Step S32: Cluster the connected components using the distance information between each connected component to obtain clusters of connected components that represent similar building structures.

[0108] Each connected component in all the target hint graphs is treated as a data point;

[0109] The same prompt word between two connected components is possible And the style similarity between the renderings corresponding to these two connected components. The negative correlation normalization of the product (e.g., first performing Z-score standardization to map the range to the [0,1] interval, and then using the unit value 1 minus the mapped value to achieve negative correlation normalization) is used as its value for... The distance metric used in clustering results in a smaller distance metric as the similarity increases.

[0110] Based on the above distance metric method, numerous data points were analyzed. Clustering, where the minimum number of clustering parameters is required. The clustering parameter radius is the total number of cue words in all target cue maps during generation (each cue word is counted once). pass Distance graph method to determine ( The value is the minimum number of points. (This can be specifically, for example, a preset constant of 4 or 5% of the total).

[0111] This leads to numerous clusters of connected domains with similar building structures, each cluster corresponding to a structural feature in the rendering.

[0112] In one embodiment of the present invention, the normalization process can specifically be, for example, maximum and minimum value normalization. Furthermore, subsequent normalization steps can all employ maximum and minimum value normalization. In other embodiments of the present invention, other normalization methods can be selected based on the specific range of values, which will not be elaborated further. The maximum and minimum value normalization in the embodiments of the present invention aims to perform standardization processing. The maximum and minimum values ​​of the normalization process can be set according to specific scenarios and data numerical characteristics. Adjusting, calibrating, or optimizing the maximum and minimum values ​​does not constitute a limitation of the present invention.

[0113] It should be noted that a positive correlation indicates that there is a unidirectional relationship between the independent and dependent variables, where the larger the independent variable is, the larger the dependent variable is; a negative correlation indicates that there is an inverse relationship between the independent and dependent variables, where the smaller the independent variable is, the larger the dependent variable is. The specific manifestation of positive and negative correlations is determined by practical applications, and this application does not impose any special restrictions.

[0114] Specifically, please refer to Figure 5In another embodiment, step S3, which uses connected component clustering to determine the raw image region blocks corresponding to the target prompt word, includes:

[0115] Step S301: Using the ratio between the number of target prompt images corresponding to any connected component cluster and the total number of target prompt images, the probability that the image features of the corresponding connected component cluster are target prompt word image features is obtained.

[0116] Step S302: The cluster of connected components corresponding to the highest probability of the prompt feature is taken as the cluster of raw image regions corresponding to the target prompt word, and the connected components in the raw image region cluster are taken as blocks of raw image regions corresponding to the target prompt word.

[0117] More specifically, step S302, which divides the connected components in the raw image region cluster as blocks of raw image regions corresponding to the target prompt word, includes:

[0118] By utilizing the number of target hint graphs corresponding to each of any two connected component clusters, we can determine the common kinship differences between the two connected component clusters and identify the common kinship clusters of any connected component cluster.

[0119] The raw image region clusters and the connected components in their respective clusters are used as raw image region blocks corresponding to the target prompt words.

[0120] In this embodiment, for the generated final image (target prompt image) containing the target prompt word, since the connected components participating in the clustering all originate from these final images, the region features of the target prompt word in the final image are the connected component features common to these images. That is, if the vast majority of target prompt images can be one-to-one corresponded to the images corresponding to the connected components in a connected component cluster, then the connected components in this cluster are the connected components corresponding to the prompt word (target prompt word). Based on this, the probability that the image features of a connected component cluster are the features of the target prompt word's original image is calculated:

[0121]

[0122] In the formula, Indicates clustering of connected components The image features are the probability of the target prompt word's image features; Indicates clustering of connected components The number of target hint graphs corresponding to the connected components (without duplicates); This indicates the total number of target hint images.

[0123] Select the cluster of connected components with the highest probability of providing a prompt feature as the cluster of the raw image region corresponding to the prompt word; if there are other clusters of the same kind, then these clusters of the same kind are used together as the cluster of the raw image region corresponding to the prompt word.

[0124] The connected components in these raw image region clusters are used as raw image region blocks corresponding to the target prompt word (the prompt word).

[0125] For determining if they belong to the same cluster:

[0126] For complex prompts, the feature structure in the generated image consists of multiple connected components. Each prompt corresponds to a feature in the raw image, and therefore the generated image must also contain this feature. Thus, for complex features, multiple connected components must coexist. Therefore, by analyzing the number of corresponding images in each connected component cluster, we can determine whether two connected component clusters belong to the same prompt.

[0127]

[0128] In the formula, Indicates clustering of connected components Clustering of connected components Differences within the same genus; Indicates clustering of connected components The total number of target hint graphs corresponding to the connected components in the graph (without duplicates); Indicates clustering of connected components The total number of target hint graphs corresponding to the connected components in the graph; Used to take the absolute value; Used for positive correlation normalization (such as maximum and minimum value normalization).

[0129] The smaller the genus difference between two connected component clusters, the greater the probability that the connected components with these two features co-occur. Therefore, these two connected component clusters are denoted as genus clusters.

[0130] Each connected component cluster is treated as a data point. Numerous connected component clusters are then grouped according to the magnitude of their differences within the same genus, and the data points are further processed. Clustering, where clustering parameters Given the total number of prompt words (each prompt word is counted once), the numerous connected component clusters are divided into... There are several categories.

[0131] Then, connected domains in the same category will be clustered together as genus clusters.

[0132] Step S4: Divide each raw image region into blocks to obtain a retrained ControlNet model. Input the current initial image and its corresponding current prompt word into the ControlNet model to obtain the current final image.

[0133] Specifically, step S4, which involves dividing the raw image into blocks to obtain a retrained ControlNet model, includes:

[0134] Each raw image region is divided into blocks and labeled according to its corresponding prompt words to obtain a semantic segmentation mask image;

[0135] The semantic segmentation mask image is used as training data for the ControlNet model to obtain a retrained ControlNet model.

[0136] In this embodiment, after obtaining numerous connected components in the final image (target prompt image) corresponding to the target prompt word, a segmentation model for segmenting the image based on the prompt word can be trained based on the target prompt word and numerous final images (which contain labels indicating the segmentation of the raw image region corresponding to the prompt word).

[0137] First, based on the method described in the above embodiment, divide all the final effect images (target prompt images) into blocks corresponding to all prompt words;

[0138] Then, using the blocks in the final effect image as the corresponding boundaries, and the prompt words corresponding to each raw image region block as annotations, all the final effect images are divided into blocks and annotated to obtain the corresponding semantic segmentation mask images;

[0139] Then, using numerous final effect images and their corresponding blocks and annotations as training data, and with existing... The model is further trained (the training method can be an existing technology), with the final result image and annotations (i.e. semantic segmentation mask image) as the input of the network, and the block information corresponding to each annotation as the output of the network.

[0140] It should be noted that the training results are... The model is designed to support semantic segmentation inputs and can have a pre-trained semantic segmentation model (such as U-Net or DeepLab) so that the model output is a mask corresponding to each prompt word.

[0141] Specifically, using the "prompt word - raw image region block" correspondence determined by S3, historical effect images are automatically labeled to generate training pairs <historical effect image, semantic segmentation mask>. Using these training pairs, a semantic segmentation model (such as U-Net or DeepLab) is trained. The model takes the effect image as input and outputs the mask corresponding to each prompt word. The "current initial effect image" is input into the trained semantic segmentation model to obtain the "current semantic segmentation mask." This "current semantic segmentation mask" is then used as a condition and input into... (In conjunction with the SD model) Generate the final rendering.

[0142] The final result is a segmented model that divides the effect image into blocks based on the effect image and prompts, which is also the retrained model. Model.

[0143] In obtaining optimization After modeling, the current initial rendering and prompts are input into the model to obtain specific control blocks for the current initial rendering. Furthermore, designers can use these architectural design requirements as the preferred options. The model's control parameters are used to finely control and adjust the initial renderings, ultimately generating the final renderings of the building structure.

[0144] This invention relies on the SD model (Stable Diffusion Model, or platform) to obtain generation information related to architectural design, involving current and historical architectural designs. Based on the designer's (user's) current prompt, it generates a rough initial rendering of the current architectural design. Combining this with historical renderings generated by the designer and their corresponding prompts, it compares the stylistic similarity between the current initial rendering and historical renderings. Then, it filters and categorizes historical renderings using any prompt (target prompt). Finally, it constructs the corresponding image regions for each prompt under different prompts using connected component clustering. Finally, it segments and labels the image regions corresponding to each prompt. Note that the ControlNet model is optimized and trained. The retrained ControlNet model, combined with the current initial rendering and corresponding prompts, generates the current corresponding area blocks. Since these area blocks are accurately correlated with the prompts, they fully consider and relate to the overall layout features. Based on these more accurate area blocks that align with the designer's intentions, the designer can further refine and adjust them to obtain the final architectural rendering. This effectively improves the precision and accuracy of the control over the generation of architectural renderings, as well as the overall control over the generated renderings, thus better aligning with the designer's actual design concept and generating the ideal rendering.

[0145] Example 2:

[0146] This invention also proposes a device for generating architectural structural renderings for architectural design. The device can be a computer, a server, or a combination of multiple devices for data analysis and computation.

[0147] like Figure 6 As shown, Figure 6 This is a schematic diagram of the hardware operating environment of the architectural structure rendering generation device for architectural design involved in the embodiments of the present invention.

[0148] like Figure 6As shown, the architectural rendering generation device for architectural design may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display or an input unit such as a control panel; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001. The memory 1005, as a computer storage medium, may include an architectural rendering generation program.

[0149] Those skilled in the art will understand that Figure 6 The hardware structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0150] Continue to refer to Figure 6 , Figure 6 The memory 1005, which is a computer-readable storage medium, may include an operating system, a user interface module, a network communication module, and a program for generating architectural renderings.

[0151] exist Figure 6 In this embodiment, the network communication module is mainly used to connect to the server and can communicate with the server for data; while the processor 1001 can call the building structure rendering generation program stored in the memory 1005 and execute the steps in the above embodiments.

[0152] Based on the hardware structure of the above-mentioned architectural design-oriented architectural structure rendering generation device, various embodiments of the architectural design-oriented architectural structure rendering generation method of the present invention are implemented.

[0153] Furthermore, this invention also provides a system for generating architectural structural renderings for architectural design (hereinafter referred to as the "architectural rendering generation system"), please refer to... Figure 7 The architectural structural rendering generation system for architectural design includes:

[0154] The image filtering module A10 is used to obtain historical renderings and current initial renderings of the building structure generated based on user input, and compare the settings of the two to determine a reference rendering in the historical renderings that is consistent with the style of the current initial rendering.

[0155] The image segmentation module A20 is used to determine target prompt images that all contain target prompt words in the reference effect images. It uses the image features corresponding to the connected components of any two target prompt images to determine the probability of the same prompt word between their respective connected components. It uses the probability of the same prompt word to cluster the connected components of the target prompt images to obtain connected component clusters that represent similar building structures. It uses the connected component clusters to determine the raw image region segment corresponding to the target prompt word. It uses each raw image region segment to obtain a retrained ControlNet model. It inputs the current initial effect image and its corresponding current prompt word into the ControlNet model to obtain the current final effect image.

[0156] Furthermore, the image filtering module A10 is also used for:

[0157] By utilizing the similarity between historical renderings and the current initial renderings at any two constraint points during generation, the style similarity between historical renderings and the current initial renderings can be determined.

[0158] Historical renderings with a style similarity greater than a preset style similarity threshold are used as reference renderings that are consistent with the style of the current initial rendering.

[0159] The limiting points include prompt words and working nodes in the stable diffusion model.

[0160] Furthermore, the image filtering module A10 is also used for:

[0161] Determine the reference constraint point with the highest cosine similarity to the target constraint point in the historical rendering and the current initial rendering, and obtain the cosine similarity between the target constraint point and the reference constraint point;

[0162] By utilizing the cosine similarity between the target constraint point and the reference constraint point, the style similarity between the historical rendering and the current initial rendering is determined.

[0163] Furthermore, the image segmentation module A20 is also used for:

[0164] Determine the shape similarity between the shape features corresponding to any connected component of any two target cue graphs, and use the shape similarity to determine the probability of the same cue word between the corresponding connected components.

[0165] Furthermore, the image segmentation module A20 is also used for:

[0166] Determine the texture similarity between texture features corresponding to any connected component of any two target cue graphs;

[0167] The probability of identical prompts between corresponding connected components is determined by using shape similarity and texture feature similarity.

[0168] Furthermore, the image segmentation module A20 is also used for:

[0169] Determine the style similarity between any two target cue images, and combine the probability of the same cue word with its corresponding style similarity to determine the distance information between the corresponding connected components;

[0170] Clustering of connected components using distance information between them yields clusters of connected components that represent similar building structures.

[0171] Furthermore, the image segmentation module A20 is also used for:

[0172] By using the ratio between the number of target cue graphs corresponding to any connected component cluster and the total number of target cue graphs, the probability that the image features of the corresponding connected component cluster are target cue word image features can be obtained.

[0173] The cluster of connected components corresponding to the highest probability of the prompt feature is taken as the cluster of raw image regions corresponding to the target prompt word, and the connected components in the raw image region cluster are taken as blocks of raw image regions corresponding to the target prompt word.

[0174] Furthermore, the image segmentation module A20 is also used for:

[0175] By utilizing the number of target hint graphs corresponding to each of any two connected component clusters, we can determine the common kinship differences between the two connected component clusters and identify the common kinship clusters of any connected component cluster.

[0176] The raw image region clusters and the connected components in their respective clusters are used as raw image region blocks corresponding to the target prompt words.

[0177] Furthermore, the image segmentation module A20 is also used for:

[0178] Each raw image region is divided into blocks and labeled according to its corresponding prompt words to obtain a semantic segmentation mask image;

[0179] The semantic segmentation mask image is used as training data for the ControlNet model to obtain a retrained ControlNet model.

[0180] The specific implementation of the architectural structure rendering generation system for architectural design of the present invention is basically the same as the embodiments of the architectural structure rendering generation method for architectural design described above, and will not be repeated here.

[0181] Furthermore, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a building structure rendering generation program, wherein, when executed by a processor, the building structure rendering generation program implements the steps of the building structure rendering generation method for architectural design described above.

[0182] The method implemented when the architectural structure rendering generation program is executed can be referred to in various embodiments of the architectural structure rendering generation method for architectural design of this invention, and will not be repeated here.

[0183] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0184] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0185] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0186] The above description is only a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. All equivalent structural / method transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. A method for generating architectural structural renderings for architectural design, characterized in that, The method is applied to a stable diffusion model, including: Obtain historical renderings and current initial renderings of the building structure generated based on user input, compare the settings of the two renderings, and determine a reference rendering in the historical renderings that has the same style as the current initial rendering. Identify target cue images that all contain target cue words in the reference effect images, and use the image features corresponding to the connected components of any two target cue images to determine the probability of the same cue word between their respective connected components. Clustering of connected components in the target prompt graph based on the probability of the same prompt word yields clusters of connected components representing similar building structures. These clusters are then used to determine the regions of the raw image corresponding to the target prompt word. The ControlNet model is retrained by dividing each raw image region into blocks. The current initial image and its corresponding current prompt word are input into the ControlNet model to obtain the current final image. The method of clustering connected components of the target cue graph based on the probability of identical cue words to obtain clusters of connected components representing similar building structures includes: Determine the style similarity between any two target cue images, and combine the probability of the same cue word with its corresponding style similarity to determine the distance information between the corresponding connected components; Clustering of connected components using distance information between them yields clusters of connected components that represent similar building structures. The method of determining the raw image region segmentation corresponding to the target prompt word by utilizing connected component clustering includes: By using the ratio between the number of target cue graphs corresponding to any connected component cluster and the total number of target cue graphs, the probability that the image features of the corresponding connected component cluster are target cue word image features can be obtained. The cluster of connected components corresponding to the highest probability of the prompt feature is taken as the cluster of the raw image region corresponding to the target prompt word, and the connected components in the raw image region cluster are taken as the blocks of the raw image region corresponding to the target prompt word. The step of dividing the raw image region into blocks based on the connected components in the cluster of raw image regions as the target prompt words includes: By utilizing the number of target hint graphs corresponding to each of any two connected component clusters, we can determine the common kinship differences between the two connected component clusters and identify the common kinship clusters of any connected component cluster. The raw image region clusters and the connected components in their respective clusters are used as raw image region blocks corresponding to the target prompt words.

2. The method for generating architectural structural renderings for architectural design according to claim 1, characterized in that, The process of comparing the settings of the two raw images to determine a reference image in the historical rendering that has the same style as the current initial rendering includes: By utilizing the similarity between historical renderings and the current initial renderings at any two constraint points during generation, the style similarity between historical renderings and the current initial renderings can be determined. Historical renderings with a style similarity greater than a preset style similarity threshold are used as reference renderings that are consistent with the style of the current initial rendering. The limiting points include prompt words and working nodes in the stable diffusion model.

3. The method for generating architectural structural renderings for architectural design according to claim 2, characterized in that, The method of determining the style similarity between historical renderings and the current initial renderings by utilizing the similarity between any two constraint points during generation includes: Determine the reference constraint point with the highest cosine similarity to the target constraint point in the historical rendering and the current initial rendering, and obtain the cosine similarity between the target constraint point and the reference constraint point; By utilizing the cosine similarity between the target constraint point and the reference constraint point, the style similarity between the historical rendering and the current initial rendering is determined.

4. The method for generating architectural structural renderings for architectural design according to claim 1, characterized in that, The method of determining the probability of identical prompt words between connected components of any two target prompt graphs using image features corresponding to their respective connected components includes: Determine the shape similarity between the shape features corresponding to any connected component of any two target cue graphs, and use the shape similarity to determine the probability of the same cue word between the corresponding connected components.

5. The method for generating architectural structural renderings for architectural design according to claim 4, characterized in that, The method of determining the probability of identical prompt words between connected components of any two target prompt graphs using image features corresponding to their respective connected components includes: Determine the texture similarity between texture features corresponding to any connected component of any two target cue graphs; The probability of identical prompts between corresponding connected components is determined by using shape similarity and texture feature similarity.

6. The method for generating architectural structural renderings for architectural design according to claim 1, characterized in that, The retrained ControlNet model obtained by dividing each raw image region into blocks includes: Each raw image region is divided into blocks and labeled according to its corresponding prompt words to obtain a semantic segmentation mask image; The semantic segmentation mask image is used as training data for the ControlNet model to obtain a retrained ControlNet model.

7. A system for generating architectural structural renderings for architectural design, characterized in that, The system is used to implement the architectural structural rendering generation method for architectural design as described in any one of claims 1 to 6; the system includes: The image filtering module is used to obtain historical renderings and current initial renderings of the building structure generated based on user input, compare the settings of the two images, and determine a reference rendering in the historical renderings that has the same style as the current initial rendering. The image segmentation module is used to determine target prompt images that contain target prompt words in the reference effect images. It uses the image features corresponding to the connected components of any two target prompt images to determine the probability of the same prompt word between their respective connected components. It uses the probability of the same prompt word to cluster the connected components of the target prompt images to obtain connected component clusters that represent similar building structures. It uses the connected component clusters to determine the raw image region segment corresponding to the target prompt word. It uses each raw image region segment to obtain a retrained ControlNet model. It inputs the current initial effect image and its corresponding current prompt word into the ControlNet model to obtain the current final effect image.

Images