Deep learning-based generative architectural drawing design module training method

Abstract

The present invention relates to a deep learning-based generative architectural drawing design module training method which enables generating a drawing reflecting requirements of a user and a structural relationship of a building in the AEC industry. A generative model training method comprises the steps of: receiving architectural drawing information including a structural element and a spatial element, and training a first conditional unit on relationships between nodes in graph-structured data on the basis of a graph convolutional network; and generating a segmentation map for the structural element and the spatial element by using the architectural drawing information including the structural element and the spatial element, and training a second conditional unit by embedding the segmentation map with a graph structure in a GAN-based model.

Description

A Learning Method for a Deep Learning-Based Generative Architectural Drawing Design Module

[0001] The present invention relates to a learning method for a deep learning-based generative architectural drawing design module capable of generating drawings that reflect user requirements and structural relationships of buildings in the AEC industry.

[0002] Generative AI is an artificial intelligence (AI) technology that utilizes existing content, such as text, audio, and images, to create new, similar content. It learns patterns in existing content and generates new content based on inferences.

[0003] While existing AI learns data and patterns to understand the subject, generative AI creates new creations through comparative learning with existing data.

[0004] In the field of images, it can recreate a photograph with a painting that imitates a specific artist's style or create an unlimited number of fake human faces; in the field of voice, it can compose a specific genre of music or recreate a specific song with the voice of a desired singer; and in the field of text, it can compose a poem or create a novel with a specific material.

[0005] In this way, generative artificial intelligence is being applied or used in various fields.

[0006] This technology can be used in the AEC (Architecture, Engineering, Construction) industry to provide a variety of services. Among these, it would be possible to create architectural drawings that reflect both user requirements and the structural relationships of the building.

[0007] This can be utilized to understand and learn architectural drawings and create architectural drawings according to user needs.

[0008] The present invention provides a learning method for a deep learning-based generative architectural drawing design module that can automatically generate an architectural drawing that meets user needs (parameters) by learning an architectural drawing divided into structural elements and spatial elements according to multiple conditions.

[0009] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0010] The learning method for a deep learning-based generative architectural drawing design module of the present invention for achieving the above object comprises a learning method based on a generative adversarial network (GAN) composed of a generative model and a discriminant model for a deep learning-based generative architectural drawing design module, wherein the generative model and the discriminant model are designed to have a structure capable of integrating a segmentation map, and a plurality of conditionals are combined to generate a new image (fake image) based on the segmentation map by the learned generative model; a step in which the discriminant model compares an actual segmentation map (real image) with the new image (fake image) and feeds back the result to the generative model; and a step in which the generative model adjusts parameters to minimize a loss function that measures a difference between the generated new image (fake image) and a target segmentation mask; wherein the learning method of the generative model comprises a step in which the generative model receives architectural drawing information having structural elements and spatial elements as input and learns a first conditional part from graph structure data to determine relationships between nodes based on a graph convolutional network; And it is characterized by including a step of generating a segmentation map for the structural elements and spatial elements using architectural drawing information having the structural elements and spatial elements, and learning a second conditional part by embedding the segmentation map with a graph structure in a GAN-based model.

[0011] In addition, the graph convolutional network-based method sets nodes and edges, which are entities of graph structure data, wherein the nodes define structural elements and spatial elements of the architectural drawing information, assign feature vectors representing properties of the elements to each node, and the edges represent relationships between nodes, and are characterized by assigning weights according to at least one of physical distance, connectivity, and interaction.

[0012] In addition, the edge is characterized in that it is divided into a connectivity edge based on adjacency between the structural element and a node physically connected thereto, and an adjacency edge indicating accessibility from the spatial element.

[0013] In addition, the above-mentioned adjacency edge is weighted by adjusting the value of the adjacency matrix, and the formula for the weight is as follows.

[0014] [formula]

[0015]

[0016] At this time, is the weight between nodes i and j, which are entities defined during the design of architectural drawings, is the distance between i and j, is a parameter that controls the distance reduction rate.

[0017] In addition, the node updates the characteristics of the node so that the structural relationship and the spatial relationship can be designed consistently and structurally based on the characteristics of the connected structural elements or spatial elements, and the update formula of the function of the node is characterized as follows.

[0018] [formula]

[0019]

[0020] At this time, refers to the feature vector of node (i) after the n+1th layer update, stands for nonlinear activation function (ReLU), denotes the learnable weight matrix of the nth layer, is a constant between nodes i and j. This means that the influence between neighboring nodes is normalized by setting .

[0021] In addition, the formula for learning to reflect the structure and relationship of all nodes of the graph convolution operation-based layer based on the graph convolutional network has the following characteristics.

[0022] [formula]

[0023]

[0024] At this time, is an adjacency matrix added to self-loops, Is A diagonal matrix consisting of the reciprocals of the sum for each row of , stands for nonlinear activation function (ReLU), is on each floor The weight matrix of, Each floor Node characteristic matrix in ( (node ​​matrix in the second).

[0025] In addition, the method for generating the segmentation map is characterized by comprising: a step of setting based on a user's input information and parameter definitions; a step of segmenting reference drawing images with architectural drawing information having structural elements and spatial elements and separating them by element to prepare a reference image set; a step of generating a reference segmentation map based on the user's input information and parameter definitions for the reference image set; a step of generating a target segmentation mask for a target image to be synthesized from the reference segmentation map; and a step of generating a segmentation map by aligning and matching the target segmentation mask to the reference segmentation map.

[0026] In addition, when learning by combining the first conditional part and the second conditional part and integrating the conditional information into the generative model, the loss function formula including the generative model and the discriminative model is characterized as follows.

[0027] [formula]

[0028]

[0029] At this time, is a potential space After applying the graph convolution operation, the graph convolution features derived from the graph structure means the connection of , and S means the segmentation map that provides layout information for the generation model, is the loss function of the generative model, is the loss function of the discriminant model, is the parameter of the generative model, refers to the parameters of the discriminant model.

[0030] As explained above,

[0031] The present invention has the effect of generating architectural drawings according to the user's needs, thereby not requiring a large amount of manpower for generating architectural drawings, and being able to generate various architectural drawings in a short period of time, thereby saving the required costs.

[0032] That is, the present invention enables automation of architectural drawing design and has the effect of increasing efficiency in drawing creation.

[0033] FIG. 1 is a block diagram illustrating a generative adversarial neural network that learns for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention.

[0034] FIG. 2 is a drawing showing a segmentation-based drawing identification in a learning method for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention.

[0035] FIG. 3 is a diagram showing a physical adjacency graph in a learning method for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention.

[0036] FIG. 4 is a flowchart illustrating a segmentation map generation method of a learning method for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention.

[0037] FIG. 5 is a diagram illustrating a second conditional generative model in a learning method for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention.

[0038] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0039] In addition, the size and shape of components illustrated in the drawings may be exaggerated for clarity and convenience of explanation, and terms specifically defined in consideration of the configuration and operation of the present invention may vary depending on the intention or custom of the user or operator, and the definitions of these terms should be made based on the contents throughout this specification.

[0040] FIG. 1 is a block diagram showing a learning process for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention.

[0041] This is a learning method based on a generative adversarial network (GAN) consisting of a generative model (100) and a discriminative model (200) for a deep learning-based generative architectural drawing design module.

[0042] As illustrated in FIG. 1, the learning method for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention first designs a generative model (100) and a discriminative model (200) in a structure that can integrate a segmentation map, and the generative model (100) learned by combining multiple conditionals generates a new image (fake image) based on the segmentation map.

[0043] As shown in Fig. 2, in order to generate a new image (fake image), it shows how to classify and identify drawings in a plane drawing such as (a) that is input, learns to identify drawings according to structural and spatial elements such as (b), and learns to classify structural and spatial elements of the drawing such as (c).

[0044] That is, the input (a) generates a new image (fake image) based on the dataset learned to classify and identify drawings, such as (c).

[0045] Here, the generation model (100) is a combination of a first conditional part (110) and a second conditional part (120), where the first conditional part (110) receives architectural drawing information having structural elements and spatial elements of the drawing and learns the relationships between nodes from graph structure data based on a graph convolutional network.

[0046] As shown in Figure 3, a physical adjacency graph is constructed to measure the adjacency of each element within the drawing, and the adjacency metric of the drawing is calculated.

[0047] That is, it calculates the location and adjacency of walls, doors, and windows.

[0048] Graph convolutional networks are designed to learn deep learning from graph-structured data to understand complex relationships and structures.

[0049] The graph convolutional network-based model establishes nodes and edges, which are entities of graph structure data. Nodes define structural and spatial elements of architectural drawing information, and assign feature vectors representing the properties of elements to each node.

[0050] At this time, structural elements refer to structural objects such as walls, windows, and doors in architectural drawings, and spatial elements refer to spatial objects according to their purpose, such as bedrooms, living rooms, bathrooms, verandas, and kitchens.

[0051] Nodes are vectorized by assigning [area, length, height, element characteristics], etc.

[0052] Here, the node's properties are updated so that the structural and spatial relationships can be designed consistently and structurally based on the characteristics of the connected structural elements or spatial elements.

[0053] This is a task that generates a design that more accurately captures the structural and spatial (functional) relationships between different architectural elements through weighting and normalization.

[0054] The formula for updating the function of a node in layer n+1 is as follows.

[0055] [formula]

[0056]

[0057] At this time, refers to the feature vector of the node after the n+1th layer update, stands for nonlinear activation function (ReLU), denotes the learnable weight matrix of the nth layer, is a node and As a constant between This means that the influence between neighboring nodes is normalized by setting .

[0058] That is, a set of node features is computed based on self-features and neighbor features using weights and regularization to describe the graph structure.

[0059] This is to vectorize and encode both feature information and graph structure during architectural design.

[0060] Edges represent relationships between nodes, and are weighted based on physical distance, connectivity, and interactions between elements within a building.

[0061] Edges are divided into connectivity edges based on adjacency between structural elements and physically connected nodes, and adjacency edges indicating accessibility from spatial elements.

[0062] Connectivity edges are edges based on adjacency, such as between a wall and a door, between a door and a room, or between a window and a wall. By analyzing the bounding box locations or segmentation locations in the dataset, a metric is defined to measure the adjacency of each element.

[0063] For example, we can determine which wall is adjacent to a door or where a window is located, and adjacency information is used to define connectivity between structural elements.

[0064] Adjacency edges are distance-based edges that calculate the distance between nodes, and calculate a metric for measuring the adjacency of each element by calculating the Euclidean distance at the bounding box location or segmentation location to determine the spatial relationship between two walls.

[0065] By setting a threshold, if the distance calculations are below a certain value, the two nodes are considered adjacent and an edge is formed between them.

[0066] Because it is based on adjacency, weights are applied by adjusting the values ​​of the adjacency matrix.

[0067] For example, if two adjacent elements are a door and a window, the weight of the corresponding edge is increased to emphasize the relationship between the door and the window.

[0068] The goal of this process is to effectively model structural connectivity within a building, enabling network analysis of complex drawing designs and the creation of new structures.

[0069] The weights in the adjacency matrix indicate the relative importance and strength of connectivity between nodes, and when assigning weights, the definition reflects the importance of the relationship between nodes and the strength of the connection in the architectural drawing design.

[0070] The edges between two physically adjacent structural elements (e.g., a wall and a door) are given a high weight, and connections that are important for the function of a particular space (e.g., accessibility between a bedroom and a bathroom) are given a higher weight to reflect the usability of the space.

[0071] At this time, the weight settings can be flexibly changed depending on the purpose and requirements of the architectural design.

[0072] For example, if the connectivity of certain structural elements (walls and windows) is important, the weights are set higher, and if the usability of the building is important, the weights for accessibility are set higher, taking into account the user's movement path.

[0073] At this time, the adjacency edge is weighted by adjusting the value of the adjacency matrix, and the formula for the weight is as follows.

[0074] [formula]

[0075]

[0076] At this time, A node is an entity defined when designing architectural drawings. and is the weight between, Is and is the distance between, is a parameter that controls the distance reduction rate.

[0077] Here, node and The smaller the distance between nodes, the greater the weight between nodes. This allows the design values ​​to be reflected based on the physical accessibility and proximity of elements, and can be varied according to the specific goals and constraints of the architectural design project.

[0078] As mentioned above, the formula for learning to reflect the structure and relationship of all nodes of the graph convolution operation-based layer based on the graph convolutional network is as follows.

[0079] [formula]

[0080]

[0081] At this time, is an adjacency matrix added to self-loops, Is A diagonal matrix consisting of the reciprocals of the sum for each row of , stands for nonlinear activation function (ReLU), is on each floor The weight matrix of, Each floor Node characteristic matrix in ( (node ​​matrix in the second).

[0082] This is a normalized adjacency matrix based on graph convolution operation, and the functions of adjacent nodes are normalized adjacency matrix and feature matrix. Nonlinear activation function by applying the product of and weights Apply to the next layer is to obtain the final feature representation for .

[0083] This prepares the learning so that the features of each node reflect complex structures and relationships as well as individual attributes.

[0084] The second conditional unit (120) uses architectural drawing information with structural and spatial elements to generate a segmentation map for the structural and spatial elements, and learns by embedding it with a graph structure in a GAN-based model.

[0085] The method for creating a segmentation map is as shown in Fig. 4, first, setting is done based on the user's input information and parameter definition (S210).

[0086] Here, the parameter definition includes the building type, the number of spaces required, and the elements to be emphasized, etc., for example, the building type indicates whether it is a residential or commercial building, etc., the number of spaces required indicates the number of spatial elements required, such as the number of bedrooms, the number of bathrooms, and the number of verandas, and the elements to be emphasized are defined so that the information entered by the user can match the parameters, such as whether it is an open living room, the length of the windows, etc.

[0087] Next, reference drawing images are segmented into architectural drawing information with structural and spatial elements, and separated by element to prepare a reference image set (S220).

[0088] More preferably, reference drawing images are segmented to separate structural elements such as walls, windows, and doors, and spatial elements such as bedrooms, living rooms, bathrooms, verandas, and kitchens, to prepare a reference image set.

[0089] Next, a reference segmentation map is created based on the user's input information and parameter definitions for the reference drawing image (S230).

[0090] At this time, the reference segmentation map indicates to which element each pixel belongs.

[0091] Class-wise segmentation of structural elements such as walls, windows, and doors, and class-wise segmentation of spatial elements such as bedrooms, living rooms, bathrooms, verandas, and kitchens.

[0092] Here, doors can be subdivided into classes such as sliding doors and swing doors, and windows can be subdivided into classes such as sliding windows and swing windows.

[0093] Next, a target segmentation mask for the target image to be synthesized is generated from the reference segmentation map (S240). At this time, the target segmentation mask specifies the layout by defining the characteristics of the structural and spatial elements of the target drawing to be generated.

[0094] This is represented as a matrix where each element corresponds to a class label indicating the architectural element present at that location.

[0095] The goal of generating a segmentation mask is to specify what each pixel or cell in the matrix should be, such as a wall, window, door, bedroom, bathroom, living room, toilet, veranda, and kitchen, and create a detailed map of the desired layout.

[0096] Finally, a segmentation map is created by aligning and matching the target segmentation mask to the reference segmentation map (S250).

[0097] At this time, the alignment and matching of the reference segmentation map and the target segmentation mask is to ensure that each part of the drawing is in the correct position, thereby minimizing the mismatch between the generated segmentation map and the target segmentation mask.

[0098] A direct editing process may be included prior to the learning phase to ensure a more accurate reflection of the user's definitions and requirements.

[0099] Through this, the finalized target segmentation mask is ready to be input into the generation model (100) and provided.

[0100] FIG. 5 is a diagram showing a second conditional generative model in a learning method for a deep learning-based generative architectural drawing design module according to an embodiment of the present invention. As shown in FIG. 5, It provides guidance on what the model should generate by specifying various functional and spatial design elements and arrangements within the scene.

[0101] Segmentation map , , , These maps are used as inputs to the model's conditions and represent design elements, which are various aspects (perspectives) of architectural design. Different colors within the maps represent different categories or types of architectural space.

[0102] In the segmentation map with transition arrows This leads to an integrated map, which is indicated by a , which describes the process by which individual design elements are combined.

[0103] represents a spatial (functional) element represents a segmentation map (not shown), where i represents a spatial (functional) element. It means each element that is subdivided from .

[0104] also, is the segmentation map of the current layer i. It is generated and refined based on .

[0105] represents a structural element It represents a segmentation map (not shown), represents a structural element It means each element that is subdivided from .

[0106] also, is the current layer As a segmentation map of It is generated and refined based on .

[0107] is a specific layer represents a segmentation map of , conditioned by information from the previous layer i, is the current layer As a segmentation map of It is generated and refined based on .

[0108] and These equations are the segmentation map This means that the input at layer i is directly related to the input at layer i. The same is true in .

[0109] is the entire segmentation map that serves as the condition for the generative model. As the final output, it becomes the target map that the generation process attempts to approximate.

[0110] In other words, the generative model (100) generates a new image (fake image) according to the first conditional part (110) based on the graph convolutional model described above and the second conditional part (120) based on the GAN.

[0111] The new image (fake image) generated in this way is passed to the discrimination model (200).

[0112] Next, the discriminant model (200) compares the actual segmentation map (real image) and a new image (fake image) and provides feedback to the generative model (100).

[0113] The generative model (100) is repeatedly trained based on feedback from the discriminative model (200), which includes adjusting the parameters of the generative model (100) to minimize a loss function that measures the difference between the generated new image and the target segmentation mask.

[0114] The adversarial training process involves a discriminant model (200) that attempts to distinguish between generated new images and real images, which leads to loss functions that both the generative model (100) and the discriminant model (200) aim to optimize.

[0115] When learning by combining the graph convolution operation information, which is the first conditional part (110), and the segmentation map information, which is the second conditional part (120), and integrating them as conditional information into the generative model (100), the loss function formula that includes both the generative model (100) and the discriminative model (200) is as follows.

[0116] [formula]

[0117]

[0118] At this time, is a potential space After applying the graph convolution operation, the graph convolution features derived from the graph structure means the connection of , and S means the segmentation map that provides layout information for the generation model, is the loss function of the generative model, is the loss function of the discriminant model, is the parameter of the generative model, refers to the parameters of the discriminant model.

[0119] In this way, the generative model (100) in the modified loss function captures not only the segmentation map but also the structural relationships and properties of the architectural elements encoded by the graph convolutional network. Conditions are specified accordingly.

[0120] Finally, the generative model (100) adjusts parameters to minimize a loss function that measures the difference between the generated new image (fake image) and the target segmentation mask.

[0121] That is, architectural drawings are generated according to the user's requirements and parameter settings.

[0122] Accordingly, the present invention has the effect of generating architectural drawings according to the user's needs, thereby not requiring a large amount of manpower for generating architectural drawings, and being able to generate various architectural drawings in a short period of time, thereby saving the required costs.

[0123] That is, the present invention enables automation of architectural drawing design and has the effect of increasing efficiency in drawing creation.

[0124] While the embodiments of the present invention have been described above, they are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true scope of technical protection of the present invention should be defined by the following claims.

[0125] The present invention relates to a learning method for a deep learning-based generative architectural drawing design module capable of generating drawings that reflect user requirements and structural relationships of buildings in the AEC industry.

[0126] The present invention has the effect of generating architectural drawings according to the user's needs, thereby not requiring a large amount of manpower for generating architectural drawings, and being able to generate various architectural drawings in a short period of time, thereby saving the required costs.

Claims

1. In a learning method based on a generative adversarial network (GAN) consisting of a generative model and a discriminative model for a deep learning-based generative architectural drawing design module, The above-mentioned generative model and discriminative model are designed in a structure that can integrate a segmentation map, and a step in which the generative model, which is learned by combining multiple conditionals, generates a new image (fake image) based on the segmentation map; The above discriminant model compares the actual segmentation map (real image) with the new image (fake image) and provides feedback to the generation model; and The above generative model comprises a step of adjusting parameters to minimize a loss function that measures the difference between a generated new image (fake image) and a target segmentation mask; The learning method of the above generative model includes a step of receiving architectural drawing information having structural elements and spatial elements as input and learning the first conditional part from graph structure data to determine the relationship between nodes based on a graph convolutional network; and A learning method for a deep learning-based generative architectural drawing design module, characterized by comprising: a step of generating a segmentation map for structural elements and spatial elements using architectural drawing information having the above structural elements and spatial elements, and embedding the segmentation map with a graph structure in a GAN-based model to learn a second conditional part; 2. In claim 1, The above graph convolutional network-based model sets up nodes and edges, which are entities of graph structure data. The above nodes define structural elements and spatial elements of the above architectural drawing information, and assign feature vectors representing the properties of the elements to each node, The above edge represents a relationship between nodes, and a learning method for a deep learning-based generative architectural drawing design module characterized in that weights are given based on at least one of physical distance, connectivity, and interaction.

3. In claim 2, The above edge is a connectivity edge based on adjacency between the structural element and the physically connected node, A learning method for a deep learning-based generative architectural drawing design module characterized in that the spatial elements are distinguished by adjacent edges indicating accessibility.

4. In claim 3, A learning method for a deep learning-based generative architectural drawing design module, wherein the above adjacency edge is weighted by adjusting the value of the adjacency matrix, and the formula for the weight is as follows. [formula] At this time, is the weight between nodes i and j, which are entities defined during the design of architectural drawings, is the distance between i and j, is a parameter that controls the distance reduction rate.

5. In claim 2, The above nodes update the characteristics of the nodes so that structural and spatial relationships can be designed consistently and structurally based on the characteristics of the connected structural elements or spatial elements. A learning method for a deep learning-based generative architectural drawing design module, wherein the update formula of the function of the above node is as follows. [formula] At this time, refers to the feature vector of node (i) after the n+1th layer update, stands for nonlinear activation function (ReLU), denotes the learnable weight matrix of the nth layer, is a constant between nodes i and j. This means that the influence between neighboring nodes is normalized by setting .

6. In claim 1, A learning method for a deep learning-based generative architectural drawing design module having the following characteristics: a formula for learning to reflect the structure and relationship of all nodes of a layer based on a graph convolution operation based on the above graph convolutional network. [formula] At this time, is an adjacency matrix added to self-loops, Is A diagonal matrix consisting of the reciprocals of the sum for each row of , stands for nonlinear activation function (ReLU), is on each floor The weight matrix of, Each floor Node characteristic matrix in ( (node ​​matrix in the second).

7. In claim 1, The method for generating the above segmentation map comprises the steps of setting based on the user's input information and parameter definition; A step of segmenting reference drawing images with architectural drawing information having the above structural elements and spatial elements and preparing a reference image set by separating them by element; A step of generating a reference segmentation map based on the user's input information and parameter definitions for the above reference image set; A step of generating a target segmentation mask for a target image to be synthesized from the above reference segmentation map; and A learning method for a deep learning-based generative architectural drawing design module, characterized by comprising the steps of: aligning and matching a target segmentation mask to the above-mentioned reference segmentation map to create a segmentation map; 8. In claim 1, A learning method for a learning-based generative architectural drawing design module, characterized in that the loss function formula including the generative model and the discriminant model is as follows when learning by combining the first conditional part and the second conditional part and integrating the conditional information into the generative model. [formula] At this time, is a potential space After applying the graph convolution operation, the graph convolution features derived from the graph structure means the connection of , and S means the segmentation map that provides layout information for the generation model, is the loss function of the generative model, is the loss function of the discriminant model, is the parameter of the generative model, refers to the parameters of the discriminant model.

Images