A method for automatically generating a vectorized house plan

Abstract

The application discloses a kind of automatically generated vectorization house type planar graph method, comprising the following steps: step 1, using node generation model to generate the node set indicating the intersection of wall in house type planar graph, each node contains its position coordinate, the semantic label of surrounding room, improve the geometric quality of generated node set simultaneously through novel loss function term;Step 2, with the node combination formed in the node set obtained in step 1 as input, the edge set is predicted, and the geometric quality of the predicted edge set is improved by a novel random self-supervised geometric enhancement method;Step 3, based on the semantic information of the room associated with the corner point, all the minimum rings are obtained by traversing the minimum ring operation of graph, i.e.all rooms.The application reduces the tedious manual design, and the intelligent degree is high.

Description

TECHNICAL FIELD

[0001] The application relates to the field of intelligent design methods of house type diagrams, and particularly relates to a method for automatically generating a vectorized house plan. BACKGROUND

[0002] In the field of computer graphics and vision, automatic design of architectural floor plans has attracted wide attention because detailed architectural floor plans are crucial for building houses and designing indoor scenes. Architects usually manually draw sketches and iteratively evaluate and adjust them to obtain satisfactory design results. Unfortunately, this process is time-consuming and requires professional architects, which is costly.

[0003] Automatic generation of architectural floor plans can enable the general public to participate in the design process and customize architectural floor plans according to their needs and preferences. In addition, it can provide inspiration and reference for architects or designers and improve design efficiency. Deep learning is the ability of computers to learn and improve themselves. Deep neural networks are the basis of deep learning. Deep neural networks have stronger representation and learning capabilities. The real world has accumulated a large number of high-quality house plan diagrams. A direct idea is to explore the inherent implicit design rules in the vector format of existing house plan diagrams to achieve automatic generation of house plan diagrams.

[0004] Therefore, it is necessary to propose a vectorized house plan diagram generation method based on a deep learning neural network to solve the problems of time-consuming, low efficiency, high cost, and dependence on professional knowledge in the manual design of vector house plan diagrams by architects, and to improve the geometric quality of the design results. SUMMARY

[0005] The application provides a method for automatically generating a vectorized house plan diagram to solve the problems of time-consuming, low efficiency, high cost, and dependence on professional knowledge in the manual design of vector house plan diagrams by architects in the prior art, and to improve the design quality and enhance the geometric quality of the generated results.

[0006] To achieve the above-mentioned purpose, the technical scheme adopted by the application is as follows:

[0007] A method for automatically generating a vectorized house plan diagram comprises the following steps:

[0008] Step 1: using a node generation model based on a diffusion model, using random noise sampled from a standard Gaussian distribution as the input of the node generation model, generating a preliminary node set from the node generation model, each node in the preliminary node set representing a wall intersection point, each node having position coordinates, semantic information of surrounding rooms, and background attributes;

[0009] Then, it is judged whether each node in the preliminary node set is a filling node according to the background attribute value of the node, and the filling node in the preliminary node set is removed to form a node set of the floor plan with the remaining nodes;

[0010] Step 2, using a neural network-based edge prediction model, a candidate edge set is constructed using any two-node combination of the node set of the floor plan obtained in step 1 to form a candidate edge set, and the candidate edge set is used as the input of the edge prediction model, and it is predicted whether there is an edge between each pair of nodes by the edge prediction model, thereby obtaining an edge set of the floor plan, and the edges in the edge set represent the walls in the floor plan. Combine the node set and the edge set of the floor plan to obtain the structure graph of the floor plan.

[0011] Step 3, by traversing the smallest ring operation of the structure graph of the floor plan, the structure graph is traversed to obtain all the smallest rings, that is, all the rooms, thereby extracting the room set from the structure graph; combine the structure graph and the room set, thereby obtaining the vectorized house plan.

[0012] Further, the neural network included in the diffusion model based on which the node generation model in step 1 is trained is a trained Transformer, denoted as node Transformer.

[0013] Further, the node Transformer adopts a multi-ary alignment loss function during training, and the alignment errors under multiple arities are optimized by hybrid optimization, thereby enhancing the geometric alignment effect of the node Transformer.

[0014] Further, the neural network-based edge prediction model in step 2 adopts a trained Transformer, denoted as edge Transformer.

[0015] Further, the edge Transformer introduces a random self-supervised geometric enhancement method during training, for each candidate edge, a point is randomly inserted on the edge, and the edge Transformer is required to predict the random coefficient of the interpolation point, thereby strengthening the understanding of the edge Transformer to the overall geometric features of the edge.

[0016] Further, in step 3, the process of traversing the smallest ring of the graph is as follows: taking each edge of the structure graph as a starting edge, and defining a certain end point to another end point as a starting direction, iteratively finding all edges in the next step that are located on the left side of the current edge and have the smallest angle with the current edge, until the next step finds the starting edge, forming a ring. If it is a non-repeating smallest ring, it is recorded, and all smallest rings are obtained by traversing each edge of the structure graph.

[0017] Further, step 3 further comprises: for each room, selecting the room semantic information with the highest frequency of occurrence in the room semantic information associated with all the corner points constituting the room as the final room category; if multiple categories have the highest frequency, considering the scarcity factor of the category, determining the room type in the following priority order: storage room > bathroom > kitchen > bedroom > balcony > living room.

[0018] The house plan can be represented by a graph structure, where the nodes of the graph represent the wall intersection points in the house plan, and the edges of the graph represent the walls in the house plan. The semantic information of the room associated with the node is taken as the attribute of the corresponding vertex. The house plan can be represented as a connected graph structure. Thus, the generation problem of the vectorized house plan can be converted into the generation problem of the graph structure. The generation problem is essentially learning the probability distribution of the data and randomly sampling from it. Due to the irregularity of the graph structure, its probability distribution is quite complex; however, the node set and the edge set are regular, and it is feasible to decouple the graph structure into the node set and the edge set for processing respectively.

[0019] The house plan has a multi-level structure, and the generation of the node set basically determines the geometric structure of the entire plan. The semantic information of the room associated with the node is also regarded as the attribute of the node, and the geometric and semantic dependencies of the plan can be learned simultaneously.

[0020] In the present application, the generation of the node set is realized by a diffusion model, which defines a series of mathematical steps to generate data by gradually transmitting noise to the target distribution, and uses a neural network to predict the coefficients of each step. Given the generated node set, the edge set is basically determined and can be simply predicted by a neural network. Therefore, the present application proposes a novel generation model that divides the generation process of the structure graph into two stages: node generation and edge prediction. First, the probability distribution of the node set is learned by the generation model, and the node set is generated by sampling. Then, the edge set is predicted according to the generated node set, and the complete structure graph is obtained. Finally, all the smallest polygonal rings (i.e. rooms) are extracted from the structure graph to obtain the final vectorized house plan.

[0021] The neural network is essentially probabilistic. In the above process, the nodes generated in the node generation stage may have some noise, for example, the generated node coordinates are not aligned. In order to solve this problem, the application introduces an alignment loss function, which takes the "misalignment degree" between nodes as the optimization object, thereby reducing the coordinate noise caused by the probabilistic nature of the neural network. At the same time, in the edge prediction stage, the neural network may output some unreasonable predictions, such as missing or false wall segments, resulting in unreasonable geometry. In order to solve this problem, the application enhances the model's understanding of the overall geometric features of the edge by randomly inserting a third point on the edge and requiring the model to predict the interpolation coefficient of the third point in a self-supervised manner, thereby improving the geometric quality.

[0022] Compared with the prior art, the application has the following advantages:

[0023] The application can automatically generate a vectorized house plan, so that the general home buyers can also participate in the design process, allowing them to customize the building plan according to their needs and preferences. In addition, it can also provide inspiration and reference for architects or designers, improving design efficiency.

[0024] The method is simple, easy to deploy on a computer, and has high automation, reducing tedious manual design. The model based on a deep neural network has sufficient reliable theoretical support, and can solve the problems of time-consuming, low efficiency, high cost, and dependence on professional knowledge in the manual design of a vector house plan by an architect, while improving the geometric quality of the design result. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 Workflow diagram for node generation in an embodiment of the application.

[0026] Figure 2 Technical solution diagram of node Transformer in an embodiment of the application.

[0027] Figure 3 Workflow diagram for edge prediction in an embodiment of the application.

[0028] Figure 4 Workflow diagram for planar graph minimum ring extraction in an embodiment of the application.

[0029] Figure 5 Workflow diagram for an embodiment of the application. DETAILED DESCRIPTION

[0030] The application will be further described below in conjunction with the drawings and embodiments.

[0031] As shown in Figure 1 , Figure 5 , the embodiment discloses a method for automatically generating a vectorized house plan, comprising the following steps:

[0032] Step 1, using a node generation model based on a diffusion model, using random noise sampled from a standard Gaussian distribution as input of the node generation model, generating a preliminary node set from the node generation model, each node in the preliminary node set representing a wall intersection, each node having position coordinates, semantic information of surrounding rooms and background attributes. Then, according to the background attribute value of each node in the preliminary node set, determine whether the corresponding node is a filler node, and remove the filler nodes in the preliminary node set to form a node set of the floor plan from the remaining nodes;

[0033] In this embodiment, the node generation model is implemented based on a diffusion model. Diffusion model is a kind of generative model, which can generate samples highly similar to the original data by learning the noise adding process of the data and reversing it from simple noise. The diffusion model includes a forward process and a reverse process. The forward process adds noise to the original sample (corresponding to time step 0). As the time step advances from 0 to the maximum value T, the noise level in the sample continues to rise, and finally becomes almost identical to Gaussian noise. The reverse process is the inverse operation of this process. Starting from the Gaussian white noise corresponding to time step T, a neural network is used to gradually remove noise and restore clear data; the neural network as a component in the reverse process is responsible for estimating the noise contained in each step t, as part of the mean parameter of the Gaussian distribution corresponding to the next time step t-1.

[0034] In this embodiment, the node generation model based on the diffusion model uses Transformer as the neural network part, denoted as node Transformer. The node Transformer consists of a decoder and multiple downstream prediction heads, where the decoder consists of multiple decoder layers, each decoder layer sequentially includes an "instance normalization layer", a "global self-attention layer", a "residual connection", an "instance normalization layer", a "fully connected layer", and a "residual connection". The model captures the dependency between nodes through the stacking of multiple Transformer decoder layers. Each downstream prediction head is a fully connected layer structure.

[0035] In this embodiment, the node Transformer is trained in advance using the RPLAN dataset, which contains more than 80,000 house plan floor plans with dense annotations, each floor plan with pixel-by-pixel room, wall, and other labels.

[0036] Specifically, in order to train the node Transformer, the RPLAN dataset is preprocessed in a series of preprocessing steps to extract clean vectorized house plan floor plans as training data, and the preprocessing process is as follows:

[0037] (1) The image is binarized, with the wall represented as white and other areas represented as black.

[0038] (2) The thickness of the wall is standardized by repeated morphological operations (erosion, dilation) and template matching. The white pixels representing the wall are iteratively eroded until the next erosion causes the number of connected regions in the image to decrease, indicating that some wall segments have been eroded to a thickness of 1 pixel.

[0039] (3) A series of template matching operations are applied, sliding a 3x3 window over the image. If a match is successful, the matching area is replaced with white. The template shape contains a line of thickness 1 pixel or a defective local wall shape. Through this process, the wall shape is gradually standardized, and the thickness becomes more uniform.

[0040] The above two steps (2) and (3) form an iteration, and this iteration process continues until all wall thicknesses are eroded to 1 pixel: the criterion is that the next erosion will cause all walls to disappear, making the image completely black. At this time, the wall intersection points and wall segments can be obtained from the image, where the wall intersection points contain the position coordinates, and the wall segments contain the indices of the two end wall intersection points, recording which two wall intersection points serve as the endpoints of the wall segment.

[0041] (4) The semantic information of the surrounding room of each wall intersection point is obtained from the four-channel image of the original RPLAN dataset.

[0042] All data samples that cannot be processed in the above steps are discarded.

[0043] After training the node Transformer on the dataset, the node generation model described in this embodiment is obtained.

[0044] In this embodiment, the node generation process is illustrated in FIG. 1. Figure 1 The input of the node generation model is a set of random noise nodes sampled from a standard Gaussian distribution, each node containing its position coordinates, the semantic label of the surrounding room, and the background attribute. The background attribute is generated because the diffusion model requires a fixed size of the node set in implementation, so the size of the node set is set to a larger fixed value, and the node set is filled to this size, and the background attribute is set to distinguish between real existing nodes and filled nodes. At this time, all attributes (position coordinates, semantic label of surrounding room, background) of all nodes are pure noise and have no information. At this time, the time step is T.

[0045] Next, the attributes of each input node are mapped to a high-dimensional representation as input to the node Transformer, where the position coordinates are mapped to a high-dimensional vector by triangular position encoding; the semantic labels and context attributes of the surrounding rooms are mapped to embedding vectors of the same dimension by fully connected layers. In addition, the current time step is also mapped to a high-dimensional representation by triangular position encoding + fully connected layers. The high-dimensional representations of the position coordinates, the semantic labels and context attributes of the surrounding rooms, and the time step are added to obtain the high-dimensional embedding of the node; the high-dimensional embeddings of all nodes form the high-dimensional embedding of the node set and are input into the node Transformer.

[0046] As shown in Figure 2 , the node Transformer includes a Transformer decoder and a downstream prediction head. The Transformer decoder processes the node set, and the decoder is composed of multiple layers stacked in order, each layer containing an "instance normalization layer", a "global self-attention layer", a "residual connection", an "instance normalization layer", a "fully connected layer", and a "residual connection" in sequence.

[0047] Through the self-attention mechanism, the node Transformer can capture complex dependencies between nodes. The output of the Transformer decoder is the high-dimensional embedding of the node set mapped by the Transformer, which is input into the downstream prediction head for attribute prediction. The downstream prediction head is based on fully connected layers, and there are a total of 2, with output dimensions of 2 and 8 respectively. The former is the noise estimate of the position coordinates, and the latter is the joint noise estimate of the semantic labels-context of the surrounding rooms; the two are concatenated to form the noise estimate of the node set (10 dimensions). This noise estimate is used to calculate the mean parameter of the Gaussian distribution of the next time step T-1 (for each time step, the variance is a preset fixed value), to obtain the Gaussian distribution of the next time step T-1, from which the node set at time T-1 is sampled. These node sets will iterate the above process until the time step reaches 0, obtaining a clean node set containing the accurate positions of each node, the semantic information of the surrounding rooms, and the context attributes. According to the context attribute value of each node, it is judged whether the node is a padding node, and the padding nodes in the preliminary node set are removed to obtain the node set of the floor plan.

[0048] The Transformer loss function comprises the MSE regression loss between the noise estimate (10-dimensional) of the node set and the real noise added during the forward pass (10-dimensional). During training, to improve the geometric quality of the generated node set, this embodiment introduces a multi-base alignment loss function as the loss function for node Transformer training. This multi-base alignment loss function optimizes the alignment error of nodes in different bases such as binary, quaternary, octal, and hexadecimal. Specifically, the numerical value of the "alignment deviation" between nodes is converted into numerical representations in different bases, the error is calculated bit by bit and accumulated, and used as a term in the loss function. This promotes accurate alignment of nodes across multiple scales, thereby enhancing the geometric accuracy of the generated planar graph. At this point, the loss function includes the MSE regression loss between the noise estimate (10-dimensional) of the node set and the real noise added during the forward pass (10-dimensional), as well as the multi-base alignment loss function.

[0049] Step 2: Using a neural network-based edge prediction model, construct a candidate edge set by combining any two nodes in the node set of the planar graph obtained in Step 1 to form candidate edges. Use the candidate edge set as input to the edge prediction model, and predict whether there is an edge between each pair of nodes. This yields the edge set of the planar graph. The edges in the edge set represent walls in the planar graph. Combine the node set and the edge set of the planar graph to obtain the structure diagram of the planar graph.

[0050] In this embodiment, the edge prediction model based on the neural network adopts a trained Transformer, denoted as the edge Transformer.

[0051] by Figure 3 Taking edge prediction as an example, the input to the edge prediction model is all node pairs formed by combining all nodes in the node set. Each node pair serves as a candidate edge, and all candidate edges form a candidate edge set. For each candidate edge, the high-dimensional embeddings of the two nodes in the corresponding node pair are extracted, including the location coordinate encoding and the semantic label-background embedding of the surrounding rooms. All high-dimensional embeddings of each node are concatenated to obtain the high-dimensional embedding of each node. Then, the high-dimensional embeddings of the two nodes are added together to obtain the high-dimensional embedding of the candidate edge.

[0052] To enhance the model's understanding of the geometric characteristics of the edges, the edge Transformer of the embodiment introduces a random self-supervised geometric enhancement method in the training process. Specifically, a point is randomly inserted on each candidate edge, and the position is determined by a random interpolation coefficient λ, that is, the point is at a random position between the two endpoints. The position coordinate code of the interpolation point is extracted, and the semantic label-background of the surrounding room of the interpolation point is set to a zero vector to extract the corresponding embedding. The high-dimensional embedding of the interpolation point is obtained by adding the two. The high-dimensional embedding of the interpolation point is added to the high-dimensional embedding of the candidate edge. The high-dimensional embedding of all candidate edges forms the high-dimensional embedding of the candidate edge set, which is input into the edge Transformer.

[0053] The structure of the edge Transformer is similar to that of the node Transformer, and it contains a Transformer decoder and a downstream prediction head. The Transformer decoder processes the candidate edge set, and the decoder is composed of multiple layers stacked in order, each containing an "instance normalization layer", a "global self-attention layer", a "residual connection", an "instance normalization layer", a "fully connected layer", and a "residual connection". Through the self-attention mechanism, the edge Transformer can capture the complex dependency between the candidate edges. The output of the Transformer decoder is the high-dimensional embedding of the candidate edge set after Transformer mapping, which is input into the downstream prediction head for attribute prediction. The downstream prediction head is based on a fully connected layer, and there are a total of 2, with output dimensions of 2 and 1 respectively. The former is the two-class probability of the true / false binary classification of the candidate edge, and the latter is the random interpolation coefficient λ.

[0054] Since the edge is undirected, to avoid ambiguity introduced by directionality, when λ is greater than 0.5, it is replaced by 1−λ in the embodiment to ensure that the value of λ is within the range of [0, 0.5]. In this way, the edge Transformer is forced to pay attention to the overall geometric characteristics of the edge during the learning process, rather than just relying on the information of the endpoints. The loss function consists of the cross-entropy loss of edge classification and the L1 regression loss of interpolation coefficient. By optimizing these two objectives simultaneously, the edge Transformer can more accurately determine the existence of the edge by more deeply understanding the geometric characteristics of the edge. The candidate edge set determined to be true is obtained as the edge set. Thus, the structural graph composed of the node set and the edge set is obtained, which contains the complete information of the floor plan.

[0055] Step 3, based on the structural graph of the floor plan obtained in step 2, the structural graph is traversed by a minimum loop operation of the structural graph of the floor plan, and all minimum loops are obtained as all rooms, thereby extracting the room set from the structural graph; the structural graph and the room set are combined, thereby obtaining the vectorized floor plan.

[0056] Specifically, the minimum loop operation process of traversing the graph is to number each node of the structure graph in an arbitrary order in advance, set all edges of the structure graph as to-be-visited edges; take each to-be-visited edge of the structure graph as a starting edge, define the starting direction as the end point number from small to large, iteratively find the edge located on the left side of the current edge and having the smallest angle with the current edge among all edges of the next step, until the starting edge is found in the next step, forming a loop; if the rotation direction of the loop is counterclockwise, it is a minimum loop, record the minimum loop, and set all edges visited according to the end point number from small to large in the process of forming the minimum loop as visited; if the rotation direction of the loop is clockwise, it is a maximum loop, do not record the loop, and set all edges visited according to the end point number from small to large in the process of forming the loop as visited; iterate the above process until there is no to-be-visited edge, and obtain all minimum loops. As shown in the following table, a formation case of a minimum loop is shown. Figure 4

[0057] For each room, select the room semantic information with the highest frequency of occurrence in the room semantic information associated with all the corner points constituting the room (excluding outdoor) as the final room category; if multiple categories have the highest frequency, consider the scarcity factor of the category, and determine the room type according to the following priority order: storage room > bathroom > kitchen > bedroom > balcony > living room. Thus, the vectorized house plan is obtained.

[0058] The preferred embodiments of the present application are described in detail above with reference to the accompanying drawings, and the examples described in the present application are only used to describe the preferred embodiments of the present application, and do not limit the concept and scope of the present application. In the above specific embodiments, each specific technical feature described above can be combined in any appropriate manner without contradiction, and such combination should also be considered as disclosed by the present disclosure as long as it does not deviate from the technical concept of the present application. In order to avoid unnecessary repetition, the present application does not further describe various possible combinations.

[0059] The present application is not limited to the specific details in the above embodiments, and various modifications and improvements of the technical solutions of the present application made by those skilled in the art within the technical concept of the present application and without departing from the design idea of the present application should fall within the protection scope of the present application. The technical content claimed by the present application has been fully recorded in the claims.​

Claims

1. A method for automatically generating a vectorized floor plan of a house, characterized in that, The method comprises the following steps: Step 1, using a node generation model based on a diffusion model, using random noise sampled from a standard Gaussian distribution as input of the node generation model, generating a preliminary node set from the node generation model, each node in the preliminary node set representing a wall intersection, each node having position coordinates, semantic information of surrounding rooms and background attributes; Then, according to the background attribute value of each node in the preliminary node set, it is judged whether the corresponding node is a filling node, and the filling nodes in the preliminary node set are removed to form a node set of the floor plan from the remaining nodes; Step 2, using an edge prediction model based on a neural network, forming a candidate edge set with any two nodes in the node set of the floor plan obtained in step 1, and using the candidate edge set as input of the edge prediction model, predicting whether there is an edge between each pair of nodes from the edge prediction model, thereby obtaining an edge set of the floor plan, the edges in the edge set representing the walls in the floor plan, combining the node set and the edge set of the floor plan, thereby obtaining a structure graph of the floor plan; Step 3, by traversing the minimum loop operation of the structure graph of the floor plan, traversing the structure graph to obtain all minimum loops, which are all rooms, thereby extracting a room set from the structure graph; combining the structure graph and the room set, thereby obtaining a vectorized house plan; The edge prediction model based on a neural network in step 2 uses a trained Transformer, denoted as edge Transformer. The edge Transformer introduces a random self-supervised geometric enhancement method during training. For each candidate edge, a point is randomly inserted on the edge, and the edge Transformer is required to predict the random coefficient of the inserted point, thereby enhancing the understanding of the edge Transformer to the overall geometric features of the edge.

2. The method for automatically generating a vectorized floor plan according to claim 1, wherein, The neural network contained in the diffusion model on which the node generation model in step 1 is based uses a trained Transformer, denoted as node Transformer.

3. The method for automatically generating a vectorized floor plan according to claim 2, wherein, The node Transformer uses a multi-precision alignment loss function during training, which optimizes the alignment error under multiple precisions by mixing, thereby enhancing the geometric alignment effect of the node Transformer.

4. The method for automatically generating a vectorized floor plan according to claim 1, wherein, In step 3, the minimum loop operation process of the graph is as follows: taking each edge of the structure graph as a starting edge, defining a certain end point to another end point as a starting direction, iteratively finding all edges in the next step, which are located on the left side of the current edge and have the smallest angle with the current edge, until the starting edge is found in the next step, forming a loop, if it is a non-repeating minimum loop, it is recorded, and all minimum loops are obtained by traversing each edge of the structure graph.

5. The method for automatically generating a vectorized floor plan according to claim 1, wherein, Step 3 further comprises: for each room, selecting the room semantic information with the highest frequency of occurrence among all the room semantic information associated with the corner points constituting the room as the final room type; if multiple categories have the highest frequency, considering the scarcity factor of the category, determining the room type in the following priority order: storage room > bathroom > kitchen > bedroom > balcony > living room.

Images