Architectural design and analysis method based on point cloud 3D reconstruction

Abstract

The present invention discloses an architectural design and analysis method based on point cloud three-dimensional reconstruction, and belongs to the technical field of building modeling. The present invention obtains the external point cloud data of the house, combines the house CAD design drawings (CAD images), and uses the collected data set to train the indoor scene inference network; uses the adversarial generative network (GANs) to infer and generate multi-view images of the indoor scene, and then generates indoor point clouds through these images, and finally fuses the indoor and outdoor point clouds, so as to infer and design the internal layout of the house without knowing the internal scene layout of the house, and establish a house model, which not only protects the privacy of the homeowner, but also improves the design efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of building modeling, and in particular to a method for architectural design and analysis based on point cloud three-dimensional reconstruction. Background Art

[0002] In the field of house space layout design, obtaining point cloud data of the interior scene of the house is crucial for design and modeling. However, due to privacy and house ownership issues, homeowners are often reluctant to allow outsiders to obtain detailed data about the interior of the house. This limits the work of designers and architects in house space layout design because they cannot obtain accurate interior layout information, especially details such as furniture placement. Traditional house interior layout design relies on direct physical measurements or detailed interior point cloud data, which is not only time-consuming and labor-intensive, but also, as mentioned above, is often difficult to implement due to privacy issues. In addition, a single point cloud data source often cannot obtain the relevant parameters of the building unit components and fully express the indoor and outdoor model reconstruction problem. This leads to problems such as incomplete information, inefficiency, and incompleteness in house space layout design.

[0003] The above problems need to be solved urgently. To this end, the present invention proposes an architectural design and analysis method based on point cloud three-dimensional reconstruction. Summary of the invention

[0004] The technical problem to be solved by the present invention is: how to infer the indoor scene layout image only through the point cloud of the building facade and the CAD design drawings (CAD images) when it is impossible to obtain the point cloud data of the interior space layout of the house, and generate the indoor point cloud through the image, and then obtain a complete indoor and outdoor integrated point cloud model, thereby providing an architectural design and analysis method based on point cloud 3D reconstruction.

[0005] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0006] Step S1: collecting point cloud data of the facade of a building, and preprocessing the collected point cloud data of the facade of a building to obtain a clean point cloud of the facade of a building;

[0007] Step S2: using the improved FusionVit multimodal deep learning network as the indoor scene inference network, inputting clean building facade point clouds and CAD images, training the indoor scene inference network, and obtaining an indoor scene inference model;

[0008] Step S3: inputting the clean building facade point cloud and the corresponding CAD image of the indoor scene to be inferred into the indoor scene inference model, using the indoor scene inference model to predict and generate the indoor scene image, and obtaining the multi-view indoor scene image;

[0009] Step S4: recovering the indoor scene point cloud of the house and building according to the multi-view indoor scene image, and then fusing the indoor scene point cloud of the house and building with the original clean facade point cloud of the house and building to obtain a point cloud containing the complete indoor and outdoor 3D structure of the house and building;

[0010] Step S5: Based on the point cloud including the complete indoor and outdoor 3D structure of the house and building, the house and building are modeled using modeling software.

[0011] Furthermore, in step S1, the specific processing process is as follows:

[0012] Step S11: Survey the surrounding environment of the building and plan the external scanning sites of the building;

[0013] Step S12: For buildings that are less than the set height, use a stand-type 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building;

[0014] Step S13: For buildings with a height greater than or equal to a set height, use a drone equipped with a 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building;

[0015] Step S14: Use the ICP algorithm to align the building facade point cloud data to obtain the aligned building facade point cloud, then use the cloth filter algorithm to filter out the ground point cloud, and then use CloudCompare software to reduce noise and crop, and finally obtain a clean building facade point cloud.

[0016] Furthermore, in step S1, the specific processing process is as follows:

[0017] Step S11: Survey the surrounding environment of the building and plan the external scanning sites of the building;

[0018] Step S12: For buildings that are less than the set height, use a stand-type 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building;

[0019] Step S13: For buildings with a height greater than or equal to a set height, use a drone equipped with a 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building;

[0020] Step S14: Use the ICP algorithm to align the building facade point cloud data to obtain the aligned building facade point cloud, then use the cloth filter algorithm to filter out the ground point cloud, and then use CloudCompare software to reduce noise and crop, and finally obtain a clean building facade point cloud.

[0021] Furthermore, in step S2, the improved FusionVit multimodal deep learning network includes a CameraViT network, a LidarViT network, a MixViT network and a 3D-GANs network, and the indoor scene inference network training process is as follows:

[0022] Step S21: Use the CameraViT network to divide the CAD image with RGB information into multiple blocks, flatten each block into a one-dimensional vector, concatenate the one-dimensional vector sequence into a matrix, add a position embedding to the one-dimensional vector to represent its relative position in the original image, and then concatenate a token for classification to obtain a sequence feature matrix with position embedding added, which is encoded through the image Transformer encoder to obtain image features;

[0023] Step S22: Use the LidarViT network to divide the clean building facade point cloud into multiple cubes and remove empty cubes, retain non-empty cubes, randomly sample the non-empty cubes, and then flatten the cubes obtained by the sampling process into one-dimensional vectors, splice the one-dimensional vector sequence into a sequence feature matrix, add a position embedding representing its relative position in the original point cloud to each one-dimensional vector, and then splice a token for classification to obtain the sequence feature matrix after adding the position embedding, and encode it through the point cloud Transformer encoder to obtain the point cloud features;

[0024] Step S23: using the MixViT network to fuse the two-dimensional image features and the three-dimensional point cloud features extracted by the CameraViT network and the LidarViT network through a splicing operation, and generating a fused feature representation through MLP;

[0025] Step S24: Input the fused feature representation output by the MixViT network into the 3D-GANs adversarial generative network for adversarial training to obtain an indoor scene inference model that meets the performance indicators after training.

[0026] Furthermore, in step S21, the specific processing process is as follows:

[0027] S211: Input the CAD image with RGB information of the building into the CameraViT network and define the image size;

[0028] S212: The CameraViT network divides the CAD image with RGB information into multiple blocks, flattens the blocks and inputs them into a multi-layer perceptron for processing to obtain multiple one-dimensional vectors, and concatenates the one-dimensional vectors into a matrix;

[0029] S213: Embed a learnable parameter E in a matrix of multiple one-dimensional vectors c , introduce a learnable class_token, initialize it to a zero vector, add it to the beginning of the matrix, and then embed the position Added to the matrix to retain the position information, the first sequence feature matrix is ​​obtained; where the position embedding The row and column index of each block in the image grid and the position information of the block in the image global coordinate system;

[0030] S214: Input the first sequence feature matrix into the image Transformer encoder for encoding to obtain image features.

[0031] Furthermore, in step S214, the image Transformer encoder includes multiple Transformer layers, each Transformer layer includes multiple alternating MSA blocks and MLP blocks, and layer normalization processing is performed before each MSA block and MLP block.

[0032] Furthermore, in step S22, the specific processing process is as follows:

[0033] S221: Segment the clean building facade point cloud to obtain multiple cubes and remove empty cubes, retaining non-empty cubes;

[0034] S222: performing random sampling of a fixed number of samples on the retained non-empty cubes to obtain sampled non-empty cubes;

[0035] S223: For each point cloud data set in the non-empty cube, including t point cloud data points, the three-dimensional coordinate mean of the point cloud data points inside each non-empty cube is calculated as the local mean, that is, the centroid of all points, and the offset of each point cloud data point relative to the centroid in the three-axis position is calculated, and added to the point cloud data set in the non-empty cube to form an input feature set. In the input feature set, each point cloud data point of each non-empty cube includes 6 coordinates, the first three coordinates are the original coordinates of each point cloud data point in the non-empty cube, and the last three coordinates are the offset of each point cloud data point relative to the centroid;

[0036] S224: After the coordinate data of each non-empty cube in the above step S223 is feature encoded and converted, a set of encoding features is obtained, that is, a one-dimensional vector sequence is obtained. The one-dimensional vector sequence is spliced ​​into a sequence feature matrix, and a learnable parameter E is embedded in the matrix. l , introduce a learnable class_token, initialize it to a zero vector, add it to the beginning of the matrix, and then embed the position Add to the matrix to retain the position information, and get the second sequence feature matrix after adding the position embedding, where the position embedding is the position coordinates of each non-empty cube in the three-dimensional space grid where the original point cloud is located;

[0037] S225: The second sequence feature matrix is ​​encoded through the point cloud Transformer encoder to obtain point cloud features.

[0038] Furthermore, in step S224, the point cloud Transformer encoder includes multiple Transformer layers, each Transformer layer includes MLP, FCN and MaxPooling layers, and the MLP, FCN and MaxPooling layers are connected in sequence.

[0039] Furthermore, in step S3, based on the multi-perspective indoor scene images, a sparse point cloud is reconstructed by using a motion structure recovery algorithm SFM, and then a dense point cloud is reconstructed by using a multi-perspective stereo vision method, thereby obtaining a point cloud of the indoor scene of the building.

[0040] Furthermore, in step S3, the specific process of fusing the indoor scene point cloud of the building with the original clean facade point cloud of the building is as follows:

[0041] S31: First, use CloudCompare software to manually coarsely align the original clean building facade point cloud and the building interior scene point cloud;

[0042] S32: Then use the ICP algorithm for precise registration to obtain a point cloud containing a complete 3D structure of indoor and outdoor areas.

[0043] Furthermore, in step S4, the specific process of building modeling is to import the point cloud containing the complete 3D structure of indoor and outdoor into the modeling software, model the point cloud model by slicing, cropping, and measuring distance, and generate a BIM model.

[0044] Compared with the prior art, the present invention has the following advantages: The architectural design and analysis method based on point cloud three-dimensional reconstruction has the following advantages compared with the prior art:

[0045] (1) By using only the exterior point cloud data and CAD design drawings of the house, the indoor scene layout image is inferred without infringing the privacy of the interior of the house, thus protecting the privacy of the homeowner.

[0046] (2) Deep learning networks and generative adversarial networks (GANs) are used to infer and generate multi-view images of indoor scenes, and then indoor point clouds are generated from the images, which greatly reduces the workload of field measurements and improves design efficiency.

[0047] (3) By integrating indoor and outdoor point cloud data, a complete indoor and outdoor integrated point cloud model is constructed, which improves the completeness and accuracy of the model.

[0048] (4) Combining 3D laser scanning technology, drone technology, deep learning network and 3D-GANs technology, it provides a novel technology integration solution and innovatively solves the problems in existing technologies.

[0049] (5) Depending on the height of the building, flexibly use a stand-mounted 3D laser scanner or a drone-mounted 3D laser scanner to collect data to meet the needs of different scenarios.

[0050] (6) Reduce manpower requirements and costs through automation and intelligent methods.

[0051] (7) It reduces physical intervention inside the house and is more environmentally friendly.

[0052] In summary, the present invention effectively solves the problems existing in the prior art through technological innovation and method improvement, and provides a more efficient, accurate and privacy-preserving architectural design and analysis method. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of a method for architectural design and analysis based on point cloud 3D reconstruction in an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of a processing flow of an indoor scene inference network in an embodiment of the present invention;

[0055] Figure 3 It is a schematic diagram of the location information embedding process in the CameraVit network in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0057] like Figure 1As shown, this embodiment provides a technical solution: an architectural design and analysis method based on point cloud 3D reconstruction, comprising the following steps:

[0058] Step S1: Data collection and preprocessing

[0059] For buildings with a height of no more than 30m, a stand-mounted 3D laser scanner is used to collect point cloud data of the building facades. For buildings with a height of more than 30m, a drone-mounted 3D laser scanner is used to collect point cloud data of the building facades. The collected point cloud data of the building facades are registered, denoised, cropped, and other operations are performed to obtain clean point clouds of the building facades.

[0060] Step S2: Indoor scene estimation network training

[0061] The improved FusionVit multimodal deep learning network is used as the indoor scene inference network. Clean building facade point clouds and CAD images are input to train the indoor scene inference network and obtain the indoor scene inference model.

[0062] Step S3: Indoor scene estimation generation

[0063] Input the clean building facade point cloud and the corresponding CAD image of the indoor scene to be inferred into the indoor scene inference model, use the trained indoor scene inference model to predict and generate indoor scene images, and obtain multi-view indoor scene images;

[0064] Step S4: Point cloud data fusion

[0065] The indoor scene point cloud of the house and building is restored based on the multi-view indoor scene images, and then the indoor scene point cloud of the house and building is fused with the original clean facade point cloud of the house and building to obtain a point cloud containing the complete indoor and outdoor 3D structure of the house and building;

[0066] Step S5: Architectural design model reconstruction

[0067] Based on the point cloud containing the complete 3D structure of the interior and exterior of the house and building, the house and building are modeled using modeling software.

[0068] In this embodiment, in step S1, the specific processing process is as follows:

[0069] Step S11: Survey the surrounding environment of the building and plan the external scanning sites of the building;

[0070] Step S12: For buildings with a height less than 30m, use a stand-type 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building;

[0071] Step S13: For buildings with a height greater than or equal to 30 m, use a drone equipped with a 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building;

[0072] Step S14: Use the commonly used ICP algorithm to align the building facade point cloud data to obtain the aligned building facade point cloud, then use the cloth filter algorithm to filter out the ground point cloud, use CloudCompare software to reduce noise and crop, and finally obtain a clean building facade point cloud.

[0073] like Figure 2 As shown, in this embodiment, in step S2, the improved FusionVit multimodal deep learning network includes a CameraViT network, a LidarViT network, a MixViT network and a 3D-GANs network, and the indoor scene inference network training process is as follows:

[0074] Step S21: The CameraViT network divides the CAD image into multiple patches and flattens them into a one-dimensional feature vector; then the one-dimensional feature vector is encoded through the Transformer encoder to obtain a two-dimensional image feature. It should be noted that the CAD image is obtained from the building design data, and each floor of the building corresponds to a CAD image. In this embodiment, the building has only one floor, so only one CAD image is input.

[0075] As a further step, the input CAD image (flat image) is divided into multiple fixed-size patches, each of which is flattened and converted into a one-dimensional vector through a linear layer. At the same time, each one-dimensional vector is attached with a position embedding representing its relative position in the original image. The one-dimensional vectors processed above are concatenated into a matrix as input and fed into the Transformer encoder (including a series of Transformer layers) for encoding. Each Transformer layer contains a multi-head self-attention block and an MLP block.

[0076] The specific process is as follows:

[0077] (1) Input and preprocessing: The CAD image (RGB image) of the building is input into the CameraViT network. The image size is defined as ,in, and Represent the height and width dimensions of the image respectively;

[0078] (2) Image segmentation and patch embedding: The CameraViT network segments the CAD image into N c Two-dimensional patches, each with a side length of v cH and v cW , so that the image I is divided into N c Patches ,in, Indicates the number of patches;

[0079] The segmented image patches are flattened and input into a multi-layer perceptron (MLP, as a linear layer) for processing to obtain a total of N c One-dimensional feature vector ; For each encoding feature i, there is , this N c One-dimensional feature vectors form a matrix.

[0080] (3) First here A learnable parameter is embedded in a matrix of one-dimensional feature vectors In the Transformer model, a special tag is needed to aggregate global information and perform classification, so the CameraViT network introduces a learnable class_token, which is initialized to a zero vector, that is, , added to the beginning of the matrix, and then embedded in the position Added to the patch embedding to preserve position information, we get +1 sequence feature matrix composed of feature vectors with added position embedding, that is ; Contains the row and column index of each patch in the original image grid, as well as the position information of the small block in the global coordinate system of the image, that is, the coordinates (x, y) of the upper left corner of each small block.

[0081] (4) Transformer encoder: The encoder consists of multiple Transformer layers, each of which consists of alternating multi-head self-attention (MSA) blocks and MLP blocks. LayerNorm (LN) is applied before each block, and residual connections are applied after each block:

[0082] ;

[0083] For all .

[0084] in, represents the feature representation after processing by the lth Transformer layer, but before applying the MLP. Here l represents the index of the block, starting from 1 and ending at ; represents the feature representation before the lth Transformer layer, i.e. The output of each block; the MSA block is used to capture the dependencies between different positions in the sequence; LN is used to normalize all features of each sample to speed up training and improve model stability; : Residual connection, that is, The output of each block is added to the features processed by the MSA block, which helps alleviate the gradient vanishing problem and allows the model to learn deeper feature representations. MLP is used as a feedforward network in the Transformer to perform further nonlinear transformations on the features processed by the self-attention mechanism. It represents the final feature representation after processing by the lth Transformer layer, including the processing results of MSA and MLP; Represents the number of Transformer encoder layers in the CameraViT model, that is, the model depth.

[0085] The entire formula describes the processing flow of each block in the Transformer encoder:

[0086] First, the output of the previous block Apply LayerNorm;

[0087] Then, the normalized features are input into the MSA block, and the processed results are again compared with Add to achieve residual connection;

[0088] Next, the result of the residual connection Apply LayerNorm;

[0089] Finally, the normalized features are input into the MLP block and combined with Add and realize residual connection again to get the final output .

[0090] (5) Output features: final image representation features is the output from the Transformer encoder: ,in, Represents the final image representation feature, which is the output after being processed by the Transformer encoder and is used for subsequent tasks; Indicates passing The feature representation after processing by Transformer layers. Represents the number of layers of the Transformer encoder in the CameraViT network, that is, the model depth.

[0091] Step S22: The LidarViT network divides the clean building facade point cloud into multiple small cubes (voxels, cubes) and processes non-empty cubes. For each non-empty cube, the 3D coordinate mean of the internal point cloud data points is calculated as the local mean (centroid), and the local mean is used as the feature input, encoded through MLP and fully connected network (FCN) (point cloud Transformer encoder) to obtain point cloud features.

[0092] The LidarVit network divides the point cloud data into multiple fixed-size patches (small cubes), each of which is flattened and converted to a one-dimensional vector through a linear layer. Each one-dimensional vector is attached with a position embedding representing its relative position in the original point cloud. The one-dimensional vectors processed above are concatenated into a matrix as input and fed into the point cloud Transformer encoder for encoding.

[0093] The specific process is as follows:

[0094] (1) Point cloud preprocessing: The LidarViT network first preprocesses the raw point cloud data, including removing empty cubes to reduce the computational burden. The point cloud data is divided into small cubes, each with a side length of Since the point cloud is a sparse representation, most of the cubes are empty, so these empty cubes are removed first.

[0095] (2) Random sampling: Random sampling is performed on cubes (non-empty cubes) containing a large number of point clouds, with a fixed number of samples R, and the obtained The purpose of this is to save computing resources and reduce the impact of point cloud density changes on detection.

[0096] (3) Feature encoding: A non-empty cube is represented by ,in, Indicates the number of cubes.

[0097] in, represents the number of non-empty cubes; X P represents the feature set of all non-empty cubes, which is a real matrix with dimension ,here , ,and are the voxel sizes of a non-empty cube in height, width, and depth, respectively; H P , WP , D P Respectively represent the dimensions of the original point cloud data in height, width, and depth.

[0098] For each point cloud data set in a non-empty cube , contains t point cloud data points, t≤T, first calculate the local mean as the centroid of all points, and then increase the relative offset of each point relative to the centroid to obtain the input feature set .

[0099] in, Represents a collection of point cloud data in a non-empty cube, each point is represented by its three-dimensional coordinates Indicates; t represents the number of point cloud data points contained in each non-empty cube; T is the maximum number of points contained in each cube; Represents the coordinates of the local mean (center of mass) on the x, y, and z axes respectively. These means are obtained by calculating the average of the coordinates of all points in the non-empty cube; represents the input feature set, which contains each point and its relative offset to the centroid of the cube. , whose input feature set is , where the first three coordinates are the original coordinates of the point, and the last three coordinates are the offset of the point relative to the centroid of the cube.

[0100] (4) MLP and FCN processing: The input features of each cube (the set of 6 coordinates of each point) are flattened into one-dimensional features and processed through MLP to obtain the encoded features. Each coded feature Transformed to feature space through a fully connected network (FCN) FCN includes linear layer, batch normalization (BN) layer and Sigmoid Linear Units (SiLU) layer. By element-by-element max pooling (MaxPooling) Get local aggregation features Finally, each and Features that combine to form point-wise connections The output encoding feature is .

[0101] ;

[0102] (5) Position embedding: Similar to CameraVit position embedding, LidarVit adds another learnable embedding before the encoded block feature sequence , plus a special embedding vector , used to represent the category information in the sequence, and finally add the position embedding , the resulting embedding vector sequence , as the input of the point cloud Transformer encoder. The final learned point cloud representation feature Output from Transformer encoder Generated by LayerNorm: . Represents the position of each non-empty cube in 3D space, that is, its coordinates in the three-dimensional grid, expressed as (x, y, z), which is used to indicate the position of the cube in 3D space.

[0103] Step S23: The MixViT network fuses the features extracted by the CameraViT network and the LidarViT network. This is usually achieved through a concatenation operation, combining the features of the two modalities and generating a fused feature representation through an MLP.

[0104] Specific:

[0105] (1) Feature representation fusion: The MixViT network first concatenates the feature representations obtained by the CameraViT network and the LidarViT network. These features are represented as H c and H l , corresponding to the features of the image and point cloud, respectively.

[0106] (2) Feature dimension adjustment: The concatenated features are converted into Encoded features ,in is the number of encoded features.

[0107] (3) Feature fusion: In this way, the MixViT network enables the features learned by the CameraViT network and the LidarViT network to be effectively fused, providing richer feature representation for subsequent indoor scene prediction tasks.

[0108] Step S24: Output features of MixViT network Input into the 3D-GANs adversarial generation network. The goal of the 3D-GANs adversarial generation network is to generate indoor scene images of houses and buildings. The 3D-GANs adversarial generation network contains a generator and a discriminator. The generator is responsible for creating indoor scene images that are as realistic as possible, while the discriminator tries to distinguish between real indoor scene images and indoor scene images generated by the generator. Through adversarial training, the generator learns to create more and more realistic indoor scene images.

[0109] In step S3, the detailed process of generating a point cloud with an indoor scene is as follows:

[0110] (1) Use scanners to collect a large amount of building facade point cloud data and preprocess them to form a training dataset;

[0111] (2) CAD images from building design data;

[0112] (3) The building facade point cloud and CAD image are input into the improved FusionVit network. The MixVit network fuses the image and point cloud features extracted by the CameraVit network and the LidarVit network, outputs the fused features into the 3D-GANs adversarial generative network, predicts the indoor scene, outputs multi-view indoor scene images, compares them with the indoor scenes of actual buildings, analyzes the accuracy of the training model, adjusts the parameters and optimizes the model.

[0113] (4) Input the facade point cloud and CAD image of the building whose indoor scene actually needs to be inferred into the trained indoor scene inference model to perform indoor scene prediction and output multi-view indoor scene images;

[0114] (5) Based on the multi-view indoor scene images, the sparse point cloud is reconstructed using the motion structure recovery algorithm (SFM), and then the dense point cloud is reconstructed using the multi-view stereo vision method (MVS) to obtain the indoor scene point cloud of the building.

[0115] It should be noted that the motion structure recovery algorithm SFM estimates the camera's pose (i.e., position and attitude) by analyzing the feature points in the image and using the correspondence between the feature points in different images, and reconstructs the 3D structure of the scene based on this pose information. The multi-view stereo vision method (MVS) specifically refers to the sparse point cloud-based multi-view stereo (Sparse-to-Dense MVS) method, which uses the sparse point cloud as the starting point to reconstruct the dense point cloud. The goal of the dense point cloud reconstruction is to fill the gaps between the sparse point clouds, making the 3D reconstruction more accurate.

[0116] In step S3, the specific process of fusing the indoor scene point cloud of the house building with the original clean facade point cloud of the house building is as follows:

[0117] (1) First, use CloudCompare software to manually coarsely align the outdoor and indoor point clouds;

[0118] (2) Then use the ICP algorithm for precise registration to obtain a point cloud containing the complete 3D structure of indoor and outdoor areas.

[0119] In step S4, the specific process of building modeling is: importing the point cloud containing the complete 3D structure of indoor and outdoor into the modeling software, modeling the point cloud model by slicing, cropping, measuring distance, etc., and generating a BIM model.

[0120] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. The architectural design and analysis method based on point cloud 3D reconstruction is characterized by: The following steps are involved: Step S1: collecting point cloud data of the facade of a building, and preprocessing the collected point cloud data of the facade of a building to obtain a clean point cloud of the facade of a building; Step S2: using the improved FusionVit multimodal deep learning network as the indoor scene inference network, inputting clean building facade point clouds and CAD images, training the indoor scene inference network, and obtaining an indoor scene inference model; In step S2, the improved FusionVit multimodal deep learning network includes a CameraViT network, a LidarViT network, a MixViT network and a 3D-GANs network, and the indoor scene inference network training process is as follows: Step S21: Use the CameraViT network to divide the CAD image with RGB information into multiple blocks, flatten each block into a one-dimensional vector, concatenate the one-dimensional vector sequence into a matrix, add a position embedding to the one-dimensional vector to represent its relative position in the original image, and then concatenate a token for classification to obtain a sequence feature matrix with position embedding added, which is encoded through the image Transformer encoder to obtain image features; Step S22: Use the LidarViT network to divide the clean building facade point cloud into multiple cubes and remove empty cubes, retain non-empty cubes, randomly sample the non-empty cubes, and then flatten the cubes obtained by the sampling process into one-dimensional vectors, splice the one-dimensional vector sequence into a sequence feature matrix, add a position embedding representing its relative position in the original point cloud to each one-dimensional vector, and then splice a token for classification to obtain the sequence feature matrix after adding the position embedding, and encode it through the point cloud Transformer encoder to obtain the point cloud features; Step S23: using the MixViT network to fuse the two-dimensional image features and the three-dimensional point cloud features extracted by the CameraViT network and the LidarViT network through a splicing operation, and generating a fused feature representation through MLP; Step S24: input the fused feature representation output by the MixViT network into the 3D-GANs adversarial generative network for adversarial training, and obtain an indoor scene inference model that meets the performance indicators after training; Step S3: inputting the clean building facade point cloud and the corresponding CAD image of the indoor scene to be inferred into the indoor scene inference model, using the indoor scene inference model to predict and generate the indoor scene image, and obtaining the multi-view indoor scene image; Step S4: recovering the indoor scene point cloud of the house and building according to the multi-view indoor scene image, and then fusing the indoor scene point cloud of the house and building with the original clean facade point cloud of the house and building to obtain a point cloud containing the complete indoor and outdoor 3D structure of the house and building; Step S5: Based on the point cloud including the complete indoor and outdoor 3D structure of the house and building, the house and building are modeled using modeling software.

2. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 1, characterized in that: In step S1, the specific processing process is as follows: Step S11: Survey the surrounding environment of the building and plan the external scanning sites of the building; Step S12: For buildings that are less than the set height, use a stand-type 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building; Step S13: For buildings with a height greater than or equal to a set height, use a drone equipped with a 3D laser scanner to scan the facade of the building to obtain point cloud data of the facade of the building; Step S14: Use the ICP algorithm to align the building facade point cloud data to obtain the aligned building facade point cloud, then use the cloth filter algorithm to filter out the ground point cloud, and then use CloudCompare software to reduce noise and crop, and finally obtain a clean building facade point cloud.

3. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 1, characterized in that: In step S21, the specific processing process is as follows: S211: Input the CAD image with RGB information of the building into the CameraViT network and define the image size; S212: The CameraViT network divides the CAD image with RGB information into multiple blocks, flattens the blocks and inputs them into a multi-layer perceptron for processing to obtain multiple one-dimensional vectors, and concatenates the one-dimensional vectors into a matrix; S213: Embed a learnable parameter E in a matrix of multiple one-dimensional vectors c , introduce a learnable class_token, initialize it to a zero vector, add it to the beginning of the matrix, and then embed the position Added to the matrix to retain the position information, the first sequence feature matrix is ​​obtained; where the position embedding The row and column index of each block in the image grid and the position information of the block in the image global coordinate system; S214: Input the first sequence feature matrix into the image Transformer encoder for encoding to obtain image features.

4. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 3 is characterized in that: In step S214, the image Transformer encoder includes multiple Transformer layers, each Transformer layer includes multiple alternating MSA blocks and MLP blocks, and layer normalization processing is performed before each MSA block and MLP block.

5. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 3, characterized in that: In step S22, the specific processing process is as follows: S221: Segment the clean building facade point cloud to obtain multiple cubes and remove empty cubes, retaining non-empty cubes; S222: performing random sampling of a fixed number of samples on the retained non-empty cubes to obtain sampled non-empty cubes; S223: For each point cloud data set in the non-empty cube, including t point cloud data points, the three-dimensional coordinate mean of the point cloud data points inside each non-empty cube is calculated as the local mean, that is, the centroid of all points, and the offset of each point cloud data point relative to the centroid in the three-axis position is calculated, and added to the point cloud data set in the non-empty cube to form an input feature set. In the input feature set, each point cloud data point of each non-empty cube includes 6 coordinates, the first three coordinates are the original coordinates of each point cloud data point in the non-empty cube, and the last three coordinates are the offset of each point cloud data point relative to the centroid; S224: After the coordinate data of each non-empty cube in the above step S223 is feature encoded and converted, a set of encoding features is obtained, that is, a one-dimensional vector sequence is obtained. The one-dimensional vector sequence is spliced ​​into a sequence feature matrix, and a learnable parameter E is embedded in the matrix. l , introduce a learnable class_token, initialize it to a zero vector, add it to the beginning of the matrix, and then embed the position Add to the matrix to retain the position information, and get the second sequence feature matrix after adding the position embedding, where the position embedding is the position coordinates of each non-empty cube in the three-dimensional space grid where the original point cloud is located; S225: The second sequence feature matrix is ​​encoded through the point cloud Transformer encoder to obtain point cloud features.

6. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 5 is characterized in that: In step S224, the point cloud Transformer encoder includes multiple Transformer layers, each Transformer layer includes MLP, FCN and MaxPooling layers, and the MLP, FCN and MaxPooling layers are connected in sequence.

7. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 1, characterized in that: In step S3, based on the multi-view indoor scene images, a sparse point cloud is reconstructed by using a motion structure recovery algorithm SFM, and then a dense point cloud is reconstructed by using a multi-view stereo vision method, thereby obtaining a point cloud of the indoor scene of a building.

8. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 1, characterized in that: In step S3, the specific process of fusing the indoor scene point cloud of the building with the original clean facade point cloud of the building is as follows: S31: First, use CloudCompare software to manually coarsely align the original clean building facade point cloud and the building interior scene point cloud; S32: Then use the ICP algorithm for precise registration to obtain a point cloud containing a complete 3D structure of indoor and outdoor areas.

9. The architectural design and analysis method based on point cloud 3D reconstruction according to claim 1, characterized in that: In step S4, the specific process of building modeling is to import the point cloud containing the complete 3D structure of indoor and outdoor into the modeling software, model the point cloud model by slicing, cropping, and measuring distance, and generate a BIM model.

Images