Depth visual perception method and system for building space analysis

By employing a deep learning approach enhanced with point cloud completion preprocessing, transfer learning, and a two-level attention mechanism, the accuracy and efficiency issues in understanding indoor spaces in existing technologies are addressed. This approach achieves high-precision point cloud data completion and semantic segmentation, thereby improving the analytical capabilities of architectural spaces.

CN118967917AActive Publication Date: 2024-11-15HUAZHONG UNIV OF SCI & TECH

Patent Information

Application Number
CN202410450865.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-04-15
Publication Date
2024-11-15
Estimated Expiration
2044-04-15

AI Technical Summary

Technical Problem

In understanding indoor spaces, existing technologies, such as image-based methods, lack sufficient accuracy, while point cloud methods, although highly accurate, suffer from incomplete data and reconstruction challenges in complex environments, making it difficult to effectively analyze the indoor spaces of completed buildings.

Method used

A point cloud understanding and segmentation network model enhanced by point cloud completion preprocessing, transfer learning pre-training, and a two-level attention mechanism is adopted. Combined with visual interpretability analysis and parametric spatial modeling, architectural space is analyzed through deep learning methods.

Benefits of technology

It achieves high-precision point cloud data completion and semantic segmentation, improves the model's generalization ability and interpretability, shortens modeling time, and enhances the accuracy and efficiency of spatial understanding.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967917A_ABST
    Figure CN118967917A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of indoor space understanding and reconstruction, and discloses a depth visual perception method and system for architectural space analysis, and the method comprises the steps: carrying out the point cloud completion preprocessing and transfer learning pre-training; constructing a double-level attention mechanism enhanced point cloud understanding and segmentation network model; performing staged training and index evaluation on the spatial analysis segmentation network model; and carrying out visual interpretation analysis and parametric modeling on the network model. According to the invention, reliable spatial semantic analysis can be provided based on the point cloud data scanned by the laser radar. The effect of the network model is verified through index evaluation and visual interpretability analysis. Furthermore, the parsed spatial point cloud data is used for establishing a parameterized spatial model in BIM software, and a practical effect is obtained. The developed method is helpful to propose a new reliable computer vision spatial analysis framework, and is helpful to propose an automatic process helpful to automatic analysis and parameterized spatial model generation so as to promote the process of engineering management automation and intelligentization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of indoor space understanding and reconstruction, and in particular relates to a depth visual perception method and system for architectural space analysis. Background Art

[0002] Indoor scene understanding is crucial for intelligent devices such as robots used in digital construction. Human perception of space is intuitive and simple. People generate spatial awareness through vision and touch, that is, the location, boundaries and composition of space. However, for construction robots, spatial perception and understanding not only rely on the raw data captured by sensors and cameras, but also need to be processed and interpreted by advanced computer vision and machine learning technologies. With the development of information technology, the spatial perception and understanding capabilities of construction robots have rapidly increased, which has not only brought innovation to the construction field, but also opened up new possibilities for the future development of robotics. With more precise spatial understanding, construction robots can more effectively help humans handle complex tasks, while improving work efficiency and safety in the construction field.

[0003] The methods for understanding indoor spaces that have been developed can be divided into two categories, namely image methods and point cloud methods. A one-stop approach to understanding and modeling semantic and spatial information through images is time-efficient, but accuracy needs to be improved. Segmenting point clouds to understand semantic information and then performing spatial understanding based on semantic information requires overcoming the difficulty of point cloud acquisition, but it can obtain a higher-precision spatial model and better understanding.

[0004] 3D reconstruction of stereo space using images is a complex process that involves analyzing a series of 2D images to reconstruct the 3D structure of an object or scene. This technology usually uses photogrammetry or structured light scanning to identify common feature points on multiple photos to reconstruct spatial scenes, which particularly involves complex computer vision algorithms for feature detection, matching, and understanding multi-view geometry. Among them, depth information is the key to reconstructing space, which is usually obtained through binocular image sensors or laser sensors. Although image-based methods provide rich texture and color details, image-based reconstruction faces many difficulties in complex indoor environments, where factors such as variable lighting, reflective surfaces, and occlusions reduce the accuracy of reconstruction.

[0005] Segmenting and reconstructing space using point clouds from laser scanners or structured light sensors is key to creating detailed 3D spatial models. This process involves extracting semantic and spatial information and building spatial models. Advanced algorithms, especially semantic segmentation algorithms, are able to automatically extract these semantic and spatial data from point clouds. LiDAR technology is used to collect raw data, and then segment and classify surface or structure-related points, including noise reduction, plane and edge detection, and geometry-based feature classification. The subsequent reconstruction stage connects these points to form a coherent spatial model, which is widely used in virtual roaming, renovation planning or building information modeling (BIM).

[0006] In summary, indoor 3D reconstruction and spatial understanding are achieved through image and point cloud data, which converts real-world space into digital information that machines can understand and process. Segmentation modeling methods make this data more orderly and structured, which is beneficial to subsequent applications such as BIM and environmental perception of autonomous vehicles. With the advancement of computing technology and machine learning algorithms, the efficiency and accuracy of these reconstruction techniques will continue to improve, making the machine's understanding of space closer to human intuitive perception.

[0007] Therefore, how to use intelligent algorithms to understand the indoor point cloud information of completed buildings and establish a parametric space model has become a crucial issue. This paper aims to explore an alternative method and propose a new attention-enhanced deep learning method to achieve the understanding and reconstruction of the indoor space of completed buildings. Summary of the invention

[0008] In view of the problems existing in the prior art, the present invention provides a depth visual perception method and system for architectural space analysis.

[0009] The present invention is implemented as follows: a method and system for deep visual perception of architectural space analysis, the method comprising:

[0010] S1: Point cloud completion preprocessing and transfer learning pre-training;

[0011] S2: Construction of point cloud understanding and segmentation network model enhanced by dual-level attention mechanism;

[0012] S3: Phased training and indicator evaluation of spatial parsing segmentation network model;

[0013] S4: Visual interpretable analysis and parameterized spatial modeling of network models.

[0014] Further, the S1 includes:

[0015] (1) Point cloud completion preprocessing: The point cloud scanning method based on the measuring station is simple to operate, but its disadvantage is that it cannot well restore objects that are far away from the measuring station and are obscured. If you want to well scan objects that cannot be scanned by a single measuring station, you need to scan from a measuring station at another angle, which requires a large amount of measuring station information and consumes more time and resources. Considering that the objects contained in the completed construction site, such as beams, floor slabs, columns, floors, walls, etc., have a single structural form, the missing point cloud is completed by using the symmetry of the component without losing the key features, that is, by copying part of the component's own point cloud to complete the missing point cloud, so as to improve the learnability of the data.

[0016] (2) Transfer learning pre-training: Appropriate transfer learning methods can effectively improve the performance and generalization ability of the network model. The public indoor space point cloud dataset S3DIS is used to remove some categories to generate a sub-dataset. The network model is pre-trained on this dataset and then trained and verified on the completed building point cloud dataset collected on site.

[0017] Furthermore, S2 includes the construction of a dual-level attention-enhanced spatial parsing network model:

[0018] The input data first goes through farthest point sampling and KNN clustering. The farthest point sampling (FPS) algorithm randomly selects a point from the input point cloud, and then selects the point farthest from the selected point. It continuously iterates so that all features of the point cloud can be considered as much as possible under limited computing resources. After sampling, the KNN algorithm is used to cluster the points near the sampling set points to retain local features. The combination of graph patterns composed of feature data is modeled through two-dimensional convolution to obtain the ability to extract point cloud features. In the process of feature extraction, the efficient channel attention mechanism ECA helps to efficiently assign attention weights to convolutional networks of different channels, making the role of effective channels more significant. The self-attention mechanism can generate globally refined attention weights according to the input features, so that the network can better understand the features under the input.

[0019] The self-attention mechanism SA enables the input features to be processed to generate refined attention features. The input feature map F is input into three independent convolutional layers with a kernel size of 1×1 to reshape into three one-dimensional vectors Fq, Fk and Fv, and then Fq and Fk are multiplied to obtain a covariance matrix of size n×n, namely the attention map; then, the attention map is multiplied with Fv to obtain the output feature map, and the residual of the input F is connected to generate a refined self-attention feature Fr; here, the self-attention feature Fr is weighted with F, that is, out=α*Fr+F; here α uses a learnable parameter, and the optimizer automatically adjusts the strength of the attention mechanism; after the refined features are generated by the self-attention mechanism, the deconvolution is upsampled layer by layer, so that the size of the feature map is enlarged, the channels are reduced, and it is gradually restored to the size of the input feature map to have the ability to segment;

[0020] The shallow features and deep features are fused, and the fused features are segmented by a one-dimensional convolutional layer; finally, the input point cloud group will be segmented into 6 categories, namely ceiling, floor, beam, column, fence and wall. These six categories represent the semantic information of the point cloud, that is, the actual meaning of spatial analysis, and are also the basic elements that constitute the architectural space.

[0021] Further, the S3 specifically includes:

[0022] (1) Evaluation of the training effect of the network model; In order to calculate the effect of training, a bidirectional symmetric loss function is proposed. The loss function is composed of the forward module Focal loss and the reverse module Revert CE loss according to a certain weight combination; the forward module uses the network output value as the predicted value and the label as the true value, and assigns different penalty weights to point clouds of different categories with different segmentation difficulties. The reverse module uses the network output value as the true value and the label as the predicted value for calculation. Such a loss function can improve the robustness of the network model to noisy data. The standard cross entropy loss function is sensitive to noisy data. Due to the noise or incorrectness of the labels in the dataset, the standard cross entropy will cause the network model to learn from these incorrect labels, resulting in a significant drop in performance; in the point cloud dataset, considering that the point clouds of objects that occlude each other have subtle noise, these points are also the key features for semantic segmentation;

[0023] CE and RCE losses can be calculated according to the following formulas (1) and (2), SCE loss L s It can be calculated according to formula (3):

[0024]

[0025]

[0026] L s=α·CE+β·RCE (3)

[0027] Where M is the number of categories, y o,c is the true label of observation o, p o,c is the predicted probability that observation o is classified as c, α and β are the weights of CE and RCE respectively;

[0028] In addition, reducing the attention paid to easy-to-classify samples is beneficial to the classification performance of male classification samples; the focal loss introduces a modulation factor based on the CE loss to reduce the weight of easy-to-classify samples; the focal loss is defined by formula (4):

[0029]

[0030] Where (1-p o,c ) γ is the modulation factor, γ is the focal parameter, and the focal loss reduces the weight of easy-to-classify samples by this modulation factor;

[0031] As a combination, formula (5) gives the newly defined loss;

[0032] Loss = θ·L f +(1-θ)·L r (5)

[0033] Where L f represents the focal loss; L r represents the RCE loss, θ and 1-θ are weights, specified as hyperparameters.

[0034] Further, the step (2) specifically includes:

[0035] (2) Multi-factor test evaluation of network models: In order to evaluate the effects of the validation and testing phases, the mean intersection over union (mIoU), as well as P value, F1 value, R value, and AUC-ROC curve indicators are introduced to comprehensively evaluate the performance of the network model on the test set; the following four common evaluation indicators, precision, recall rate, f1 score, and IoU, can be expressed by formulas (6 to 9):

[0036]

[0037]

[0038]

[0039]

[0040] Here it refers to the index of the class; TP, FP, FN represent true positive, false positive, and false negative respectively; N is the number of points; among them, all classes use overall accuracy (OA), weighted average (WA) and macro average (MA) precision, recall, F1 score and mean IoU (MIoU); their calculation results are given by the formula:

[0041]

[0042]

[0043]

[0044]

[0045] The ROC curve reflects the curve of the continuous change relationship between sensitivity and specificity. The horizontal axis of the ROC curve is the false positive rate (FPR), that is, the proportion of samples that are actually negative and are incorrectly predicted as positive. The vertical axis is the true positive rate (TPR), that is, the proportion of samples that are actually positive and are correctly predicted as positive. The principle of drawing the ROC curve is to specify a set of thresholds, divide the predicted probability output by the network model into binary classification prediction results according to the threshold, calculate the FPR and TPR under each threshold division, and then determine the points corresponding to each threshold in the coordinate system and connect them in sequence. Finally, calculate the area of ​​the curve about the horizontal axis, that is, the AUC value. The value range of AUC is between 0 and 1. The closer the AUC value is to 1, the better the performance of the network model.

[0046] Further, the S4 specifically includes:

[0047] (1) Visual interpretability analysis of network models:

[0048] In order to more intuitively understand the state of the machine when learning features and the basis for feature extraction, a visual interpretability analysis is performed on the proposed network; Grad-CAM is a popular interpretability analysis method, and the calculation formula of Grad-CAM is as follows:

[0049]

[0050]

[0051]

[0052] Among them, Grad refers to the calculation of the category activation value Y for the target category c c Relative to feature map A kThe gradient of each feature map channel is then globally averaged and pooled to obtain the importance weight of the channel. The final calculated The heatmap also needs to be upsampled to the size of the input tensor to highlight the areas that are most important for the network model to make a decision for a specific category;

[0053] (2) Parametric reconstruction of space:

[0054] In Revit, the FARO As Build for Revit plug-in helps complete the process of building a spatial model after importing a classified point cloud; to build the model, first create a component template library and develop materials for various structural components (such as beams, columns, plates, etc.) that appear in the scene; next, randomly select points classified into categories on the point cloud for fitting. Finally, combine each component according to the boundary conditions to form a complete spatial model;

[0055] Building a spatial model on a segmented point cloud rather than raw point cloud data can obtain faster and clearer boundaries, which can avoid the influence of other categories of components when fitting a certain type of component based on the point cloud; in the indoor scenes of completed buildings, the combination of various components often follows a certain pattern, which reduces the difficulty of building a spatial model.

[0056] Another object of the present invention is to provide a depth visual perception system for architectural space analysis that implements the depth visual perception method for architectural space analysis, the system comprising:

[0057] Preprocessing training module, used for point cloud completion preprocessing and transfer learning pretraining;

[0058] The network model building module is connected to the preprocessing training module and is used to build a point cloud understanding and segmentation network model enhanced by a two-level attention mechanism;

[0059] The training and evaluation module is connected to the network model building module for deep learning evaluation and interpretability and parameterized spatial model reconstruction;

[0060] The modeling module, connected with the training and evaluation modules, is used for visual interpretable analysis and parameterized spatial modeling of the network model.

[0061] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the depth visual perception method for architectural space analysis.

[0062] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0063] First, the present invention verifies the effectiveness of the proposed method through a completed building point cloud dataset collected on site. The completed building point cloud collected by a laser scanner is parsed and segmented, and parameterized modeling is performed on the segmentation results. The results show that: (1) In cross-validation, the average mIoU achieved by this method is 0.970, the variance is 4.05E-05, and the maximum value is 0.983, which has excellent performance overall. The network model of this method achieves an mIoU of 0.9713 on a representative dataset, which is better than the 0.9445 achieved by the current advanced point cloud semantic segmentation network PointNet++; (2) Visual interpretability analysis shows that the feature extraction layer correctly distinguishes most categories, showing good overall confidence; (4) The time for establishing a spatial model after parsing is shortened by about 37% compared to manual modeling without parsing, and the occurrence of errors is effectively reduced, indicating that segmentation and modeling play different but complementary roles in the process of spatial understanding. This invention provides a new deep machine vision method for accurate spatial parsing and rapid reconstruction.

[0064] Second, the present invention develops a new machine vision method for architectural space analysis and reconstruction. The present invention proposes a point cloud analysis network enhanced by a dual-level attention mechanism, which can provide reliable spatial semantic analysis based on point cloud data scanned by LiDAR. The effect of the network model was verified by indicator evaluation and visual interpretability analysis. Furthermore, the parsed spatial point cloud data was used to establish a parametric spatial model in the BIM software, and practical effects were achieved. The developed method helps to propose a new reliable computer vision spatial analysis framework, and helps to propose an automated process that helps to automatically analyze and generate parametric spatial models, so as to promote the process of automated and intelligent engineering management.

[0065] Third, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0066] (1) The expected benefits and commercial value after this transformation are significant. The proposed technical solution can effectively improve the understanding and modeling accuracy of indoor scenes. For unmanned autonomous systems with large market demand, such as construction robots and self-driving engineering machinery, the present invention can effectively meet the intelligent needs of these devices, and therefore has significant economic benefits and market application prospects.

[0067] (2) The present invention solves a technical problem that has long been urgently needed to be solved in the industry but has not been successfully overcome, namely, how to accurately and efficiently achieve object segmentation and scene understanding from point cloud data in complex indoor environments.

[0068] (3) Overcoming technical bias, the present invention not only combines the dual-level attention mechanism and transfer learning methods, but also incorporates visual interpretability analysis technology. This innovative technical solution not only enhances the segmentation efficiency and accuracy of the model, but also improves the model's interpretability, enabling it to better understand and analyze indoor scenes. Through this method, the present invention effectively solves the multiple challenges of point cloud data processing in complex indoor environments, including point cloud data overlap, uneven distribution, and data imbalance.

[0069] Fourth, the deep visual perception method for architectural space analysis provided by the present invention solves many technical problems of the prior art and brings significant technical progress. The following is a detailed analysis of these aspects:

[0070] First, this method solves the problem of incomplete point cloud data. In architectural space analysis, point cloud data is often missing due to various reasons (such as sensor limitations, occlusion, etc.), which seriously affects the accuracy of subsequent spatial analysis and visual perception. By using the symmetry of components to perform point cloud completion preprocessing, these missing points can be effectively filled, the data quality can be improved, and a more accurate basis can be provided for subsequent deep visual perception.

[0071] Secondly, this method uses transfer learning technology to improve the performance and generalization ability of the network model. Transfer learning is an effective machine learning method that can transfer knowledge learned from one task to another related task, thereby accelerating the learning process of the new task and improving the performance of the model. In this method, by pre-training the network model on a public indoor space point cloud dataset and then using data collected on site for training and verification, the existing knowledge can be effectively utilized to improve the performance of the model on new data.

[0072] In addition, this method improves the ability to understand and segment point cloud data by building a point cloud understanding and segmentation network model enhanced by a dual-level attention mechanism. The attention mechanism can help the model focus more on important information when processing data, thereby improving the performance of the model. The introduction of the dual-level attention mechanism enables the model to focus on different information at different levels, further improving the accuracy of point cloud understanding and segmentation.

[0073] Finally, the method also focuses on visual interpretability analysis and parametric spatial modeling of the network model. This can not only improve the interpretability of the model and allow people to better understand the decision-making process of the model, but also present the analysis results in a more intuitive and understandable way through parametric spatial modeling, providing strong support for the design and optimization of architectural space.

[0074] In summary, this deep visual perception method for architectural space analysis has achieved significant technological progress by solving the problem of incomplete point cloud data, using transfer learning to improve model performance, building a network model enhanced by a two-level attention mechanism, and focusing on the visual interpretability and parametric spatial modeling of the model. It provides a more accurate and efficient method for the analysis and design of architectural space. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is an overall framework diagram of spatial analysis and reconstruction provided by an embodiment of the present invention;

[0076] Figure 2 is a structural diagram of a dual-level attention-enhanced spatial parsing network provided by an embodiment of the present invention;

[0077] Figure 3 is a structural diagram of an efficiency channel attention module provided by an embodiment of the present invention;

[0078] Figure 4 is the structure of the self-attention module provided by an embodiment of the present invention;

[0079] Figure 5 It is a main flow chart of parametric modeling provided by an embodiment of the present invention;

[0080] Figure 6 : are the index curve (a) of the pre-training stage and the index curve (b) of the single-region transfer learning stage provided in the embodiment of the present invention, and the loss value loss is a relative value;

[0081] Figure 7 is the AUC-ROC curve provided by an embodiment of the present invention;

[0082] Figure 8 This is a visual interpretability analysis provided by an embodiment of the present invention, ceiling (a), ground (b), beam (c), column (d), guardrail (e), wall (f);

[0083] Fig. 9 It is the parametric modeling of the region 11 provided by the embodiment of the present invention;

[0084] Fig.10 This is an error that is prone to occur in the traditional modeling method provided by the embodiment of the present invention;

[0085] Fig.11 It is a structural diagram of a deep visual perception system for architectural space analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0087] Reference Figure 1 As shown, the following is the detailed working principle of the mentioned architectural space parsing deep visual perception method:

[0088] S1: Point cloud completion preprocessing and transfer learning pre-training

[0089] #(1) Point cloud completion preprocessing

[0090] Working principle: In the real world, due to occlusion, scanning angle and other reasons, the collected architectural space point cloud data is incomplete. The point cloud completion preprocessing stage uses the symmetry of the components to identify the missing parts and completes the entire point cloud by copying the existing symmetric parts. In this way, a complete point cloud model can be obtained, providing a more accurate data basis for subsequent processing.

[0091] #(2) Transfer learning pre-training

[0092] How it works: Transfer learning is a common strategy in deep learning, which allows the model to transfer the learned features and knowledge to another related task after being trained on one task. In this stage, pre-training is first performed on the public indoor space point cloud dataset S3DIS. The S3DIS dataset is a large-scale point cloud dataset containing multiple categories. By pre-training on this dataset, the model can learn the basic features and patterns of the space. Then, further training and verification are performed on the actual point cloud dataset collected on site to adapt to the specific application environment and needs.

[0093] S2: Point cloud understanding and segmentation network model construction enhanced by two-level attention mechanism

[0094] How it works: The network model built in this step includes a two-level attention mechanism to improve the accuracy and efficiency of point cloud understanding. The attention mechanism allows the network model to automatically focus on the most important parts when processing data, which is especially important in point cloud processing because point cloud data is usually large and contains a lot of redundant information. The two-level attention mechanism is divided into local attention and global attention, where local attention focuses on specific areas in the point cloud, while global attention focuses on the entire point cloud space.

[0095] S3: Phased training and indicator evaluation of spatial parsing segmentation network model

[0096] How it works: In this step, the network model is trained in stages to gradually improve the model's performance on spatial parsing tasks. Usually, this includes multiple training cycles, each of which optimizes the model's parameters and uses a series of evaluation metrics (such as precision, recall, IoU, etc.) to evaluate the model's performance. Through this iterative process, the segmentation accuracy of the model can be continuously improved.

[0097] S4: Visual Interpretation Analysis and Parameterized Spatial Modeling of Network Models

[0098] Working principle: Finally, a visual interpretability analysis of the model is performed to understand the decision-making process of the model, which is usually achieved through techniques such as feature mapping and activation maps. Visual interpretability helps researchers and users understand how the model works and how to improve it. At the same time, parametric spatial modeling is performed, which means converting the output of the model into a parametric form that can be used for actual building design and analysis, such as a CAD model or BIM (Building Information Model).

[0099] This approach improves the ability to automatically analyze building spaces by combining advanced deep learning techniques and point cloud processing technologies, thereby supporting applications such as smart city planning, automated building monitoring, and augmented reality.

[0100] This embodiment provides a new method for architectural space analysis and parametric space model establishment, including the following steps:

[0101] Step 1: Preprocessing of point cloud data completion and pre-training of network models. Specifically including:

[0102] Step 1.1 Point cloud data completion and preprocessing. In this example, the Trimble X7 station-type laser point cloud scanner scans and collects point clouds on the 5th and 6th floors of a completed building site in Wuhan, Hubei Province. This example uses the collected data as a case study. The building point cloud is 2 layers. The original point cloud data is cleaned, and the total number of points in the initial data set is about 16.53 million after removing irrelevant point clouds. The point cloud is divided into 6 categories by manual labeling: ceiling, floor, beam, column, guardrail, and wall. In order to divide the data set into training and test sets, the 5th layer is divided into areas 1 to 8 more evenly, and the 6th layer is divided into areas 9 to 16. The number of points of each category in each layer is shown in Table 1.

[0103] Table 1. Number of points in each category of the 5th and 6th layer point clouds.

[0104]

[0105]

[0106] Based on the marked point cloud dataset, the obscured point clouds of columns, beams, ceilings, and walls are completed. For column point clouds, the component point cloud is divided into 20 parts along the direction to be completed. The program automatically finds the falling edge of the point cloud quantity statistics, and the subsequent parts are completed from the falling edge of the part with the largest statistics. The scanned column part point cloud is used to complete the part of the point cloud obscured by the fence to the junction with the floor, and the unclear parts of the fence and column are distinguished. All categories of point clouds are processed according to boundaries and symmetry. Finally, all point clouds are downsampled according to the minimum distance of 0.01m to obtain the data set used in the experiment.

[0107] Step 1.2 Transfer learning data preparation. The public large-scale indoor space point cloud dataset S3DIS has 13 categories. In order to better adapt to the data mode in this example, the S3DIS dataset is cleared of irrelevant points and only retains 5 categories: ceiling, floor, beam, column, and wall. Compared with the point cloud dataset collected by the lidar, the guardrail category is missing. The sub-dataset of some categories of this S3DIS is downsampled according to the minimum distance of 0.0.1m and used as the data source for the pre-training stage.

[0108] Step 2: Construction of point cloud understanding and segmentation network model enhanced by dual-level attention mechanism, including:

[0109] Step 2.1: Two-level attention-enhanced spatial parsing network;

[0110] The dual-level attention-enhanced spatial parsing network proposed in the present invention for parsing space and generating semantic labels consists of an encoder, a decoder, and a label predictor. Its structure is as follows: Figure 2 As shown in the figure, the input data passes through an encoder consisting of 4 efficient channel attention (ECA) enhanced feature extraction modules, and then passes through a decoder consisting of a self-attention (SA) feature screening module and 3 upsampling and propagation modules. The encoder and decoder are jump-connected to each other at their respective levels, and finally the output of the decoder is used by the segmentation category predictor for semantic category determination.

[0111] In the encoder, the feature extraction module first performs farthest point sampling (FPS) on the input data according to a certain number of sampling points, randomly selects a point from the input point cloud, and then selects the point farthest from the selected point. It continuously iterates so that all features of the point cloud can be considered as much as possible under limited computing resources. Subsequently, each sampling point is clustered by KNN according to a certain number of points to enrich the local features around the key sampling points. The sampled and clustered point set needs to be subjected to a set of two-dimensional convolutional layers for feature extraction. In this process, the size of the feature set continues to shrink and the number of channels continues to increase. In order to enhance the weight of key features and reduce the weight of irrelevant features, the efficient channel attention mechanism ECA is introduced in this module. Its structure is as follows: Figure 3 As shown in Figure 1, it helps to efficiently assign attention weights to convolutional networks of different channels, making the role of effective channels more significant.

[0112] In the decoder, the data first passes through a self-attention mechanism module. The structure of the self-attention mechanism SA is as follows Figure 4 As shown in the figure, it enables the input features to be processed to produce refined attention features. The input feature map F is input into three independent convolutional layers with a kernel size of 1×1 to reshape into three one-dimensional vectors Fq, Fk and Fv, and then Fq and Fk are multiplied to obtain a covariance matrix of size n×n, i.e., the attention map. After that, the attention map is multiplied with Fv to obtain the output feature map, and the residual of the input F is connected to produce a refined self-attention feature Fr. Here, the self-attention feature Fr is weighted with F, i.e., out = α*Fr+F. Here, α uses a learnable parameter, and the optimizer automatically adjusts the strength of the attention mechanism. After that, the data is restored to its original size through 3 groups of upsampling modules.

[0113] Both the encoder and decoder have a 4-layer structure. In order to achieve better network performance, the hierarchical skip connection technology is introduced, that is, the top-down encoder layer is connected to the corresponding bottom-up decoder layer, that is, the output of each encoder layer not only enters the encoder of the next layer, but also merges with the underlying data of the corresponding decoder and enters the decoder of that layer. This feature fusion process helps to generate specific features from abstract features.

[0114] Finally, the shallow features and deep features are fused, and the fused features are segmented by a one-dimensional convolutional layer. Finally, the input point cloud group will be segmented into 6 categories, namely ceiling, floor, beam, column, fence and wall. These six categories represent the semantic information of the point cloud, that is, the actual meaning of spatial analysis, and are also the basic elements that constitute the architectural space.

[0115] Step 2.2: Bidirectional symmetry considering the loss function of imbalanced categories;

[0116] The present invention combines two loss functions, namely Symmetric Cross Entropy loss and focal loss. The standard cross entropy loss function is sensitive to noisy data. Due to the noise or incorrectness of the labels in the data set, the standard cross entropy will cause the network model to learn from these incorrect labels, resulting in a significant drop in performance. In the point cloud data set, considering that the point clouds of mutually occluding objects have subtle noise, but these points are the key features for semantic segmentation. For this reason, the Symmetric Cross Entropy loss is introduced, and the SCE loss is obtained by summing the standard cross entropy loss and the reverse cross entropy loss. Reverse cross entropy is to exchange the predicted value of the standard cross entropy loss with the true value for calculation, which encourages the predicted probability distribution to be close to a uniform distribution in the case of real label noise, which can make the network model more robust to label noise. Combining the forward and reverse cross entropy loss functions can reduce the dependence on noisy labels and help the network model better generalize to unseen data, especially when these new data contain noise patterns different from the training data, so it is very useful when training large-scale data sets. Focal loss is used to distinguish indistinguishable categories and reduce the impact of unbalanced data in each category. These two loss functions are combined in the consistency cost weight γ to train a better network model.

[0117] CE and RCE losses can be calculated according to the following formulas (1) and (2), SCE loss L s It can be calculated according to formula (3):

[0118]

[0119]

[0120] L s =α·CE+β·RCE (3)

[0121] Where M is the number of categories, y o,c is the true label of observation o, p o,c is the predicted probability that observation o is classified as c, α and β are the weights of CE and RCE respectively;

[0122] In addition, reducing the attention paid to easy-to-classify samples is beneficial to the classification performance of difficult-to-classify samples. The focal loss introduces a modulation factor based on the CE loss to reduce the weight of easy-to-classify samples. The focal loss is defined by formula (4):

[0123]

[0124] Where (1-p o,c ) γis the modulation factor, γ is the focal parameter, and the focal loss reduces the weight of easy-to-classify samples by this modulation factor;

[0125] As a combination, formula (5) gives the newly defined loss;

[0126] Loss = θ·L f +(1-θ)·L r (5)

[0127] Where L f represents the focal loss; L r represents the RCE loss, θ and 1-θ are weights, specified as hyperparameters.

[0128] Further, the step (3) specifically includes:

[0129] The following four common evaluation indicators are introduced: precision, recall, f1 score, IoU, and ROC, which can be expressed by formulas (6 to 9):

[0130]

[0131]

[0132]

[0133]

[0134] Here it refers to the index of the class; TP, FP, FN represent true positive, false positive, and false negative respectively; N is the number of points; among them, all classes use overall accuracy (OA), weighted average (WA) and macro average (MA) precision, recall, F1 score and mean IoU (MIoU); their calculation results are given by the formula:

[0135]

[0136]

[0137]

[0138]

[0139] In addition, ROC curve (Receiver Operating Characteristic curve) and AUC (Area Under the Curve) are also important indicators for measuring classification network models. The ROC curve reflects the curve of the continuous change relationship between sensitivity and specificity. The horizontal axis of the ROC curve is the false positive rate (False Positive Rate, FPR), that is, the proportion of samples that are actually negative and are incorrectly predicted as positive, and the vertical axis is the true positive rate (True Positive Rate, TPR), that is, the proportion of samples that are actually positive and are correctly predicted as positive. The principle of drawing the ROC curve is to specify a set of thresholds, divide the predicted probability output by the network model into binary classification prediction results according to the threshold, calculate the FPR and TPR under each threshold division, and then determine the points corresponding to each threshold in the coordinate system and connect them in sequence. Finally, calculate the area of ​​the curve about the horizontal axis, that is, the AUC value. The value range of AUC is between 0 and 1. The closer the AUC value is to 1, the better the performance of the network model. If the prediction result of the network model is a label instead of a predicted probability, then the ROC curve can still be drawn and the AUC value can be calculated. In summary, the ROC curve provides an intuitive and comprehensive way to evaluate and compare the performance of classifiers, visualize the network model, and determine the optimal classification threshold.

[0140] Step 3: Phased training and indicator evaluation of the spatial parsing segmentation network model;

[0141] The computational experiment uses the PyTorch framework developed by Facebook to build the network model under Python coding. It is implemented in the hardware environment of CPUi712700, GPU RTX 2080Ti 22GB, RAM 32GB and Ubuntu22.04 operating system. After grid search, the optimal hyperparameters in the training stage are determined as shown in Table 2.

[0142] Table 2. Hyperparameter table

[0143]

[0144]

[0145] The values ​​of these basic hyperparameters remain constant in all subsequent training stages. The training process is divided into three stages: pre-training stage, transfer learning cross-validation stage, and single-region transfer learning stage.

[0146] Pre-training stage. Pre-training is performed on the prepared S3DIS dataset, and the weights saved in the training results of this stage will be used as the initial weights of all subsequent training stages. During the training process of the S3DIS training set and test set, the relative loss value, training set accuracy and test set mIoU of the proposed network are as follows Figure 6 At the end of the training, the network model weights with the best performance in this stage are selected as the initial weights for the subsequent training stages.

[0147] Transfer learning cross-validation stage. In the cross-validation stage, in order to evaluate the overall performance of the network model, the proposed network was cross-validated 16 times on the indoor completed building scene dataset. Each cross-validation uses a single area as the test set, other areas as the training set, and controlled variables to eliminate the interference of other factors. In this stage, each training uses the pre-trained weights of the S3DIS dataset as the initial weights. Finally, a representative area is selected, and the performance indicator mIoU shown in this area should be closest to the average mIoU of the cross-validation. The data in this area can better reflect the average performance of the entire dataset, thereby providing standardization and capabilities in subsequent experiments. The experimental results of this stage are shown in Table 3.

[0148] Table 3. Cross-validation experimental results

[0149]

[0150]

[0151] From the results, we can see that the network model performs well on each test set, with an overall average cross-over ratio of 0.970, indicating that the network model has high accuracy in identifying and segmenting point cloud data in different regions, with a variance of 4.05244E-05. The small variance indicates that the performance of the network model on different test sets is relatively stable and has no significant fluctuations. The mIoU of region 11 is 0.9713, which is closest to the average value among all regions, so region 11 will be used as the representative of the data set collected on site in the next experimental stage.

[0152] Single-region transfer learning stage. In this stage, the point cloud data of region 11 provided by the cross-validation results is used as the test set, and the other 15 scene data of the building point cloud are used as the training set. The relative loss value, training set accuracy and test set mIoU of the proposed network during the training process in this stage are as follows: Figure 6(b) is shown. On this platform, the training time for each round in this stage is about 1 minute. The optimal weights of transfer learning are used as the initial weights, which effectively reduces the training time in the subsequent stages. Subsequently, this area is used as a test set for ablation experiments, interpretability analysis, modeling understanding, etc. The results calculated according to the evaluation indicator formula are shown in Table 4. The AUC-ROC curve is shown in Figure 7 The comparison of mIoU with other advanced methods is shown in Table 5.

[0153] Table 4. Test results for region 11

[0154]

[0155] Table 5. Comparison with other advanced methods

[0156]

[0157]

[0158] From the results, it can be seen that the mIoU value achieved by this method for area 11 is 0.9713, which is higher than the mIoU value of 0.9445 achieved by PointNet++. The higher accuracy proves that the machine has an effective semantic understanding of the space and has high practical value. In the visualization of the segmentation results, a small part of the guardrail in contact with the ground is incorrectly predicted as the ground. This is due to the noise in the dataset calibration process. Some points are incorrectly marked at the interface of the two types of point clouds, but this does not affect the overall recognition effect. From the AUC-ROC curve image, it can be seen that the curve has a high envelope area with the x-axis. The method performs well in the point cloud segmentation of the standard layer of the completed building and has great application potential. From the evaluation index table, it can be seen that most of the point clouds are successfully segmented, indicating that the method can process and learn point cloud data features well. Test results of different networks under the same dataset distribution. By comparing the results given in Table 5, it can be observed that the proposed network shows better performance compared with the advanced point cloud networks dgcnn, pointnet, pointnet++ and aspcnet.

[0159] Step 4: Visual interpretative analysis and parametric modeling of the network model; specifically including:

[0160] Step 4.1. Visual interpretability analysis of network models

[0161] In order to more intuitively understand the state of the machine when learning features and the basis for feature extraction, a visual interpretable analysis of the proposed network was performed. Grad-CAM (Gradient-weighted Class Activation Mapping) is a popular interpretable analysis method that is used to highlight the areas where convolutional neural networks contribute most to specific category decisions. The basic idea of ​​Grad-CAM is to use the gradient of the target category for the output of the last convolutional layer of the network. These gradients are considered to be an indication of the importance of category decisions. The calculation formula of Grad-CAM is as follows:

[0162]

[0163]

[0164]

[0165] Among them, Grad refers to the calculation of the category activation value Y for the target category c c Relative to feature map A k The gradient of each feature map channel is then globally averaged and pooled to obtain the importance weight of the channel. The final calculated The heatmap also needs to be upsampled to the size of the input tensor to highlight the areas that are most important for the network model to make a decision for a specific class.

[0166] The data of the last feature extraction layer of the proposed network is extracted, and the heat map is made according to the above formula as follows Figure 8 Among the six categories, the ceiling, floor, and beam categories have relatively good interpretability, as shown in Figure 8 (a), (b) and (c). The orange-red part indicates that the Grad-CAM heat value of the point cloud segmentation network in this category is high, indicating that the network model can accurately learn and classify based on these significant features. In the column, guard and wall categories, as shown in Figure 8 As shown in (d), (e), and (f), although the highlighted red areas are not widely distributed, the final segmentation results are acceptable. This is because these categories are more difficult to segment. The point cloud needs more attention mechanisms to help distinguish. For the more difficult categories of beams and columns, the point cloud segmentation network shows the features that can be explained in the image. Figure 8 In (d), the vertically continuous point clouds in this column are highlighted in red. This means that the network pays attention to the vertically continuous features of these point clouds and uses them to determine the categories of these points. However, it can be seen from Figure 8It can be observed in (c), (d), and (e) that point clouds that do not belong to this category are also the areas that the neural network pays more attention to. This is because features are extracted in the intermediate layers, and operations will be performed based on these features in the subsequent layers of the neural network.

[0167] Step 4.2: Parametric reconstruction of space

[0168] In Revit, the FARO As Build for Revit plugin helps complete the process of building a spatial model after importing a classified point cloud, such as Figure 5 To build a spatial model, first create a component template library and develop materials for various structural components (such as beams, columns, plates, etc.) that appear in the scene. Next, randomly select points divided into categories on the point cloud for fitting. Finally, combine each component according to the boundary conditions to form a complete spatial model.

[0169] Building a spatial model on the segmented point cloud instead of the original point cloud data can obtain faster and clearer boundaries, which can avoid being affected by other types of components when fitting a certain type of component based on the point cloud. In the indoor scenes of completed buildings, the combination of various components often follows a certain pattern, which reduces the difficulty of building a spatial model.

[0170] The modeling process of the point cloud segmentation results of region 11 took about 5 minutes to generate a fine parameterized spatial model, while the traditional method took about 8 minutes to model the region. The automatic semantic segmentation of the point cloud gives the point cloud labeling information and clusters the point cloud into different spatial elements, which is the key to spatial understanding. Errors will occur in the manual reconstruction of the ring beam, such as Fig.10 As shown in the figure, the beams are incorrectly modeled at some ceiling points. Based on the segmentation result of area 11, the point clouds are fitted into the parametric spatial model according to the component category with the help of the FARO As Built for Autodesk Revit plug-in. Finally, the point clouds are merged according to their positions to obtain the parametric spatial model of area 11. The visualization image of the model is shown in the figure below. Fig. 9 The dimensions of each structural component are shown in Table 6.

[0171] Table 6. Component dimensions and modeling time for parametric modeling

[0172]

[0173] In the modeling results of a single area, the size of the floor slab protrusion is 2000*70mm. Because only the data of this area is considered as the source of modeling, the thickness information of the wall and floor slab cannot be obtained, and the thickness of the wall and the guard plate is uniformly set to 200mm, and the thickness of the floor slab is set to 300mm. In the process of spatial understanding, segmentation provides abstract semantic understanding, modeling provides specific physical location understanding, and spatial understanding is achieved through classification and clustering; when these two technologies are combined, they complement each other and provide a comprehensive and integrated spatial understanding. This comprehensive understanding enables computer systems to better support complex design and planning tasks, improve decision-making quality, and promote automation and intelligence. In short, segmentation and modeling play different but complementary roles in the process of spatial understanding. Segmentation enables computers to identify and interpret various elements in the space and their functions by providing semantic understanding. On the other hand, modeling complements this understanding by providing a specific and accurate representation of the physical space, including size, shape, and relative position.

[0174] like Fig.11 As shown, an embodiment of the present invention provides a depth visual perception system for architectural space analysis that implements the depth visual perception method for architectural space analysis, the system comprising:

[0175] Preprocessing training module, used for point cloud completion preprocessing and transfer learning pretraining;

[0176] The network model building module is connected to the preprocessing training module and is used to build a point cloud understanding and segmentation network model enhanced by a two-level attention mechanism;

[0177] The training and evaluation module is connected to the network model building module for deep learning evaluation and interpretability and parameterized spatial model reconstruction;

[0178] The modeling module, connected with the training and evaluation modules, is used for visual interpretable analysis and parameterized spatial modeling of the network model.

[0179] An embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the depth visual perception method for architectural space analysis.

[0180] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0181] The spatial parsing network enhanced by the dual-level attention mechanism provided by the implementation of the present invention achieved an mIoU value of 0.9713 in the field test, which is higher than the mIoU value of 0.9445 achieved by PointNet++. The higher accuracy proves that the machine has an effective semantic understanding of the space and has a high practical value. As can be seen from the AUC-ROC curve image, the curve has a high envelope area with the x-axis. The method performs well in the point cloud segmentation of the standard layer of the completed building and has great application potential. As can be seen from the evaluation index Table 4, most of the point clouds are successfully segmented, indicating that the method can process and learn point cloud data features well. Test results of different networks under the same data set distribution. By comparing the results given in Table 5, it can be observed that the proposed network model shows better performance than the advanced point cloud networks dgcnn, pointnet, pointnet++ and ASPCNET.

[0182] The modeling process of the semantic segmentation results of the point cloud took about 5 minutes to generate a fine parameterized spatial model, while the traditional method took about 8 minutes to model the area. The automatic semantic segmentation of the point cloud gives the point cloud label information and clusters the point cloud into different spatial elements, which is the key to spatial understanding. Errors will occur in the manual reconstruction of the ring beam, such as Fig.10 As shown in the figure, the beams are modeled incorrectly at some ceiling points. This method can greatly improve the modeling speed and avoid errors.

[0183] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A deep visual perception method for architectural space analysis, characterized in that: The method includes: S1: Point cloud completion preprocessing and transfer learning pre-training; S2: Construction of point cloud understanding and segmentation network model enhanced by dual-level attention mechanism; S3: Phased training and indicator evaluation of spatial parsing segmentation network model; S4: Visual interpretable analysis and parameterized spatial modeling of network models.

2. The method for deep visual perception of architectural space analysis according to claim 1, characterized in that: The S1 includes: (1) Point cloud completion preprocessing: using the symmetry of the component to complete the missing point cloud, that is, copying part of the component’s own point cloud to complete the missing point cloud; (2) Transfer learning pre-training: Appropriate transfer learning methods can effectively improve the performance and generalization ability of the network model. The public indoor space point cloud dataset S3DIS is used to remove some categories to generate a sub-dataset. The network model is pre-trained on this dataset and then trained and verified on the completed building point cloud dataset collected on site.

3. The method for depth visual perception of architectural space analysis according to claim 1, characterized in that: The S2 includes: (1) A two-level attention-enhanced spatial parsing network; (2) Bidirectional symmetric loss function considering imbalanced categories; (3) Multi-factor evaluation indicators and determination.

4. The method for depth visual perception of architectural space analysis according to claim 3, characterized in that: The step (1) specifically comprises: The input data first goes through farthest point sampling and KNN clustering. The farthest point sampling (FPS) algorithm randomly selects a point from the input point cloud, and then selects the point farthest from the selected point. It continuously iterates so that all features of the point cloud can be considered as much as possible under limited computing resources. After sampling, the KNN algorithm is used to cluster the points near the sampling set points to retain local features. The combination of graph patterns composed of feature data is modeled through two-dimensional convolution to obtain the ability to extract point cloud features. In the process of feature extraction, the efficient channel attention mechanism ECA helps to efficiently assign attention weights to convolutional networks of different channels, making the role of effective channels more significant. The self-attention mechanism can generate globally refined attention weights according to the input features, so that the network can better understand the features under the input. The self-attention mechanism SA enables the input features to be processed to generate refined attention features. The input feature map F is input into three independent convolutional layers with a kernel size of 1×1 to reshape into three one-dimensional vectors Fq, Fk and Fv, and then Fq and Fk are multiplied to obtain a covariance matrix of size n×n, namely the attention map; then, the attention map is multiplied with Fv to obtain the output feature map, and the residual of the input F is connected to generate a refined self-attention feature Fr; the self-attention feature Fr is weighted with F, that is, out = α*Fr+F; here α uses a learnable parameter, and the optimizer automatically adjusts the strength of the attention mechanism; after the refined features are generated by the self-attention mechanism, the deconvolution is upsampled layer by layer, so that the size of the feature map is enlarged, the channels are reduced, and it is gradually restored to the size of the input feature map to have the ability to segment; The shallow features and deep features are fused, and the fused features are segmented by a one-dimensional convolutional layer; finally, the input point cloud group will be segmented into 6 categories, namely ceiling, floor, beam, column, fence and wall. These six categories represent the semantic information of the point cloud, that is, the actual meaning of spatial analysis, and are also the basic elements that constitute the architectural space.

5. The method for depth visual perception of architectural space analysis according to claim 3, characterized in that: The step (2) specifically comprises: It combines two loss functions, namely Symmetric Cross Entropy loss and focal loss. The standard cross entropy loss function is more sensitive to noisy data. Due to the noise or incorrect labels in the dataset, the standard cross entropy will cause the network model to learn from these incorrect labels, resulting in a significant drop in performance. In the point cloud dataset, considering that the point clouds of mutually occluded objects have subtle noise, these points are also the key features for semantic segmentation. CE and RCE losses can be calculated according to the following formulas (1) and (2), SCE loss L s It can be calculated according to formula (3): L s =α·CE+β·RCE (3) Where M is the number of categories, y o,c is the true label of observation o, p o,c is the predicted probability that observation o is classified as c, α and β are the weights of CE and RCE respectively; In addition, reducing the attention paid to easy-to-classify samples is beneficial to the classification performance of male classification samples; the focal loss introduces a modulation factor based on the CE loss to reduce the weight of easy-to-classify samples; the focal loss is defined by formula (4): Where (1-ρ o,c ) γ is the modulation factor, γ is the focal parameter, and the focal loss reduces the weight of easy-to-classify samples by this modulation factor; As a combination, formula (5) gives the newly defined loss; Loss=θ·L f +(1-θ)·L r (5) Where L f represents the focal loss; L r represents the RCE loss, θ and 1-θ are weights, specified as hyperparameters. Further, the step (3) specifically includes: The following four common evaluation indicators are introduced: precision, recall, f1 score, IoU, and ROC, which can be expressed by formulas (6 to 9): Here it refers to the index of the class; TP, FP, FN represent true positive, false positive, and false negative respectively; N is the number of points; among them, all classes use overall accuracy (OA), weighted average (WA) and macro average (MA) precision, recall, F1 score and mean IoU (MIoU); their calculation results are given by the formula:

6. The method for depth visual perception of architectural space analysis according to claim 1, characterized in that: The S3 specifically includes: (1) Evaluation of the training effect of the network model. In order to calculate the effect of training, a bidirectional symmetric loss function is proposed. The loss function is composed of the forward module Focal loss and the reverse module Revert CE loss according to a certain weight. The forward module takes the network output value as the predicted value and the label as the true value, and assigns different penalty weights to point clouds of different segmentation difficulty levels. The reverse module takes the network output value as the true value and the label as the predicted value for calculation. Such a loss function can improve the robustness of the network model to noisy data. Afterwards, in order to evaluate the effect of the verification and testing stages, the mean intersection over union (mIoU), as well as P value, F1 value, R value, and AUC-ROC curve indicators are introduced to comprehensively evaluate the performance of the network model on the test set. (2) Interpretability analysis of the network model: In order to more intuitively understand the state of the machine when learning features and the basis for feature extraction, a visual interpretability analysis of the proposed network was performed. The Grad-CAM algorithm was used to back-propagate the output of the network model to calculate the heat map of the feature extraction layer, highlighting the area where the convolutional neural network contributes most to the decision of a specific category, in order to verify that the corresponding area is correctly recognized and understood by the neural network. (3) Parametric BIM spatial model reconstruction: In order to make the spatial analysis practical, the analyzed point cloud is imported into Autodesk, and the FARO As-built for Revit plug-in is used to complete the process of establishing the spatial model. In this process, a component template library is first established for various structural components that appear in the scene. Secondly, points are randomly selected on the point cloud divided into categories for fitting. Finally, the components are combined according to the boundary conditions to form a complete parametric spatial model.

7. The method for depth visual perception of architectural space analysis according to claim 1, characterized in that: The S4 specifically includes: (1) Visual interpretability analysis of network models: In order to more intuitively understand the state of the machine when learning features and the basis for feature extraction, a visual interpretability analysis is performed on the proposed network; Grad-CAM is a popular interpretability analysis method, and the calculation formula of Grad-CAM is as follows: Among them, Grad refers to the calculation of the category activation value Y for the target category c c Relative to feature map A k The gradient of each feature map channel is then globally averaged and pooled to obtain the importance weight of the channel. The final calculated The heatmap also needs to be upsampled to the size of the input tensor to highlight the areas that are most important for the network model to make a decision for a specific category; (2) Parametric reconstruction of space: In Revit, the FARO As Build for Revit plug-in helps complete the process of building a spatial model after importing a classified point cloud; to build the model, first create a component template library to develop materials for the various structural components that appear in the scene; next, randomly select points classified into categories on the point cloud for fitting. Finally, combine each component according to the boundary conditions to form a complete spatial model; Building a spatial model on a segmented point cloud rather than raw point cloud data can obtain faster and clearer boundaries, which can avoid the influence of other categories of components when fitting a certain type of component based on the point cloud; in the indoor scenes of completed buildings, the combination of various components often follows a certain pattern, which reduces the difficulty of building a spatial model.

8. A depth visual perception system for architectural space analysis implementing the depth visual perception method for architectural space analysis as claimed in any one of claims 1 to 7, characterized in that: The system includes: Preprocessing training module, used for point cloud completion preprocessing and transfer learning pretraining; The network model building module is connected to the preprocessing training module and is used to build a point cloud understanding and segmentation network model enhanced by a two-level attention mechanism; The training and evaluation module is connected to the network model building module for deep learning evaluation and interpretability and parameterized spatial model reconstruction; The modeling module, connected with the training and evaluation modules, is used for visual interpretable analysis and parameterized spatial modeling of the network model.

9. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the depth visual perception method for architectural space analysis as described in any one of claims 1-7.

Citation Information

Patent Citations

  • 3D point cloud segmentation method based on attention network

    CN110111345A

  • Scene three-dimensional intelligent reconstruction system and method based on BIM and deep learning

    CN115147545A

  • Three-dimensional point cloud semantic segmentation method and system based on visual assistance and feature enhancement

    CN116229079A

  • Performing semantic segmentation of 3D data using deep learning

    US11694333B1

  • Few-shot urban remote sensing image information extraction method based on meta learning and attention

    US20230215166A1

Cited By

  • Oral cavity image segmentation method and system based on attention mechanism

    CN120070460A

  • BIM generation method based on AI, computer equipment and computer program product

    CN121167866A