BIM-based three-dimensional surveying and mapping data fusion processing method and system

Abstract

The application discloses a BIM-based three-dimensional surveying and mapping data fusion processing method and system, multi-source surveying and mapping data are collected; geometric features of point cloud data are extracted through a point cloud branch, and texture features of image data are extracted through an image branch; two branch features are aligned based on an AFM attention fusion module, a cross-modal feature vector is generated, a registration matrix of point cloud and image is calculated in combination with POS position and attitude data, the registration matrix is optimized by using an iterative closest point neural network, and a three-dimensional semantic point cloud set is generated; the three-dimensional semantic point cloud set is imported into BIM software, building components in the point cloud are identified based on a semantic segmentation network, geometric properties of the components in the point cloud are associated with component properties in the BIM model, a lightweight three-dimensional model is output, and the model is used for surveying and mapping viewing and interaction. The application can improve surveying and mapping data processing efficiency and engineering investigation and design efficiency.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to a method and system for fusion processing of three-dimensional surveying data based on BIM. Background Technology

[0002] Traditional 3D surveying data processing often relies on a single data source, resulting in incomplete information dimensions and insufficient accuracy. Existing fusion methods mostly employ simple stitching or traditional feature matching, which struggles to accurately extract the correlation information between point cloud geometric features and image texture features. The registration process is also susceptible to noise interference, leading to significant errors. Furthermore, current technologies have shortcomings in associating attributes between multi-source data and BIM models. The models lack sufficient lightweightness, making it difficult to meet the demands of rapid viewing and interaction on mobile devices. This hinders the efficient application of 3D surveying data in engineering practice and results in low real-time processing efficiency. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a BIM-based three-dimensional mapping data fusion processing method and system.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned BIM-based three-dimensional mapping data fusion processing method: Point cloud data, multi-view images, and POS position and pose data of the target area are collected. The three types of data are associated by timestamps. The collected data are preprocessed to obtain multi-source mapping data. A multimodal feature extraction network is established. The geometric features of point cloud data are extracted through the PointNet++ point cloud branch, and the texture features of image data are extracted through the ResNet-101 image branch. Based on the AFM attention fusion module, the two branches of features are aligned to generate cross-modal feature vectors. The registration matrix of point cloud and image is calculated by combining POS position and pose data. The registration matrix is ​​optimized by ICP-Net iterative nearest point neural network to generate a three-dimensional semantic point cloud set. The three-dimensional semantic point cloud is imported into BIM software, and building components in the point cloud are identified based on a semantic segmentation network. The geometric attributes of the components in the point cloud are associated with the component attributes in the BIM model, and a lightweight three-dimensional model is output. The model is then used for surveying, viewing, and interaction.

[0005] Furthermore, in the aforementioned BIM-based 3D mapping data fusion processing method, the point cloud data, multi-view images, and POS position and pose data of the target area are collected, and the three types of data are associated through timestamps. The collected data is then preprocessed to obtain multi-source mapping data, including: Point cloud data, multi-view images, and POS position and pose data of the target area are collected. The timestamp information of the three types of raw data is extracted. Based on the timestamp of the POS data, the point cloud data, multi-view image data and POS data of the same time dimension are associated with each other through a time matching algorithm to obtain associated data. A pass-through filtering algorithm is used to remove invalid point clouds from the point cloud data, a statistical filtering algorithm is used to remove isolated noise points, and the valid point clouds are downsampled. The image is processed for distortion correction, size normalization and grayscale equalization, while blurry and overexposed invalid images are removed; Abnormal data points caused by equipment vibration and signal interference during data acquisition are removed, and a sliding window filtering algorithm is used to smooth the valid data. The three types of preprocessed data are then standardized to obtain multi-source mapping data.

[0006] Furthermore, in the above-mentioned BIM-based 3D mapping data fusion processing method, the establishment of a multimodal feature extraction network, and the extraction of geometric features of point cloud data through the PointNet++ point cloud branch, includes: A point cloud feature extraction branch is constructed based on the PointNet++ network framework. The branch contains three core modules: sampling layer, grouping layer, and feature propagation layer. A multi-scale hierarchical extraction strategy is used to accurately capture the geometric features of point clouds. The preprocessed point cloud data is input into the branch. The sampling layer uses the FPS farthest point sampling algorithm to select key sampling points in the point cloud. Then, the grouping layer constructs local neighborhoods with the sampling points as the center and performs feature encoding on the point cloud data in each neighborhood. By fusing local features at different scales through a feature propagation layer, the level of feature abstraction is gradually improved, and a point cloud geometric feature vector with uniform dimension is output.

[0007] Furthermore, in the above-mentioned BIM-based 3D mapping data fusion processing method, the step of extracting texture features from image data using the ResNet-101 image branch includes: An image feature extraction branch is constructed based on the ResNet-101 network framework. The original 50 convolutional layers, 16 residual blocks and fully connected layers of the network are retained, the classification layer of the original network is removed and a feature adaptation layer is added. The preprocessed multi-view image data is input into the branch, and the texture features of the image are initially extracted through the convolutional layer. The residual block is used to solve the gradient vanishing problem in the deep training of the network. The deep convolutional features are compressed and integrated through a fully connected layer, and then the output features are adjusted to an image texture feature vector with the same dimension as the point cloud geometric feature vector through a feature adaptation layer.

[0008] Furthermore, in the above-mentioned BIM-based 3D mapping data fusion processing method, the AFM attention fusion module aligns the two-branch features, generates cross-modal feature vectors, and calculates the registration matrix of point cloud and image using POS position and pose data. The registration matrix is ​​then optimized using the ICP-Net iterative nearest-point neural network to generate a 3D semantic point cloud set, including: The point cloud geometric feature vector and the image texture feature vector are input into the module. The importance of the two types of features is analyzed by the computing unit, and weights are assigned to key features and redundant features. Based on the attention weight results, the spatial dimension of the two types of features is adjusted, and the bilinear interpolation algorithm is used to correct the spatial offset between the features. We use a weighted summation method to fuse the aligned features of the two classes to generate a cross-modal feature vector.

[0009] Furthermore, in the above-mentioned BIM-based 3D mapping data fusion processing method, the step of importing the 3D semantic point cloud set into BIM software and identifying building components in the point cloud based on a semantic segmentation network includes: A semantic segmentation network is built based on the U-Net architecture. The 3D semantic point cloud set imported from BIM software is input into the network. Through deep convolution and deconvolution operations of the network, various building components in the point cloud are identified, and different types of components are assigned unique identification codes and color labels to distinguish the semantics of the components.

[0010] Furthermore, in the aforementioned BIM-based 3D mapping data fusion processing method, the step of associating the geometric attributes of components in the point cloud with the component attributes in the BIM model to output a lightweight 3D model, and using the model for mapping viewing and interaction, includes: By matching identifier codes, the attributes of point cloud components are associated and fused with the corresponding component attributes in the BIM model, supplementing the detailed attribute information of the BIM model and outputting a lightweight 3D model.

[0011] Furthermore, in the BIM-based 3D mapping data fusion processing system, the BIM-based 3D mapping data fusion processing system includes the following modules: The surveying data acquisition module is used to collect point cloud data, multi-view images, and POS position and attitude data of the target area. It associates the three types of data through timestamps and preprocesses the collected data to obtain multi-source surveying data. The data feature extraction module is used to establish a multimodal feature extraction network. It extracts the geometric features of point cloud data through the PointNet++ point cloud branch and extracts the texture features of image data through the ResNet-101 image branch. The mapping data processing module is used to align the two-branch features based on the AFM attention fusion module, generate cross-modal feature vectors, and calculate the registration matrix of point cloud and image by combining POS position and pose data. The registration matrix is ​​optimized by using ICP-Net iterative nearest point neural network to generate a three-dimensional semantic point cloud set. The data viewing and interaction module is used to import the three-dimensional semantic point cloud into BIM software, identify building components in the point cloud based on the semantic segmentation network, associate the geometric attributes of the components in the point cloud with the component attributes in the BIM model, output a lightweight three-dimensional model, and use the model for surveying, viewing and interaction.

[0012] Furthermore, in the BIM-based 3D surveying data fusion and processing system, the surveying data processing module includes the following sub-modules: The processing submodule is used to build a semantic segmentation network based on the U-Net architecture. It inputs the 3D semantic point cloud set imported from BIM software into the network, identifies various building components in the point cloud through deep convolution and deconvolution operations of the network, and assigns unique identification codes and color labels to different types of components to perform semantic differentiation of components.

[0013] Furthermore, in the BIM-based 3D surveying data fusion and processing system, the surveying data processing module includes the following sub-modules: The matching submodule is used to associate and fuse point cloud component attributes with corresponding component attributes in the BIM model by matching identifier codes, supplementing the detailed attribute information of the BIM model and outputting a lightweight 3D model.

[0014] Its beneficial effects lie in achieving efficient integration of multi-source surveying and mapping data through timestamp association and hierarchical preprocessing, improving data quality, and solving the fusion problems of data heterogeneity and spatiotemporal misalignment in traditional methods. The collaborative application of a multimodal feature extraction network and an AFM attention fusion module accurately captures point cloud geometric features and image texture features. Combined with ICP-Net optimized registration, it significantly improves registration accuracy and stability, and reduces the impact of noise interference. It achieves deep attribute association between multi-source data features and BIM models, enriching the information dimensions of BIM models. The generated lightweight model balances accuracy and loading efficiency, supporting rapid viewing and interaction on multiple devices. This invention can improve the efficiency of surveying and mapping data processing, significantly enhance the efficiency of engineering surveying and design, improve the accuracy of construction and maintenance, and reduce project costs. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0016] Figure 1This is a schematic diagram of the first embodiment of the BIM-based three-dimensional mapping data fusion processing method in this invention. Figure 2 This is a schematic diagram of the second embodiment of the BIM-based three-dimensional surveying data fusion processing method in this invention. Figure 3 This is a schematic diagram of the first embodiment of the BIM-based three-dimensional mapping data fusion processing system in this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, the BIM-based 3D surveying data fusion processing method includes the following steps: Step 101: Collect point cloud data, multi-view images, and POS position and pose data of the target area. Associate the three types of data through timestamps and preprocess the collected data to obtain multi-source mapping data. Specifically, in this embodiment, point cloud data, multi-view images, and POS position and pose data of the target area are collected. Timestamp information of the three types of raw data is extracted. Using the timestamp of the POS data as a benchmark, a time matching algorithm is used to associate the point cloud data, multi-view image data, and POS data at the same time dimension to obtain associated data. A direct-pass filtering algorithm is used to remove invalid point clouds from the point cloud data, and a statistical filtering algorithm is used to remove isolated noise points. Valid point clouds are downsampled. The images undergo distortion correction, size normalization, and grayscale equalization, while blurry and overexposed invalid images are removed. Abnormal data points caused by equipment jitter and signal interference during data acquisition are removed, and a sliding window filtering algorithm is used to smooth the valid data. The three types of preprocessed data are standardized to obtain multi-source mapping data.

[0020] Raw data acquisition involves simultaneously collecting three types of core data for the target mapping area: point cloud data acquired by LiDAR equipment, recording the 3D spatial coordinates of objects within the area; multi-view image data acquired by high-definition cameras, covering visual texture information from different angles of the target area; and position and attitude data acquired by the positioning and attitude system (POS), including real-time latitude, longitude, elevation, heading angle, pitch angle, roll angle, and other parameters of the acquisition equipment. During the acquisition process, time synchronization of the three types of equipment is ensured to guarantee the consistency of data timestamps.

[0021] Data timestamp association involves extracting timestamp information from three types of raw data. Using the timestamp of POS data as a benchmark, due to its high positioning and orientation accuracy and strong temporal stability, a time-matching algorithm is used to associate point cloud data, multi-view image data, and POS data at the same time dimension. For minor timestamp discrepancies, linear interpolation is used for correction, ensuring that the three types of data collected at the same time form associated data groups, accurately corresponding to the same spatial state of the target area.

[0022] Raw data preprocessing involves quality optimization of the three types of correlated data. Point cloud data preprocessing uses a pass-through filtering algorithm to remove invalid point clouds such as ground and noise, and a statistical filtering algorithm to remove isolated noise points. The remaining valid point clouds are then downsampled to reduce the data volume while maintaining data accuracy. Multi-view image data preprocessing includes distortion correction to eliminate errors caused by camera lens distortion, size normalization to unify all images to the same resolution, grayscale equalization to enhance image texture contrast, and removal of blurry, overexposed / underexposed invalid images. POS data preprocessing removes abnormal data points caused by equipment jitter and signal interference during data acquisition, and uses a sliding window filtering algorithm to smooth the valid data, improving the stability of position and attitude data.

[0023] The multi-source data format is unified by converting the preprocessed point cloud data into LAS format, the multi-view image data into JPG format, and the POS data into TXT format for standardization processing, generating a multi-source mapping dataset containing unified data identifiers and timestamp information for subsequent network model input.

[0024] Step 102: Establish a multimodal feature extraction network, extract geometric features of point cloud data through the PointNet++ point cloud branch, and extract texture features of image data through the ResNet-101 image branch; Specifically, in this embodiment, a point cloud feature extraction branch is constructed based on the PointNet++ network framework. The branch includes three core modules: a sampling layer, a grouping layer, and a feature propagation layer. A multi-scale hierarchical extraction strategy is used to accurately capture the geometric features of the point cloud. The preprocessed point cloud data is input into the branch. The sampling layer uses the FPS farthest point sampling algorithm to select key sampling points in the point cloud. Then, the grouping layer constructs local neighborhoods centered on the sampling points and performs feature encoding on the point cloud data in each neighborhood. The feature propagation layer fuses local features at different scales, gradually improving the abstraction level of the features and outputting a point cloud geometric feature vector with unified dimensions. An image feature extraction branch is constructed based on the ResNet-101 network framework. The original 50 convolutional layers, 16 residual blocks, and fully connected layers are retained, the classification layer of the original network is removed, and a feature adaptation layer is added. The preprocessed multi-view image data is input into the branch, and the texture features of the image are initially extracted through the convolutional layers. The residual blocks are used to solve the gradient vanishing problem in the deep training of the network. The deep convolutional features are compressed and integrated through the fully connected layers, and then the output features are adjusted to the image texture feature vector with the same dimension as the point cloud geometric feature vector through the feature adaptation layer.

[0025] Multimodal feature extraction network construction and feature extraction are the core technical support links in the entire 3D mapping data fusion processing flow. Its core objective is to achieve effective separation and accurate extraction of features from two heterogeneous data types—point clouds and images—by constructing a targeted multimodal network architecture, laying a high-quality feature foundation for subsequent cross-modal fusion. The design logic of this step closely revolves around the characteristics of multi-source data. The core advantage of point cloud data lies in its accurate 3D spatial geometric information, while image data possesses rich texture and color information. The feature dimensions and representation forms of the two types of data differ significantly. Therefore, a dual-branch parallel network architecture design is adopted to maximize the preservation of the original feature information of each type of data and avoid information loss caused by a single feature extraction mode. During the overall network architecture construction phase, it is crucial to ensure the independence of the two parallel branches and their compatibility with subsequent fusion. To this end, a standardized connection channel with the AFM attention fusion module is specifically reserved to ensure that point cloud geometric feature vectors and image texture feature vectors can be smoothly input into the fusion module. Simultaneously, the network input and output specifications are clearly defined: the input consists of preprocessed point cloud data and multi-view image data in a unified format, and the output consists of two types of feature vectors with matched dimensions, eliminating dimensionality barriers for subsequent fusion operations. The construction of the PointNet++ point cloud branch and geometric feature extraction are crucial for ensuring accurate capture of spatial information. The choice of the PointNet++ network framework stems primarily from its mature advantages in point cloud processing. This framework can effectively handle the disorder and sparsity of point cloud data, achieving multi-scale feature extraction through the synergistic effect of sampling layers, grouping layers, and feature propagation layers. The sampling layer employs the FPS (Farthest Point Sampling) algorithm, which can select key sampling points more evenly compared to random sampling, ensuring a complete representation of the overall shape of the target region. The grouping layer constructs local neighborhoods centered on sampling points, enabling the network to focus on local geometric details and capture the relative positional relationships between points by encoding the features of point cloud data within the neighborhood. The feature propagation layer is responsible for multi-scale feature fusion, gradually integrating local features at different levels to achieve a progression from low-level geometric features to high-level abstract features. The final output point cloud geometric feature vector fully contains core information such as spatial location, shape contour, and topological relationships. The ResNet-101 image branch focuses on in-depth texture feature extraction. The core reason for choosing the ResNet-101 framework is that its residual block structure can effectively solve the gradient vanishing problem in deep network training, ensuring efficient transfer and extraction of texture features. In the branch architecture adjustment, the core structure of the original network—50 convolutional layers, 16 residual blocks, and fully connected layers—is retained to ensure the depth and accuracy of texture feature extraction. The classification layer is removed and a feature adaptation layer is added to abandon the image classification function of the original network and shift the output target to feature vector generation. At the same time, by adjusting the dimension of the feature adaptation layer, the dimensions of the image texture feature vector and the point cloud geometric feature vector are perfectly matched, preparing for subsequent fusion.During feature extraction, convolutional layers are responsible for initially extracting basic features such as edges and textures of the image. Residual blocks are connected via shortcuts to ensure efficient feature transfer. Fully connected layers compress and integrate deep features, ultimately generating image texture feature vectors that accurately contain key information such as color distribution, texture details, and local contours. To ensure the quality of feature extraction, feature validity verification is indispensable. By calculating the variance and information entropy of the feature vector, variance reflects the dispersion of the feature, while information entropy characterizes the richness of information in the feature. The comprehensive evaluation of these two indicators can effectively determine whether the feature accurately represents the information of the original data. If the validity is insufficient, feature extraction needs to be repeated by adjusting key parameters such as the convolutional kernel size and learning rate until a high-quality feature vector is obtained.

[0026] Step 103: Align the two branches of features based on the AFM attention fusion module to generate cross-modal feature vectors, and calculate the registration matrix of point cloud and image by combining POS position and pose data. Optimize the registration matrix using ICP-Net iterative nearest point neural network to generate a 3D semantic point cloud set. Specifically, in this embodiment, the point cloud geometric feature vector and the image texture feature vector are input into the module. The importance of the two types of features is analyzed by the computing unit, and weights are assigned to key features and redundant features. Based on the attention weight results, the spatial dimension of the two types of features is adjusted, and the spatial offset between features is corrected by using a bilinear interpolation algorithm. The aligned two types of features are fused by weighted summation to generate a cross-modal feature vector.

[0027] The construction and feature alignment of the AFM attention fusion module are crucial prerequisites for achieving cross-modal fusion. Since point cloud geometric features and image texture features originate from different modalities and exhibit spatial offsets and semantic differences, direct fusion can easily lead to feature confusion. Therefore, an attention mechanism is needed to achieve accurate feature matching and weight allocation. In the AFM module construction, the feature attention weight calculation unit is the core component. It analyzes the contribution of the two types of features during the fusion process, assigning higher weights to key features and lower weights to redundant features, achieving selective feature fusion and improving the effectiveness of the fused features. The feature alignment unit addresses the spatial offset problem of the two types of features by using a bilinear interpolation algorithm to adjust the spatial dimension, accurately mapping image texture features to the point cloud coordinate system, ensuring that the two types of features match in the same spatial dimension. The feature fusion unit employs a weighted summation fusion strategy, integrating the aligned two types of features based on attention weights. The generated cross-modal feature vector possesses both geometric and textural information, providing accurate feature basis for subsequent spatial registration. Initial calculation and iterative optimization of the registration matrix are core steps in ensuring spatial fusion accuracy. The initial registration matrix is ​​calculated based on POS position and pose data, primarily because POS data possesses high-precision position and pose information, providing a reliable initial reference for spatial alignment between the point cloud and the image. By establishing the transformation relationship between the point cloud coordinate system and the image pixel coordinate system, the initial registration matrix is ​​solved using the least squares method, achieving preliminary spatial alignment of the two types of data and laying the foundation for subsequent optimization. ICP-Net iterative optimization is a key method for improving registration accuracy. Compared to the traditional ICP algorithm, ICP-Net uses neural networks to automatically calculate registration errors and adaptively adjust matrix parameters, significantly improving registration efficiency and accuracy. During the registration optimization process, the feature matching unit finds corresponding feature point pairs based on cross-modal feature vectors and determines the registration deviation by calculating the spatial distance error between feature point pairs. The iterative optimization unit dynamically adjusts the registration matrix parameters based on error feedback, continuously reducing the distance error between feature point pairs until the error value is lower than the preset threshold of 0.01m, ensuring that the registration accuracy meets the requirements of engineering applications. The registration result verification stage calculates the reprojection error, projects the image feature points onto the point cloud coordinate system, and verifies the distance deviation between the projected points and the corresponding point clouds, further ensuring complete spatial matching. The generation of the 3D semantic point cloud set is the final goal of this stage. Based on the optimal registration matrix, image texture feature information is first accurately assigned to the corresponding point cloud data, giving the point cloud, which originally only contained spatial information, rich color and texture details. Then, combined with the semantic information in the cross-modal feature vectors, the target categories are initially divided through feature clustering algorithms to achieve preliminary semantic differentiation of building components. Finally, each point cloud data is assigned a specific semantic label, such as building walls, doors and windows, beams and columns, giving the point cloud data clear semantic attributes.The resulting 3D semantic point cloud integrates spatial coordinates, texture information, and semantic labels, achieving not only deep fusion of multi-source surveying data but also providing high-quality core data support for subsequent attribute association, lightweight processing, and interactive applications with BIM models. It serves as a crucial bridge connecting 3D surveying data and BIM applications.

[0028] The ICP-Net iterative optimization of the registration matrix involves constructing an ICP-Net iterative nearest-point neural network. This network, based on the iterative approach of the traditional ICP algorithm, uses a neural network to automatically calculate registration errors and adaptively adjust matrix parameters. In the registration optimization process, the initial registration matrix and cross-modal feature vectors are input into the ICP-Net network. First, the network's feature matching unit finds corresponding feature point pairs in the point cloud and image, calculating the spatial distance error between these pairs. Then, the iterative optimization unit adjusts the parameters of the registration matrix to reduce the distance error between feature point pairs. This iterative process is repeated until the error value is below a preset threshold, such as 0.01m, yielding the optimal registration matrix. For registration result verification, the optimal registration matrix is ​​applied to the point cloud and image data. The reprojection error—the error of projecting image feature points onto the point cloud coordinate system—verifies the registration accuracy, ensuring a perfect spatial match between the point cloud and the image.

[0029] The generation of 3D semantic point cloud sets is based on the optimal registration matrix. Image texture feature information is assigned to the corresponding point cloud data. At the same time, semantic information in cross-modal feature vectors is combined to initially classify target categories through feature clustering. Semantic labels are assigned to each point cloud data, such as building walls, doors and windows, beams and columns. Finally, a 3D semantic point cloud set with spatial coordinates, texture information and semantic labels is generated.

[0030] Step 104: Import the 3D semantic point cloud into the BIM software, identify the building components in the point cloud based on the semantic segmentation network, associate the geometric attributes of the components in the point cloud with the component attributes in the BIM model, output a lightweight 3D model, and use the model for surveying, viewing and interaction.

[0031] Specifically, in this embodiment, a semantic segmentation network is built based on the U-Net architecture. The 3D semantic point cloud set imported from BIM software is input into the network. Through deep convolution and deconvolution operations, the network identifies various building components in the point cloud and assigns unique identifier codes and color labels to different types of components for semantic differentiation. The point cloud component attributes are then correlated and fused with the corresponding component attributes in the BIM model through identifier code matching, supplementing the detailed attribute information of the BIM model and outputting a lightweight 3D model.

[0032] The 3D semantic point cloud is imported into BIM software. The generated 3D semantic point cloud set in LAS format is imported into mainstream BIM software such as Revit and Bentley. The point cloud data is loaded and displayed using the software's built-in point cloud processing plugin, ensuring that the point cloud data can be accurately mapped into the 3D coordinate system of the BIM software. A semantic segmentation network is built, based on the optimized U-Net architecture. The 3D semantic point cloud set imported into the BIM software is input into the network. Through deep convolution and deconvolution operations of the network, various building components in the point cloud, such as walls, floors, beams, columns, doors, windows, and pipelines, are accurately identified. Different types of components are assigned unique identification codes and color labels, realizing semantic differentiation of components.

[0033] If no existing BIM model exists for the target area, a basic BIM model is constructed in the BIM software based on the identified semantic information of the building components. This model includes basic attributes such as the geometric shape and spatial location of the components. If an existing BIM model exists, the model is standardized to ensure that the coordinate system is consistent with the point cloud data. Attribute association matching is used to extract the geometric attributes of various components in the 3D semantic point cloud, such as dimensions, volume, position coordinates, and material parameters. By matching identifier codes, the point cloud component attributes are associated and fused with the corresponding component attributes in the BIM model, supplementing the detailed attribute information of the BIM model and improving the completeness and accuracy of the model.

[0034] The associated BIM model is lightweighted by using a model simplification algorithm to remove redundant geometric surfaces and attribute information, while retaining the core component structures and key attributes. The model format is converted to a lightweight format, such as glTF, to reduce the model's data size and ensure that the model can be loaded and displayed quickly on different devices.

[0035] Deploying lightweight 3D models onto a visualization platform enables the following interactive functions: multi-angle viewing, allowing users to view the 3D mapping results of the target area from different perspectives through rotation, translation, and zoom; component query, allowing users to click on any component in the model to view its corresponding geometric attributes, material information, semantic tags, and other detailed data; spatial analysis, supporting spatial analysis operations such as distance measurement, area calculation, and collision detection based on the model; and data export, supporting the export of viewing and analysis results into report formats to provide data support for engineering design, construction management, operation and maintenance, and other work.

[0036] Its beneficial effects lie in achieving efficient integration of multi-source surveying and mapping data through timestamp association and hierarchical preprocessing, improving data quality, and solving the fusion problems of data heterogeneity and spatiotemporal misalignment in traditional methods. The collaborative application of a multimodal feature extraction network and an AFM attention fusion module accurately captures point cloud geometric features and image texture features. Combined with ICP-Net optimized registration, it significantly improves registration accuracy and stability, and reduces the impact of noise interference. It achieves deep attribute association between multi-source data features and BIM models, enriching the information dimensions of BIM models. The generated lightweight model balances accuracy and loading efficiency, supporting rapid viewing and interaction on multiple devices. Improving surveying and mapping data processing efficiency can significantly enhance engineering survey and design efficiency, construction and maintenance accuracy, and reduce project costs.

[0037] Please see Figure 2 In the BIM-based 3D mapping data fusion processing method, point cloud data, multi-view images, and POS position and pose data of the target area are collected. These three types of data are correlated using timestamps. The collected data undergoes preprocessing to obtain multi-source mapping data, including the following steps: Step 201: Collect point cloud data, multi-view images and POS position and pose data of the target area, extract the timestamp information of the three types of raw data, and use the timestamp of POS data as the basis to associate the point cloud data, multi-view image data and POS data of the same time dimension through time matching algorithm to obtain associated data; Step 202: Use a pass-through filtering algorithm to remove invalid point clouds from the point cloud data, use a statistical filtering algorithm to remove isolated noise points, and downsample the valid point clouds. Step 203: Perform distortion correction, size normalization and grayscale equalization on the image, and remove blurry or overexposed invalid images. Step 204: Remove abnormal data points caused by equipment vibration and signal interference during data acquisition, and smooth the valid data using a sliding window filtering algorithm; standardize the three types of preprocessed data to obtain multi-source mapping data.

[0038] The above describes embodiments of the BIM-based 3D mapping data fusion processing method of the present invention. Please refer to [link / reference]. Figure 3 In a BIM-based 3D mapping data fusion processing system, the system includes the following modules: The surveying data acquisition module is used to collect point cloud data, multi-view images, and POS position and attitude data of the target area. It associates the three types of data through timestamps and preprocesses the collected data to obtain multi-source surveying data. The data feature extraction module is used to establish a multimodal feature extraction network. It extracts the geometric features of point cloud data through the PointNet++ point cloud branch and extracts the texture features of image data through the ResNet-101 image branch. The mapping data processing module is used to align the two-branch features based on the AFM attention fusion module, generate cross-modal feature vectors, and calculate the registration matrix of point cloud and image by combining POS position and pose data. The registration matrix is ​​optimized by using ICP-Net iterative nearest point neural network to generate a three-dimensional semantic point cloud set. The data viewing and interaction module is used to import 3D semantic point clouds into BIM software, identify building components in the point cloud based on semantic segmentation networks, associate the geometric attributes of the components in the point cloud with the component attributes in the BIM model, output a lightweight 3D model, and use the model for surveying, viewing and interaction.

[0039] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended technical solutions and their equivalents.

Claims

1. A BIM-based method for fusing and processing 3D surveying data, characterized in that, The BIM-based 3D mapping data fusion processing method includes the following steps: Point cloud data, multi-view images, and POS position and pose data of the target area are collected. The three types of data are associated by timestamps. The collected data are preprocessed to obtain multi-source mapping data. A multimodal feature extraction network is established. The geometric features of point cloud data are extracted through the PointNet++ point cloud branch, and the texture features of image data are extracted through the ResNet-101 image branch. Based on the AFM attention fusion module, the two branches of features are aligned to generate cross-modal feature vectors. The registration matrix of point cloud and image is calculated by combining POS position and pose data. The registration matrix is ​​optimized by ICP-Net iterative nearest point neural network to generate a three-dimensional semantic point cloud set. The three-dimensional semantic point cloud is imported into BIM software, and building components in the point cloud are identified based on a semantic segmentation network. The geometric attributes of the components in the point cloud are associated with the component attributes in the BIM model, and a lightweight three-dimensional model is output. The model is then used for surveying, viewing, and interaction.

2. The BIM-based three-dimensional mapping data fusion processing method as described in claim 1, characterized in that, The point cloud data, multi-view images, and POS position and pose data of the target area are collected and correlated with each other using timestamps. The collected data is then preprocessed to obtain multi-source mapping data, including: Point cloud data, multi-view images, and POS position and pose data of the target area are collected. The timestamp information of the three types of raw data is extracted. Based on the timestamp of the POS data, the point cloud data, multi-view image data and POS data of the same time dimension are associated with each other through a time matching algorithm to obtain associated data. A pass-through filtering algorithm is used to remove invalid point clouds from the point cloud data, a statistical filtering algorithm is used to remove isolated noise points, and the valid point clouds are downsampled. The image is processed for distortion correction, size normalization and grayscale equalization, while blurry and overexposed invalid images are removed; Abnormal data points caused by equipment vibration and signal interference during data acquisition are removed, and a sliding window filtering algorithm is used to smooth the valid data. The three types of preprocessed data are then standardized to obtain multi-source mapping data.

3. The BIM-based three-dimensional mapping data fusion processing method as described in claim 1, characterized in that, The establishment of a multimodal feature extraction network, which extracts geometric features from point cloud data through the PointNet++ point cloud branch, includes: A point cloud feature extraction branch is constructed based on the PointNet++ network framework. The branch contains three core modules: sampling layer, grouping layer, and feature propagation layer. A multi-scale hierarchical extraction strategy is used to accurately capture the geometric features of point clouds. The preprocessed point cloud data is input into the branch. The sampling layer uses the FPS farthest point sampling algorithm to select key sampling points in the point cloud. Then, the grouping layer constructs local neighborhoods with the sampling points as the center and performs feature encoding on the point cloud data in each neighborhood. By fusing local features at different scales through a feature propagation layer, the level of feature abstraction is gradually improved, and a point cloud geometric feature vector with uniform dimension is output.

4. The BIM-based three-dimensional mapping data fusion processing method as described in claim 1, characterized in that, The extraction of texture features from image data using the ResNet-101 image branch includes: An image feature extraction branch is constructed based on the ResNet-101 network framework. The original 50 convolutional layers, 16 residual blocks and fully connected layers of the network are retained, the classification layer of the original network is removed and a feature adaptation layer is added. The preprocessed multi-view image data is input into the branch, and the texture features of the image are initially extracted through the convolutional layer. The residual block is used to solve the gradient vanishing problem in the deep training of the network. The deep convolutional features are compressed and integrated through a fully connected layer, and then the output features are adjusted to an image texture feature vector with the same dimension as the point cloud geometric feature vector through a feature adaptation layer.

5. The BIM-based three-dimensional mapping data fusion processing method as described in claim 1, characterized in that, The AFM-based attention fusion module aligns the two-branch features, generates cross-modal feature vectors, and calculates the registration matrix between the point cloud and the image using POS position and pose data. The registration matrix is ​​then optimized using the ICP-Net iterative nearest-point neural network to generate a 3D semantic point cloud set, including: The point cloud geometric feature vector and the image texture feature vector are input into the module. The importance of the two types of features is analyzed by the computing unit, and weights are assigned to key features and redundant features. Based on the attention weight results, the spatial dimension of the two types of features is adjusted, and the bilinear interpolation algorithm is used to correct the spatial offset between the features. We use a weighted summation method to fuse the aligned features of the two classes to generate a cross-modal feature vector.

6. The BIM-based three-dimensional mapping data fusion processing method as described in claim 1, characterized in that, The step of importing the three-dimensional semantic point cloud set into BIM software and identifying building components in the point cloud based on a semantic segmentation network includes: A semantic segmentation network is built based on the U-Net architecture. The 3D semantic point cloud set imported from BIM software is input into the network. Through deep convolution and deconvolution operations of the network, various building components in the point cloud are identified, and different types of components are assigned unique identification codes and color labels to distinguish the semantics of the components.

7. The BIM-based three-dimensional mapping data fusion processing method as described in claim 1, characterized in that, The process of associating the geometric attributes of components in the point cloud with the component attributes in the BIM model to output a lightweight 3D model, and then using the model for surveying, viewing, and interaction, includes: By matching identifier codes, the attributes of point cloud components are associated and fused with the corresponding component attributes in the BIM model, supplementing the detailed attribute information of the BIM model and outputting a lightweight 3D model.

8. A BIM-based 3D surveying data fusion and processing system, characterized in that, The BIM-based 3D mapping data fusion and processing system includes the following modules: The surveying data acquisition module is used to collect point cloud data, multi-view images, and POS position and attitude data of the target area. It associates the three types of data through timestamps and preprocesses the collected data to obtain multi-source surveying data. The data feature extraction module is used to establish a multimodal feature extraction network. It extracts the geometric features of point cloud data through the PointNet++ point cloud branch and extracts the texture features of image data through the ResNet-101 image branch. The mapping data processing module is used to align the two-branch features based on the AFM attention fusion module, generate cross-modal feature vectors, and calculate the registration matrix of point cloud and image by combining POS position and pose data. The registration matrix is ​​optimized by using ICP-Net iterative nearest point neural network to generate a three-dimensional semantic point cloud set. The data viewing and interaction module is used to import the three-dimensional semantic point cloud into BIM software, identify building components in the point cloud based on the semantic segmentation network, associate the geometric attributes of the components in the point cloud with the component attributes in the BIM model, output a lightweight three-dimensional model, and use the model for surveying, viewing and interaction.

9. The BIM-based three-dimensional mapping data fusion processing system as described in claim 8, characterized in that, The surveying data processing module includes the following sub-modules: The processing submodule is used to build a semantic segmentation network based on the U-Net architecture. It inputs the 3D semantic point cloud set imported from BIM software into the network, identifies various building components in the point cloud through deep convolution and deconvolution operations of the network, and assigns unique identification codes and color labels to different types of components to perform semantic differentiation of components.

10. The BIM-based three-dimensional mapping data fusion processing system as described in claim 8, characterized in that, The surveying data processing module includes the following sub-modules: The matching submodule is used to associate and fuse point cloud component attributes with corresponding component attributes in the BIM model by matching identifier codes, supplementing the detailed attribute information of the BIM model and outputting a lightweight 3D model.

Images