Old building digital surveying and mapping system and method based on air-ground coordination point cloud fusion

Abstract

The application relates to the technical field of building surveying and mapping, and provides an old building digital surveying and mapping system and method based on air-ground cooperative point cloud fusion, which comprises the following steps: blind area detection is performed on spatiotemporal fusion data scanned by a unmanned aerial vehicle device and a ground robot, and a dynamic master-slave mechanism is started when there is a blind area view angle in top point cloud data / bottom point cloud data; wherein the dynamic master-slave mechanism takes a point cloud data corresponding device with a blind area as a slave device, takes a point cloud data corresponding device without a blind area as a master device, controls the slave device to perform point cloud filling, determines blind area compensation data, and generates target spatiotemporal fusion data loaded to a visual interface after the blind area compensation data is fused to the spatiotemporal fusion data. Through a dynamic master-slave control mechanism, a multi-dimensional synchronization strategy and an intelligent compensation algorithm driven by building features, a closed loop working flow from data acquisition to quality verification is constructed.

Description

Technical Field

[0001] This invention relates to the field of architectural surveying technology, and in particular to a digital surveying system and method for old buildings based on air-ground collaborative point cloud fusion. Background Technology

[0002] The surveying of old buildings, due to their complex structural features, intricate decorative components, and irregular geometric shapes, places extremely high demands on the accuracy, completeness, and adaptability of surveying techniques. Traditional surveying techniques have significant limitations in areas such as multi-device collaboration, dynamic environmental adaptability, and intelligent compensation mechanisms, making it difficult to meet the high standards required for the protection and restoration of historical buildings.

[0003] In existing technologies, single-platform scanning solutions are limited by physical perspective constraints and cannot effectively cover blind spots at the top and bottom of buildings. Drone scanning easily misses hidden areas such as eaves and the backs of carvings, while ground-based equipment struggles to capture details at higher elevations like rooftops. These methods rely on human experience to identify blind spots, easily leading to missed detections or misjudgments. Furthermore, compensation data generation often uses simple interpolation algorithms, causing geometric distortion and texture feature breaks. When fusion heterogeneous data, the spatiotemporal asynchrony between devices causes coordinate offsets and motion blur. Traditional registration algorithms are insufficiently adaptable to complex structures, easily leading to error accumulation. In addition, fixed master-slave control modes lack flexibility in dynamic environments, resulting in low device collaboration efficiency and weak data recovery capabilities after data loss.

[0004] Existing compensation techniques largely rely on manual modeling or general generation algorithms, failing to adequately consider architectural feature constraints. This results in insufficient matching between the generated data and the original structure's form, texture, and mechanical properties. The lack of a quality verification process makes error traceability difficult, and data quality depends on manual screening in the later stages, leading to inefficiency and insufficient reliability. The industry has long faced core contradictions such as the mismatch between high-precision requirements and equipment capabilities, low efficiency in processing massive amounts of data, and poor adaptability to dynamic environments.

[0005] In recent years, technological improvements have focused on enhancing hardware performance or optimizing local algorithms, such as increasing the power of scanning equipment, improving the efficiency of registration algorithms, or introducing additional mobile platforms. However, these solutions have failed to overcome the inherent limitations of single-device data acquisition, nor have they established a cross-platform intelligent collaborative framework, thus failing to systematically solve the collaborative problems of blind spot compensation, multi-source data fusion, and dynamic control. Summary of the Invention

[0006] This invention proposes a digital mapping system and method for old buildings based on air-ground collaborative point cloud fusion. Through a dynamic master-slave control mechanism, multi-dimensional synchronization strategies, and intelligent compensation algorithms driven by building features, a closed-loop workflow is constructed from data acquisition to quality verification. This solution effectively integrates the spatial coverage advantages of UAVs and ground robots, combined with real-time fusion and adaptive compensation technologies, providing a high-precision, high-efficiency, and highly robust end-to-end solution for the digitization of old buildings.

[0007] In a first aspect, the embodiments of this application provide a method for digital mapping of old buildings based on air-ground collaborative point cloud fusion, including:

[0008] Blind spot detection is performed on the spatiotemporal fusion data scanned by UAV equipment and ground robots, and a dynamic master-slave mechanism is activated when there is a blind spot view in the top point cloud data / bottom point cloud data;

[0009] The dynamic master-slave mechanism uses the device corresponding to the point cloud data with blind spots as the slave device and the device corresponding to the point cloud data without blind spots as the master device. It controls the slave device to perform point cloud filling, determines the blind spot compensation data, and then merges the blind spot compensation data into the spatiotemporal fusion data to generate the target spatiotemporal fusion data loaded onto the visualization interface.

[0010] In this embodiment, during the digital mapping of old buildings, when blind spots exist, data filling of the blind spots from different perspectives is achieved by dynamically switching between master and slave devices. During data filling, because there are always synchronized master and slave control terminals, master-slave verification and synchronization are performed constantly, enabling high-precision spatiotemporal synchronization compensation and achieving digital visualization output. Because of spatiotemporal synchronization, mapping deviations are avoided due to a single device or a lack of correlation between devices during blind spot compensation.

[0011] In conjunction with the first aspect, before performing blind spot detection on the spatiotemporal fusion data scanned by the UAV equipment and ground robot, the method further includes:

[0012] A pre-configured synchronous data transmission mechanism is used to receive top point cloud data from UAV equipment scanning old buildings and bottom point cloud data from ground robots scanning them. The synchronous transmission mechanism includes a spatiotemporal synchronization mechanism, a master-slave dynamic synchronization mechanism, and an interface perspective synchronization mechanism.

[0013] In this embodiment of the application, for data synchronization before blind spot monitoring, the synchronization transmission mechanism configured in this application can achieve three-layer synchronization in terms of spatiotemporal acquisition, master-slave control and interface perspective, so as to ensure the accuracy of the data.

[0014] In conjunction with the first aspect, the spatiotemporal synchronization mechanism is configured with a unified coordinate system based on a joint calibration target; wherein, the ground robot is timed in real time by the UAV equipment within a sliding window of the unified coordinate system, generating spatiotemporal fusion data of the top point cloud and the bottom point cloud.

[0015] In the embodiments of this application, the spatiotemporal synchronization mechanism is configured with a unified coordinate system. By setting a sliding window for spatiotemporal synchronization, real-time data synchronization is performed. The synchronized data exists in a unified coordinate system and in the same data window, so no spatiotemporal deviation will occur during fusion.

[0016] In conjunction with the first aspect, the interface perspective synchronization mechanism performs multi-view segmentation of the target spatiotemporal fusion data based on a sliding time axis according to the unified coordinate system of the spatiotemporal synchronization mechanism; wherein, the segmented perspectives include UAV view, ground robot view and fusion view, the sliding time axis is used for perspective synchronization backtracking, and the point cloud curvature similarity of the target spatiotemporal fusion data after perspective synchronization backtracking is the same.

[0017] In this embodiment, the interface view synchronization mechanism can perform three-way segmentation of UAV, ground robot and fused view when performing multi-view segmentation. During the segmentation process, because there is a sliding time axis, the view synchronization backtracking can be achieved based on the sliding time axis. During the view synchronization backtracking process, the similarity of point cloud curvature is fused, which can achieve master-slave device scanning data fusion with extremely low deviation or even no deviation. It can also realize the backtracking judgment of any event and determine whether there is a point cloud data error.

[0018] In conjunction with the first aspect, the fusion of blind spot compensation data into spatiotemporal fusion data includes:

[0019] Determine the overlapping area of ​​the blind zone edge based on the host device and slave device;

[0020] Extract the spatiotemporally aligned coordinates of the overlapping boundary regions that overlap with the blind zone edge in the top and bottom point cloud data on a unified coordinate system;

[0021] Based on the coordinates of the overlapping boundary, the spatiotemporal fusion data of the blind zone compensation data and the overlapping area of ​​the blind zone edge are bidirectionally projected to determine whether there are any anomalies in the compensation data; when the compensation data is anomaly, the texture feature parameters of the bidirectional projection are different.

[0022] In this embodiment of the application, during the process of blind zone compensation data fusion into spatiotemporal fusion data, which is blind zone data compensation, the boundary compensation judgment of spatiotemporal alignment of blind zone compensation data on a unified coordinate system is performed through the overlapping area of ​​blind zone edges. During compensation, the blind zone data and spatiotemporal fusion data can be bidirectionally projected to determine whether there are anomalies in the overlapping area of ​​the edges, thereby determining whether there are differences in texture features.

[0023] In conjunction with the first aspect, the fusion of blind spot compensation data into spatiotemporal fusion data further includes:

[0024] Construct a database of architectural features for old buildings and identify functional areas with different architectural features;

[0025] When there are blind spot compensation data that are unidentifiable feature blind spots, the functional areas in the building feature library are matched based on the target area where the feature blind spot is located, and active prediction compensation is performed when the match is consistent.

[0026] In the embodiments of this application, when performing blind spot compensation, the application will actively predict the blind spot compensation data. Based on the active prediction of blind spot functionality, the application will actively predict the blind spot compensation data to achieve blind spot compensation.

[0027] In conjunction with the first aspect, the process of fusing blind spot compensation data into spatiotemporal fusion data further includes:

[0028] Match the geometric and topological features of adjacent areas of the blind zone to generate a virtual point cloud with shape rules for functional areas based on architectural features;

[0029] The curvature consistency of the virtual point cloud and the point cloud data of the host device is verified to determine whether the curvature error exceeds a preset threshold, and the virtual point cloud that does not exceed the preset threshold is used as the target data for blind zone compensation data.

[0030] In this embodiment of the application, when compensating for blind spots, the application will determine the curvature consistency based on the shape rules of the old building and the characteristics of the virtual point cloud. Through curvature consistency, the matching shape features and the characteristics of the blind spots of the old building will be determined to determine the data for blind spot compensation.

[0031] In conjunction with the first aspect, when the curvature error does not exceed a preset threshold, it includes:

[0032] The target region is divided into three sub-regions according to curvature distribution, and a differential error threshold is set.

[0033] By fusing infrared thermal imaging and historical drawing data, a candidate point cloud is generated through a graph neural network.

[0034] Candidate data are selected based on dynamic scoring of curvature continuity, texture consistency, and structural compliance.

[0035] Generative adversarial networks are used for iterative optimization until the discriminant confidence difference is less than a preset threshold.

[0036] In this embodiment of the application, in order to minimize the occurrence of errors during blind zone compensation, this application uses an adversarial network generator to iteratively optimize the blind zone compensation data by calculating the confidence level to determine the most suitable compensation data.

[0037] In conjunction with the first aspect, before performing blind spot detection on the spatiotemporal fusion data scanned by the UAV equipment and ground robot, the method further includes:

[0038] The point cloud data collected by the UAV is divided into spatial grids, and the rate of change of point cloud density in each grid is detected. Only grids with a rate of change exceeding the threshold are incrementally encoded.

[0039] High curvature feature points are extracted from the point cloud data collected by the ground robot as keyframes, and keyframe data is encoded and transmitted first.

[0040] Establish a spatiotemporal mapping relationship between UAV incremental data blocks and robot keyframes;

[0041] When data loss is detected in a certain device, related data is extracted from the data stream of another device based on the spatiotemporal mapping relationship and interpolated to complete the data.

[0042] In this embodiment, prior to blind spot monitoring, the data collected by the UAV and ground robot devices is subjected to real-time scanning. The data collected by the UAV can be divided into spatial grids, and incremental encoding is achieved based on the point cloud density within the grid, i.e., incremental data. For the data collected by the ground robot, keyframe determination of high curvature feature points is required. The two are mapped in a spatiotemporal manner. If the two are consistent after mapping, data compensation will be performed if either the master or slave device loses data, based on the spatiotemporal mapping relationship.

[0043] Secondly, this application proposes a digital mapping system for old buildings based on air-ground collaborative point cloud fusion, comprising:

[0044] Blind Spot Dynamic Master-Slave Control Module: Used to perform blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, and to activate the dynamic master-slave mechanism when there is a blind spot view in the top point cloud data / bottom point cloud data;

[0045] Terminal loading module: Used in dynamic master-slave mechanism to use the device corresponding to the point cloud data with blind spots as the slave device and the device corresponding to the point cloud data without blind spots as the master device, and control the slave device to perform point cloud filling, determine the blind spot compensation data, and after the blind spot compensation data is fused into the spatiotemporal fusion data, generate the target spatiotemporal fusion data loaded onto the visualization interface.

[0046] In this embodiment, during the digital mapping of old buildings, when blind spots exist, data filling of the blind spots from different perspectives is achieved by dynamically switching between master and slave devices. During data filling, because there are constantly synchronized master and slave control terminals, master-slave verification and synchronization are performed continuously, enabling high-precision spatiotemporal synchronization compensation and achieving digital visualization output. Because of spatiotemporal synchronization, mapping deviations are avoided due to a single device or a lack of correlation between devices during blind spot compensation.

[0047] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0048] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0050] Figure 1 This is a flowchart illustrating a method for digitally mapping old buildings based on air-ground collaborative point cloud fusion, as described in an embodiment of the present invention.

[0051] Figure 2 This is a flowchart illustrating the implementation of the synchronous data transmission mechanism in this embodiment of the invention.

[0052] Figure 3 This is a flowchart illustrating the implementation of the spatiotemporal synchronization mechanism in this invention.

[0053] Figure 4 This is a flowchart illustrating the implementation of the interface perspective synchronization mechanism in this embodiment of the invention.

[0054] Figure 5 This is a diagram illustrating the detection and execution of abnormal compensation data in an embodiment of the present invention.

[0055] Figure 6 This is a flowchart illustrating the implementation of the feature library-driven compensation mechanism in this embodiment of the invention.

[0056] Figure 7 This is an implementation diagram of the curvature error determination process in an embodiment of the present invention;

[0057] Figure 8 This is a process implementation diagram of the multi-source data fusion path in an embodiment of the present invention;

[0058] Figure 9 This is an implementation diagram of the data compensation mechanism in this invention embodiment;

[0059] Figure 10 This is a diagram illustrating the spatiotemporal mapping relationship in an embodiment of the present invention.

[0060] Figure 11 This is a system composition diagram of an old building digital mapping system based on air-ground collaborative point cloud fusion, as described in an embodiment of the present invention. Detailed Implementation

[0061] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0062] In the surveying of old buildings, traditional surveying of old buildings mainly relies on a single platform, such as scanning in stages using only ground-based mobile equipment, and then manually judging the blind spots that appear during the scanning process. The location of the blind spots is mainly predicted based on human experience, and then the scanning path is planned.

[0063] In the blind spot compensation stage, missing data is filled in using spatial interpolation algorithms based on nearby point clouds, or details are supplemented through manual modeling later. The acquisition devices coordinate their operations using a fixed master-slave mode, with the master device always undertaking the core data acquisition task. For example, a ground-based mobile device primarily acquires data, and then, based on missing data, another ground-based mobile device or drone is manually assigned to assist in acquisition. The auxiliary slave devices only perform supplementary scans, which can easily lead to data gaps.

[0064] Traditional solutions, mostly relying on single-device scanning, are limited by physical viewing angles and cannot eliminate inherent blind spots at the top and bottom. Manual judgment of blind spots is subjective, inefficient, and prone to missing data on critical components. Compensation data generated by simple interpolation algorithms does not match the original structural geometry well enough, resulting in frequent texture breaks and surface distortion. Fixed master-slave models lack dynamic responsiveness; when the master device encounters environmental interference, the system needs to reconstruct the workflow for an extended period, causing data gaps and a surge in time costs.

[0065] Traditional compensation methods primarily employ rigid registration algorithms (such as ICP) to stitch together multi-source data, relying on manual calibration of feature points to achieve coordinate system unification, and using offline batch processing to complete data fusion. The quality verification process uses only curvature continuity as a single evaluation metric, relying on manual visual inspection to filter out abnormal data, lacking a systematic verification mechanism.

[0066] Furthermore, rigid registration algorithms cannot adapt to the irregular structural changes of old buildings, easily leading to error accumulation and coordinate shifts. The accuracy of manually calibrated feature points is significantly affected by the operator's experience, and differences in heterogeneous data density result in obvious transitional breaks at the fusion interface. A single curvature index cannot comprehensively assess texture alignment and structural compliance, and manual screening is inefficient and has a high rate of missed detections. Therefore, the lack of a closed-loop verification system makes it difficult to trace the causes of errors, and abnormal data repair relies on repeated scanning, severely restricting surveying efficiency.

[0067] In view of this, this application solves the core defects of traditional technologies, such as incomplete handling of blind spots, low data fusion accuracy, and poor dynamic adaptability, by constructing an intelligent surveying and mapping system that integrates air and ground.

[0068] This application primarily utilizes a dynamic master-slave control mechanism, enabling surveying equipment to autonomously switch master and slave roles based on real-time blind spot detection results. This achieves intelligent optimization of scanning paths and efficient resource allocation, preventing data discontinuity issues common in traditional fixed-mode surveying. Furthermore, this application combines multi-source data synchronous transmission and joint calibration technologies to establish a unified spatiotemporal reference framework, eliminating coordinate offsets and motion blur caused by equipment heterogeneity, thereby improving the accuracy of multi-view data fusion. By introducing a building feature-driven compensation mechanism, virtual point clouds conforming to architectural rules are generated using a pre-set component feature library and a deep learning model. This not only aligns with the aesthetic characteristics of historical architecture but also resolves geometric distortion and texture breakage issues caused by traditional interpolation compensation. During closed-loop verification, multi-dimensional evaluation indicators such as curvature, texture, and structure are integrated, combined with iterative optimization using generative adversarial networks, to achieve a complete control chain from data generation to quality feedback, comprehensively ensuring the morphological compliance and feature consistency of the compensated data. The collaborative work of the real-time visualization engine and adaptive processing algorithm in this application can significantly shorten the processing delay from data acquisition to output, thereby enabling dynamic adjustment and on-site automatic or proactive decision-making in complex environments, providing high-precision digital mapping, and solving the problem of acquisition accuracy in blind areas.

[0069] Example 1:

[0070] This application provides a method for digital mapping of old buildings based on air-ground collaborative point cloud fusion. (See reference...) Figure 1 In the specific real-time process:

[0071] First, blind spot detection is performed on the spatiotemporal fusion data scanned by UAV equipment and ground robots, and a dynamic master-slave mechanism is activated when there is a blind spot view in the top point cloud data / bottom point cloud data;

[0072] The scanning device of this application combines ground equipment with UAV equipment. The UAV scanning 100 obtains the outline and appearance features of the old building by performing a real-time aerial view of the old building, thus achieving the first mapping. The UAV scanning of this application is not limited to high-altitude UAV scanning. For the special texture of the old building, both the UAV and the ground robot can perform synchronous spatiotemporal scanning indoors. The UAV has the advantage of aerial scanning, and the ground robot has the advantage of scanning areas that the UAV cannot access, such as gaps. The ground robot scanning 101 achieves spatiotemporal synchronous scanning by combining ground robots.

[0073] The first data from the drone scan 100 and the second data from the ground robot scan 101 are combined in a spatiotemporal manner, based on the air-ground collaborative spatiotemporal data acquisition 103, to scan the old building data in the same time and space domains, achieving adaptation and integration between the two.

[0074] During this process, blind zone detection of spatiotemporal fusion data will be performed 104. Specifically, the scanning coverage areas of drones and ground robots from different inspection and scanning perspectives will be compared. Through point cloud density gradient analysis and curvature continuity detection algorithms, the top / bottom blind zone boundaries in old buildings and bottom areas will be identified. As for the area between the bottom and the top, the two scans will cross data to achieve feature acquisition of the middle area.

[0075] If a blind spot is detected, a dynamic master-slave switching protocol is triggered, and the dynamic master-slave mechanism is started 105. If no blind spot is detected, the collected data can be directly used to generate spatiotemporal fusion data 111, which means fusing the data collected by the UAV equipment with the data scanned by the ground robot. The two data are spatiotemporally fused, duplicate data is removed, and the combined accurate mapping data is generated.

[0076] The perspective loss compensation process is illustrated in the following case: A Qing Dynasty hip-roof building features multiple layers of carved eaves and bracket sets. A drone cannot scan the blind spot under the eaves. When a ground robot scans the bottom data, it detects the top blind spot, triggering a dynamic master-slave switch. The drone becomes the slave, generating a spiral-progressive path along the blind spot boundary and performing a high-density compensation scan. During fusion, feature-constrained interpolation is used, matching the bracket set shape rule library to generate a virtual point cloud. Through a master-slave device data verification mechanism, the curvature difference between the compensated data and the original structure is reduced to an imperceptible level.

[0077] In the process of determining the master and slave devices 106, the dynamic master-slave mechanism in this application marks the device corresponding to the point cloud data with blind spots as the slave device 108 and the device corresponding to the point cloud data without blind spots as the master device 107. It also controls the slave device to perform point cloud filling compensation 109 to generate blind spot compensation data 110. After fusing the blind spot compensation data 112 into the spatiotemporal fusion data, it generates the target spatiotemporal fusion data 113 that is loaded onto the visualization interface 114.

[0078] In practice, the point cloud data of the current blind-spot-free device is used as the primary reference system, and devices with blind-spot data in the primary reference system are converted into slave devices. The primary device maintains the baseline data stream, and the slave devices generate compensation paths based on the characteristics of the primary data, such as adjusting the laser scanning angle and increasing the scanning overlap rate.

[0079] During blind spot compensation, a two-way projection verification method is employed, such as orthogonal projection combined with perspective projection. This involves matching the compensation data with the data collected by the main device in the same coordinate system to verify texture features and curvature continuity, ensuring geometric consistency of the fused interface. For example, this ensures no misalignment of the textures on the carved components of ancient buildings. However, sudden movement of personnel on-site can interrupt drone scanning, creating dynamic blind spots.

[0080] The device determination mechanism in the master-slave controller switches the ground robot to the master in 0.5 seconds or even less. It generates a predicted point cloud based on historical scan data. After the UAV recovers, it quickly aligns the data streams before and after the interruption through a spatiotemporal mapping table and uses a sliding window verification method to automatically repair the discontinuous point cloud in the temporal domain.

[0081] In this process, multi-dimensional verification indicators are automatically introduced, such as curvature, texture, and structure, and the abnormal compensation data is automatically removed to avoid secondary errors caused by manual intervention.

[0082] Example 2:

[0083] Before performing blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, this application will also set up a data synchronization mechanism, see [link / reference]. Figure 2 :

[0084] When this application is implemented, during the process of air-ground collaborative spatiotemporal data acquisition 103, a pre-configured synchronous data transmission mechanism will be used to receive the top point cloud data of the old building scanned by the UAV equipment and the bottom point cloud data scanned by the ground robot; among which, the synchronous transmission mechanism includes a spatiotemporal synchronization mechanism, a master-slave dynamic synchronization mechanism and an interface perspective synchronization mechanism.

[0085] The synchronous data transmission mechanism configuration 1031 is designed to achieve hierarchical synchronous transmission of the top point cloud data scanned by the UAV 100 and the bottom point cloud data scanned by the ground robot 101. This generates a highly consistent data foundation during blind spot detection, fuses the point clouds of the top and bottom data, and establishes a unified coordinate system based on physical calibration objects during the spatiotemporal synchronous transmission process. This achieves coordinate system alignment, and by jointly calibrating the spatiotemporal parameters of the sensors of the UAV and the ground robot, it achieves timestamp synchronization, eliminates pose deviations between devices, mainly differences in monitoring accuracy, and achieves mutual matching of scanning frequencies.

[0086] Hardware-level clock synchronization is used in the time dimension to ensure strict alignment of the scanning sequence. During the master-slave device switchover, the data dependencies between devices are updated in real time. That is, the master-slave synchronization mechanism 1051 maintains the stability of the collaborative link through heartbeat detection and status broadcast synchronization.

[0087] The master device continuously pushes a reference data stream, while the slave device adaptively adjusts the scanning frequency and data format. Multi-source data is mapped to a unified visualization space, and through dynamic binding technology of viewpoint parameters, the perspective relationship between the UAV's overhead view and the ground's upward view is automatically corrected, ensuring spatial consistency in the operating interface.

[0088] For spatiotemporal synchronization in joint scanning, a spatiotemporal synchronization mechanism is deployed: infrared calibration targets are set at the four corners of the building, and the UAV and ground robot automatically complete joint calibration after being powered on, with master-slave dynamic synchronization starting up.

[0089] The ground robot recognizes the need for top scanning and proactively requests to switch to a slave role, receiving the scanning path from the drone.

[0090] Interface perspective synchronization display: Based on the interface perspective synchronization mechanism 1032, the control operation platform renders a fused view of the UAV's top-down point cloud and the ground's bottom-up point cloud in real time, automatically hiding overlapping and redundant data. In this process, the scanning perspectives of the UAV and the ground robot are synchronized and then mapped onto a unified coordinate system to generate a visualized coordinate system. Rendering technology is used for real-time synchronous rendering to highlight the overall scanning and mapping of the current old building, improving the clarity and accuracy of the scan.

[0091] Example 3:

[0092] The spatiotemporal synchronization mechanism in this application is configured with a unified coordinate system based on a joint calibration target (see reference). Figure 3 Among them, the ground robot is given real-time time synchronization through the UAV equipment in a sliding window of a unified coordinate system, generating spatiotemporal fusion data of top point cloud and bottom point cloud.

[0093] This application deploys uniquely coded physical targets, such as QR codes / reflective balls, at key locations on the building. Drones and ground robots, using multiple sensors (e.g., combining LiDAR and visual monitoring), jointly identify these targets, establishing a unified spatial reference across devices and forming a unified coordinate system. The joint identification targets, based on the world coordinate system, measure data from three angles: horizontal, vertical, and the main facade of the building, ensuring the accuracy of the measurement data during the measurement process.

[0094] The target's three-dimensional coordinates are absolute reference coordinates, which can eliminate the cumulative errors of the equipment's local coordinate system. The UAV system 1001, which consists of UAV scanning, and the ground robot 1011 constitute the scanning system. The PPS signal of the Beidou satellite is used as the reference signal for real-time authorization to determine the spatiotemporal reference node, and broadcast timing signals to the lidar of the ground robot and the UAV to achieve timestamp alignment.

[0095] That is, the ground equipment dynamically adjusts the scanning sequence within the sliding time window to ensure that the timestamps of the top and bottom point clouds are strictly aligned.

[0096] The window size adaptively adjusts according to environmental complexity, balancing real-time performance and accuracy requirements. Under a unified coordinate system, the pitch scan data from the UAV and the elevation angle data from the ground robot are spatially interpolated and fused over time to generate a continuous 3D point cloud model. The fusion process preserves the original data characteristics of the equipment, avoiding information loss.

[0097] Example 4:

[0098] The interface perspective synchronization mechanism 1032 of this application performs multi-view segmentation of the target spatiotemporal fusion data based on a sliding time axis according to the unified coordinate system of the spatiotemporal synchronization mechanism; see reference Figure 4 The multi-view segmentation module 10321 segments the view into a drone view, a ground robot view, and a fused view. During this process, the view synchronization backtracking is performed by controlling the sliding time axis 10322. After the view synchronization backtracking 10323, the curvature similarity of the target spatiotemporal fused data point cloud is the same. Through the curvature similarity verification method, the drone view and the ground robot view with different curvatures can also be fine-tuned based on the time axis. This is controlled during the multi-view segmentation.

[0099] In the specific implementation process, based on the time-space mapping relationship of a unified coordinate system, the fused data is segmented into UAV overhead view, ground-based overhead view, and fused view according to the acquisition perspective. Each perspective retains the original data characteristics while establishing perspective correlations. Through time-stamp-driven data slicing, this process allows for synchronous playback of perspectives at any time period, ensuring strict alignment of the spatiotemporal evolution of point clouds from different perspectives. During the playback process, the curvature distribution similarity of point clouds from multiple perspectives is calculated in real time. When local differences are detected, a curvature compensation algorithm is automatically triggered to eliminate morphological deviations caused by differences in device perspectives.

[0100] In actual implementation:

[0101] The top view of the drone is segmented in a unified coordinate system, for example, to show the beam structure presented by the roof fold outline and the side view of the ground robot;

[0102] Slide the timeline to trace back data from key construction stages and compare structural deformation trends under different procedures;

[0103] Automatically align the curvature distribution of top-view and side-view data to generate a 3D roof model with a continuous transition;

[0104] The achievable technical effects are: significantly improved accuracy in restoring fold curves, clear identification of the spatial relationships between mortise and tenon joints in wooden components, and accurate mapping and display of old buildings.

[0105] Example 5:

[0106] This application integrates blind spot compensation data into spatiotemporal fusion data to establish an intelligent perception and compensation verification mechanism for blind spot edges. (See reference...) Figure 5 ,include:

[0107] This application determines the overlapping area of ​​the blind zone edge based on the host device and the slave device;

[0108] Then, extract the spatiotemporally aligned coordinates of the overlapping boundary regions with the blind zone edge from the top and bottom point cloud data in a unified coordinate system.

[0109] Finally, based on the coordinates of the overlapping boundary, the spatiotemporal fusion data of the blind zone compensation data and the overlapping area of ​​the blind zone edge are bidirectionally projected to determine whether there are any anomalies in the compensation data; when the compensation data is anomaly, the texture feature parameters of the bidirectional projection are different.

[0110] Based on the spatial relative pose and scanning path historical data of master and slave devices, this application dynamically identifies the transition edge between blind areas and effective areas through feature point density gradient analysis. The boundary identification process integrates device motion trajectory prediction to avoid misjudgment caused by dynamic changes in the environment.

[0111] This application employs an improved iterative nearest point algorithm under a unified coordinate system, specifically Unified Coordinate System System 1042, to align the edge point clouds of master and slave devices using the ICP algorithm. Specifically, it aligns the edge point clouds of the UAV scanning data and the ground robot scanning data in this application, and then extracts a set of boundary feature points with spatiotemporal continuity. Subsequently, stable overlapping coordinates are selected through curvature similarity constraints as the reference anchor points for data fusion.

[0112] The compensation data is mapped bidirectionally along the perspectives of the master device (e.g., orthographic projection) and the slave device (e.g., perspective projection). Multi-scale texture feature comparisons, such as texture feature comparisons of SIFT keypoint distributions and texture feature comparisons of local binary patterns, are used to determine the presence of abnormal regions. When the difference in texture features between the bidirectional projections exceeds a tolerance threshold, the compensation data regeneration process is triggered.

[0113] In practical implementation, for example, the mortise and tenon joints of Song Dynasty palaces often encounter scanning blind spots due to obstruction by the top beams, resulting in a mismatch between traditional compensation data and the original structural texture. This application addresses this by using a ground robot as the master unit to scan along the column bases and identify the complete outline of the mortise and tenon joint's bottom; and a drone as the slave unit to perform compensation scanning. This application extracts the bracket feature points at the beam-column junction as overlapping boundaries, aligning the mortise spatial pose of the master and slave device point clouds. Bidirectional projection verification reveals anomalies in the texture direction of the compensation data, including differences between traditional carving patterns and modern processing traces generated through compensation. This triggers compensation regeneration based on a historical form library, generating a virtual point cloud of mortise and tenon joints conforming to the Song Dynasty's *Yingzao Fashi* (Building Standards). The repaired mortise and tenon joints exhibit coherent textures, and historical craftsmanship features are fully preserved.

[0114] Example 6:

[0115] This application integrates blind spot compensation data into spatiotemporal fusion data, and constructs a collaborative system of historical building feature knowledge base and intelligent compensation to build a blind spot restoration based on cultural heritage. (See reference...) Figure 6 :

[0116] This application first constructs a database of architectural features of old buildings (200), and identifies functional areas of different architectural features. Based on historical documents, architectural codes, and existing surveying results, a multi-dimensional feature library covering component shapes, material characteristics, and craftsmanship is established. Each functional area, such as brackets, roof tiles, and carvings, is associated with its morphological rules; for example, mortise and tenon proportions and pattern symmetry; physical attributes, such as wood grain direction and stone weathering patterns; and cultural semantics, such as official rank and regional style. These functional areas can be functionally matched with the data in the feature blind area. If the functional matching is successful, the texture features of the blind area can be actively predicted and compensated (203).

[0117] That is, when the blind area compensation data is a characteristic blind area that cannot be recognized, the functional area in the building feature library is matched based on the target area where the characteristic blind area is located, and when the match is consistent, active prediction compensation is performed.

[0118] In this process, through point cloud semantic segmentation technology, the remaining features around the blind area are extracted, such as the remaining tenon joints of the bucket arches and the fractured edges of the eaves tiles. Then, combined with the spatial topological relationship, such as the position of the eaves and the level of the beam frame, the category of the functional area to which the blind area belongs is inferred. When the blind area features match the feature library, the generation rules of the corresponding area are called, such as the proportion constraints of the upturned and out-turned brackets of the bucket arches in Qing Dynasty official architecture, and a virtual data that conforms to historical authenticity is generated by combining the point cloud completion network based on the attention mechanism in the deep learning model. The blind area type monitoring 104201 will transmit the characteristic data of the blind area to the building feature library for matching. At this time, the main data transmitted is the multi-source data around the blind area, that is, the point cloud data obtained by scanning. By extracting the contour features of these data, the blind area type is identified. The blind area types include characteristic blind areas, for which active prediction compensation can be achieved. There are also conventional blind areas, that is, blind areas that cannot be identified by features, which are either specific blind areas that have never appeared or temporary blind areas caused by interference and blocking. For these, through the standard compensation process, after the compensation data is verified and the verification is passed, compensation is performed to achieve spatio-temporal data fusion, and the fused data is visually output.

[0119] Embodiment 7:

[0120] After the blind area compensation data is fused into the spatio-temporal fusion data in this application, an intelligent compensation verification system under the form constraint is further constructed. Refer to Figure 7 :

[0121] In this application, the collected point cloud data forms the host point cloud data, and then the geometric features or form features of the adjacent area are extracted by means of feature matching according to the geometric topological features of the area adjacent to the blind area. Combined with the form rule library 3011, a virtual point cloud with a form rule of a functional area based on building features is generated; in this process, based on the component topological rules of the building feature library 200, such as the spatial relationship between the upturned, out-turned brackets and the play head of the bucket arch, the geometric features of the area adjacent to the blind area are extracted, such as the angle of the mortise and tenon joints and the trend of the carved texture, to form the form rule library 3011, that is, the rule library of different building form features in old buildings, and a virtual point cloud that conforms to the historical form is generated. In this process, the virtual point cloud generator 3012 is used to execute. During the generation process, traditional craftsmanship technique parameters are fused, such as the roof curvature gradient of the method of lifting and folding, to ensure the historical authenticity of the virtual data.

[0122] At the interface of virtual point cloud and real data fusion, a joint analysis method of Gaussian curvature and mean curvature is used to detect the continuity of the surface and the naturalness of the transition, and the micro-curvature jump area is identified through the sliding window scanning mechanism.

[0123] The curvature consistency verification 3021 is performed between the virtual point cloud and the point cloud data of the host device to determine whether the curvature error exceeds a preset threshold. The virtual point cloud with a curvature error not exceeding the preset threshold is used as the target data for blind zone compensation. When the curvature difference exceeds the limit, a feature recombination algorithm based on shape rules is triggered to adjust the generation parameters of the virtual point cloud, such as the bracket set projection ratio and tile spacing, until the curvature consistency verification is passed.

[0124] Example 8:

[0125] When the curvature error in this application does not exceed a preset threshold, a multimodal data-driven intelligent optimization system is also constructed, achieving refined control of cultural heritage restoration. (See attached document.) Figure 8 :

[0126] First, the target area is divided into three sub-regions based on curvature distribution, and a differentiated error threshold is set. Data fusion is then achieved as long as the curvature error does not exceed the threshold. For example, data fusion and generation of 400 data points generates a candidate point cluster. For instance, if the curvature of the target area does not exceed the threshold, curvature sensitivity levels are determined based on the functional attributes of building components, such as load-bearing structures, decorative parts, and texture styles, with differentiated verification standards set.

[0127] Highly sensitive areas, such as the bracket arch nodes, are detected using micro-curvature continuity testing.

[0128] Low-sensitivity areas, such as walls, focus on macroscopic shape matching.

[0129] Then, infrared thermal imaging and historical drawing data are fused together, and candidate point clusters are generated through graph neural networks. The internal structural information revealed by infrared thermal imaging, such as decay and cavities in wood, is combined with the shape rules recorded in historical drawings. A feature association model that spans time and space is established through graph neural networks to generate candidate data that takes into account both historical authenticity and structural safety.

[0130] Then, the dynamic optimization system 401 built into the server that controls the host and slave of this application realizes the scoring and sorting, and filters candidate data based on curvature continuity, texture consistency and structural compliance. A three-dimensional scoring matrix of curvature, texture and structure is constructed, and the index weights are dynamically adjusted based on the attention mechanism. For example, the decorative area focuses on texture and the load-bearing area focuses on structure. The optimal candidate set is generally compensable data with high consistency and a score of more than 85 on a 100-point scale.

[0131] Finally, an adversarial network is generated and iteratively optimized using the 402 algorithm until the discriminator confidence difference is less than a preset threshold. A dual discriminator architecture is adopted, for example, combining a form discriminator and an engineering discriminator, to iteratively optimize the generated data until both historical style fidelity and engineering reliability are simultaneously satisfied.

[0132] Example 9:

[0133] Before performing blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, this application also constructs an intelligent hierarchical data processing system to preprocess the scanned data for both UAV and ground robot purposes, enabling efficient data collaboration in complex scenarios. (See [reference needed]). Figure 9 and Figure 10 :

[0134] During the UAV preprocessing process 1001, the point cloud data collected by the UAV is divided into spatial grids, and the density change rate of the point cloud within each grid is detected. Only grids with a change rate exceeding the threshold are incrementally encoded. The UAV point cloud is divided into spatial grids, and active areas of building deformation are identified based on the density change rate, such as eaves bending areas and crack expansion areas. Only the density change rate in the spatial grid is judged, and incremental encoding is performed on areas with significant changes to reduce redundant data transmission.

[0135] During the preprocessing of the ground robot 1003, high curvature feature points are extracted from the point cloud data collected by the ground robot as key frames, and key frame data is encoded and transmitted with priority. For the ground robot data, high curvature points reflecting key architectural features are extracted, such as carved turning edges and mortise and tenon joints, and a key frame priority transmission channel is established to ensure the integrity and real-time performance of core feature data.

[0136] Finally, a spatiotemporal mapping relationship between UAV incremental data blocks and robot keyframes is established 1002; through the bidirectional binding of timestamps and spatial coordinates, the association relationship between UAV incremental data blocks and robot keyframes is constructed, forming a resilient network in which data between devices are mutually backed up.

[0137] When data loss is detected in a device, related data is extracted from the data stream of another device based on the spatiotemporal mapping relationship and interpolated to complete the data. When a single device's data stream is abnormal, related data segments are quickly located based on the spatiotemporal mapping table, and temporary compensation data is generated using a feature-driven interpolation algorithm to maintain the continuity of the surveying process.

[0138] Example 10:

[0139] This application proposes a digital mapping system for old buildings based on air-ground collaborative point cloud fusion. (See reference...) Figure 11 include:

[0140] Blind Spot Dynamic Master-Slave Control Module: This module performs blind spot detection on the spatiotemporal fusion data scanned by UAVs and ground robots. When a blind spot exists in the top / bottom point cloud data, a dynamic master-slave mechanism is activated. Based on real-time point cloud density and feature continuity analysis, the roles of the master and slave devices are dynamically determined. The master device maintains a high-confidence data stream, while the slave device generates a compensation scan path based on the master data characteristics, forming a bidirectional verification relationship from the baseline to the compensation.

[0141] The terminal loading module uses a dynamic master-slave mechanism to assign devices with point cloud data containing blind spots as slave devices and devices with point cloud data without blind spots as master devices. It controls the slave devices to perform point cloud filling, determine blind spot compensation data, and fuse this compensation data with the spatiotemporal fusion data to generate target spatiotemporal fusion data loaded onto the visualization interface. Guided by the master data, the slave devices perform progressive scanning along the blind spot boundaries, generating compensated point clouds through shape rules. The compensated data is verified through bidirectional projection; after forward / backward projection, it is fused with the master data under a unified spatiotemporal reference. The visualization interface renders the fusion results in real time, supporting manual annotation of key areas of interest, such as vulnerable parts of cultural relics. The system dynamically adjusts the compensation strategy accordingly to achieve localized encrypted scanning.

[0142] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for digitally mapping old buildings based on air-ground collaborative point cloud fusion, characterized in that, include: Blind spot detection is performed on the spatiotemporal fusion data scanned by UAV equipment and ground robots, and a dynamic master-slave mechanism is activated when there is a blind spot view in the top point cloud data / bottom point cloud data; The dynamic master-slave mechanism uses devices with point cloud data containing blind spots as slave devices and devices with point cloud data without blind spots as master devices. It controls the slave devices to perform point cloud filling, determine blind spot compensation data, and then fuses this compensation data with the spatiotemporal fusion data to generate the target spatiotemporal fusion data loaded onto the visualization interface. The blind zone compensation data is a compensation point cloud generated by the slave device performing a progressive scan along the blind zone boundary under the guidance of the host data, and constrained by shape rules. The blind spot compensation data is determined by filling in the data of the old building blind spot from different perspectives of the slave device in a time-space synchronized manner. Before performing blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, the following steps are also included: A pre-configured synchronous data transmission mechanism is used to receive top point cloud data from UAV scanning of old buildings and bottom point cloud data from ground robots scanning them. The synchronous transmission mechanism includes a spatiotemporal synchronization mechanism, a master-slave dynamic synchronization mechanism, and an interface perspective synchronization mechanism. The spatiotemporal synchronization mechanism is configured with a unified coordinate system based on a joint calibration target; wherein, the ground robot is timed in real time by the UAV equipment within a sliding window of the unified coordinate system, generating spatiotemporal fusion data of the top point cloud and the bottom point cloud.

2. The method for digitally mapping old buildings based on air-ground collaborative point cloud fusion as described in claim 1, characterized in that, The interface perspective synchronization mechanism performs multi-view segmentation of the target spatiotemporal fusion data based on a sliding time axis according to the unified coordinate system of the spatiotemporal synchronization mechanism. The segmented perspectives include UAV view, ground robot view, and fused view. The sliding time axis is used for perspective synchronization backtracking. The point cloud curvature similarity of the target spatiotemporal fusion data after perspective synchronization backtracking is the same.

3. The method for digitally mapping old buildings based on air-ground collaborative point cloud fusion as described in claim 1, characterized in that, The process of fusing blind spot compensation data into spatiotemporal fusion data includes: Determine the overlapping area of ​​the blind zone edge based on the host device and slave device; Extract the spatiotemporally aligned coordinates of the overlapping boundary regions that overlap with the blind zone edge in the top and bottom point cloud data on a unified coordinate system; Based on the coordinates of the overlapping boundary, the spatiotemporal fusion data of the blind zone compensation data and the overlapping area of ​​the blind zone edge are bidirectionally projected to determine whether there are any anomalies in the compensation data; when the compensation data is anomaly, the texture feature parameters of the bidirectional projection are different.

4. The method for digital mapping of old buildings based on air-ground collaborative point cloud fusion as described in claim 1, characterized in that, The process of fusing blind spot compensation data into spatiotemporal fusion data also includes: Construct a database of architectural features for old buildings and identify functional areas with different architectural features; When there are blind spot compensation data that are unidentifiable feature blind spots, the functional areas in the building feature library are matched based on the target area where the feature blind spot is located, and active prediction compensation is performed when the match is consistent.

5. The method for digitally mapping old buildings based on air-ground collaborative point cloud fusion as described in claim 4, characterized in that, After fusing the blind spot compensation data into the spatiotemporal fusion data, the method further includes: Match the geometric and topological features of adjacent areas of the blind zone to generate a virtual point cloud with shape rules for functional areas based on architectural features; The curvature consistency of the virtual point cloud and the point cloud data of the host device is verified to determine whether the curvature error exceeds a preset threshold, and the virtual point cloud that does not exceed the preset threshold is used as the target data for blind zone compensation data.

6. The method for digitally mapping old buildings based on air-ground collaborative point cloud fusion as described in claim 5, characterized in that, When the curvature error does not exceed a preset threshold, it includes: The target region is divided into three sub-regions according to curvature distribution, and a differential error threshold is set. By fusing infrared thermal imaging and historical drawing data, a candidate point cloud is generated through a graph neural network. Candidate data are selected based on dynamic scoring of curvature continuity, texture consistency, and structural compliance. Generative adversarial networks are used for iterative optimization until the discriminant confidence difference is less than a preset threshold.

7. The method for digital mapping of old buildings based on air-ground collaborative point cloud fusion as described in claim 1, characterized in that, Before performing blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, the following steps are also included: The point cloud data collected by the UAV is divided into spatial grids, and the rate of change of point cloud density in each grid is detected. Only grids with a rate of change exceeding the threshold are incrementally encoded. High curvature feature points are extracted from the point cloud data collected by the ground robot as keyframes, and keyframe data is encoded and transmitted first. Establish a spatiotemporal mapping relationship between UAV incremental data blocks and robot keyframes; When data loss is detected in a certain device, related data is extracted from the data stream of another device based on the spatiotemporal mapping relationship and interpolated to complete the data.

8. A digital mapping system for old buildings based on air-ground collaborative point cloud fusion, characterized in that, include: Blind Spot Dynamic Master-Slave Control Module: Used to perform blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, and to activate the dynamic master-slave mechanism when there is a blind spot view in the top point cloud data / bottom point cloud data; The terminal loading module is used in a dynamic master-slave mechanism to designate devices with point cloud data containing blind spots as slave devices and devices with point cloud data without blind spots as master devices. It controls the slave devices to perform point cloud filling, determine blind spot compensation data, and fuse the blind spot compensation data with the spatiotemporal fusion data to generate the target spatiotemporal fusion data loaded onto the visualization interface. The blind zone compensation data is a compensation point cloud generated by the slave device performing a progressive scan along the blind zone boundary under the guidance of the host data, and constrained by shape rules. The blind spot compensation data is determined by filling in the data of the old building blind spot from different perspectives of the slave device in a time-space synchronized manner. Before performing blind spot detection on the spatiotemporal fusion data scanned by UAV equipment and ground robots, the following steps are also included: A pre-configured synchronous data transmission mechanism is used to receive top point cloud data from UAV scanning of old buildings and bottom point cloud data from ground robots scanning them. The synchronous transmission mechanism includes a spatiotemporal synchronization mechanism, a master-slave dynamic synchronization mechanism, and an interface perspective synchronization mechanism. The spatiotemporal synchronization mechanism is configured with a unified coordinate system based on a joint calibration target; wherein, the ground robot is timed in real time by the UAV equipment within a sliding window of the unified coordinate system, generating spatiotemporal fusion data of the top point cloud and the bottom point cloud.

Images