A building surveying data fusion processing method

Abstract

The application discloses a kind of building survey data fusion processing methods, specifically related to building survey data processing technical field, for building surveying and mapping in pose drift and topological distortion problem;Extract priori building information same name anchor point to generate initial mapping matrix, generate space-time alignment tensor containing global absolute coordinates by hardware trigger and inertial interpolation.Based on semantic inflation mask, spatial boolean operation is carried out, and irrelevant area point cloud is filtered out;Good static grid is used to strip dynamic interference by back projection depth range and optical flow active map, calculate heterogeneous boundary degradation confidence and covariance principal component decay flag, and generate topological connectivity enable mask;In graph optimization phase, according to the mask, the invalid observation edge is executed to block;After detecting the degradation of geometric features, the regular feature registration is suspended, and the Manhattan orthogonal constraint and the Lagrange forced penalty factor are introduced to take over the global trajectory.

Description

Technical Field

[0001] This invention relates to the field of building surveying data processing technology, and more specifically, to a method for fusion processing of building surveying data. Background Technology

[0002] Spatial geometric reconstruction at construction sites typically requires mobile mapping equipment equipped with multimodal sensors. Current SLAM (Simultaneous Localization and Mapping) front-end odometry architectures are based on a simple yet rigorous operating logic: assuming the physical observation environment is approximately static, any building surface can uniformly diffusely reflect probe waves, and the system iteratively matches and calculates a smooth and continuous spatial coordinate trajectory after extracting local extrema. However, real construction environments are severely unstructured; large areas of low-emissivity glass curtain walls, unpainted aluminum alloy ducts, and pure white corridors are typical heterogeneous boundaries. When probe waves act on these areas, the attenuation characteristics of light and sound waves are vastly different. Visual sensors are prone to instantaneous overexposure or loss of feature points; LiDAR pulses can experience multipath penetration or specular refraction, forming large-area deep voids, causing optical flow drift and depth jumps to contradict observations within the same spatial grid. Worse still, when the mapping platform traverses flat, degraded areas such as long straight tunnels, spatial constraints become drastically homogenized, and the principal component eigenvalues ​​in the underlying covariance matrix also show significant attenuation, leading to the loss of physical constraints in the normal dimension. Once signal failures and geometric degradations seep into the back-end optimization network, they can cause sensor tangential shifts along the wall, and even result in rank deficiency of the solution matrix, map topology distortion, and ghosting errors. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of existing technologies, this invention provides a method for fusing and processing building survey data. This method involves anchoring the perceived data to prior global coordinates and obtaining a spatiotemporally aligned tensor for spatial Boolean dimensionality reduction and generating a dynamic gating graph based on back-projection depth jumps and optical flow ranges. Simultaneously, it studies heterogeneous boundary degradation and covariance flattening degradation to form topological connectivity enabling masks that block observation edges. When geometric degradation is detected, it introduces absolute orthogonal constraints and uses a Lagrange forced penalty factor to take over the global pose to address the problems mentioned in the background technology.

[0004] To achieve the above objectives, the present invention provides a method for fusing and processing building survey data, comprising the following steps:

[0005] First, an initial mapping matrix is ​​generated based on the sparse absolute control reference point set and the envelope of the building's digital prior entities. Then, a cross-modal absolute spatiotemporal alignment tensor is generated by combining hardware trigger timestamps and IMU interpolation compensation.

[0006] A 3D semantic dilation mask is generated based on the envelope of the building's digital prior entities. Spatial Boolean NOT operation is performed on the cross-modal absolute spatiotemporal alignment tensor to filter out static candidate meshes. Based on the cross-modal absolute spatiotemporal alignment tensor within the static candidate meshes, depth-temporal jump range meshes and non-rigid optical flow activity maps are extracted, and cross-modal dynamic free gating maps are generated through back projection intersection. For static candidate meshes not blocked by the cross-modal dynamic free gating map, radar echo intensity extreme value sequences and image gray-level co-occurrence boundaries are extracted to generate heterogeneous boundary degradation confidence. Simultaneously, the underlying covariance matrix is ​​extracted from the local neighborhood point group, and local flat degradation markers are generated in parallel based on the principal component directional eigenvalue attenuation relationship.

[0007] A topological connectivity enable mask is formed based on the cross-modal dynamic free gating graph and the heterogeneous boundary degradation confidence.

[0008] Based on the topological connectivity enabling mask, the observed edges are preserved, disconnected, or blocked; when the local flatness degradation flag is set, the regular feature registration edges are actively suspended, and the absolute orthogonality constraint and the augmented Lagrange forced penalty factor are forcibly introduced into the objective equation of the graph optimization registration error to perform global pose takeover, and the building survey reconstruction results are output.

[0009] Preferably, the edge computing gateway reads the control point number, grid position, and absolute engineering coordinates from the sparse absolute control reference point set, and reads the corresponding spatial anchor points from the building digital prior entity envelope to form an anchor point pairing table; based on the anchor point pairing table, it extracts the on-site anchor point center and the prior anchor point center, performs position proximity comparison and axial direction consistency comparison to generate engineering coordinate pre-handshake admission quantity; when the engineering coordinate pre-handshake admission quantity is within the allowable range, it generates the initial mapping matrix based on the anchor point pairing table, and writes the initial mapping matrix into the cross-modal absolute spatiotemporal alignment tensor.

[0010] Preferably, the three-dimensional semantic expansion mask is generated by reading the nominal error of the measuring equipment and the allowable expansion extreme value of the corresponding target component during construction and acceptance, and merging them to generate a dynamic composite expansion tolerance (reading the nominal mean square error, acceptance expansion extreme value and safety margin according to the component number, and taking the maximum envelope of the three as the expansion distance); based on the dynamic composite expansion tolerance, the three-dimensional semantic expansion mask is generated by performing an outward normal offset on the envelope of the building digital prior entity; spatial points located outside the three-dimensional semantic expansion mask, or falling into the prohibited area defined by the engineering coordinate pre-handshake admission amount, are directly eliminated or latched to the bypass cache channel; spatial points located within the three-dimensional semantic expansion mask and not falling into the prohibited area are retained as static candidate meshes.

[0011] Preferably, when acquiring the cross-modal dynamic free-gated map, a depth temporal jump range grid is formed by reading the continuous depth of the same engineering coordinate voxel within the edge computing gateway. Furthermore, a non-rigid optical flow activity map can be read from the local displacement of the image block after deducting the rigid motion of the platform; and the depth temporal jump range grid can be back-projected and intersected with the aforementioned non-rigid optical flow activity map to obtain the cross-modal dynamic free-gated map containing the release, blocking, and transparent penetration tracking states.

[0012] Preferably, when acquiring the heterogeneous boundary degradation confidence value, the edge computing gateway reads the radar echo intensity extreme value, depth hole ratio and multi-echo spacing to form LiDAR side degradation based on the static candidate grid that is not blocked by the cross-modal dynamic free gating graph, and reads the overexposed connected components, image gray-level co-occurrence boundary and edge direction stability to form visual degradation side value. The heterogeneous boundary degradation confidence value is realized by performing hysteresis classification based on the LiDAR side degradation and the visual degradation side value.

[0013] Preferably, the method for obtaining the local flatness degradation flag is as follows: extract local neighborhood point groups from the retained component fragments, and form the underlying covariance matrix based on the distribution of the local neighborhood point groups in the horizontal, vertical and longitudinal directions of the project; read the eigenvalue decay relationship corresponding to the principal component direction in the underlying covariance matrix, and when the eigenvalue decay relationship indicates that the component fragment has lost multidimensional orthogonal constraints and only retains a single plane or a single edge feature, lock the local flatness degradation flag as a pipe connection.

[0014] Preferably, when the edge computing gateway forms a feature admission step flag based on the cross-modal dynamic free-gated graph, forms a backward ray substitution state based on the transparent penetration tracking state, and forms a one-sided modal circuit breaker pruning instruction based on the heterogeneous boundary degradation confidence, the feature admission step flag, the backward ray substitution state, and the one-sided modal circuit breaker pruning instruction are then encoded according to the observation edge number and the modal channel number to generate the topological connectivity enable mask.

[0015] Preferably, when the local flatness degradation flag is latched as a takeover, the edge computing gateway suspends the regular observation edges registered based on point cloud features and writes the absolute orthogonal constraint representing the Manhattan world hypothesis as a rigid condition into the registration iteration error objective equation of the graph optimization backend, forming a forced registration trajectory record; based on the forced registration trajectory record and the continuous error change state, the augmented Lagrange forced penalty factor is generated. The augmented Lagrange forced penalty factor is written with a first takeover strength in the takeover initiation stage, a second takeover strength with the highest restriction level in the takeover maintenance stage, and a third takeover strength in the takeover release stage, and the global pose estimation takeover is completed through a progressive penalty function.

[0016] Preferably, when the inertial measurement unit corresponding to the IMU motion data loses connection at high frequency, the cross-modal absolute spatiotemporal alignment tensor will be compensated through a pure visual constant velocity motion interpolation path; when the LiDAR 3D point cloud loses connection, the edge computing gateway experiences thermal overload, or the heterogeneous boundary degradation confidence quantity exceeds the limits on both sides, the update of the fine branch of the heterogeneous boundary degradation confidence quantity will be stopped and a temporary reconstruction result with degradation source markers will be output based on the previous valid pose; before outputting the result, a bypass pullback verification will be performed, and if the continuously striped retained data in the bypass buffer channel is abnormal, a hidden quality defect marker will be added and output together.

[0017] Preferably, the edge computing gateway is connected to a LiDAR, a vision camera, an inertial measurement unit, and a hardware trigger board; the hardware trigger board outputs a common trigger edge to the LiDAR and the vision camera; the edge computing gateway reads the readback time of the common trigger edge and writes the readback time, along with the LiDAR 3D point cloud flow, the vision RGB image sequence, and the IMU motion data, into the cross-modal absolute spatiotemporal alignment tensor.

[0018] The technical effects and advantages of this invention are as follows:

[0019] (1) This invention effectively overcomes the drawback of traditional SLAM systems relying on relative coordinates. Before the data enters the solution link, the edge computing gateway generates a pre-handshake admission quantity by extracting the corresponding anchor points of the prior building entity envelope and combines the underlying hardware homogeneous triggering and inertial pre-integration to generate a spatiotemporal alignment tensor with global absolute coordinates; then, a three-dimensional semantic dilation mask is introduced to perform spatial Boolean NOT operation to eliminate the loss of edge computing power caused by irrelevant occlusions such as temporary scaffolding, so that the effective point cloud data can be accurately mapped to the engineering absolute coordinate system in the initial stage, thereby truly preventing the pose accumulation and drift problem in a large-scale environment.

[0020] (2) This invention cross-validates the optical flow activity map with the depth range grid back projection to effectively remove dynamic disturbances. For heterogeneous boundaries such as glass and metal, based on the asymmetry of visual and radar degradation features, the observation edge on one side is disconnected according to the topological connectivity mask. When the mapping platform enters a flat degradation area such as a long straight corridor, and the range ratio of the bottom covariance eigenvalue exceeds the preset degradation threshold, conventional feature registration is suspended, and Manhattan orthogonal constraints and piecewise Lagrange forced penalty factors are forcibly introduced into the graph optimization objective equation to eliminate the implicit geometric slippage generated along the wall tangent and ensure the topological consistency of the global 3D model reconstruction. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the overall process of the building survey data fusion processing method of the present invention.

[0022] Figure 2This is a schematic diagram illustrating the multi-source asynchronous on-site data acquisition and spatiotemporal absolute alignment of the present invention.

[0023] Figure 3 This is a schematic diagram of the gating degradation of the present invention.

[0024] Figure 4 This is a schematic diagram of the topology control of the present invention.

[0025] Figure 5 This is a schematic diagram of the reconstructed output of the present invention. Detailed Implementation

[0026] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0027] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0028] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0029] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0030] See Figure 1This embodiment presents a general flowchart of a building surveying data fusion processing method applicable to building surveying scenarios involving dynamic construction, large-area glass curtain walls, reflective metal surfaces, glazed tiles, plain white walls without texture, and vibrations from mobile measurement platforms. The method is executed by an edge computing gateway connected to a LiDAR, a vision camera, an inertial measurement unit (IMU), and a hardware trigger board. The hardware trigger board outputs a common trigger edge to the LiDAR and the vision camera. The edge computing gateway reads the readback time of this common trigger edge and, together with the LiDAR 3D point cloud flow, the vision RGB image sequence, and the IMU motion data, writes a cross-modal absolute spatiotemporal alignment tensor based on an initial mapping matrix. This tensor contains global anchor coordinates (which, after transformation by the initial mapping matrix, are spatial coordinates consistent with the building engineering coordinate system, floor elevation, and grid direction, used to ensure that point cloud voxels, image pixel blocks, component fragments, and pose nodes all fall under the same absolute engineering coordinate reference). The method includes steps one through four.

[0031] Step 1: Acquire asynchronous data from multiple sources and achieve absolute spatiotemporal alignment;

[0032] Step 1 occurs before the field data enters the fusion and resolution link. It is used to unify the non-hard synchronization LiDAR 3D point cloud stream, visual RGB image sequence and IMU motion data to the same absolute time scale and the same absolute engineering coordinate reference.

[0033] For further details, please refer to [link / reference]. Figure 2 A schematic diagram of multi-source asynchronous on-site data acquisition and absolute spatiotemporal alignment, wherein step one includes the following sub-steps:

[0034] Step 101: Engineering coordinate pre-handshake and absolute anchoring based on digital prior envelope;

[0035] Based on the digital prior envelope, the engineering coordinate pre-handshake and absolute anchoring edge computing gateway obtain a sparse absolute control reference point set and a building digital prior entity envelope. According to the control point numbers or grid positions in the control reference point set described above, spatial anchor points with the same name are read in conjunction with the building digital prior entity envelope, forming an anchor point pairing table. Then, the anchor point centers of these two sets of points are extracted, and singular value decomposition (SVD) matrix operations are performed on the constructed anchor point pairing table to produce an initial mapping matrix. The engineering coordinate pre-handshake admission quantity has several admission levels: allowed, delayed, and prohibited. When generating this admission quantity, it is necessary to determine whether the anchor point center position deviation is less than the set proximity threshold and whether the direction order is completely consistent. If these conditions are met and the acquisition end attitude is stable, the allowed level is written. If there are anchor points to be verified, slight attitude changes, or boundaries at the edge of adjacent areas, the delayed level is written. If there are anchor point mismatches or grid axial order misalignments, the prohibited level is written, controlling whether subsequent dense point cloud spatial Boolean operations can be performed normally.

[0036] The sparse absolute control reference point set refers to a spatial reference data set consisting of a small number of control points with known engineering coordinates on site, including control point numbers, absolute engineering coordinates, acquisition time, and reliable status; the building digital prior entity envelope refers to a set of digital building shape and structural boundaries pre-formed from architectural design drawings, grid lines, floor elevations, wall, column, beam and slab outlines, and door and window opening boundaries, used to constrain the spatial ownership and entity boundary range of on-site measurement data.

[0037] Step 102: Perform absolute timestamp extraction and hard-triggered edge readback of multi-source asynchronous frames;

[0038] During the mobile acquisition process of building surveying equipment, LiDAR scan frames, camera exposure frames, and IMU sampling frames are not naturally formed at the same time. Instantaneous platform vibration can cause time misalignment between point cloud boundaries and image boundaries. Therefore, it is necessary to first extract the absolute timestamps of each modality data and perform hard-triggered edge readback.

[0039] The acquisition controller uses the underlying shutter hardware trigger signal as the main time base and reads the timestamps from the LiDAR data packet, RGB image frame header, IMU circular buffer and trigger readback pin respectively. The edge computing gateway performs same-source clock conversion and edge sequence relationship verification on the above timestamps to form a timestamp queue before entering interpolation compensation.

[0040] Hard-triggered edge extraction is achieved by the acquisition terminal controller latching the local crystal oscillator count value when the shutter trigger line generates a rising edge, and writing the count value into the trigger readback field; LiDAR frames are identified by both the scan start time and the scan end time, visual frames are identified by the exposure center time, and IMU frames are identified by the midpoint between two adjacent sampling points, so that the three types of data have comparable physical time positions.

[0041] Asynchronous frame header merging refers to the edge computing gateway establishing a circular frame table based on the time period of engineering measurement, merging point cloud frames, image frames, and IMU increments within the same security window (e.g., 5 milliseconds to 15 milliseconds) into the candidate frame group; if LiDAR occurs outside the window range, the center of the window is used as the interpolation target to avoid crudely classifying the entire frame into adjacent periods.

[0042] The absolute time residual is obtained as follows: The edge computing gateway first reads the scan start time and scan end time of the LiDAR frame, and takes the midpoint between the two as the scan center time; then it reads the exposure center time, the main trigger time latched by the hardware trigger readback pin, and the inertial sampling midpoint time corresponding to the IMU increment from the RGB image frame header; subsequently, it calculates the deviation between the scan center time and the main trigger time, the deviation between the exposure center time and the main trigger time, and the deviation between the inertial sampling midpoint time and the main trigger time. The edge computing gateway takes the larger of the scan-side deviation and the exposure-side deviation as the cross-modal main deviation, and adds the inertial-side deviation to the cross-modal main deviation after converting it according to a preset inertial conversion factor to obtain the absolute time residual. The absolute time residual is used as the time-side input of the subsequent hard alignment admission, and is not used as a simple display parameter.

[0043] Step 103: Perform six-axis kinematic pre-integration and dilation interpolation compensation;

[0044] When a mobile surveying platform at a building surveying site passes through paving stone seams, threshold strips, and cable slope protection, it experiences brief pitch, yaw, and vertical jitter. These jitters may be smaller than what is perceptible to humans, but are sufficient to cause misalignment of reflective curtain walls, metal door frames, and narrow beam boundaries in point clouds and images, inducing divergence in subsequent fusion solution matrices. The edge computing gateway reads the candidate frame groups and absolute temporal residuals formed in step 102, calls the IMU angular velocity, linear acceleration, and installation extrinsic parameters to perform dilation interpolation on the LiDAR pose and visual pose at the main trigger time, and combines this with the initial mapping matrix formed in step 101 to output a cross-modal absolute spatiotemporal aligned tensor with an absolute engineering coordinate base. Inertial incremental sliding window pre-integration is performed by the edge computing gateway with a sliding step size of 1 to 3 milliseconds, reading the IMU circular buffer and accumulating the angular velocity direction change and linear acceleration displacement trend within each candidate frame group. Inertial sampling points with saturation, breakpoints, or reverse jumps are eliminated, ensuring that compensation is based on continuous physical motion rather than single-frame estimation.

[0045] The dilation interpolation refers to the process by which the edge computing gateway performs a simultaneous pullback on the LiDAR scan center pose and the visual exposure center pose based on the changes in IMU angular velocity and linear acceleration, the extrinsic parameters installed at the acquisition end, and the master trigger time of the candidate frame group, while retaining a vibration redundancy boundary outside the pulled-back pose. When generating the vibration redundancy boundary, the edge computing gateway first reads the peak values ​​of IMU angular velocity and linear acceleration within the candidate frame group, then converts these peak values ​​into spatial offset ranges for point cloud voxels and image pixel blocks according to the device's installed extrinsic parameters. Subsequently, the spatial offset ranges are written into the pose increment field of the cross-modal absolute spatiotemporal alignment tensor. This dilation interpolation serves as the pose reference read during the back-projection intersection and degradation determination in step two.

[0046] The engineering coordinate anchoring output is generated by the edge computing gateway reading device extrinsic parameters, initial mapping matrix, and acquisition end attitude. Point cloud voxels and image pixel blocks are bound to attitude nodes in the same engineering coordinate system, forming a globally anchored cross-modal absolute spatiotemporal alignment tensor. This cross-modal absolute spatiotemporal alignment tensor is stored using frame group numbers as the primary index. Each frame group number corresponds to a unified timescale, engineering coordinates, LiDAR extrinsic parameters, visual camera extrinsic parameters, IMU installation extrinsic parameters, attitude increment, absolute time residual, trigger readback field, and frame group status field. The frame group status field includes at least three states: allowed, delayed, and discarded. When the frame group status field is allowed, step two can read the voxel coordinates, pixel block coordinates, and attitude increment of that frame group; when the frame group status field is delayed, step two can only read the frame group number and trigger readback field; when the frame group status field is discarded, step two prohibits reading the point cloud voxels and image pixel blocks of that frame group. Therefore, the cross-modal absolute spatiotemporal alignment tensor serves as the sole input base for subsequent 3D Boolean mask dimensionality reduction, cross-modal dynamic free gating graphs, and heterogeneous boundary degradation confidence.

[0047] Step 104: Generate hard-aligned admission parameters and perform abnormal frame discarding loop closure;

[0048] The hard alignment admission value is an allow, delay, or discard decision value obtained by residual classification, dead zone determination, and frame group state latching of multi-source frame timestamps, inertial motion continuity, and spatial anchoring integrity. It is used to control the write qualification of the alignment cache.

[0049] The hard alignment admission parameter is obtained as follows: the edge computing gateway reads the absolute time residual, IMU compensation continuity, LiDAR echo integrity, RGB frame header exposure status, and engineering coordinate anchor point availability status, and sequentially performs time residual classification, inertial continuity verification, echo integrity verification, and anchor point availability verification. When the absolute time residual does not exceed the allowable threshold, and IMU compensation is continuous, LiDAR echo is complete, no abnormal exposure marker appears in the RGB frame header, and the engineering coordinate anchor point is available, the hard alignment admission parameter is set to the allowable state; when the absolute time residual is between the allowable threshold and the discard threshold, or when there is a single input waiting for neighboring frames to complete, the hard alignment admission parameter is set to the delayed state; when the absolute time residual exceeds the discard threshold, or when any of the following persists: IMU compensation breakpoint, LiDAR echo missing, or engineering coordinate anchor point unavailability, the hard alignment admission parameter is set to the discard state. The allow threshold can be set to 200 to 800 microseconds, and the discard threshold can be set to 1.5 to 3 milliseconds.

[0050] Step 2: Used to generate cross-modal dynamic free gating graphs, heterogeneous boundary degradation confidence values, and local flattening degradation indicators, and to provide deterministic inputs for feature admission blocking, unilateral modal circuit breaking pruning, and subsequent graph optimization control in Step 3.

[0051] For further details, please refer to [link / reference]. Figure 3 The diagram illustrates the gating degradation, and step two includes the following sub-steps:

[0052] Step 201: Three-dimensional semantic dilation mask based on prior envelope and spatial Boolean NOT operation;

[0053] Construction sites often contain numerous temporary obstructions (such as scaffolding and stockpiled materials), and directly performing optical flow calculations on such global point clouds would generate a huge computational load. Therefore, the system introduces a dimensionality reduction strategy at the macroscopic level.

[0054] The edge computing gateway reads the nominal mean square error of the current measuring device and combines it with the maximum allowable expansion value (the maximum outward expansion dimension of the target component that is allowed to deviate from the design outline in the construction acceptance specification or the project's preset quality boundary) of the corresponding target component (such as column, wall, beam, slab or door and window opening boundary). It then generates a dynamic composite expansion tolerance (controlling the distance of the building digital prior entity envelope to the outward normal spatial offset). Based on this dynamic composite expansion tolerance, the edge computing gateway performs an outward normal spatial offset on the building digital prior entity envelope to obtain a three-dimensional semantic expansion mask. It then performs a streaming spatial Boolean NOT operation on the point cloud within the cross-modal absolute spatiotemporal alignment tensor. If the coordinates of this spatial point are completely outside the safe area defined by the mask and simultaneously fall within the prohibited area defined by the project coordinate pre-handshake admission quantity, it is judged as an irrelevant construction interference and directly eliminated or latched into a bypass cache. The remaining mesh after the spatial Boolean NOT operation filtering is used as a static candidate mesh for fine processing.

[0055] The 3D semantic expansion mask refers to the 3D spatial mask formed by the edge computing gateway after performing spatial offset along the outward normal of walls, columns, beams, slabs, door and window openings, and floor boundaries in the building's digital prior entity envelope based on the dynamic composite expansion tolerance. The 3D semantic expansion mask uses component number, engineering coordinate range, and expansion boundary state as indexes to determine whether the current spatial point still belongs to the acceptable measurement range of the building's main structure. The spatial Boolean NOT operation refers to the edge computing gateway comparing the point cloud voxel coordinates in the cross-modal absolute spatiotemporal alignment tensor with the 3D semantic expansion mask to determine spatial inclusion relationships, and removing spatial points that fall completely outside the 3D semantic expansion mask as temporary construction objects or anomalies.

[0056] Step 202: Generate depth-time jump range mesh and non-rigid optical flow activity map;

[0057] Construction workers, suspended platforms, temporary fencing, and reflective protective films at building surveying sites often undergo short-term displacements between LiDAR scanning and visual exposure. If these changes are simply smoothed as ordinary noise, the actual load-bearing wall boundaries, the background behind glass, and temporary dynamic objects will be mixed into the same observation edge, causing the backend fusion solution matrix to be dragged by dynamic features. The edge computing gateway first reads the static candidate mesh output after spatial Boolean dimensionality reduction, then divides the point cloud voxels and image pixel blocks into the same spatial mesh according to engineering coordinates, extracting depth jumps, echo discontinuities, optical flow displacement, and exposure saturation edges to form two types of pre-projection physical maps for backprojection intersection calculation.

[0058] The method for obtaining the depth temporal jump range grid is as follows: the edge computing gateway divides the measurement space into multiple voxel grids according to engineering coordinates and uses the voxel number as an index; for each voxel, the LiDAR near-end depth, far-end depth, and echo hit count in the most recent few admission cycles are read, and its maximum depth jump amplitude, echo discontinuity count, and hole duration period are written into the corresponding grid cell; when the same voxel shows an increased depth jump and a decreased echo hit count in consecutive admission cycles, the voxel is marked as having an active depth jump, otherwise it is marked as having stable depth, which serves as the LiDAR-side input for generating cross-modal dynamic free gating maps. The non-rigid optical flow activity map is obtained as follows: the visual acquisition end extracts corner points, line segment endpoints, and grayscale boundary blocks from the RGB image sequence, and records the displacements stored in adjacent admission cycles; the edge computing gateway obtains the local residual displacement by subtracting the rigid displacement caused by the overall platform movement based on the acquisition end's attitude increment; if the local residual displacement of an image block continuously exceeds a preset pixel displacement boundary and does not move in the same direction as the wall, column, or floor boundary, the block is marked as non-rigid optical flow active, otherwise it is marked as non-rigid optical flow inactive, and stored with the image block number and engineering coordinate ray number. In addition, the edge computing gateway also reads the exposure time and pixel saturation ratio in the image frame header, and retains the boundary index for some large overexposed but continuous glass, metal door frame, or glazed tile areas in the image, which are used to distinguish moving obstructions from static transparent reflective surfaces in subsequent cross-modal dynamic free gating maps.

[0059] In one possible embodiment, the most recent admission cycles can be three to five consecutive admission cycles. The edge computing gateway records the near-end depth, far-end depth, echo hit count, and depth hole persistence status of the same voxel within each admission cycle, and writes the changes in near-end depth, far-end depth, and echo hit count between the current admission cycle and the previous valid admission cycle into the voxel grid. If the same voxel simultaneously satisfies the following conditions in two or more consecutive admission cycles: depth change exceeding the depth jump boundary, echo hit count decreasing, and depth hole persisting, then the voxel is marked as having an active depth jump. If the same voxel's depth change does not exceed the depth jump boundary and the echo hit count remains stable in multiple consecutive admission cycles, then the voxel is marked as having stable depth. The depth jump boundary can be preset to 20 to 100 millimeters based on the measurement distance and LiDAR nominal ranging error, or it can be adjusted according to the component measurement accuracy requirements, but its input data, continuous cycle verification, and output status remain unchanged.

[0060] Step 203: Generate a cross-modal dynamic free gating graph and perform front-end admission marking;

[0061] The final spatial blocking state is determined through cross-validation of cross-modal data. The edge computing gateway back-projects the LiDAR voxel centers onto the RGB image coordinate system. A voxel is marked as blocking if and only if it is marked as having an active depth jump in the depth range grid and its corresponding image patch has non-rigid ionization features. If a voxel experiences a depth jump but the corresponding image region boundary is continuous and there is no active optical flow (indicating that the probe wave penetrates the transparent material), these voxels are marked as transparent penetration tracking states, retaining their ray directions for background tracking to avoid excessive removal of effective features. The cross-modal dynamic ionization gating map described in this paper is a spatial blocking discrimination map obtained by voxel marking based on aligned LiDAR depth changes, local non-rigid optical flow, and cross-modal back-projection overlap. The output values ​​include open state, blocking state, and transparent penetration tracking state, which are used to veto the feature extraction admission qualification of the corresponding voxel. The cross-modal back-projection intersection refers to the following: the edge computing gateway reads the LiDAR extrinsic parameters, visual camera extrinsic parameters, camera intrinsic parameters, engineering coordinates, and attitude increments from the cross-modal absolute spatiotemporal alignment tensor, transforms the LiDAR voxel center from the engineering coordinate system to the visual camera coordinate system, and then projects it into the RGB image coordinate system to determine the image block number corresponding to the LiDAR voxel; at the same time, the edge computing gateway pulls the image blocks in the non-rigid optical flow active map back to the engineering coordinate system according to the camera ray number, so that the depth jump evidence on the LiDAR side and the local free motion evidence on the visual side fall under the same voxel number or the same ray number.

[0062] When a voxel is marked as having an active depth jump in the depth temporal jump range grid, and the corresponding image patch, after back projection, is marked as having active non-rigid optical flow in the non-rigid optical flow active map, the edge computing gateway marks the voxel as blocked in the cross-modal dynamic free-gated map. When the same voxel has stable depth and the corresponding image patch has inactive non-rigid optical flow, it is marked as allowed. When the same voxel exhibits a depth jump or discontinuous echo, but the corresponding image patch boundary is continuous, the exposure saturation region is continuous, and the non-rigid optical flow is inactive, it is marked as transparent penetration tracking. The blocked state prevents the voxel from entering feature extraction, the allowed state allows the voxel to enter the static candidate grid, and the transparent penetration tracking state prevents the voxel from being used as a strongly constrained feature, while retaining its ray number for subsequent backward ray tracing.

[0063] The gating map is written to the disk and the interface is encapsulated by the edge computing gateway. The cross-modal dynamic free gating map is written to the front-end feature extraction shared cache. The cache fields include at least the voxel number, gating state, ray number, trigger readback frame group number and depth jump evidence that forms the state. When the cross-modal dynamic free gating map is in a blocked state, the feature extractor at the acquisition end must not read the point cloud corner or image corner of the voxel.

[0064] Step 204: Generate heterogeneous boundary degradation confidence and pre-set unilateral circuit breaker criteria;

[0065] For static heterogeneous boundaries such as glass and multilayer metals, the performance degradation of LiDAR and vision sensors is often asymmetrical. The edge computing gateway extracts and merges the LiDAR-side echo extrema, hole ratio, and multi-echo spacing to obtain radar degradation parameters based on the allowed area; then it extracts and merges the overexposed connected components, gray-scale co-occurrence boundaries, and edge orientation stability from the vision side to obtain visual degradation parameters. The degradation parameters obtained from both sides are then hysterically categorized: only when the degradation parameter on one side exceeds the uplink trigger threshold and remains above it within consecutive admission cycles is the heterogeneous boundary degradation confidence value categorized as a high degradation state. This hysteresis filtering logic reduces the probability of false triggers caused by single-frame transient noise.

[0066] The heterogeneous boundary degradation confidence value refers to the boundary degradation discrimination value obtained by extreme value truncation, hysteresis classification and asymmetric hysteresis merging of LiDAR multipath penetration features and visual overexposure, low texture or edge discontinuity features in the static candidate mesh. It is used to indicate whether the single-sided modal disconnect clipping command is triggered in step three.

[0067] The heterogeneous boundary degradation confidence is obtained as follows: First, the extreme values ​​of LiDAR echo intensity, the proportion of deep holes, and the multi-echo spacing are read and merged to form the LiDAR-side degradation; then, the overexposed connected components, the image gray-level co-occurrence boundary, and the edge direction stability are read and merged to form the visual degradation side quantity; subsequently, uplink trigger threshold comparison and downlink release threshold comparison are performed on the LiDAR-side degradation and the visual degradation side quantity, respectively. When either side degradation side quantity exceeds the uplink trigger threshold and remains so within a continuous admission period, the heterogeneous boundary degradation confidence is set to a high degradation state; when the degradation side quantity is between the uplink trigger threshold and the downlink release threshold, the heterogeneous boundary degradation confidence is set to a hysteresis-held state; when both sides degradation side quantities are below the downlink release threshold and continuously meet the release period, the heterogeneous boundary degradation confidence is set to a low degradation state. The high degradation state is used to generate a single-sided modal circuit breaker trimming command in step three; the hysteresis hold state is used to maintain the circuit breaker state of the previous cycle in step three; and the low degradation state is used to release the corresponding observation edge in step three.

[0068] The LiDAR side degradation refers to the LiDAR side degradation discrimination value formed by truncating, archiving, and periodically verifying the extreme values ​​of LiDAR echo intensity, the proportion of deep holes, and the multi-echo spacing within the same static candidate grid. When generating the LiDAR side degradation, the edge computing gateway removes isolated spikes from a single period, retains continuously occurring echo intensity anomalies, deep holes, and multi-echo spacing anomalies, and writes them into the LiDAR side degradation field of that grid.

[0069] The visual degradation measure refers to the visual degradation discrimination value formed by merging overexposed connected components, image gray-level co-occurrence boundaries, and edge orientation stability within the same static candidate grid, after boundary continuity verification and orientation drift determination. When generating the visual degradation measure, the edge computing gateway marks continuously overexposed areas with stable boundary orientations as static reflective degradation, and areas with chaotic boundary orientations and out of sync with LiDAR holes as low-confidence areas awaiting verification.

[0070] Radar extreme value truncation sampling involves the edge computing gateway reading the upper and lower edges of echo intensity, depth void segments, and multi-echo spacing within each candidate grid. Isolated peaks significantly exceeding the sensor's effective range are written into the truncation dead zone to prevent them from directly amplifying the confidence result. Image co-occurrence boundary verification involves the edge computing gateway extracting grayscale adjacency relationships, edge direction consistency, and saturated connected component lengths from image blocks corresponding to the same grid to distinguish between static reflection degradation and low-confidence waiting-for-verification states.

[0071] Step 205: Parallel detection of local flattening degradation based on the decay of principal components of underlying covariance;

[0072] This step involves the edge computing gateway extracting local neighborhood point groups from the retained component fragments. Based on the distribution of these local neighborhood point groups in the horizontal, vertical, and other directions of the project, a bottom-level covariance matrix is ​​calculated. This matrix represents a three-dimensional point distribution matrix formed by the edge computing gateway after discretizing the distribution of local neighborhood point groups in the static candidate mesh or retained component fragments along the horizontal, vertical, and other directions of the project, respectively. It characterizes the geometric constraint capability of the local neighborhood point groups in different spatial directions. The edge computing gateway reads the bottom-level covariance matrix and performs principal component decomposition on it to obtain three eigenvalues ​​along the first, second, and third principal directions. The edge computing gateway sorts the three eigenvalues ​​according to their magnitude and reads the decay relationship between the largest, second largest, and smallest eigenvalues. If both the largest and second-largest eigenvalues ​​are significantly greater than the smallest eigenvalue, and the smallest eigenvalue remains below the planar degradation threshold, it indicates that the current component segment retains only a single planar constraint, and the local flatness degradation flag is set to planar takeover. If only the largest eigenvalue is significantly greater than the second-largest and smallest eigenvalues, and both the second-largest and smallest eigenvalues ​​remain below the edge degradation threshold, it indicates that the current component segment retains only a single edge constraint, and the local flatness degradation flag is set to edge takeover. If all three principal directional eigenvalues ​​remain within their corresponding effective constraint ranges, the local flatness degradation flag remains untaken. The planar degradation threshold and edge degradation threshold can be preset according to the point cloud density, component scale, and measurement platform movement speed, and are latched and confirmed through two to three consecutive admission cycles. The local flatness degradation flag includes at least the component segment number, degradation type, trigger cycle, eigenvalue decay evidence, and takeover status. When the local flatness degradation flag is set to planar takeover or edge takeover, it is output to the topology connectivity enable mask to trigger subsequent forced takeover control values.

[0073] Step 206: Perform equivalent implementation and downgraded fallback parameter encapsulation;

[0074] The materials, lighting, and sensor placement vary significantly across different building sites. If the cross-modal dynamic free-gated map and heterogeneous boundary degradation confidence quantity rely solely on a fixed bimodal combination, insufficient support may occur under conditions such as pure white walls, strong backlighting, or single-channel sensor disconnection. The edge computing gateway, while maintaining the names, interfaces, and admission functions of the cross-modal dynamic free-gated map and heterogeneous boundary degradation confidence quantity, pre-configures equivalent extraction paths and degradation fallback paths.

[0075] In another alternative equivalent implementation, when all image patches have continuous low texture and LiDAR echo stability, the heterogeneous boundary degradation confidence is converted into a one-dimensional lookup table attenuation determination that depends only on the ratio between the LiDAR range and the echo hole, and these are used as degradation sources for step three to limit the triggering strength of the unilateral modal disconnect clipping command.

[0076] The one-dimensional lookup table attenuation determination refers to the process where, when image patches have continuous low texture and the visual side cannot stably extract gray-level co-occurrence boundaries, the edge computing gateway reads a LiDAR range-degradation level lookup table pre-written into local memory and outputs a degradation level based on the LiDAR depth range and echo hole ratio. The LiDAR range-degradation level lookup table includes at least a depth range interval, an echo hole ratio interval, and corresponding low degradation, hysteresis-preserving, and high degradation levels. The result obtained through the one-dimensional lookup table attenuation determination is still written into the degradation source field of the heterogeneous boundary degradation confidence value, for use in step three to limit the trigger strength of the unilateral modal disconnect pruning command.

[0077] When a single LiDAR connection fails or the edge computing gateway triggers a thermal overload alarm, the system suspends fine updates of the heterogeneous boundary degradation confidence, retaining only the valid value of the previous cycle and its hysteresis holding time, while allowing cross-modal dynamic free gating graphs to perform coarse screening based on the visual optical flow activity graph.

[0078] The interface is consistently encapsulated by the edge computing gateway, which encapsulates the cross-modal dynamic free-gated map, heterogeneous boundary degradation confidence, local flattening degradation flag, degradation source flag, frame group number, and evidence index into a readable parameter packet in step three. This parameter packet includes at least the voxel number, gating state, ray number, degradation side source, frame group number, hysteresis state, and evidence index. The parameter packet does not output the final mapping result; it only outputs the admission, blocking, tracking, circuit breaker warning, and degradation states.

[0079] Step two outputs the cross-modal dynamic free-gated map, heterogeneous boundary degradation confidence and its evidence index, local flat degradation flag, degradation source marker, and frame group number. After reading the cross-modal dynamic free-gated map, the blocking state voxels are admitted by feature extraction, and the transparent penetration tracking state voxels are subjected to backward ray tracing. After reading the heterogeneous boundary degradation confidence, it is compared with the hysteresis control dead zone. If the limit boundary is touched, a one-sided modal circuit breaker pruning instruction is obtained and converted into a topological connectivity enable mask along with the local flat degradation flag.

[0080] Step 3: Receive the cross-modal dynamic free-gated graph, heterogeneous boundary degradation confidence, local flatness degradation flag, evidence index, degradation source marker, and frame group number output from Step 2. The task is to transform the aforementioned decision parameters from readable discriminant values ​​into executable control branches. This step is used to convert the aforementioned decision parameters into feature admission, backward ray substitution, one-sided circuit breaker pruning, and topology connectivity control.

[0081] For further details, please refer to [link / reference]. Figure 4 The topology control diagram shows that step three includes the following sub-steps:

[0082] Step 301: Perform front-end feature admission step determination based on cross-modal dynamic free gating graph;

[0083] Pedestrians, scaffold crossbars, temporary material piles, and protective film on a building survey site may form visible but unusable strong features within the same spatial grid. If the feature extractor continues to read only by the number of corner points, edge strength, or point cloud density, it will send the pseudo-boundaries of dynamic objects and transparent reflective surfaces into the graph optimization solver, causing the backend to be contaminated before ill-conditioned residuals are identified.

[0084] The edge computing gateway reads the cross-modal dynamic free-gated map encapsulated in step two, checks the gating status one by one according to the voxel number, maps the blocking status, allowing status, and transparent penetration tracking status to different feature reading permissions, and generates a feature admission step flag. The feature admission step flag refers to the feature extraction admission discrimination value obtained by hard blocking, conditional allowing, and ray rerouting processing of the voxel gating status of the cross-modal dynamic free-gated map. Its output is used to control whether point cloud corners, planar patches, image corners, and line segment endpoints enter the observation edge construction.

[0085] The gating state reading is performed by the edge computing gateway reading the voxel number, ray number, depth jump evidence and non-rigid optical flow evidence that form the gating from the front-end shared cache, and comparing it with the frame group number of the current measurement cycle; if the evidence index comes from a delayed or discarded frame group, the corresponding voxel is not released in this cycle.

[0086] The feature admission step flag is generated as follows: when the voxel gating state in the cross-modal dynamic detached gating graph is in the blocking state, the edge computing gateway sets the feature admission step flag corresponding to the voxel to the blocking position and disables the voxel's access permissions for point cloud corners, planar patches, image corners, and line segment endpoints; when the voxel gating state is in the allowing state, the edge computing gateway sets the feature admission step flag to the allowing position and allows the voxel to enter the candidate observation edge buffer; when the voxel gating state is in the transparent penetration tracking state, the edge computing gateway sets the feature admission step flag to the tracking position, closes the current voxel's strong constraint entry, and opens the backward ray search permission. The feature admission step flag serves as the input for generating the backward ray substitution state and topological connectivity enable mask in step three.

[0087] At the acquisition end, the latched edge computing gateway writes the feature admission step flag into the read gate of the feature extractor; for voxels corresponding to the release file, only their original corner points and faces can be read; for voxels corresponding to the blocking file, their point cloud and image feature addresses are set to unreadable; for voxels corresponding to the tracking file, the strong constraint entry of the current voxel is closed, and only the ray number and background search permission are retained.

[0088] Step 302: Perform backward ray tracing and background substitution admission based on the transparent penetration tracking state;

[0089] In measurements, glass curtain walls, glossy stone, and coated metal panels often exhibit both abrupt changes in local depth and continuous visual boundaries. If these areas are directly blocked, the system will lose the stable background behind the glass or the reflective surface. If they are allowed to pass directly, the transparent or mirrored surface will be treated as the actual load-bearing boundary.

[0090] The edge computing gateway initiates transparent backtracking for voxels whose feature admission step flag is in the tracking mode, searches for stable background voxels along the camera ray and the near and far depth segments of LiDAR, and generates a backtracking ray replacement state.

[0091] The backward ray substitution state refers to the substitution admission state formed by the ray number of the transparent penetrating tracking voxel, the backward depth continuity segment, and the background boundary stability through distance threshold, angle threshold, and inter-frame preservation determination. It is used to determine whether to replace the current transparent voxel with the background voxel to participate in weakly constrained observation.

[0092] The ray index pullback is performed by the edge computing gateway, which reads the camera intrinsic parameters, LiDAR to camera extrinsic parameters and engineering coordinate anchor points based on the ray number. It searches layer by layer from the current transparent voxel along the line of sight to the far end, prioritizing the search for the depth segment behind which it exists for three consecutive admission cycles.

[0093] Background stabilization threshold verification is performed by an edge computing gateway that compares the depth jitter, edge direction, and inter-frame displacement of candidate voxels. Substitution admission is only allowed if the depth change does not exceed a backward depth threshold and the edge direction remains within an angle threshold. The backward depth threshold is 20 to 80 millimeters, and the angle threshold can be set to 3 to 8 degrees. The backward ray substitution status includes at least three categories: successful substitution, failed substitution, and pending verification.

[0094] Specifically, when the edge computing gateway finds a background voxel that exists in multiple consecutive admission cycles along the ray number at the far end, and the background voxel satisfies the conditions of depth stability, edge direction stability, and inter-frame displacement stability, the backward ray substitution state is considered a successful substitution. When no stable background voxel is found along the same ray, or when the found background voxel exceeds the limit in any of the depth, edge direction, or inter-frame displacement conditions, the backward ray substitution state is considered a failed substitution. If it is only a single admission cycle and does not meet the requirement of consecutive cycles, the backward ray substitution state is considered to be awaiting verification. Successful substitution allows the formation of backward background constraint edges, failed substitution is written to the abnormal evidence cache, and the waiting-for-verification state is delayed in the construction of observation edges.

[0095] The alternative observation permission is written by the edge computing gateway, which writes the backward ray substitution state into the observation edge candidate cache, so that the background voxel can only form low-level geometry-preserving edges and is not allowed to replace the current transparent voxel as a strong positioning anchor point; if a stable background cannot be found, the current voxel is blocked and the ray number is written into the abnormal evidence cache.

[0096] Step 303: Generate a one-sided modal circuit breaker trimming command based on the heterogeneous boundary degradation confidence.

[0097] After processing in steps 301 and 302, most of the candidate observation edge buffers are static or background substitute features. However, glass, multi-layered metal, highly reflective glazed bricks, and pure white walls may still have ill-conditioned residuals in a single mode. If they are uniformly weighted or completely deleted, it will damage the modes that are still credible on the other side, causing the available constraints to be excessively sacrificed.

[0098] The edge computing gateway reads the heterogeneous boundary degradation confidence, LiDAR-side degradation, visual degradation side value, hysteresis hold-up status and degradation source markers, determines for each candidate observation edge whether to cut off only one side of the modal constraint, and generates a single-side modal disconnect pruning instruction.

[0099] The single-sided modal disconnect pruning command refers to the topology pruning control command generated by the heterogeneous boundary degradation confidence and its LiDAR-side and visual-side degradation sources after hysteresis control dead zone determination. Its output is used to indicate that the graph optimization solver retains the reliable side observation edge and disconnects the failed side observation edge.

[0100] Hysteresis dead zone triggering is configured by the edge computing gateway with an upper limit for triggering circuit breaker and a lower limit for releasing circuit breaker. Circuit breaker is initiated when the heterogeneous boundary degradation confidence value reaches the upper limit for two consecutive admission cycles; circuit breaker is released when the heterogeneous boundary degradation confidence value falls below the lower limit for three consecutive admission cycles. The upper limit for triggering circuit breaker can be set from 0.70 to 0.90, and the lower limit for releasing circuit breaker can be set from 0.45 to 0.65.

[0101] Degradation source orientation is formed by the edge computing gateway comparing LiDAR-side degradation and visual degradation measurements as evidence. If there are multiple echo spacing anomalies, depth holes, and intensity solitaries on the LiDAR side while the visual boundary is stable, a LiDAR-side disconnect command is generated. If there are overexposed connected regions, gray-scale co-occurrence breaks, and edge direction drift on the visual side but the LiDAR depth is continuous, a visual-side disconnect command is generated. If both sides cross the boundary simultaneously, no single-sided disconnect is generated, and the system enters a fully blind suspended coasting state.

[0102] The single-sided modal disconnect pruning instruction includes at least the observation edge number, voxel number, failed modal identifier, retained modal identifier, disconnect set flag, release cycle count, hysteresis state, evidence index, and frame group number. The failed modal identifier indicates the failed side in the LiDAR channel, visual channel, or backward background channel; the retained modal identifier indicates the trustworthy side still allowed to participate in observation edge construction; the disconnect set flag indicates whether step four clears the corresponding residual block; and the release cycle count indicates whether the continuous cycle condition for releasing the disconnect has been met. After the edge computing gateway writes the single-sided modal disconnect pruning instruction into the observation edge construction interface of the graph optimization solver, it reads back the receive confirmation bit of that interface.

[0103] Step 304: Generate a topological connectivity enable mask and establish a degradation-guaranteed closed loop;

[0104] If the unilateral modal circuit breaker trimming command only remains at the command layer, the graph optimization solver may misread old caches or continue to construct bilateral constraints in situations such as sensor disconnection, end-side thermal overload, and complete blind zones. The edge computing gateway encodes the feature admission step flag, backward ray substitution state, unilateral modal circuit breaker trimming command, and local flatness degradation flag into a unified topological connectivity enabling mask, and uses it as the sole interface for step four to clear residual blocks, retain constraint edges, force takeover, and freeze the graph optimization solver.

[0105] The topological connectivity enable mask is a network connectivity control value formed by mapping voxel admission states, backward substitution states, one-sided disconnected states, and geometrically degraded states according to edge numbers, modal channel mapping, and degraded state latching. It is used to ensure that the observed edge is always in a connected, one-sided disconnected, blocked, takeover replacement, or frozen sliding state in the graph-optimized network.

[0106] The topology connectivity enable mask is encoded using observation edge numbers as row indices and LiDAR channels, visual channels, and backward background channels as column indices. Each column index contains at least one control value for connectivity, one-sided disconnect, blocking, forced takeover, and freeze. The connectivity control value indicates that the observation edge of the corresponding channel is allowed to enter the residual block assembly in step four; the one-sided disconnect control value indicates that the observation edge of the corresponding channel is not allowed to enter the residual block assembly in step four, but other reliable channels of the same observation edge can be left; the blocking control value indicates that all modal channels of the observation edge cannot enter the residual block assembly; the forced takeover control value is triggered by a local flattening degradation flag, indicating that regular registration needs to be suspended at the mathematical level; the freeze control value indicates that no new observation edges are added in this cycle, only the previous valid pose and front-end odometry results are retained. After reading the topology connectivity enable mask in step four, the relevant solution operations are performed according to the provided control values.

[0107] The forced takeover control value refers to the control level in the topological connectivity enable mask triggered by the local flatness degradation flag, which is used to suspend the graph optimization solver from the regular observation edges based on point cloud feature registration and initiate the pose takeover process based on Manhattan orthogonal constraints and augmented Lagrange forced penalty factors.

[0108] The fully blind suspended coasting state is a degraded control state initiated by the edge computing gateway when, during the current measurement cycle, both LiDAR-side degradation and visual-side degradation simultaneously reach the circuit-breaking trigger condition, or when the proportion of blocking voxels in the cross-modal dynamic free gating graph exceeds a preset area ratio. In this state, the edge computing gateway prevents the graph optimization solver from assembling newly added observation edges, stops updating the fine branch of the heterogeneous boundary degradation confidence, reads the previous valid pose and front-end odometry increment to generate a temporary trajectory, and writes a freeze control value into the topology connectivity enable mask. The fully blind suspended coasting state is released after the effective LiDAR depth, effective visual boundary, and available engineering coordinate anchor points are restored within several consecutive admission cycles. Interface latching and readback are handled by the edge computing gateway writing the topology connectivity enable mask, freeze flag, and corresponding evidence index into the graph optimization solver input queue, while simultaneously reading back the queue confirmation bit. If the confirmation bit is not returned, the system prevents step four from continuing to write the pose file and transfers the data for this cycle to the exception cache.

[0109] Step 3 outputs the feature admission step flag, backward ray substitution state, one-sided modal disconnect pruning instruction, topology connectivity enable mask, freeze flag, and evidence index. Step 4 reads the topology connectivity enable mask, clears the residual block for observation edges marked as one-sided disconnect, retains the solution for observation edges marked as connected, introduces a penalty mechanism for states marked as forced takeover, and suspends the assembly of new edges corresponding to the freeze flag period.

[0110] Step 4: The graph optimization solver works in collaboration with the edge computing gateway to perform topology pruning, forced takeover, anti-counterfeiting disk dumping, and output reconstruction.

[0111] For further details, please refer to [link / reference]. Figure 5 The reconstruction output diagram shows that step four includes the following sub-steps:

[0112] Step 401: Perform observation edge assembly and residual block clearing based on the topological connectivity enable mask;

[0113] After the aforementioned steps, candidate observation edges may still contain unilateral ill-conditioned constraints caused by glass multipath, specular reflection, or low texture of pure white walls. If the graph optimization solver uses the old cache or only smooths the residual values, the failure mode will still propagate to the global pose nodes through the Jacobian matrix, causing model ghosting and wall drift.

[0114] The graph optimization solver reads the topology connectivity enable mask, one-sided modal disconnect pruning instructions, and frame group numbers. It locates the residual blocks to be retained, one-sidedly disconnected, blocked, or frozen according to the observation edge number, and performs deterministic zeroing before matrix assembly. Mask-driven edge table assembly is performed by the graph optimization solver reading the topology connectivity enable mask using the observation edge number as the row index and the LiDAR channel, visual channel, and backward background channel as the column index. When the corresponding position is a connectivity control value, the observation edge of that channel is retained. When the corresponding position is a one-sided disconnect control value, the channel is not written to the edge table for this period, but other trusted channels of the same observation edge are allowed to continue writing. When the corresponding position is a blocking control value, all channels of that observation edge are not written to the edge table for this period. When the corresponding position is a freeze control value, no new observation edges are added for this period.

[0115] The process of clearing the failed residual block involves the graph optimization solver calling the residual block access interface. Based on the one-sided modal break-circuit trimming command, the residual block of the failed mode is located, and its diagonal and cross blocks in the Jacobian matrix are simultaneously cleared. The residual block access interface uses the observation edge number, the failed mode identifier, and the pose node number as indexes. When the break-circuit setting flag in the one-sided modal break-circuit trimming command is valid, the graph optimization solver no longer writes the corresponding residual block into the current period matrix assembly queue.

[0116] The freeze cycle protection is implemented by the edge computing gateway, which prohibits the graph optimization solver from assembling new observation edges when the freeze flag is valid. It only allows short-term gliding using the previous valid pose and front-end odometry, and writes the original frame of the current cycle, the topological connectivity enable mask, and the circuit break evidence into the exception cache.

[0117] Step 402: Pose takeover based on Manhattan orthogonal constraints and augmented Lagrange forced penalty factor;

[0118] This step primarily addresses the loss of normal degree-of-freedom constraints and global slippage caused by local flattening degradation in graph optimization. It is triggered only when the topological connectivity enable mask reads the forced takeover control value (i.e., the local flattening degradation flag is set). Upon triggering takeover, regular observation edges registered based on point cloud features are forcibly suspended. The edge computing gateway suspends these edges, reads the stable and confirmed ground plane, main wall plane, and ceiling plane references from historical valid poses, and writes the orthogonal relationships between these planes into the constraint queue of the graph optimization solver, forming a forced registration trajectory record. To ensure a smooth exit when restoring normal texture after solution convergence, and to avoid numerical oscillations during nonlinear optimization, an augmented Lagrange forced penalty factor is introduced based on the forced registration trajectory record and the continuous error change state.

[0119] The absolute orthogonal constraints are extracted by the edge computing gateway from the stable and confirmed ground plane, main wall plane, column plane, beam edge, and ceiling plane references in the historical valid poses. During extraction, the edge computing gateway reads the plane pieces that have not been blocked by the topological connectivity enable mask in multiple consecutive valid admission cycles, and compares their normal directions with the floor elevation direction and axis grid direction in the building engineering coordinate system; if the plane piece maintains a stable direction in consecutive cycles, it is used as the orthogonal reference plane. The absolute orthogonal constraints include at least the vertical relationship between the ground plane and the main wall plane, the orthogonal relationship between adjacent main wall planes, the vertical relationship between the floor slab boundary and the wall boundary, and the component number and pose node number corresponding to the above orthogonal relationships.

[0120] The augmented Lagrange forced penalty factor is used to control the graph optimization solver to apply different orthogonal constraint strengths in the three stages of dosing gate opening, dosing gate holding, and dosing gate release. The edge computing gateway reads the current pose node, the previous valid pose node, the orthogonal reference plane number, and the tangential slip distance from the forced registration trajectory record, and uses the tangential slip distance, orthogonal constraint deviation, and continuous iteration convergence state as inputs to the penalty factor.

[0121] When the topology connectivity enable mask is first written into the forced takeover control value and the tangential slip distance does not exceed the takeover holding boundary, the edge computing gateway sets the penalty factor to the first takeover strength. The graph optimization solver allows small-scale attitude perturbations but will not drift along a single wall direction; when the tangential slip distance does not decrease for more than two consecutive iterations or the orthogonal constraint deviation is higher than the takeover holding boundary for multiple consecutive iterations, the edge computing gateway sets the penalty factor to the second takeover strength and restricts a larger orthogonal constraint term in the graph optimization iterative error objective equation, with pose nodes preferentially conforming to the rigid orthogonal relationship formed between the ground plane, main wall, and axis grid; when the front end recaptures a valid corner point, edge line, or multi-dimensional planar patch, and the tangential slip distance and orthogonal constraint deviation also fall back to within the release boundary for multiple consecutive iterations, the edge computing gateway reduces the penalty factor to the third takeover strength and gradually restores the regular feature registration edges.

[0122] The augmented Lagrange forced penalty factor includes three discrete intensity levels: a first control strength, a second control strength, and a third control strength. The first control strength corresponds to the control activation stage, the second control strength corresponds to the control holding stage, and the third control strength corresponds to the control release stage. The first, second, and third control strengths can be preset as three discrete levels by the edge computing gateway or read from a control strength table in local memory. The control strength table includes at least the control stage, tangential slip distance range, orthogonal constraint deviation range, continuous periodic conditions, and corresponding intensity levels. The augmented Lagrange forced penalty factor is output and written to the constraint queue of the graph optimization solver without directly modifying the original point cloud or image data.

[0123] Step 403: Compare the global rigid body translation with the local load-bearing plane orthogonal normal curvature sequence to generate a topological distortion latch flag;

[0124] In scenarios involving highly reflective halls or large glass curtain walls, the aforementioned circuit breakers may still miss some false features in the solution process. These false features can cause unreasonable large translations of the entire model in a short period of time, while local load-bearing planes may exhibit normal bending that contradicts the building's physical structure. If only the magnitude of the residuals is checked, this anomaly may be mistakenly attributed to normal turning or vibrations from the moving measurement platform, resulting in incorrect poses that are then written into the engineering model.

[0125] The topology distortion latch flag refers to a system-level anti-counterfeiting discrimination value formed by the superposition judgment of global rigid body displacement, abnormal bending of local load-bearing plane normal and previous circuit breaker evidence, continuous periodic latching and release condition verification. It is used to control the writing of pose file, the disking of abnormal evidence and the output of electromechanical braking.

[0126] The topology distortion latch flag setting method is as follows: The edge computing gateway reads the spatial displacement between the current cycle's solved pose and the previous valid pose to form a global rigid body translation; the edge computing gateway reads the normal direction of the local load-bearing plane corresponding to the boundary of the wall, column, or floor, and continuously records the normal changes of adjacent local load-bearing planes as a local load-bearing plane orthogonal normal curvature sequence; several one-sided modal disconnect clipping commands, transparent penetration tracking states, or disconnection control values ​​in the topology connectivity enable mask are read in this cycle to form a disconnection evidence state; if the global rigid body translation exceeds the translation threshold, the local load-bearing plane orthogonal normal curvature sequence exceeds the normal threshold, and the disconnection evidence state is valid, then the edge computing gateway sets the topology distortion latch flag to latching anomaly; if none of the above conditions are met within a continuous release cycle, it indicates that the topology distortion latch flag has been released from latching. After the topology distortion latch flag is set, this step prohibits the user from writing the current pose file to the main project model. The main engineering model refers to the set of 3D reconstruction data of the building continuously generated by the edge computing gateway during the building measurement process and saved as a formal result. It includes at least pose data of the reliable period, point cloud tiles, texture index, component spatial ownership information and engineering coordinate mapping relationship. It is different from the abnormal evidence package and temporary trajectory. Only data that is not intercepted by the topology distortion latch flag and is not in a frozen or discarded state can be written into the main engineering model.

[0127] The local load-bearing plane orthogonal normal curvature sequence is obtained as follows: the edge computing gateway extracts local planar patches corresponding to wall, column, floor, or beam boundaries from the point cloud tiles and image boundaries of the current period, and records the normal direction of each local planar patch according to the engineering coordinate system; then, the local planar patches are arranged according to spatial proximity and floor grid order, and the change in normal direction between adjacent local planar patches is calculated. If the change in normal direction of consecutive adjacent local planar patches is inconsistent with the orthogonality of building physics, the change is written into the local load-bearing plane orthogonal normal curvature sequence. The local load-bearing plane orthogonal normal curvature sequence serves as the microstructure input for the topological distortion latch flag.

[0128] The translation threshold can be set to 30 to 120 mm based on the upper limit of the device's walking speed; the normal threshold can be set to 5 to 15 degrees; and the continuous release cycle can be set to three admission cycles. These thresholds can be preset according to the measurement platform's travel speed, floor grid spacing, and acquisition frequency, but their input, processing, and output logic remain unchanged.

[0129] Step 404: Based on the topology distortion latch flag, execute the file write pointer suspension and anti-counterfeiting disk write;

[0130] Architectural reverse mapping typically involves writing continuous pose files, point cloud tiles, and texture indexes to the project directory in an incremental manner. Once an incorrect pose enters the main file, subsequent floor stitching and project coordinate verification will be continuously contaminated. Simply popping up an alarm at the application layer cannot stop the cache thread from continuing to write to disk.

[0131] After the topology distortion latch flag is set, the edge computing gateway calls the local file system write control interface, suspends the underlying I / O write pointer of the current error pose file, and writes the original frame, cross-modal absolute spatiotemporal alignment tensor, cross-modal dynamic free gating graph, heterogeneous boundary degradation confidence, topology connectivity enable mask and circuit breaker evidence index into the read-only anomaly evidence packet.

[0132] The file write pointer suspension refers to the edge computing gateway reading the corresponding file handle, cache page number, and offset address to be written before submitting the pose file, point cloud tile file, or texture index file to the main engineering model directory, and pausing the submission of the offset address to be written when the topology distortion latch flag is in a latching abnormal state. The edge computing gateway returns a suspension confirmation bit to the graph optimization solver and writes the file handle, offset address, frame group number, and trigger reason of the suspended file into the abnormal evidence package. Pose files in a suspended state cannot participate in subsequent floor stitching and engineering coordinate verification until the topology distortion latch flag is released from latching.

[0133] The anomaly evidence package contains at least the following: hardware trigger timestamp, frame group number, cross-modal absolute spatiotemporal alignment tensor index, cross-modal dynamic free-gated graph, heterogeneous boundary degradation confidence, feature admission step flag, back ray substitution state, unilateral modal circuit breaker pruning instruction, topology connectivity enable mask, topology distortion latch flag, observation edge number, file write pointer suspension record, and read-only verification field. The read-only verification field is written by the edge computing gateway when the anomaly evidence package is encapsulated to indicate whether this anomaly evidence is stored outside the main engineering model update link. The aforementioned anomaly evidence includes the input source for subsequent verification of anomaly cycles, parameter generation process, topology pruning, and file suspension process.

[0134] In an alternative equivalent implementation, the file write pointer is released when the global rigid body translation and the local load-bearing plane orthogonal normal curvature sequence both fall below the release boundary within three consecutive admission cycles and no new disconnection occurs in the topological connectivity enable mask; otherwise, the latch is held and the main project model is prohibited from being updated.

[0135] Step 405: Execute CAN bus deceleration alarm, bypass pullback verification, and system-level reconfiguration output;

[0136] When the mobile measurement platform is still moving on the construction site, if only the file writing is suspended without restricting the chassis movement, the sensors will continue to collect a large amount of unusable data, which may create continuous distortion areas in narrow passages, next to scaffolding, or next to glass curtain walls.

[0137] The edge computing gateway outputs deceleration alarm hardware level and CAN bus braking message to the lower-level chassis controller based on the topology distortion latch flag, freeze flag and file write pointer suspension confirmation bit, and at the same time outputs the system-level reconstruction result.

[0138] The CAN bus braking message is generated by the edge computing gateway after the topology distortion latch flag is set and the file write pointer is returned with a pending confirmation bit. The CAN bus braking message includes at least the frame group number, anomaly level, suggested speed level, braking hold period, and a checksum. The anomaly level is determined by the latching state of the topology distortion latch flag, the duration of the freeze control value in the topology connectivity enable mask, and the write state of the anomaly evidence packet. The suggested speed levels are: maintain current low speed, reduce to low-speed cruising, and stop. After receiving the CAN bus braking message, the chassis controller changes the travel state of the mobile measurement platform according to the suggested speed level, returns an execution receipt to the edge computing gateway, and the reconstruction results are distributed by the edge computing gateway to write the pose, point cloud tiles, and texture index of the reliable period into the main engineering model, write the original data of the latching period into the anomaly evidence packet, and mark the short-term odometer calculation result of the freeze period as a temporary trajectory, not as a formal engineering result. When LiDAR single-path disconnection, end-side thermal overload, or complete blind suspension and sliding state still exist, the system no longer updates the fine branch of the heterogeneous boundary degradation confidence quantity, but only retains the coarse screening result of the cross-modal dynamic free gating map and the previous valid pose, and outputs a temporary reconstruction result with degradation source markers. In summary, step one provides an absolutely anchored cross-modal absolute spatiotemporal alignment base; step two generates the cross-modal dynamic free gating map, heterogeneous boundary degradation confidence quantity, and local flat degradation marker; step three converts the aforementioned parameters into a topological connectivity enabling mask; and step four performs residual block zeroing, forced takeover, topological distortion latching, file write interception, and electromechanical braking output based on the topological connectivity enabling mask, forming a complete building surveying data fusion processing flow.

[0139] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for fusing and processing building survey data, characterized in that, include: First, an initial mapping matrix is ​​generated based on the sparse absolute control reference point set and the envelope of the building's digital prior entities. Then, a cross-modal absolute spatiotemporal alignment tensor is generated by combining hardware trigger timestamps and IMU interpolation compensation. A 3D semantic dilation mask is generated based on the building digital prior entity envelope. Spatial Boolean NOT operation is performed on the cross-modal absolute spatiotemporal alignment tensor to filter out static candidate meshes. Based on the cross-modal absolute spatiotemporal alignment tensor within the static candidate grid, a depth-temporal jump range grid and a non-rigid optical flow activity map are extracted. A cross-modal dynamic free-gated map is generated through back-projection intersection. For static candidate grids not blocked by the cross-modal dynamic free-gated map, radar echo intensity extreme value sequences and image gray-level co-occurrence boundaries are extracted to generate heterogeneous boundary degradation confidence. Simultaneously, the underlying covariance matrix is ​​extracted from local neighborhood point groups, and local flat degradation markers are generated in parallel based on the principal component directional eigenvalue attenuation relationship. A topological connectivity enable mask is formed based on the cross-modal dynamic free gating graph and the heterogeneous boundary degradation confidence. Based on the topological connectivity enabling mask, the observed edges are preserved, disconnected, or blocked; when the local flatness degradation flag is set, the regular feature registration edges are actively suspended, and the absolute orthogonality constraint and the augmented Lagrange forced penalty factor are forcibly introduced into the objective equation of the graph optimization registration error to perform global pose takeover, and the building survey reconstruction results are output.

2. The building survey data fusion processing method according to claim 1, characterized in that, The edge computing gateway reads the control point number, grid position, and absolute engineering coordinates from the sparse absolute control reference point set, and reads the corresponding spatial anchor points from the building digital prior entity envelope to form an anchor point pairing table. Based on the anchor point pairing table, it extracts the on-site anchor point center and the prior anchor point center, performs position proximity comparison and axial direction consistency comparison to generate engineering coordinate pre-handshake admission quantity. When the engineering coordinate pre-handshake admission quantity is within the allowable range, it generates the initial mapping matrix based on the anchor point pairing table and writes the initial mapping matrix into the cross-modal absolute spatiotemporal alignment tensor.

3. The building survey data fusion processing method according to claim 2, characterized in that, The three-dimensional semantic expansion mask is generated by reading the nominal error of the measuring equipment and the extreme value of the allowable expansion of the corresponding target component during construction and acceptance, and then merging them to generate a dynamic composite expansion tolerance difference; based on the dynamic composite expansion tolerance difference, the three-dimensional semantic expansion mask is generated by performing an outward normal offset on the envelope of the building digital prior entity. Spatial points located outside the three-dimensional semantic expansion mask, or falling into the prohibited area defined by the engineering coordinate pre-handshake admission quantity, are directly eliminated or latched into the bypass cache channel. Spatial points located within the three-dimensional semantic expansion mask but not falling into the forbidden region are retained as static candidate meshes.

4. The building survey data fusion processing method according to claim 1, characterized in that, When acquiring a cross-modal dynamic free gating map, read the continuous depth of the same engineering coordinate voxel to form a depth temporal jump range grid; For the local displacement of the image block after deducting the rigid motion of the platform, the non-rigid optical flow activity map is read; and the depth temporal jump range grid is back-projected and intersected with the above non-rigid optical flow activity map to obtain the cross-modal dynamic free gating map containing the release, blocking and transparent penetration tracking states.

5. The building survey data fusion processing method according to claim 4, characterized in that, When acquiring the heterogeneous boundary degradation confidence, the edge computing gateway reads the radar echo intensity extremes, depth hole ratios, and multi-echo spacing from the static candidate grid that is not blocked by the cross-modal dynamic free gating graph to form LiDAR side degradation, and reads the overexposed connected components, image gray-level co-occurrence boundaries, and edge direction stability to form visual degradation side quantity. Based on the LiDAR side degradation and the visual degradation side quantity, hysteresis classification is performed to realize the heterogeneous boundary degradation confidence.

6. The building survey data fusion processing method according to claim 1, characterized in that, The method for obtaining the local flatness degradation flag is as follows: extract local neighborhood point groups from the retained component fragments, and form the underlying covariance matrix based on the distribution of the local neighborhood point groups in the horizontal, vertical and longitudinal directions of the project; read the eigenvalue decay relationship corresponding to the principal component direction in the underlying covariance matrix, and when the eigenvalue decay relationship indicates that the component fragment loses multidimensional orthogonal constraints and only retains a single plane or a single edge feature, the local flatness degradation flag is locked as a pipe connection.

7. The building survey data fusion processing method according to claim 5, characterized in that, When the edge computing gateway forms a feature admission step flag based on the cross-modal dynamic free-gated graph, forms a backward ray substitution state based on the transparent penetration tracking state, and forms a one-sided modal circuit breaker pruning instruction based on the heterogeneous boundary degradation confidence, the feature admission step flag, the backward ray substitution state, and the one-sided modal circuit breaker pruning instruction are then encoded according to the observation edge number and the modal channel number to generate the topological connectivity enable mask.

8. The building survey data fusion processing method according to claim 7, characterized in that, When the local flattening degradation flag is latched as a takeover, the edge computing gateway suspends the regular observation edges based on point cloud feature registration and writes the absolute orthogonal constraint representing the Manhattan world hypothesis as a rigid condition into the registration iteration error objective equation of the graph optimization backend, forming a forced registration trajectory record. Based on the forced registration trajectory record and the continuous error change state, the augmented Lagrange forced penalty factor is generated. The augmented Lagrange forced penalty factor is written with a first takeover strength in the takeover initiation phase, a second takeover strength with the highest restriction level in the takeover maintenance phase, and a third takeover strength in the takeover release phase. The global pose estimation takeover is completed through a progressive penalty function.

9. The building survey data fusion processing method according to claim 8, characterized in that, When the inertial measurement unit corresponding to the IMU motion data loses connection at high frequency, the cross-modal absolute spatiotemporal alignment tensor will be compensated through a pure visual constant velocity motion interpolation path. When the LiDAR 3D point cloud loses connection, the edge computing gateway experiences thermal overload, or the heterogeneous boundary degradation confidence quantity exceeds the limits on both sides, the update of the fine branch of the heterogeneous boundary degradation confidence quantity will be stopped, and a temporary reconstruction result with degradation source markers will be output based on the previous valid pose. Before outputting the result, a bypass pullback verification will be performed. If the continuously strip-shaped retained data in the bypass buffer channel is abnormal, a hidden quality defect marker will be added and output together.

10. The building survey data fusion processing method according to claim 1, characterized in that, The edge computing gateway is connected to a LiDAR, a vision camera, an inertial measurement unit, and a hardware trigger board. The hardware trigger board outputs a common trigger edge to the LiDAR and the vision camera. The edge computing gateway reads the readback time of the common trigger edge and writes the readback time, along with the LiDAR 3D point cloud flow, the vision RGB image sequence, and the IMU motion data, into the cross-modal absolute spatiotemporal alignment tensor.

Images