Tunnel structure defect identification method and system based on multispectral perception and slam

Abstract

This invention belongs to the field of tunnel detection technology and provides a method and system for identifying tunnel structural defects based on multispectral sensing and SLAM. Based on multiple response timestamp data, a residual squared sum objective function is constructed to represent the difference between the theoretically predicted clock vector and the actual measured timestamp matrix. Through iterative optimization, the optimal smooth estimate of the clock drift rate is determined, reducing the combined effects of electromagnetic interference and temperature / humidity variations. A three-dimensional point cloud map is constructed based on the coupling of axial integrals of inertial data and radial constraints of laser data, achieving long-distance, high-precision passive positioning under tunnel corridor effects. Furthermore, a lightweight identification model using a multidimensional knowledge distillation architecture and geometrically guided fraction fusion reduces edge memory consumption. By combining the reduction of electromagnetic interference and temperature / humidity variations, overcoming positioning drift under the dual constraints of corridor effects, and breaking through edge computing power limitations, the tunnel defect identification effect is improved.

Description

Technical Field

[0001] This invention belongs to the field of tunnel inspection technology, and in particular relates to a method and system for identifying tunnel structural defects based on multispectral sensing and SLAM. Background Technology

[0002] Due to geological subsidence, groundwater erosion, material aging, and dynamic loads, structural defects are prone to develop on the surface and inside of tunnel linings. Currently, the industry's methods for detecting tunnel defects are transitioning from traditional manual inspections to automated inspections.

[0003] In existing technologies, inertial measurement units (IMUs), binocular cameras, and lidar are used for fusion positioning and 3D modeling to detect tunnel defects. However, when faced with the extreme physical environment of real and extremely complex underground tunnels, existing automatic identification methods suffer from problems such as hardware clock drift and multimodal synchronization failure under the combined effects of extreme electromagnetic interference and temperature and humidity changes, SLAM geometric degradation and axial cumulative drift induced by the corridor effect, and missed / false alarms of defects caused by edge computing resource constraints and multidimensional feature fragmentation, resulting in poor defect identification performance. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a method and system for identifying tunnel structural defects based on multispectral sensing and SLAM. This invention improves the tunnel defect identification effect by reducing the influence of electromagnetic interference and temperature and humidity fluctuations, overcoming the positioning drift under the dual constraints of the corridor effect, and breaking the limitation of edge computing power.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, the present invention provides a method for identifying tunnel structure defects based on multispectral sensing and SLAM, comprising: Acquire relevant laser data, visible light data, infrared data, and inertial data within the tunnel. During data acquisition, the unidirectional propagation delay of a single transmission and the discrete deviation of the corresponding sensor clock relative to the absolute clock are determined based on the local arrival time of the trigger pulse packet and the response time of the response data packet. Based on multiple response timestamp data, a residual sum of squares objective function is constructed to compare the theoretically predicted clock vector with the actual measured timestamp matrix. Through iterative optimization, the optimal smooth estimate of the clock drift rate is determined. A three-dimensional point cloud map is constructed by coupling the axial integral of inertial data with the radial constraint of laser data. Features are extracted from laser data, visible light data, and infrared data to obtain an analysis matrix; Based on the analysis matrix and the preset recognition model, the defect recognition result is obtained; the defect is then mapped inversely to the three-dimensional point cloud coordinate system of the three-dimensional point cloud map; wherein, the recognition model is a lightweight recognition model that fuses a trained multidimensional knowledge distillation architecture with geometric guided scores.

[0006] Furthermore, impedance compensation bridging circuits are deployed on each sensor that acquires laser data, visible light data, infrared data, and inertial data. By utilizing the common-mode rejection characteristics of differential signals, the low-frequency and high-frequency common-mode noise generated by the ventilation fan and high-voltage line in the tunnel is attenuated to a preset value.

[0007] Furthermore, continuous pre-integration is performed using inertial data to determine the prior pose of the aircraft. After extracting edge and planar features from the new frame of point cloud acquired by the lidar, point-to-plane registration is performed with the maintained local sub-image. When solving the least squares incremental equation, the corresponding high-dimensional information matrix is ​​defined and extracted, and the high-dimensional information matrix is ​​divided into rotation blocks, translation blocks, and cross terms. Using the Schur complement elimination method, cross terms are removed, rotation states are marginalized, and the translation information matrix of the pure translation state is extracted separately. Eigenvalue decomposition is performed on the translation information matrix to obtain three eigenvalues ​​and corresponding eigenvectors.

[0008] Furthermore, the total degradation index is obtained by the ratio of the maximum eigenvalue to the minimum eigenvalue. When the total degradation index exceeds the preset tunnel degradation threshold, an adaptive diagonal matrix of the lidar odometer, including rotation weights and translation weights, is constructed. The translation weights are forcibly subjected to a nonlinear attenuation coefficient, which actively weakens the registration constraint of the lidar along the tunnel axis. At tunnel intersections with rich structures, the attenuation coefficient returns to normal.

[0009] Furthermore, in the objective function of the weighted residual sum of squares for global factor graph optimization, the confidence weight of the inertial data pre-integration factor on the degradation axis is increased.

[0010] Furthermore, in the analysis matrix, visible light data corresponds to surface texture edge features and color change features, targeting surface micro-cracks and carbonization discoloration; infrared data corresponds to local abnormal low temperature features and evaporation heat absorption temperature drop features, targeting early hidden water seepage and damp water stains; laser data corresponds to local surface curvature change features and normal vector anomaly features, targeting surface peeling, concrete depressions, and spalling.

[0011] Furthermore, each discrete point in the 3D point cloud generated by the LiDAR is inversely projected onto the pixel plane of the visible light camera and the infrared thermal imager. The continuous color value and absolute physical temperature scalar of the 3D point in the real physical world are determined by the weighted average of the four adjacent physical pixels surrounding the projected point. The enhanced point cloud is divided into a regular voxel grid, and deep feature encoding is performed for the center point of each voxel: a local difference sign-amplitude transform is introduced, and the grayscale information of the voxel center and its adjacent sub-voxels is compared. Through global thresholding, voxel-based local binary pattern features are generated. A local covariance matrix is ​​constructed based on the point cloud, and the normal vector of each point is extracted. The spatial relative angle change between the normal vectors of the center point and neighboring points is determined, and a feature histogram descriptor is constructed. The temperature gradient and average temperature drop within a specified spatial range are then calculated with the current point as the center.

[0012] Furthermore, the recognition model includes a multi-branch feature extractor to independently process texture, geometric, and temperature feature data; a deep residual structure and a multi-head self-attention mechanism are embedded in the teacher network of the recognition model to suppress background noise from ventilation ducts and joint mud strips within the tunnel.

[0013] Furthermore, on an airborne edge computing platform, a student network is deployed, and the feature distribution of the hidden layer in the middle of the student network is constrained to be consistent with that of the teacher model. A geometrically guided score fusion framework is introduced, which dynamically transforms the fixed distance measure that is prone to misjudgment in a single modality into a distinctive abnormal feature component by predicting the local scale factor of orientation awareness.

[0014] Secondly, the present invention also provides a tunnel structure defect identification system based on multispectral sensing and SLAM, comprising: The data acquisition module is configured to acquire relevant laser data, visible light data, infrared data, and inertial data within the tunnel. Specifically, during data acquisition, it determines the unidirectional propagation delay of a single transmission and the discrete deviation of the corresponding sensor clock relative to the absolute clock based on the local arrival time of the trigger pulse packet and the response time of the response data packet. Based on multiple response timestamp data, it constructs a residual squared sum objective function of the difference between the theoretically predicted clock vector and the actual measured timestamp matrix. Through iterative optimization, it determines the optimal smooth estimate of the clock drift rate. The 3D point cloud map construction module is configured to construct a 3D point cloud map based on the coupling of the axial integral of inertial data and the radial constraint of laser data. The feature extraction module is configured to extract features from laser data, visible light data, and infrared data to obtain an analysis matrix. The defect identification module is configured to: obtain defect identification results based on the analysis matrix and the preset identification model; and reverse map the defect to the three-dimensional point cloud coordinate system of the three-dimensional point cloud map; wherein the identification model is a lightweight identification model that fuses a trained multidimensional knowledge distillation architecture with geometric guided scores.

[0015] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the tunnel structure defect identification method based on multispectral sensing and SLAM described in the first aspect.

[0016] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the steps of the tunnel structure defect identification method based on multispectral sensing and SLAM described in the first aspect.

[0017] Fifthly, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the tunnel structure defect identification method based on multispectral sensing and SLAM described in the first aspect.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention constructs a residual squared sum objective function based on multiple response timestamp data, which is the difference between the theoretically predicted clock vector and the actual measured timestamp matrix. Through iterative optimization, it determines the optimal smooth estimate of the clock drift rate, reducing the combined effects of electromagnetic interference and temperature and humidity fluctuations. Based on the coupling of the axial integral of inertial data and the radial constraint of laser data, a three-dimensional point cloud map is constructed, achieving long-distance, high-precision passive positioning under tunnel corridor effects. Furthermore, a lightweight recognition model using a multi-dimensional knowledge distillation architecture and geometrically guided fraction fusion reduces edge memory consumption. By combining the reduction of electromagnetic interference and temperature and humidity fluctuations, overcoming positioning drift under the dual constraints of corridor effects, and breaking through edge computing power limitations, the invention improves the tunnel defect recognition effect. Attached Figure Description

[0019] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0020] Figure 1 This is a hardware topology and anti-interference closed-loop delay compensation synchronization circuit diagram of Embodiment 1 of the present invention; Figure 2 This is a logic block diagram of degradation-sensing laser inertial SLAM according to Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the training and inference of a multispectral fusion AI model based on knowledge distillation according to Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the UI interface for binding a three-dimensional visual interactive detection report with dynamic diseases in Embodiment 1 of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0023] SLAM (Simultaneous Localization and Mapping) refers to the technology that allows a mobile platform to estimate its own pose in real time and build a three-dimensional map of its surroundings using sensor data in an unknown environment.

[0024] Multispectral sensing: Specifically refers to a multimodal data sensing method that simultaneously acquires visible light (RGB texture information), infrared thermal imaging (temperature information), and solid-state lidar (three-dimensional geometric depth information).

[0025] Corridor Effect: Geometrically degenerate environments such as tunnels that are long, narrow, straight, and have a uniform surface texture. In such environments, sensors struggle to extract sufficient effective geometric constraint features along the tunnel axis (direction of travel), easily leading to severe cumulative drift in the localization algorithm at this degree of freedom.

[0026] Geometric Degeneracy-aware: A technique that can automatically quantify and assess whether the current environmental features are sufficient to support full-degree-of-freedom pose estimation through mathematical models (such as Hessian matrix eigenvalue decomposition) and make adaptive algorithm adjustments accordingly.

[0027] Knowledge Distillation (KD) is a deep learning model compression and acceleration technique that transfers knowledge from a large-scale, feature-extraction-capable "Teacher Model" to train a lightweight "Student Model," enabling it to achieve near-large model accuracy on resource-constrained edge computing devices.

[0028] VLBP (Voxel-Based Local Binary Pattern): Based on voxels, this local binary pattern extracts robust 3D microtexture features to illumination variations through local difference sign-amplitude transformation in 3D space.

[0029] FPFH (Fast Point Feature Histogram): A robust feature descriptor used to describe the local geometric topology and curvature abrupt changes in 3D point clouds.

[0030] G2SF (Geometry-Guided Score Fusion): This geometry-guided score fusion technique transforms the equidistant Euclidean distance of a single modality into an anisotropic local distance metric by predicting a direction-aware scale factor, enabling efficient fusion of multimodal features and anomaly detection.

[0031] Example 1: As a key component of modern transportation infrastructure, the structural health of tunnels directly affects people's lives and property and operational safety. Affected by geological subsidence, groundwater erosion, material aging, and dynamic loads, tunnel lining surfaces and interiors are prone to structural defects such as cracks, water seepage, hollowing, and spalling. Currently, the industry's methods for detecting tunnel defects are transitioning from traditional manual inspections to automated inspections. While existing technologies utilize drones to perform fusion positioning and 3D modeling for tunnel defect detection using inertial measurement units (IMUs), binocular cameras, and lidar, improving automation to some extent, current automatic identification methods exhibit severe technical bottlenecks when facing the extreme physical environment of real and extremely complex underground tunnels. These bottlenecks include hardware clock drift and multimodal synchronization failure under the combined effects of extreme electromagnetic interference and temperature and humidity fluctuations; SLAM geometric degradation and axial cumulative drift induced by the corridor effect; and issues such as missed / false alarms due to edge computing resource constraints and multidimensional feature fragmentation. Specifically: The interior of a tunnel is far from an ideal laboratory environment. It is typically densely packed with high-voltage power cables, high-power variable frequency fans, large lighting arrays, and overhead contact lines for rail transit. These devices radiate strong low-frequency and high-frequency electromagnetic interference (EMI) into the space during operation. Existing multi-sensor fusion detection systems often simplify their hardware time synchronization schemes to "using a common microcontroller chip as a central hardware synchronization trigger to send TTL synchronization trigger pulse signals to each sensor." However, under the combined effects of strong EMI and the complex temperature and humidity fluctuations within the tunnel, this unidirectional open-loop pulse signal is highly susceptible to waveform distortion, signal attenuation, and uncontrollable delays in the nanosecond to microsecond range when transmitted along the cables. Furthermore, the exposure and sampling response times of different heterogeneous sensors (such as lidar, uncooled thermal imagers, and global shutter cameras) inherently differ due to the temperature drift of their internal crystal oscillators. When a conventional hard trigger reaches the receiver, it can no longer guarantee strict microsecond-level time alignment. This slight misalignment of the underlying timestamp directly causes severe ghosting and pixel tearing when the visible light texture, infrared temperature field, and laser point cloud collected by the drone are spatially mapped during high-speed inspection (such as 30km / h) flight. This fundamentally destroys the accuracy foundation of subsequent multimodal disease fusion identification.

[0032] Because the tunnel interior shields all GPS / BeiDou satellite navigation signals, existing automated inspection equipment heavily relies on Simultaneous Localization and Mapping (SLAM) algorithms for self-localization. The current common practice is to directly apply existing tightly coupled laser inertial SLAM algorithms (such as LOAM, LIO-SAM, or the FAST-LIO2 framework). However, the tunnel environment exhibits a typical "corridor effect" in terms of geometry and topology: the space presents an infinitely extending tubular shape, with nearly parallel lining walls on both sides and highly repetitive, monotonous concrete textures. In this geometrically degenerate environment, the effective surface features scanned by the lidar along the tunnel axis (i.e., the direction of the UAV's movement) are extremely scarce, resulting in insufficient constraints on the state estimation observation model in this direction. Mathematically, the Jacobian matrix in the SLAM backend tends to be singular in the corresponding translational degrees of freedom, making the optimization process prone to getting trapped in gradual local minima. Directly applying existing SLAM algorithms to handle such ill-conditioned matrices amplifies minute measurement noise and IMU integration errors, leading to catastrophic cumulative drift of tens or even hundreds of meters in the UAV's forward direction. The direct consequence is severe axial stretching and distortion in the reconstructed 3D tunnel point cloud map, rendering it unable to provide absolutely accurate spatial coordinates for identified minor defects and thus losing its reference value for engineering maintenance and targeted repair.

[0033] Existing defect identification modules exhibit significant polarization and limitations. On one hand, most systems rely solely on a single visible light camera and a conventional lightweight visual network (such as YOLO Basic) to identify two-dimensional cracks. This single-modality approach is ineffective in dimly lit, dusty tunnels, failing to distinguish early water seepage disguised as shadows or detect hidden spalling caused by internal voids, resulting in extremely high false alarm and false positive rates. On the other hand, while academia has proposed using 3D point clouds, visible light, and infrared thermal imaging for multimodal fusion detection, complex fusion networks (such as the massive Transformer architecture or multi-branch 3D-CNN) require hundreds of megabytes of parameters, necessitating substantial GPU memory and computing power. Due to limitations in payload and power consumption, the onboard edge computing modules (such as the Jetson series) on tunnel inspection drones have very limited computing power and typically less than 4GB of memory. Directly applying existing large models would lead to edge computing power collapse, extremely low frame rates, and an inability to achieve real-time identification while in flight. However, if the number of network layers is simply reduced or traditional lightweight models are used to splice multimodal features, the network's ability to extract high-order nonlinear collaborative relationships between geometry (depth of depression), texture (edge ​​of crack), and temperature (temperature drop due to water seepage) will be destroyed, resulting in superficial feature fusion that cannot adapt to the complex and ever-changing real-world tunnel scene.

[0034] In summary, existing tunnel detection technologies have significant technical shortcomings that urgently need to be overcome in terms of the ability of underlying hardware to synchronously resist interference, the geometric degradation robustness of the core SLAM algorithm, and the balance between lightweight and accuracy in multimodal edge computing.

[0035] To address at least one of the aforementioned problems, this embodiment provides a tunnel structure defect identification method based on multispectral sensing and SLAM. Optionally, the identification device implementing the method is integrated onto a tunnel inspection aircraft. Its system architecture includes an anti-interference closed-loop time delay compensation synchronization and multispectral sensing module, a degradation sensing synchronous localization and mapping (Tunnel-SLAM) module, an airborne edge computing and defect identification module based on multidimensional knowledge distillation, and an autonomous navigation and obstacle avoidance module for occupied grids. The identification method includes: S1. Hardware architecture and closed-loop synchronization mechanism based on least squares difference compensation: Optional sensing hardware includes a forward-facing solid-state LiDAR, a global shutter visible light camera, an uncooled infrared thermal imaging camera, and a high-precision tactical-grade inertial measurement unit (IMU). All sensors are rigidly mounted via low-resonance rigid alloy brackets to ensure constant relative pose between the coordinate systems of each sensor.

[0036] To address the issues of electromagnetic interference and response delay in tunnels caused by conventional microcontroller-based synchronization trigger pulses, this embodiment abandons the traditional unidirectional open-loop hard triggering and proposes an EMI-resistant closed-loop hardware synchronization data processing method with least-squares delay compensation, such as... Figure 1 As shown, the specific implementation steps and circuit processing methods are as follows: S1.1, Physical Anti-interference Layer: The central hardware synchronous trigger does not directly output TTL level. The pulse signal is first input to the high-speed differential driver chip and converted into a low-voltage differential signal. At the data access pin of each sensor, a push-pull impedance compensation bridge circuit is deployed. This hardware topology utilizes the common-mode rejection characteristics of differential signals to attenuate the low-frequency and high-frequency common-mode noise generated by the ventilation fan and high-voltage line in the tunnel by at least 25dB, ensuring extremely high sharpness of the pulse edge from the physical layer.

[0037] S1.2, Two-way timestamp response protocol: To eliminate cable distributed capacitance delay and internal crystal oscillator temperature drift in the sensor, the system establishes a strict closed-loop timestamp interaction mechanism at the data link layer.

[0038] The central synchronizer sends a trigger pulse packet. The heterogeneous sensor captures the rising edge, records its local arrival time, and after completing a full exposure / scan, returns a response data packet to the central synchronizer, including the arrival time and acquisition completion time. Upon receiving the response packet, the central synchronizer records the reception time. Based on the time differences between the sending, receiving, and responding points, the system can determine the unidirectional propagation delay of a single transmission, as well as the discrete deviation of the sensor's clock relative to the central absolute clock.

[0039] S1.3 Dynamic delay compensation based on least squares method: Due to aircraft vibration and drastic temperature changes in the airflow within the tunnel, the deviation determined in a single instance exhibits high-frequency jitter. This embodiment introduces the least squares method for nonlinear fitting and optimal estimation of the clock drift rate. Specifically, the central synchronizer maintains a sliding window, buffering the most recent response timestamp data; an objective function is constructed based on the sum of squared residuals between the theoretically predicted clock vector and the actual measured timestamp matrix; and the optimal smooth estimate of the clock drift rate is determined through iterative optimization of the objective function.

[0040] The objective function can be expressed in the form of the sum of squared residuals from existing least-squares estimation. In this embodiment, let the first... The absolute time when the central synchronizer triggers transmission during secondary synchronization is The absolute time when the central synchronizer receives the response packet is The sensor records the local arrival time of the trigger. The local time when the sensor completes one exposure or scan is Single-pass unidirectional propagation delay It can be represented as: ; Corresponding discrete clock bias It can be represented as: ; Let the absolute start time of the sliding window be... The absolute time interval is Clock drift rate Then the first Theoretical predictions of local timestamps during secondary synchronization It can be represented as: ; Based on the difference between the predicted and measured values, a residual sum of squares objective function is constructed: ; in, The second term is a smoothing constraint term for the clock drift rate, used to smooth the weights. The clock drift rate is obtained by iteratively optimizing this objective function. The optimal smoothed estimate.

[0041] At the system driver node, for each frame of raw multimodal data uploaded by each sensor, the final globally unified timestamp is assigned by the underlying driver by subtracting the fitted smooth clock drift rate from the sensor's local time. Strict reverse compensation correction is performed.

[0042] This processing method completely solves the uncontrollable time delay in the tunnel environment, compresses the alignment error of multimodal data on the time axis from the millisecond level to less than 50 nanoseconds, and fundamentally eliminates the spatial coordinate mapping trail caused by flight motion.

[0043] S2. Degradation-sensing laser inertial SLAM algorithm adapted to tunnel corridor effects: This module abandons the direct application of existing SLAM algorithms (such as LIO-SAM) and performs deep low-level reconstruction to address the geometric degradation characteristics of tunnels. This embodiment proposes an Adaptive-LIO for Tunnels algorithm based on Hessian matrix eigenvalue decomposition and degradation direction awareness. Figure 2 As shown, specifically: S2.1 Construction of Nonlinear Observation Model: The system uses high-frequency IMU data for continuous pre-integration to determine the aircraft's prior pose. A new frame of point cloud acquired by the lidar is then used to extract edge and planar features, and subsequently registered with the maintained local sub-image from point to surface. At this point, the system constructs a linearized nonlinear observation equation that includes the pose state vector, measurement residuals, and the Jacobian matrix.

[0044] The linearized nonlinear observation equation can be expressed as a linearized form obtained by performing a first-order Taylor expansion on the nonlinear observation model in existing SLAM backend optimization. Specifically, let the current frame state increment be... Regarding the first Point-to-surface constraints, and their laser geometric residuals It can be represented as: ; in, The normal vector of the local plane. For the feature points in the point cloud of the current frame, This refers to the projection point on the reference plane corresponding to the feature point in the local sub-image. and These are the rotation matrix and translation vector for the current frame, respectively.

[0045] IMU pre-integrated residuals Stacking these residuals with the individual laser geometric residuals yields the total residual vector: ; In the prior state Performing a first-order Taylor expansion in the vicinity, we have: ; This leads to the corresponding least squares increment equation: ; in, For Jacobian matrices, This is the weight matrix. This serves as the information matrix for subsequent block partitioning, Schul complement elimination, and degradation direction analysis.

[0046] S2.2 Automatic Quantitative Detection of Geometric Degradation Direction in Tunnel Environment: To mathematically pinpoint the direction of constraint loss caused by the tunnel corridor effect, this embodiment defines and extracts an information matrix when solving the least squares incremental equation. The system is divided into rotation blocks, translation blocks, and intersection terms. Then, using Schur Complement elimination, intersection terms are removed, rotation states are marginalized, and the information matrix of the pure translation states is extracted separately. Eigenvalue decomposition is performed on the translation information matrix to obtain three eigenvalues ​​and their corresponding eigenvectors. In long, straight tunnels, because both side walls are parallel to the direction of travel, the normal vector constraints collected by the lidar in the direction of travel are minimal, resulting in the smallest eigenvalue representing the constraint strength in the direction of travel. It plummeted and approached zero.

[0047] S2.3 Adaptive Weighted Factor Graph Optimization Strategy: Existing algorithms assume that all three degrees of freedom in the space are equally reliable. When a feature degenerates in a certain direction, even small noise in that direction can be amplified, leading to catastrophic drift. This embodiment introduces an adaptive penalty mechanism driven by the degradation exponent.

[0048] The total degradation index is quantified by calculating the ratio of the largest eigenvalue to the smallest eigenvalue. Let the largest eigenvalue be... The smallest eigenvalue is The degradation threshold is When the total degradation index exceeds a preset threshold, dynamic penalty logic is triggered; an adaptive diagonal matrix for lidar odometry, including rotation and translation weights, is constructed. In the optimizer of this embodiment, the translation weights corresponding to the degradation feature vector direction are subject to a nonlinear attenuation coefficient to actively weaken unreliable lidar axial registration constraints.

[0049] S2.4 Cross-compensation of IMU pre-integral covariance: In the comprehensive residual objective function of global factor graph optimization, since the observation weight of laser in the tunnel axis is greatly weakened, the system adaptively increases the confidence weight of IMU pre-integration factor in the degenerate axis. Through this cross-complementary improvement at the underlying mathematical architecture, the algorithm is completely immune to the tunnel corridor effect. Even in a pure straight cylindrical featureless tunnel that is several kilometers long, it can rely entirely on the coupling of IMU axial integration and laser radial constraint to achieve centimeter-level drift-free 3D point cloud map construction.

[0050] In this embodiment, the comprehensive residual objective function can adopt the weighted residual sum of squares form used in existing factor graph optimization, and further incorporate adaptive weight adjustment based on the degradation direction. Let the set of states to be optimized be... Then the global factor graph optimization objective function It can be represented as: ; ; in, For the first Frame lidar registration residual, For the first Frame IMU pre-integration residual, and These are the weight matrices for the corresponding factors.

[0051] Let the degenerate direction unit vector corresponding to the smallest eigenvalue of the translation information matrix in step S2.2 be . The total degradation index is: ; in, To prevent division by zero minimum constant. When At that time, the weight of the laser factor in the degradation direction is adjusted as follows: ; The weights of the IMU pre-integration factors in the same degradation direction are adjusted as follows: ; That is, the laser registration constraint weight is actively reduced along the degradation axis, while the IMU pre-integration constraint weight is increased simultaneously. Through the above cross-compensation, robust suppression of the tunnel corridor effect is achieved.

[0052] S3. Deep fusion and local feature encoding process in multidimensional feature space: Simple stitching of multimodal data cannot enable AI models to understand the three-dimensional nature of diseases. As shown in Table 1, this module not only involves multimodal fusion, but also defines in detail the specific extraction and analysis processes of geometric, textural, and temperature features.

[0053] Table 1 Multimodal disease characteristic dimension analysis matrix

[0054] S3.1, Three-dimensional back projection and bilinear interpolation mapping based on calibration extrinsic parameters: For the same frame dataset captured via nanosecond-level hard synchronization, each discrete point in the high-density 3D point cloud generated by the LiDAR is inversely projected onto the pixel plane of the visible light camera and the infrared thermal imager, strictly according to the optical distortion model, using a high-precision calibrated joint extrinsic parameter matrix. Addressing the issue that projected points typically fall between discrete pixel grids, this invention does not employ the nearest neighbor method, which leads to jagged edges, but instead incorporates a bilinear interpolation algorithm for high-fidelity sampling. By obtaining the weighted average of the four adjacent physical pixels around the projected point, the continuous color value and absolute physical temperature scalar of the 3D point in the real physical world are accurately calculated. At this point, the original pure geometric point cloud is endowed with multimodal properties, upgraded to an information-rich "enhanced 3D point cloud."

[0055] S3.2, VLBP and FPFH Local Feature Encoding: Simple point attributes cannot describe the geometric topology of the disease. The system divides the enhanced point cloud into a regular voxel grid and performs deep feature encoding for the center point of each voxel.

[0056] Texture feature encoding: A local difference sign-amplitude transform is introduced to compare the grayscale / color information of the voxel center with that of adjacent sub-voxels. Subsequently, global thresholding is applied to generate voxel-based Local Binary Pattern (VLBP) features. This feature is unaffected by local overexposure or dead-angle shadows caused by tunnel point light sources and can extremely sharpen the edge texture of cracks.

[0057] Geometric feature encoding: A local covariance matrix is ​​constructed based on the point cloud, and the normal vector of each point is extracted. The spatial relative angle change between the normal vectors of the center point and its neighboring points is calculated, and a Fast Point Feature Histogram (FPFH) descriptor is constructed. This histogram is extremely sensitive to surface smoothness and can accurately and quantitatively describe the dramatic local curvature changes caused by spalling.

[0058] Temperature feature encoding: Calculates the temperature gradient and average temperature drop within a specified spatial range centered on the current point. Due to the slow evaporation and heat absorption of moisture, the seepage area exhibits a persistent negative temperature anomaly in uncooled infrared data, distinctly different from the surrounding dry concrete.

[0059] S4. Edge computing defect identification model based on multidimensional knowledge distillation and geometry-guided: This embodiment proposes a lightweight recognition model that incorporates a multidimensional knowledge distillation (MD-KD) architecture and geometric guided score fusion (G2SF). Figure 3 As shown, specifically: S4.1 Construction and training of offline and cloud-based teacher models: On a cloud server with unlimited computing power, a teacher network with an extremely deep hierarchical structure (based on an advanced architecture) is constructed. This network employs a multi-branch feature extractor to independently process the extracted texture, geometric, and temperature feature data. The teacher network embeds a high-level module combining a deep residual structure and a multi-head self-attention mechanism to powerfully suppress complex background noise such as ventilation ducts and joint mud bands within the tunnel, achieving an extremely high overall accuracy detection limit.

[0060] S4.2, Multi-dimensional knowledge distillation architecture enables lightweight deployment at the edge: On an onboard edge computing platform with only a few gigabytes of memory and limited floating-point computing power, a lightweight student network with a parameter count that is only a tiny fraction of the teacher model is deployed. To compensate for the accuracy drop caused by model scaling, a rigorous multi-level knowledge distillation mechanism is introduced during the system's offline training phase: Feature Dimension Mask Alignment: This forces the feature distribution of the intermediate hidden layers of the student network to be consistent with that of the teacher model. This compels the lightweight student model to learn the attention mechanisms of the larger model in the spatial and channel dimensions, i.e., to learn "which key regions to focus on".

[0061] Output logical soft label alignment: By optimizing the category probability divergence distribution, the student network learns the potential connections between categories at the output layer. For example, it not only learns that the target is "crack," but also learns through the soft distribution that "it has a certain probability of being a water stain boundary," thus possessing the high-dimensional discrimination ability to handle ambiguous diseases.

[0062] S4.3 Real-time classification and recognition logic and multimodal decision tree based on G2SF: To address the issue of modal conflicts arising from concealed defects during real-time inference in airborne equipment, this invention introduces a geometry-guided fractional fusion (G2SF) framework at the detection end. Traditional fusion often involves simple weighting of various scores, while G2SF dynamically transforms fixed distance measures that are prone to misjudgment in a single modality into highly discriminative anomalous feature components by predicting direction-aware local scale factors.

[0063] The logical discriminant tree for the detailed classification and recognition process is as follows: High-confidence determination of surface spalling: When the spatial features of a local area revealed a significant abrupt change in the FPFH geometric feature histogram within a specific curvature range (physically characterized by obvious pitting on the concrete surface), and simultaneously detected a sudden change in VLBP texture grayscale in this area, while the infrared temperature field remained stable, the G2SF fusion operator evaluated a geometric anomaly score with extremely high accuracy, and the comprehensive classification output was "spalling". Simultaneously, extreme depth data of the area was extracted to calculate the accurate spalling amount.

[0064] Anti-interference judgment of water seepage / stains: This is a hidden defect that is easily misjudged as "shadows" by humans. If the network detects dark patches on a two-dimensional plane, but attribute analysis reveals that the area is geometrically smooth and has not peeled off, and infrared channel calculations show that there is a continuous temperature difference between the interior and edges of this area and the surrounding dry area (due to the heat absorption of water evaporation), the network uses a thermodynamic anomaly factor to forcibly increase the confidence score of the water stain, thus accurately identifying this seepage area disguised as a shadow.

[0065] Microstructural Crack Fusion Determination: For tiny cracks with widths on the millimeter scale, their three-dimensional geometric features exhibit minimal fluctuations. The system primarily relies on elongated, high-frequency alternating VLBP edge textures extracted via the visible light channel for localization. Simultaneously, the fusion framework utilizes a separation penalty mechanism to polarize and distance these features from normal tunnel construction joints (which possess regular, straight geometric constraints) in the feature space, thereby effectively eliminating false alarms about construction joints.

[0066] Finally, as shown in Figure 4, all high-confidence defects selected by the student network using the Intersection over Union (IoU, which is the ratio of the intersection and union of the predicted bounding box and the actual defect area) are directly back-mapped back to the three-dimensional point cloud coordinate system output by the SLAM module, realizing real-time and accurate binding between defect type and tunnel three-dimensional absolute spatial coordinates.

[0067] S5, Dynamic grid update and autonomous navigation obstacle avoidance: Leveraging the efficiency of edge computing, the real-time drift-free point cloud map output from SLAM is projected and converted into a 3D occupancy grid map. The UAV's multi-rotor flight controller dynamically replans the route by combining a pre-set safe inspection path along the tunnel centerline with real-time updated obstacle probabilities in the grid space. While maintaining a stable observation distance, the system can safely avoid suddenly appearing overhead cables or construction trolleys within the tunnel.

[0068] As shown in Table 2, this embodiment not only achieves a breakthrough at the hardware level, but also achieves highly valuable engineering results through in-depth optimization of the underlying model and algorithm architecture: Table 2. Performance Comparison Index Matrix Between the Invention and Traditional Tunnel Detection Schemes

[0069] This embodiment proposes a closed-loop hardware synchronization technology based on least squares difference compensation, which overcomes chip pulse delay caused by strong EMI in tunnels and drastic temperature changes. Compared with traditional single-chip hard triggering, this method successfully compresses the heterogeneous time alignment error of sensors to the 50 nanosecond level, ensuring a perfect rigid mapping between the 3D point cloud depth and 2D visible / infrared pixels of the UAV in flight, thus eliminating the reference error for defect localization from a physical level.

[0070] This embodiment reconstructs the laser inertial SLAM algorithm for the special characteristics of tunnel environments. It actively senses the geometric degradation direction through matrix eigenvalue decomposition and dynamically generates adaptive penalty weights. This degradation perception mechanism completely solves the problem of severe axial cumulative drift that traditional vision / laser SLAM is prone to in tubular structures lacking features, and achieves stable centimeter-level mapping accuracy in long-distance environments without GPS.

[0071] This embodiment completely abandons simple visible light two-dimensional recognition and coarse model stitching. On the one hand, it utilizes deep local coding technology to achieve deep spatial representation of geometry, texture, and temperature. On the other hand, it innovatively adopts multi-dimensional knowledge distillation technology and a geometry-guided fractional fusion framework, enabling a very small-scale airborne edge model to acquire the nonlinear reasoning capabilities of a deep cloud-based large model. Under limited computing power, it achieves a significant reduction in the number of parameters and a doubling of inference speed, especially demonstrating strong anti-interference detection capabilities for "early water seepage due to temperature anomalies" and "hidden peeling due to deep anomalies," which are easily missed by manual detection.

[0072] This embodiment achieves on-the-fly identification and real-time accurate registration through deep closed-loop collaboration of underlying airborne edge computing, multispectral fusion, and a degradation-aware SLAM framework. Any structural defect can acquire high-precision absolute 3D coordinates in an extremely short instant after identification, completely eliminating the tedious and costly manual offline data comparison work later. The system automatically generates a visualized inspection report containing a 3D model, precise defect location, and a multispectral physical evidence matrix, greatly improving the decision-making efficiency for tunnel engineering acceptance and daily maintenance.

[0073] In other embodiments, due to the strong generalization ability of the synchronization mechanism and recognition framework, the physical carriers on which they are mounted, in addition to multi-rotor aircraft, can seamlessly replace the intelligent inspection robots or quadrupedal bionic wheeled unmanned vehicle platforms deployed on the catenary track; the lidar can be replaced by a solid-state type with a high-beam mechanical rotating lidar (more suitable for large two-way multi-lane tunnels or underground caverns with ultra-wide cross-sections); the visible light camera can be upgraded from a monocular global shutter to a binocular stereo vision camera array; and for very early high-risk fine seepage, the thermal imaging camera can also be replaced by a cooled infrared thermal imager with higher thermal sensitivity (NETD) and finer resolution.

[0074] In other embodiments, for complex, superimposed defects (such as large-scale lining collapse accompanied by severe water inrush) or non-standard special underground structures, the computational architecture for defect identification can be degraded from or evolved from a purely "independent and complete processing by the airborne edge computing module" to a "cloud-edge dynamic collaborative mode." In this alternative mode, the lightweight network on the airborne end only runs light inference, responsible for the efficient compression of the enhanced point cloud and the preliminary target screening of suspected defect feature regions with low precision and high recall. Subsequently, the airborne communication module rapidly transmits these high-brightness feature slice data back to the ground control station or cloud computing center, where it is handed over to the deep teacher network (large model) with unlimited computing power for extremely high-precision secondary nonlinear inference and diagnosis, thereby achieving a perfect balance between edge power consumption and extremely high precision in the cloud.

[0075] The high-fidelity point cloud model of the tunnel malfunction distribution generated in this embodiment, with absolute three-dimensional coordinates, can be displayed and analyzed on a conventional PC terminal 2D screen system. Its data format is natively compatible and can be directly imported into a standard BIM (Building Information Modeling) digital twin operation and maintenance platform. In other embodiments, further, during the subsequent repair and construction phase, the positioning data and point cloud model can be combined with AR (Augmented Reality) or VR (Virtual Reality) hybrid spatial head-mounted displays. After on-site construction personnel wear these devices and enter the tunnel, the system uses the model data for spatial reverse positioning and anchoring, achieving a 1:1 immersive holographic overlay of the malfunction coordinates onto the actual physical wall surface. This will completely change the traditional work method, enabling intuitive, perspective-based review of tunnel malfunction locations and high-precision repair guidance.

[0076] Example 2: This embodiment provides a tunnel structure defect identification system based on multispectral sensing and SLAM, including: The data acquisition module is configured to acquire relevant laser data, visible light data, infrared data, and inertial data within the tunnel. Specifically, during data acquisition, it determines the unidirectional propagation delay of a single transmission and the discrete deviation of the corresponding sensor clock relative to the absolute clock based on the local arrival time of the trigger pulse packet and the response time of the response data packet. Based on multiple response timestamp data, it constructs a residual squared sum objective function of the difference between the theoretically predicted clock vector and the actual measured timestamp matrix. Through iterative optimization, it determines the optimal smooth estimate of the clock drift rate. The 3D point cloud map construction module is configured to construct a 3D point cloud map based on the coupling of the axial integral of inertial data and the radial constraint of laser data. The feature extraction module is configured to extract features from laser data, visible light data, and infrared data to obtain an analysis matrix. The defect identification module is configured to: obtain defect identification results based on the analysis matrix and the preset identification model; and reverse map the defect to the three-dimensional point cloud coordinate system of the three-dimensional point cloud map; wherein the identification model is a lightweight identification model that fuses a trained multidimensional knowledge distillation architecture with geometric guided scores.

[0077] The working method of the system is the same as that of the tunnel structure defect identification method based on multispectral sensing and SLAM in Example 1, and will not be repeated here.

[0078] Example 3: This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the tunnel structure defect identification method based on multispectral sensing and SLAM described in Embodiment 1.

[0079] Example 4: This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements the steps of the tunnel structure defect identification method based on multispectral sensing and SLAM described in Embodiment 1.

[0080] Example 5: This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the tunnel structure defect identification method based on multispectral sensing and SLAM described in Embodiment 1.

[0081] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for tunnel structure defect identification based on multi-spectral perception and SLAM, characterized in that, include: Acquire relevant laser data, visible light data, infrared data, and inertial data within the tunnel. During data acquisition, the unidirectional propagation delay of a single transmission and the discrete deviation of the corresponding sensor clock relative to the absolute clock are determined based on the local arrival time of the trigger pulse packet and the response time of the response data packet. Based on multiple response timestamp data, a residual sum of squares objective function is constructed to compare the theoretically predicted clock vector with the actual measured timestamp matrix. Through iterative optimization, the optimal smooth estimate of the clock drift rate is determined. A three-dimensional point cloud map is constructed by coupling the axial integral of inertial data with the radial constraint of laser data. Features are extracted from laser data, visible light data, and infrared data to obtain an analysis matrix; Based on the analysis matrix and the preset recognition model, the defect recognition result is obtained; the defect is then mapped inversely to the three-dimensional point cloud coordinate system of the three-dimensional point cloud map; wherein, the recognition model is a lightweight recognition model that fuses a trained multidimensional knowledge distillation architecture with geometric guided scores.

2. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 1, characterized in that, Impedance compensation bridging circuits are deployed for each sensor that acquires laser data, visible light data, infrared data, and inertial data. By utilizing the common-mode rejection characteristics of differential signals, the low-frequency and high-frequency common-mode noise generated by the ventilation fan and high-voltage line in the tunnel is attenuated to a preset value.

3. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 1, characterized in that, Continuous pre-integration of inertial data is used to determine the prior pose of the aircraft. A new frame of point cloud acquired by lidar is used to extract edge and planar features, and then the points are registered to surfaces with the maintained local sub-image. When solving the least squares incremental equation, the corresponding high-dimensional information matrix is ​​defined and extracted. The high-dimensional information matrix is ​​divided into rotation blocks, translation blocks, and cross terms. Using the Schur complement elimination method, the cross terms are removed, the rotation state is marginalized, and the translation information matrix of the pure translation state is extracted separately. Eigenvalue decomposition is performed on the translation information matrix to obtain three eigenvalues ​​and corresponding eigenvectors.

4. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 3, characterized in that, The total degradation index is obtained by the ratio of the maximum eigenvalue to the minimum eigenvalue. When the total degradation index exceeds the preset tunnel degradation threshold, an adaptive diagonal matrix for lidar odometers, including rotation weights and translation weights, is constructed. The translation weights are forcibly subjected to a nonlinear attenuation coefficient, which actively weakens the registration constraint of lidar along the tunnel axis. At tunnel intersections with rich structures, the attenuation coefficient returns to normal.

5. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 3, characterized in that, In the weighted residual sum of squares objective function of global factor graph optimization, the confidence weight of the inertial data pre-integration factor on the degradation axis is increased.

6. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 1, characterized in that, In the analysis matrix, visible light data corresponds to surface texture edge features and color change features, targeting surface micro-cracks and carbonization discoloration; infrared data corresponds to local abnormal low temperature features and evaporation heat absorption temperature drop features, targeting early hidden water seepage and damp water stains; laser data corresponds to local surface curvature change features and normal vector anomaly features, targeting surface peeling, concrete depression and spalling.

7. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 6, characterized in that, Each discrete point in the 3D point cloud generated by the lidar is inversely projected onto the pixel plane of the visible light camera and the infrared thermal imager; the continuous color value and absolute physical temperature scalar of the 3D point in the real physical world are determined by the weighted average of the four adjacent physical pixels around the projected point; the enhanced point cloud is divided into a regular voxel grid, and deep feature encoding is performed for the center point of each voxel: a local difference sign-amplitude transform is introduced, the grayscale information of the voxel center and the adjacent sub-voxels are compared, and a voxel-based local binary pattern feature is generated through global thresholding; Construct a local covariance matrix based on point cloud and extract the normal vector of each point; Determine the spatial relative angle change between the normal vectors of the center point and neighboring points, and construct a feature histogram descriptor; with the current point as the center, specify the temperature gradient and average temperature drop within the spatial range.

8. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 1, characterized in that, The recognition model includes a multi-branch feature extractor to independently process texture, geometric, and temperature feature data; the teacher network of the recognition model embeds a deep residual structure and a multi-head self-attention mechanism to suppress background noise from ventilation ducts and joint mud strips in the tunnel.

9. The tunnel structure defect identification method based on multispectral sensing and SLAM as described in claim 1, characterized in that, On an airborne edge computing platform, a student network is deployed, and the feature distribution of the hidden layer in the middle of the student network is constrained to be consistent with that of the teacher model. A geometrically guided score fusion framework is introduced, which dynamically transforms the fixed distance measure that is prone to misjudgment in a single modality into a distinctive abnormal feature component by predicting the local scale factor of orientation awareness.

10. A tunnel structure defect identification system based on multispectral sensing and SLAM, characterized in that, include: The data acquisition module is configured to acquire relevant laser data, visible light data, infrared data, and inertial data within the tunnel. Specifically, during data acquisition, it determines the unidirectional propagation delay of a single transmission and the discrete deviation of the corresponding sensor clock relative to the absolute clock based on the local arrival time of the trigger pulse packet and the response time of the response data packet. Based on multiple response timestamp data, it constructs a residual squared sum objective function of the difference between the theoretically predicted clock vector and the actual measured timestamp matrix. Through iterative optimization, it determines the optimal smooth estimate of the clock drift rate. The 3D point cloud map construction module is configured to construct a 3D point cloud map based on the coupling of the axial integral of inertial data and the radial constraint of laser data. The feature extraction module is configured to extract features from laser data, visible light data, and infrared data to obtain an analysis matrix. The defect identification module is configured to: obtain defect identification results based on the analysis matrix and the preset identification model; and reverse map the defect to the three-dimensional point cloud coordinate system of the three-dimensional point cloud map; wherein the identification model is a lightweight identification model that fuses a trained multidimensional knowledge distillation architecture with geometric guided scores.

Images