A pipe tunnel leakage detection method and system based on laser radar reflectivity

Abstract

This invention provides a method and system for detecting leaks in pipe tunnels based on lidar reflectivity. First, a point cloud of the pipe tunnel is acquired, and spatially aligned reflectivity and signal intensity feature streams are extracted. These feature streams are then input into a target detection model embedded with a cross-attention module to perform feature fusion. Finally, the detection result of the leaking area in the pipe tunnel is output based on the fused features. This invention eliminates the reliance on manual parameter design through deep learning, allowing weights to adapt to various environments, significantly improving detection efficiency and reducing labor costs. The system supports direct processing of streaming point clouds, achieving efficient real-time on-site detection. By utilizing dual-channel feature fusion of reflectivity and signal intensity, this method achieves higher precision and recall than traditional single-signal algorithms in complex environments, significantly enhancing the accuracy of leak detection.

Description

Technical Field

[0001] This invention relates to the field of pipe and tunnel inspection technology, specifically to a pipe and tunnel leakage detection method and system based on lidar reflectivity, and particularly to a pipe and tunnel leakage detection method and system employing a deep learning method based on convolutional neural networks. Background Technology

[0002] To ensure urban water supply security, my country has constructed numerous long-distance water supply pipelines and tunnels. As operating time increases, these long-distance pipelines and tunnels are prone to dangerous conditions such as pipe wall leakage, necessitating regular safety assessments of the entire pipeline. Currently, in the field of pipeline and tunnel inspection, common leakage detection methods primarily rely on optical imaging technology.

[0003] Common optical imaging techniques include camera-based visible light or infrared dual-spectrum detection methods. In these methods, inspection robots are typically equipped with optical cameras to acquire images of the pipe walls. However, this approach has significant limitations: First, the camera's shooting range is limited. Since a camera can only cover a specific shooting direction at a time, most inspection robots must rotate the camera to capture a panoramic image of the pipe. This easily causes image blurring and other problems during mobile inspections, making the process time-consuming and complex. Second, cameras struggle to accurately distinguish leakage characteristics. Visible light cameras cannot effectively differentiate between the inherently colored texture of the pipe wall and the actual leakage area, easily leading to false detections. While infrared cameras utilize temperature differences to identify leaks, in scenarios like pipe tunnels where the pipe wall and water are in the same constant temperature environment for extended periods, the temperature difference is often not significant, easily resulting in missed detections.

[0004] Laser scanning feature-based detection is another relatively mature technology. For example, in the scenario of inspecting seepage areas in tunnels, this approach uses laser scanning equipment to collect tunnel point cloud data and identifies areas based on the difference in reflection intensity between seepage areas and dry areas. In practice, this typically involves first removing impurity points using methods such as Gaussian distribution, then filtering the target point cloud using a preset reflection intensity threshold, and finally marking the seepage areas using density clustering algorithms or mathematical morphology methods such as dilation, corrosion, and connected component analysis.

[0005] However, the aforementioned traditional methods based on laser point clouds have revealed several shortcomings in practical applications. On one hand, these methods heavily rely on pre-designed parameters or thresholds, such as neighborhood radii and binarization thresholds in clustering algorithms. This makes the algorithms poorly adaptable to seepage targets in different environments, resulting in poor generalization and easily influenced by human design experience in terms of recognition accuracy. On the other hand, traditional methods suffer from accuracy issues when processing data. Since traditional schemes often use the original point cloud reflectance intensity for calculations, they fail to fully consider the factory differences between different receivers of multi-line lidar, leading to inconsistent reconstructed surface reflectance benchmarks. Furthermore, reconstructing reflectance typically requires decoupling the influence of the incident angle. Traditional methods mostly presuppose the lidar-illuminated surface as an ideal plane, thus inferring the incident angle from the distance. This simplified model is clearly unsuitable for tunnel environments with complex surface geometries, severely impacting recognition accuracy.

[0006] In summary, existing methods for detecting leakage in pipeline tunnels have several technical problems that urgently need to be addressed, including detection efficiency, environmental adaptability, and data calculation accuracy in complex scenarios. These issues make it difficult to meet the current demand for high-precision and automated safety assessments in long-distance water supply pipeline tunnel projects. Summary of the Invention

[0007] This invention aims to overcome the aforementioned shortcomings by employing a deep learning method based on convolutional neural networks to directly identify leakage areas in pipe tunnels. This method, trained on a large dataset of leakage areas in pipe tunnels under various environments, effectively overcomes the problem of traditional methods being unable to adapt to diverse pipe tunnel environments. Furthermore, the deep learning method eliminates the need for manually designed thresholds; it uses the model weights derived from the training dataset as the basis for identification, thus eliminating the influence of human factors on accuracy. Additionally, this invention directly utilizes the reflectivity channel provided by radar, which has undergone factory calibration, and the deep learning-based method does not need to consider issues such as surface incidence angle, resulting in stronger environmental adaptability.

[0008] Specifically, this invention proposes a method for detecting leakage in pipe tunnels based on lidar reflectivity. The process includes: first, using a mobile vehicle equipped with lidar to record point cloud files of the tunnel interior; second, splitting the point cloud files frame-by-frame into two-dimensional point cloud images, which are then processed to include channels for reflectivity, signal strength, etc., and the leakage areas are manually labeled to obtain bounding boxes for the leakage areas; then, training the images using a neural network; and finally, testing unknown two-dimensional point cloud images or streaming point cloud images based on the model weights to obtain the bounding boxes for the leakage areas.

[0009] In the preprocessing of point cloud data, to ensure the continuity of image features, this invention employs buffer image filling logic. Specifically, a storage module is established to extract single-frame point cloud data from the point cloud video and maintain a buffer image. When a new frame of point cloud data arrives, the information of non-missing pixels is identified and the corresponding pixels in the buffer image are refreshed, while the historical pixel information corresponding to missing points in the buffer image is retained, until a complete, hole-free single-frame image is synthesized.

[0010] To achieve efficient conversion of physical properties into visual features, this invention designs a linear normalization method for reflectivity and signal intensity. This method first locks the maximum and minimum values ​​of the current channel data, calculates the quantization interval step size, and maps the original physical values ​​to an 8-bit grayscale space, thereby converting the reflectivity map or signal intensity map into an 8-bit grayscale image that conforms to the neural network input specification.

[0011] The key focus of this invention lies in the improved deep learning model architecture. A cross-attention module is embedded in the middle of the backbone network of the YOLO v5s model. This module constructs a basic cross-attention layer, receiving dual-input stream features from the reflectivity and signal intensity channels, and utilizes a multi-head attention mechanism to capture the long-distance dependency and nonlinear correlation between the two physical channels. During feature fusion, query vectors, key vectors, and value vectors are generated through linear projection. Combined with residual connections and learnable positional embeddings, deep cross-fusion of the two-dimensional visual feature streams is achieved, and the fused features are output to the model's detection head.

[0012] In detail, this invention provides a method for detecting pipe and tunnel leakage based on lidar reflectivity. The method first uses lidar to acquire point cloud data of the pipe and tunnel, and extracts reflectivity and signal intensity feature streams that are spatially aligned. Then, these two feature streams are synchronously input into a pre-built target detection model, and a cross-flow feature fusion is performed using a cross-attention module embedded in the model, thus forming a fused feature coupled with dual physical attributes. Finally, the model accurately identifies and outputs the detection result of the pipe and tunnel leakage area based on this fused feature. This technical solution, by introducing and deeply fusing the physical features of reflectivity and signal intensity, effectively solves the technical problem of insufficient recognition accuracy in dark areas or low-contrast environments using traditional visual detection, thus improving the robustness of the detection.

[0013] Preferably, the cross-attention module employs asymmetric projection logic, using the reflectivity feature stream reflecting material properties as the query stream and generating a query vector through an unbiased linear projection layer; it uses the signal intensity feature stream reflecting signal energy intensity as the key / value stream, generating corresponding key and value vectors. A multi-head attention mechanism is used to calculate the similarity weights between the query vector and the key vector, and these weights are applied to the value vector, ultimately outputting the fused features. This achieves the goal of calibrating the original sensed signal using high-quality physical properties.

[0014] Furthermore, the cross-attention module is precisely embedded in the P3 / 8 scale position of the backbone network of the target detection model, thereby performing dual-stream fusion at the feature level with the highest image resolution and the richest spatial texture information, which greatly improves the model's ability to capture small targets in subtle and sporadic leakage areas.

[0015] Preferably, before performing the fusion computation, the cross-attention module first flattens the two-dimensional feature map into a sequence format and performs dimensional permutation. Then, it adds learnable positional embedding information to the feature sequence through a combination of linear layers and residual connections. This technical solution, by reintroducing spatial constraints into the serialized data, compensates for the potential loss of spatial structure that may occur when the attention mechanism processes flattened data, ensuring the accuracy of leakage localization.

[0016] Preferably, before inputting the data into the model, the system performs linear normalization on the original physical attribute values. By statistically analyzing the dynamic extrema of the physical attribute values ​​within the image and calculating the quantization interval step size, the original high-bit-depth physical values ​​are linearly mapped to the standard grayscale feature space. This further eliminates dimensional differences between different sensors and maximizes the contrast between the leaking area and the dry background, thus adapting to the input specifications of deep learning models.

[0017] Furthermore, the cross-attention module employs a sophisticated dual residual fusion structure. First, the output of the multi-head attention is coupled with the original query stream through a first-layer residual connection. Then, this result is input into a feedforward network consisting of two unbiased linear layers, where a second residual connection is executed. This approach effectively mitigates the vanishing gradient problem during deep network training while enabling deep feature interaction, and preserves the original low-level details of the reflectivity channel to the greatest extent possible.

[0018] Preferably, to address packet loss during acquisition, a buffered image refresh mechanism is implemented in the preprocessing stage. By maintaining a global buffered image, the corresponding coordinate values ​​are refreshed in real time using valid pixels from new frames, while missing positions are filled using historical pixel information. This technical solution utilizes the continuity of the temporal dimension to compensate for spatial data gaps in single-frame images, ensuring the spatiotemporal integrity of the data sequence input to the model.

[0019] Furthermore, the cross-attention module features channel adaptation during forward propagation, automatically configuring the convolutional layer execution dimension alignment based on whether the number of input channels for reflectivity and signal intensity feature streams are consistent. This enhances the module's versatility and flexibility, enabling it to be compatible with LiDAR feature inputs of different specifications or from different sources.

[0020] Preferably, when constructing the training dataset, the system samples frames at a fixed frequency and uses a deduplication algorithm to remove duplicate or highly redundant image samples. Then, it synchronously performs annotation on pixel-aligned dual-channel images. This reduces the annotation workload while optimizing the diversity of training samples and effectively lowers the risk of model overfitting.

[0021] Meanwhile, this invention also provides a tunnel leakage detection system, which includes four main modules: data acquisition, data preprocessing, feature fusion, and detection output, used to collaboratively execute the aforementioned detection methods. Through deep integration of hardware and software algorithm modules, this system achieves a complete process from acquiring tunnel point clouds to automatic leakage identification, significantly improving the automation level and operational efficiency of underground engineering safety inspections.

[0022] Compared with the prior art, the present invention has the following beneficial effects: First, by introducing deep learning methods, the problem of dependence on manually designed parameters in traditional methods is solved, enabling the model to adapt to various complex tunnel environments through training weights, which significantly improves generalization ability and detection efficiency, and reduces labor costs.

[0023] Secondly, this invention directly utilizes the factory-calibrated reflectivity channel output by the lidar hardware, combined with the signal strength channel, to perform dual-stream feature fusion. This effectively avoids the factory differences of multi-line receivers and the problem of inaccurate incident angle decoupling under complex geometric surfaces, greatly improving the accuracy and recall rate of leakage identification.

[0024] Third, through a buffered image refresh mechanism and a lightweight model design, this method can effectively handle the point cloud packet loss problem and support real-time detection of streaming point clouds, meeting the needs of real-time on-site safety evaluation in industrial settings. Attached Figure Description

[0025] Figure 1 A schematic diagram of the overall structure of a pipe and tunnel leakage detection method based on lidar reflectivity provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of using a buffered image to fill holes in a newly arrived image in an embodiment of the present invention; Figure 3 This is a diagram of the improved YOLO v5s model architecture based on the cross-attention mechanism in an embodiment of the present invention; Figure 4 This is a schematic diagram of the internal architecture and feature fusion process of the cross-attention module in an embodiment of the present invention. Detailed Implementation

[0026] To better understand the technical solutions of the present invention, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that those skilled in the art can implement the present invention in other forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0027] This embodiment provides a method for detecting pipe and tunnel leakage based on lidar reflectivity, with data acquisition based on a specific lidar hardware architecture. In this embodiment, the lidar uses multi-beam Flash LiDAR technology and the Time-of-Flight (ToF) principle to achieve point cloud acquisition. Its core hardware architecture relies on a vertical-cavity surface-emitting laser (VCSEL) array and a single-photon avalanche diode (SPAD) array.

[0028] The lidar precisely controls the emission parameters using digital beam manipulation technology, emitting 865nm eye-safe laser pulses. The SPAD detector captures the echo photons with single-photon-level sensitivity and picosecond-level time resolution. Subsequently, an integrated CMOS ASIC chip records the time difference between laser emission and reception in real time and calculates the target distance using the distance formula d = c × t / 2. Combining the horizontal 360° rotation angle of the laser beam with the vertical fixed channel angle, complete 3D point cloud data is finally generated.

[0029] While acquiring point cloud data, the lidar simultaneously outputs data from three core channels: signal strength (SIGNAL), reflectivity (REFLECTIVITY), and near-infrared (NEAR_IR). (1) Near-infrared channel: This channel focuses on measuring the intensity of natural near-infrared light in the environment. During the acquisition process, the lidar first disables laser emission, and the SPAD detector counts and outputs the ambient infrared photons. The data from this channel is mainly used to assess ambient lighting conditions and distinguish between day and night scenes, thereby assisting in object recognition in low-light environments.

[0030] (2) Signal Strength Channel: This channel is the original signal layer. The lidar counts the total number of photons received by the SPAD during the return of each laser pulse using a counter, then subtracts the number of ambient infrared photons to obtain the signal photon count, which is then output. This data directly reflects the intensity of the reflected light from the target and is related to the target material, surface roughness, and distance. It is suitable for ranging quality assessment and noise filtering.

[0031] (3) Reflectivity Channel: This channel outputs the standardized calibrated reflectivity. Its calculation logic is: (signal photon count × distance) 2 The preset formula "× factory calibration factor ÷ emission energy" yields reflectivity data decoupled from factors such as distance and sensor manufacturing differences. This channel eliminates the influence of distance and sensor sensitivity differences, and is the key basis for object classification and leakage material identification in this embodiment.

[0032] Based on the aforementioned hardware output capabilities, this method can directly acquire the physical property characteristics of the target surface during the acquisition process. Because the surface of the leaking area inside the pipe tunnel is covered with a water layer, the surface reflectivity of this part exhibits characteristics significantly different from those of the normal dry pipe wall area. This invention fully utilizes the reflectivity and signal intensity data acquired by the lidar to identify water seepage defects in the pipe wall from a physical property perspective.

[0033] See the instruction manual appendix Figure 1 The pipe and tunnel leakage detection method based on lidar reflectivity provided in this embodiment mainly involves three parts: the construction of pipe and tunnel leakage dataset, the training of neural network, and online / offline detection application.

[0034] First, in the dataset construction phase, a mobile platform, such as a detection vehicle equipped with the aforementioned LiDAR, was used to perform a full-line scan inside the tunnel, recording point cloud video files. Subsequently, the recorded point cloud files were split into two-dimensional point cloud images frame-by-frame according to the acquisition sequence, and then deconstructed into reflectivity and signal strength channels. The split images underwent preprocessing to eliminate holes and packet loss, and standardization and normalization were implemented. Finally, leak areas in the images were manually labeled to generate sample data with location bounding boxes, thus completing the construction of the tunnel leak dataset.

[0035] Secondly, in the model training part, the constructed dataset is input into the improved YOLO v5s model based on the cross-attention mechanism. During the training process, the model uses the cross-attention module embedded in its backbone network to perform in-depth comparison and fusion of reflectivity features and signal intensity features in the same scene. Through multiple rounds of iteration, it learns the physical property features of the leakage area and finally generates the optimal model weights.

[0036] Third, in the detection application section, the unknown two-dimensional point cloud image to be identified or the real-time streaming point cloud image is input into the trained model. Based on the model weights obtained from training, the model automatically identifies pixel regions with leakage characteristics in the image and outputs the location bounding boxes of the leakage regions in real time, thereby realizing automated detection of pipe and tunnel leakage.

[0037] Through the above overall architecture, this invention achieves a complete closed loop from the bottom-level physical point cloud acquisition to the high-level semantic recognition.

[0038] Furthermore, this embodiment performs in-depth preprocessing on the acquired raw data to ensure that the data input to the deep learning model possesses high-quality physical feature representation. Firstly, for the compressed packages stored as "point cloud videos" from LiDAR data acquisition, this invention utilizes a specially designed storage module to extract and save each frame of the point cloud file in real time during point cloud data playback. Considering that during the mobile acquisition process of LiDAR, it is affected by environmental occlusion, signal loss, or data transmission fluctuations, and that single-frame point cloud projections often exhibit pixel information loss or voids when projected as a two-dimensional image, this embodiment introduces a time-series-based buffered image refresh mechanism.

[0039] See the instruction manual appendix Figure 2 The system continuously maintains a global buffer image. When a new frame of point cloud image arrives, the system does not output it directly, but instead iterates through every pixel coordinate point of that frame. For valid pixels that are not missing, the system extracts their corresponding reflectivity or signal strength information and refreshes the corresponding pixels in the buffer image in real time. If a hole or missing pixel appears at a certain position in the new frame, the historical pixel information of that position in the buffer image is retained until the corresponding valid information arrives in a subsequent new frame before being refreshed. Through this incremental buffer update logic, the system can synthesize single-frame images that are complete and without holes, effectively solving the problem of feature loss caused by packet loss.

[0040] After obtaining a complete single-frame image, since the physical values ​​of reflectivity and signal intensity output by the LiDAR typically exceed the 8-bit (0-255) grayscale space range commonly used in standard convolutional neural networks, this embodiment further designs a linear normalization method. The system locks the dynamic range of the physical attribute values ​​in the current image, that is, identifies their maximum and minimum values, and strictly calculates the mapping threshold within each integer segment according to (maximum value - minimum value) / 256 as a single-segment quantization interval. Through this linear mapping relationship, the original physical attribute values ​​are accurately converted into 8-bit grayscale values ​​within the range of 0-255, thereby generating reflectivity grayscale maps and signal intensity grayscale maps that conform to the neural network input specifications. This normalization method not only eliminates factory differences between different sensors, but also significantly enhances the contrast between the leakage area and the dry background through dynamic interval locking.

[0041] In constructing the pipeline leakage dataset, this embodiment performs multi-dimensional optimization on the processed two-dimensional image sequence. First, the system performs frame sampling at a fixed time frequency to obtain a preliminary sample set with temporal representativeness. Subsequently, a deduplication algorithm is introduced to perform similarity analysis on the sampled images, eliminating redundant images caused by the detection carrier's pauses or repeated scanning, thereby improving the quality of the dataset from the source and preventing overfitting during model training.

[0042] For the high-quality samples after deduplication, this embodiment performs a manual annotation process. Since the reflectance map and signal intensity map originate from the synchronous deconstruction of the same frame of point cloud data, they have strict pixel-level alignment characteristics in spatial dimension. Annotators use bounding boxes to accurately mark the location of the leakage area in the image, generating location annotation box information. This annotation information serves as the core basis for supervised learning, and together with the aforementioned calibrated and normalized dual-channel grayscale image, constitutes the pipe and tunnel leakage dataset used to train the improved YOLO v5s model.

[0043] This embodiment employs an improved YOLOv5 model based on a cross-attention mechanism to perform the detection task. The underlying architecture is the YOLOv5 algorithm framework released by the Ultralytics team in 2020. This framework boasts excellent industrial applicability due to its end-to-end detection, balance between speed and accuracy, and highly engineered design. Specifically, the algorithm framework follows the classic "Backbone-Neck-Head" paradigm: 1. The backbone network (Backbone) uses a CSPDarknet structure, extracting semantic and spatial features of different scales from the original input 2D image through integrated C3 modules, fast spatial pyramid pooling (SPPF), and other components. 2. The neck network (Neck) is based on the Path Aggregation Network (PANet) architecture, achieving multi-scale feature fusion through upsampling, feature concatenation (Concat), and downsampling operations to adapt to the detection needs of targets of different sizes. 3. The head (Head) completes the final prediction of target category, bounding box coordinates, and confidence score through the Detect layer.

[0044] Furthermore, to accommodate the deployment needs of edge devices such as tunnel inspection robots, this embodiment specifically selects the lightweight version YOLOv5s from the YOLOv5 series (the "s" stands for small, a lightweight version within the YOLOv5 framework. The YOLOv5 series includes four versions: s / m / l / x, with the core differences being the depth multiplier, which controls the number of repetitions of the C3 module in the Backbone / Neck, and the width multiplier, which controls the number of feature channels in each layer). This version achieves an optimal balance between performance and speed through specific parametric design, with its depth multiplier (depth_multiple) set to 0.33 and its width multiplier (width_multiple) set to 0.50. The advantages of choosing YOLOv5s as the base model are that it has the smallest depth and width coefficients, only about 7M parameters, the fastest inference speed on GPU (FPS over 100), an accuracy of about 82% mAP@0.5 on the COCO dataset, and low GPU memory usage (training only requires 4-8G of GPU memory). It is suitable for real-time detection scenarios such as security monitoring, autonomous driving edge computing, and drone aerial photography, as well as resource-constrained devices such as mobile devices and smart cameras.

[0045] This invention embeds a cross-attention module in the middle of the YOLO v5s backbone. By comparing the reflectance and signal intensity channels of the same image, it achieves feature fusion between two two-dimensional visual feature maps (input streams). The fused features are then incorporated into the lower half of the backbone and the detection head, enabling more accurate identification of leakage areas. For the specific network architecture, please refer to the appendix of the specification. Figure 3 As shown.

[0046] First, in the backbone network, the model receives dual input stream data: a preprocessed and normalized reflectance map and a signal intensity map. The backbone network adopts a CSPDarknet structure, consisting of multiple functional modules stacked hierarchically. Specifically, the input image first passes through an initial convolutional layer (Conv), followed by a series of feature extraction layers composed of C3 modules. Notably, in this embodiment, a CrossAttentionBlock is implanted at the P3 / 8 scale of the backbone network, corresponding to the critical layer before high-level image feature compression. This module is configured in the middle of the backbone network, between the front C3 modules and the rear C3 modules and the Fast Spatial Pyramid Pooling (SPPF) component. This layout ensures that after the model extracts the initial features, it can immediately perform in-depth comparison and fusion of physical features across different channels.

[0047] Secondly, in the neck network portion, this embodiment retains the classic PANet architecture of YOLOv5s. This part includes upsampling layers, feature concatenation layers, and convolutional layers. By extracting feature maps from different layers of the backbone network (including P3 / 8 scale and deeper layers), multi-scale upsampling and downsampling fusion is performed in the neck network. This multi-scale path aggregation design can combine high-level semantic information with low-level spatial information that incorporates cross-attention features, thereby enhancing the model's ability to represent leakage regions of different sizes.

[0048] Finally, in the detection head section, the model outputs through the Detect layer. This detection layer, based on the fused features from the neck network, performs target category prediction (such as leakage area), bounding box coordinates, and confidence calculation. Due to the introduction of a cross-attention mechanism at the P3 / 8 scale of the backbone network (which is typically responsible for capturing small and medium-sized targets), the model achieves extremely high accuracy in capturing subtle leakage features commonly found in pipe tunnels.

[0049] This is based on the instruction manual. Figure 3 The improved architecture shown retains the lightweight advantages of the original YOLOv5s parameters depth_multiple (0.33) and width_multiple (0.50), while achieving lightweight performance enhancement of the original algorithm by embedding a module for physical channel fusion in the backbone network.

[0050] See the instruction manual appendix for further details. Figure 4 This embodiment implements a cross-attention module specifically for dual-input stream visual feature fusion. This module captures long-distance dependencies of features through multi-head attention and combines residual connections and position embedding techniques to achieve deep cross-fusion of two-dimensional visual feature streams. The cross-attention module mainly consists of two core classes (both inheriting from nn.Module): CrossAttention as the basic cross-attention layer, and CrossAttentionBlock as a cross-attention block that encapsulates multiple basic layers and adapts to two-dimensional visual feature maps.

[0051] The basic cross-attention layer, during the initialization of the CrossAttention class, receives the number of input / output channels *c* and the number of attention heads *num_heads*. Internally, this class includes an unbiased linear projection layer that maps the input stream *x* to the query vector *q*, and the input stream *y* to the key vector *k* and the value vector *v*. Furthermore, this class also includes a multi-head attention mechanism (nn.MultiheadAttention) and a feedforward network (FFN) consisting of two unbiased linear layers.

[0052] During the forward propagation, this layer first performs QKV linear projection on the two inputs x (query) and y (key / value), and realizes feature interaction across the input stream through a multi-head attention mechanism. After computation, the module combines the attention output with the original input x using residual connections, then performs feature transformation through a feedforward network, and completes the final feature processing through a second residual connection, outputting the fused feature tensor.

[0053] The CrossAttentionBlock class, a 2D visual feature fusion block, is designed for 2D visual feature maps (B, C1, H, W). During initialization, this class can flexibly configure convolutional layers based on whether the input and output channels are consistent (c1 and c2) to achieve channel adaptation. Simultaneously, this class constructs learnable positional embedding linear layers and a list of a specified number of CrossAttention layers.

[0054] During its forward propagation, the following steps are performed: (1) Channel adaptation: If the number of channels in the input feature map does not match the preset, channel adaptation is performed first. (2) Spatial dimension flattening and format conversion: The spatial dimension of the two-dimensional feature map is flattened and the dimension is replaced with the sequence format (H×W, B, C) required by the multi-head attention mechanism. (3) Position embedding supplement: Learnable position embedding information is added through linear layers and residual connections to compensate for the loss of spatial information caused by the flattening operation and restore the spatial structure. (4) Multi-round cross-fusion: The feature sequence passes through all CrossAttention layers in the layer list in turn to complete multi-round deep cross-attention fusion. (5) Dimension restoration: The processed sequence format features are restored to the two-dimensional feature map format (B, C2, H, W) and output.

[0055] This module preserves the original feature information through residual connections and enhances the feature interaction capabilities between different physical channels (reflectivity and signal strength) using cross-attention. Due to its excellent encapsulation, this module can not only be embedded in the YOLOv5s model of this embodiment, but also be extended to other visual models such as object detection or multimodal fusion.

[0056] To more intuitively understand the collaborative relationship between the two core classes at the algorithm level, please refer to the appendix of the instruction manual. Figure 4 It details the entire process from raw image input to fused feature output.

[0057] In the forward propagation process of CrossAttentionBlock, the details of the dimensional transformation after channel adaptation are as follows: After flattening the spatial dimensions, the system performs a permutation operation permute(2, 0, 1) to transform the tensor format, which was originally batch-based, into a format based on the sequence length (H × W), i.e., (H × W, B, C2). This is the physical prerequisite for adapting to standard multi-head attention computation. In the stage of adding position embeddings, such as... Figure 4 As shown, the input sequences are not directly added together. Instead, they are linearly mapped by self.linear1(x) and self.linear2(y) respectively, and then spatial location information is implanted by adding the residuals of x = x + self.linear1(x), which ensures the model's sensitivity to the spatial location of the leakage point.

[0058] Inside the core CrossAttention layer (see appendix) Figure 4 (Right subgraph) The computational details of the forward propagation are further broken down: First, the projection layer uses three independent projection operators self.q(x), self.k(y) and self.v(y) to simultaneously generate q, k, and v vectors. Then, the system calls the multi-head attention function self.ma(q, k, v)[0] for computation and explicitly specifies that only the feature output with index 0 is extracted.

[0059] In the feedforward network (FFN) processing stage after feature interaction, its internal cascaded logic includes: First residual connection layer: directly summing the output of the multi-head attention with the original input sequence x. Two-stage linear transformation: sequentially inputting the residual-fused sequence into the first linear transformation layer self.fc1(x) and the second linear transformation layer self.fc2(x) to achieve high-dimensional feature mapping and nonlinear transformation. Second residual connection layer: superimposing the output of the FFN with the result of the first residual connection layer, ultimately outputting a highly fused feature sequence x(seq_len, B, C) through this dual residual structure.

[0060] After completing the loop traversal and feature update of all self.ca_layers, the module enters the dimension restoration stage. The system first uses permute(1, 2, 0) to adjust the dimensional order of the feature sequence back to (B, C2, H × W), and finally calls the reshape function to reconstruct the spatial dimensions into a two-dimensional image matrix (B, C2, H, W).

[0061] Through precise control based on specific operators (such as permute, reshape, and residual addition), the cross-attention module of this embodiment can perform pixel-level nonlinear coupling between the reflectivity channel and the signal intensity channel of the lidar on the same spatial scale, thereby laying a solid feature foundation for the subsequent detection head to output accurate leakage location bounding boxes.

[0062] This invention constructs a highly efficient and accurate automated detection system for pipe and tunnel leakage by deeply integrating low-level physical sensing with high-level deep learning algorithms. Specifically, it includes: 1. This invention directly utilizes reflectivity channel data output from the lidar hardware, which is factory-calibrated and decoupled from distance and sensor differences. This strategy avoids the complex incident angle compensation calculations and data distortion caused by factory differences in multi-line receivers in traditional methods, providing a stable and reliable physical property benchmark for material identification in leakage areas.

[0063] 2. To address the unavoidable packet loss and voiding issues during point cloud acquisition, this invention implements a time-series-based buffered image refresh mechanism. By maintaining a global buffer and performing incremental pixel updates, complete feature restoration of a single frame image is achieved. Combined with a dynamic extremum locking linear normalization technique, lossless transformation of physical properties into deep learning input specifications is realized.

[0064] 3. This invention innovatively incorporates a cross-attention module based on the lightweight YOLOv5s architecture. This module achieves nonlinear deep fusion of dual-channel features by using query data derived from reflectivity and key / value pairs derived from signal strength, combined with learnable positional embeddings and a dual residual structure. This mechanism enables the model to capture long-distance dependencies between features, significantly improving the model's accuracy and robustness in capturing subtle leakage targets in complex backgrounds.

[0065] In summary, this invention, through comprehensive innovation in data source, data preprocessing, and feature fusion algorithms, effectively solves the pain points of existing technologies, such as strong dependence on manual parameters, poor environmental adaptability, and low detection accuracy, providing technical support with industrial-grade application value for the overall safety evaluation of long-distance tunnels.

[0066] The technical solutions of the present invention have been described in detail above with reference to specific embodiments. However, the above embodiments are only used to illustrate the technical concept of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or substitutions can be made to the technical solutions involved in the present invention without departing from the overall technical concept of the present invention. All equivalent substitutions, modifications, or improvements made to the present invention by those skilled in the art based on their understanding of the technical solutions of the present invention, as long as they do not depart from the technical concept and essence of the present invention, should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for detecting pipe and tunnel leakage based on lidar reflectivity, characterized in that, Includes the following steps: Acquire point cloud data of the tunnel collected by lidar and extract spatially aligned reflectivity feature stream and signal strength feature stream; The reflectivity feature stream and the signal intensity feature stream are input into the target detection model; Using the cross-attention module embedded in the target detection model, feature fusion is performed between the reflectivity feature stream and the signal intensity feature stream to form fused features; The detection results of the leakage area of ​​the pipe tunnel are output based on the fusion characteristics.

2. The method according to claim 1, characterized in that, The logic for performing feature fusion in the cross-attention module includes: The reflectivity feature stream is used as the query stream, and a query vector is generated using an unbiased linear projection layer. The signal intensity feature stream is used as a key / value stream, and a key vector and value vector are generated using an unbiased linear projection layer. The similarity weights between the query vector and the key vector are calculated using a multi-head attention mechanism and applied to the value vector to output the fused features.

3. The method according to claim 1 or 2, characterized in that, The cross-attention module is implanted at the P3 / 8 scale position of the backbone network of the object detection model.

4. The method according to claim 1 or 2, characterized in that, Before performing feature fusion, the cross-attention module further includes the following steps: Flatten the two-dimensional feature map into a sequence format and perform dimension permutation; Learnable positional embedding information is added to the feature sequence by combining linear layers with residual connections.

5. The method according to claim 1 or 2, characterized in that, Before inputting the feature stream into the target detection model, a linear normalization process is also performed, including: The dynamic extreme values ​​of physical attribute values ​​are statistically analyzed and the quantization interval step size is calculated. The original physical attribute values ​​are then linearly mapped to an 8-bit grayscale feature space.

6. The method according to claim 2, characterized in that, The cross-attention module employs a dual residual fusion structure: the attention output is connected to the original query stream through a first-layer residual connection; the features transformed by the feedforward network consisting of two unbiased linear layers are connected to the results of the first-layer residual connection through a second-layer residual connection.

7. The method according to claim 1 or 2, characterized in that, When preprocessing the point cloud data, a buffered image refresh mechanism is executed: Maintain a global buffer image, identify valid pixels in the current frame and refresh the pixel values ​​at the corresponding coordinates in the buffer image, and retain the historical pixel information in the buffer image for pixels missing in the current frame.

8. The method according to claim 1 or 2, characterized in that, The cross-attention module performs channel adaptation during forward propagation: Based on whether the number of input channels of the reflectivity feature stream and the signal intensity feature stream are consistent, the convolutional layer is automatically configured to perform dimension alignment.

9. The method according to claim 1 or 2, characterized in that, Constructing the dataset for training the object detection model includes: Frame sampling is performed at a fixed frequency, redundant image samples are removed using a deduplication algorithm, and leakage areas are simultaneously marked on spatially aligned reflectance and signal intensity maps.

10. A pipe and tunnel leakage detection system based on lidar reflectivity, used to perform the detection method as described in any one of claims 1-9, comprising: Data acquisition module: used to acquire point clouds of the tunnel using a mobile carrier equipped with a lidar, and output mutually aligned reflectivity data and signal strength data; Data preprocessing module: used to perform operations such as refreshing the buffered image, linear normalization, or sampling frames at a fixed frequency and removing redundant image samples using a deduplication algorithm; Feature fusion module: This is a cross-attention module embedded in the backbone network of the target detection model, used to perform feature fusion between the reflectivity feature stream and the signal intensity feature stream; Detection output module: used to identify leakage targets based on the fused features and output detection result annotation boxes.

Images