A method for detecting water leakage through a tunnel based on heterogeneous computing and physical perception

Abstract

The application relates to the technical field of underground traffic infrastructure intelligent operation and maintenance and monitoring, and discloses a tunnel leakage water detection method based on heterogeneous computing and physical sensing, which comprises the following steps: obtaining laser radar point cloud data and visible light image data collected by an inspection robot, constructing a positioning task flow and a disease identification task flow, and performing asynchronous alignment on the two task flows based on timestamps; performing enhancement processing on the visible light image to obtain an enhanced image; inputting the enhanced image into a leakage water identification model, extracting image features and performing global context modeling, and outputting a leakage water pixel-level binary mask; and based on camera calibration parameters, extrinsic parameter between the camera and the laser radar, and robot pose, performing space back projection and coordinate transformation on the binary mask combined with laser radar point cloud depth information to generate a tunnel leakage water three-dimensional semantic model. The method is suitable for leakage water identification and spatial positioning in a tunnel environment.

Description

Technical Field

[0001] This application relates to the field of intelligent operation and maintenance and monitoring technology for underground transportation infrastructure, such as a method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing. Background Technology

[0002] As a vital artery for cross-sea transportation, the operational safety of undersea subway tunnels is of paramount importance. However, due to high water pressure, geological subsidence, and the deterioration of concrete materials, the tunnel lining surface is highly susceptible to water leakage. Water leakage not only leads to corrosion and deterioration of the lining concrete and other structural materials, causing concrete expansion and crack formation, thus reducing the structural strength of the underground tunnel, but long-term water accumulation can also damage electrical, lighting, and ventilation equipment, causing short circuits in the track circuits and seriously threatening train operation safety.

[0003] Currently, although automated data collection and identification using inspection robots equipped with cameras has become the mainstream trend, existing water leakage identification technologies still face multiple severe challenges in actual tunnels or complex underground environments.

[0004] First, due to the unique characteristics of underground enclosed spaces, acquired images often suffer from dim lighting, uneven distribution, and low contrast. Conventional image enhancement methods based on histograms or traditional Retinex theory, while brightening dark areas, easily amplify sensor noise, leading to image distortion and severely interfering with subsequent feature extraction. Second, the surface environment of tunnel linings is extremely complex. Widespread cable and pipe shadows, oil stains, and efflorescence (white spots) are easily confused with seepage stains, making it difficult for traditional deep learning algorithms to effectively distinguish between background interference and defect features, resulting in a high false detection rate. Furthermore, seepage water often presents as thin, discontinuous flow marks under gravity, or is distributed laterally along the joints of tunnel segments. Existing standard convolution operations are limited by local receptive fields, making it difficult to capture such long-distance dependencies spanning large areas of space. This easily leads to broken and discontinuous water marks, making it impossible to accurately assess the actual scale of the seepage. Finally, existing high-precision models often rely on a large number of parameters and computational overhead, while edge devices such as tunnel inspection robots have limited computing power and power consumption, making it difficult to support large models for real-time inference and failing to meet the urgent need for rapid and accurate monitoring at engineering sites. In addition, existing technologies often adopt a "data-level fusion" strategy, forcibly synchronizing radar point clouds and image data into a massive multimodal network. This not only further exacerbates the computing burden but also requires extremely high time synchronization accuracy, resulting in poor system fault tolerance.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides a method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing, which adapts to the harsh imaging environment of undersea tunnels, has strong anti-interference capabilities, and improves detection accuracy.

[0008] In some embodiments, the method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical perception includes: S10, acquiring lidar point cloud data and visible light image data collected by an inspection robot, constructing a positioning task flow and a defect identification task flow, and asynchronously aligning the positioning task flow and the defect identification task flow based on timestamps; S20, enhancing the visible light image, extracting the initial features of the visible light image, performing feature evolution and illumination reconstruction on the initial features to obtain an enhanced image; S30, inputting the enhanced image into a water leakage identification model, extracting image features and performing global context modeling, and outputting a pixel-level binary mask for water leakage; S40, based on camera calibration parameters, extrinsic parameters between the camera and lidar, and robot pose, combining the binary mask with lidar point cloud depth information to perform spatial back projection and coordinate transformation to generate a three-dimensional semantic model of tunnel water leakage.

[0009] In some embodiments, the underwater tunnel leakage detection system based on heterogeneous computing and physical perception includes: a task flow construction module, configured to acquire lidar point cloud data and visible light image data, construct a positioning task flow and a defect identification task flow, and asynchronously align the positioning task flow and the defect identification task flow based on timestamps; an image enhancement module, configured to enhance the visible light image to obtain an enhanced image; a defect identification module, configured to extract features and perform global context modeling on the enhanced image, and output a pixel-level binary mask for leakage; and a 3D fusion module, configured to perform spatial back projection and coordinate transformation on the binary mask combined with lidar point cloud depth information based on camera calibration parameters, extrinsic parameters between the camera and lidar, and robot pose, to generate a 3D semantic model of tunnel leakage.

[0010] The present disclosure provides a method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing, which can achieve the following technical effects:

[0011] 1. By extracting initial features, evolving features, and reconstructing the original visible light image, online image enhancement is achieved, providing high-quality input for subsequent feature extraction and improving detection accuracy.

[0012] 2. The refined leakage water identification model dynamically suppresses background noise and outputs the purified features, and performs global context modeling and outputs a high-precision pixel-level binary mask for leakage water. While ensuring the real-time nature of the inspection, it achieves pixel-level delineation of the disease outline, thereby further improving the detection accuracy.

[0013] 3. By using three-dimensional asynchronous projection technology, two-dimensional pixel-level semantic masks are combined with radar depth information to restore a spatial real-scene semantic model with physical attributes. This solves the problems of spatiotemporal misalignment between the positioning task flow and the defect identification task flow, as well as the inability to quantify defects, and provides refined data support for the intelligent operation and maintenance of tunnels.

[0014] 4. The SCI++ self-calibrating illumination enhancement module with physical perception is used to enhance the original visible light image online. A compact residual architecture and MobileNetV3-Large based on depth-separable convolution are used as the encoder backbone network. These are matched with the computing power and actual constraints of the robot edge, which can reduce the computing power and memory occupation of the algorithm on the quadruped robot's onboard terminal and ensure the real-time performance of the inspection process.

[0015] 5. By constructing a joint architecture of heterogeneous computing and physical perception, this method achieves strong anti-interference capabilities in multiple dimensions: In the environmental perception dimension, it overcomes the extreme low-light interference unique to underground enclosed spaces by utilizing a lighting self-calibration mechanism based on the Retinex physical model. Combined with gated bottleneck convolution (GBC) and the state space module (CSMamba) in the refined leakage water identification model, it performs dynamic background noise filtering and global semantic completion, dynamically suppressing complex background noise such as efflorescence crystallization and pipeline shadows, and effectively repairing discontinuous flow marks. In the system operation dimension, it effectively eliminates data synchronization and spatiotemporal misalignment interference caused by the non-equivalent sampling frequencies of radar and camera during the high-dynamic inspection process of the quadruped robot using an asynchronous spatiotemporal alignment mechanism based on a sliding time window. Considering the above anti-interference designs at both the environmental and system levels, this method can still accurately distinguish between disease features and background interference even under harsh conditions of dim lighting and complex backgrounds, greatly improving the robustness of the 3D real-scene semantic model generation. Therefore, it has strong anti-interference capabilities.

[0016] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0017] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0018] Figure 1 This is a schematic diagram of a method for detecting water leakage in a sea tunnel based on heterogeneous computing and physical sensing, provided in an embodiment of this disclosure.

[0019] Figure 2 This is an example image of the original visible light image of water seepage in the lining of a cross-sea tunnel, collected by a robot according to an embodiment of this disclosure;

[0020] Figure 3 This is a logical structure diagram of the SCI++ lighting enhancement module provided in this embodiment, which illustrates the reconstruction process of the residual lighting map;

[0021] Figure 4 This is a schematic diagram of the overall architecture of the improved lightweight hybrid vision network provided in the embodiments of this disclosure;

[0022] Figure 5 This is a detailed structural diagram of the gated bottleneck convolutional module (GBC) provided in the embodiments of this disclosure, illustrating the dual-branch gating mechanism;

[0023] Figure 6 This is an internal schematic diagram of the CSMamba module provided in this embodiment, which includes a four-way scanning and a CS attention unit;

[0024] Figure 7 This is a schematic diagram illustrating the principle of coordinate transformation and spatiotemporal alignment in asynchronous post-fusion of three-dimensional space provided in this embodiment of the disclosure;

[0025] Figure 8 This is a comparison chart of pixel-level segmentation accuracy of different methods provided in the embodiments of this disclosure on a dataset of seepage water in undersea tunnels;

[0026] Figure 9 This is a graph showing the intersection and union ratio of the validation sets of the baseline network provided in this embodiment before and after the introduction of the SCI++ module;

[0027] Figure 10 This is a graph showing the change in verification loss of the baseline network provided in this embodiment before and after the introduction of the SCI++ module;

[0028] Figure 11 This is a comparison of the image enhancement and water leakage segmentation effects of traditional U-Net and the method of this application in typical samples;

[0029] Figure 12This is a comparison of the absolute error of three-dimensional spatial projection between the traditional strongly coupled alignment method and the asynchronous sliding window alignment method of this application. Detailed Implementation

[0030] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0031] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0032] Unless otherwise stated, the term "multiple" means two or more.

[0033] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0034] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0035] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0036] Unless otherwise stated, in this document: and These represent the height and width of the image, respectively. This represents the original visible light image input. Represents the pixel coordinates in the image plane, where Horizontal pixel coordinates Vertical pixel coordinates; Indicates the coordinates of the camera's principal point; Represents a 3D point in the camera coordinate system. Represents a three-dimensional point in the world coordinate system; This indicates moving the point from the coordinate system Transform to coordinate system The homogeneous rigid body transformation matrix; `clamp` indicates element-wise multiplication, and `clamp` indicates clamping.

[0037] Combination Figure 1 As shown, this disclosure provides a method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing, including:

[0038] S10: Acquire the lidar point cloud data and visible light image data collected by the inspection robot, construct the positioning task flow and the disease identification task flow, and asynchronously align the positioning task flow and the disease identification task flow based on the timestamp.

[0039] S20, enhance the visible light image by extracting the initial features of the visible light image, performing feature evolution and illumination reconstruction on the initial features, and obtaining the enhanced image;

[0040] S30: Input the enhanced image into the water leakage identification model, extract image features and perform global context modeling, and output the pixel-level binary mask of the water leakage.

[0041] S40, based on camera calibration parameters, extrinsic parameters between the camera and LiDAR, and robot pose, combines a binarized mask with LiDAR point cloud depth information to perform spatial back projection and coordinate transformation, generating a three-dimensional semantic model of tunnel water leakage.

[0042] This method is a lightweight 3D recognition approach for edge-side water leakage in quadruped robots designed for undersea tunnel environments. It is implemented on a quadruped inspection robot platform with edge computing capabilities. Figure 2 As shown, the core onboard computing unit of this platform adopts a high-performance heterogeneous computing terminal (such as the NVIDIA Jetson Orin series), with computing power specifications including: an integer computing power of over 100 TOPS and corresponding tensor processing cores. The platform's supporting hardware includes: a 360°×90° hemispherical ultra-wide-angle LiDAR (sampling frequency typically 10Hz-20Hz) and a high-resolution visible light industrial camera (frame rate no less than 30fps).

[0043] During the detection process, S10 is executed first, which constructs the task flow of the four-legged inspection platform based on a heterogeneous parallel computing architecture:

[0044] At the software level, the computing resources of the edge computing unit are logically decoupled into two independent task flows: one is the localization task flow that processes LiDAR point cloud data, which is allocated to the multi-core CPU logic core to run the LiDAR SLAM algorithm and output the robot's three-dimensional spatial pose in the tunnel in real time. The second task is to process the disease identification task flow of visual images, which is allocated to the GPU tensor core to run a lightweight semantic segmentation network.

[0045] To address the data synchronization issue caused by inconsistent sampling frequencies between the two task streams, timestamps are used for asynchronous caching and alignment of the localization task stream and the disease identification task stream. Specifically, this includes:

[0046] S11, Establish a sliding time window based on a circular queue in memory, and use the timestamps of the acquired visual frame images as indexes;

[0047] S12, retrieve pose data corresponding to the visual frame timestamp within the sliding window; when there is no pose data corresponding to the timestamp within the sliding window, calculate the robot pose at that moment based on two adjacent pose data before and after the timestamp, thereby completing the asynchronous spatiotemporal alignment of the localization task flow and the recognition flow, and constructing a heterogeneous parallel task architecture that supports subsequent 3D mapping.

[0048] In this way, S10 constructs two independent task flows: the localization task flow uses the CPU logic core to run the LiDAR SLAM algorithm to calculate the robot's pose in real time, and the disease identification task flow uses the GPU tensor core to run a lightweight visual recognition model, thus ensuring that the navigation and detection tasks do not interfere with each other and run in real time. At the same time, by using timestamps to asynchronously cache and align the two task flows, the problem of data synchronization loss caused by inconsistent sampling frequencies between the two task flows is solved.

[0049] Then, S20 is performed, physically sensed SCI++ self-calibrated illumination enhancement processing:

[0050] First, a robot is used to collect raw visible light images of the tunnel (such as...). Figure 2 As shown in the figure, initial features are extracted from the original visible light image. These initial features are then subjected to feature evolution and reconstruction. While brightening the dark areas of the image using a weight-sharing mechanism, high-frequency random noise generated by the sensor is suppressed through physical smoothing constraints, resulting in an enhanced image. Specifically, this includes:

[0051] S21, combined Figure 3 As shown, the original visible light image of the tunnel will be obtained. Input to a pre-built ultra-lightweight lighting estimation network In order to adapt to the computing power limitations of the edge computing terminal of the quadrupedal inspection robot, this network... A compact residual architecture is adopted, containing only one input convolutional layer, one residual convolutional block with shared weights, and one output convolutional layer, with the number of feature channels uniformly set to 3. To adapt to the real-time requirements of edge computing, the input image first passes through the input convolutional layer and activation function to extract initial features. :

[0052] ,

[0053] In the formula, Represents the ReLU activation function. It is a convolution operator with a kernel size of 3×3 and 3 feature channels.

[0054] S22, subsequently, the initial features are input into a convolutional block with residual connection structures for feature evolution, resulting in deep residual features:

[0055] ,

[0056] This represents the deep residual characteristics after evolution through residual convolution blocks; This represents a residual convolutional block with shared weights. By employing residual connections, the vanishing gradient problem commonly found in extremely deep or shallow networks can be effectively avoided, ensuring sensitivity to low-light features.

[0057] S23, Based on this, the pixel-level illumination map is reconstructed through a residual illumination learning mechanism. Its mathematical expression is:

[0058] ,

[0059] In the formula, Indicates the output convolution operator; This indicates that the network output is mapped to... Activation function for the interval; Indicates amplitude limiting operation; A very small constant (value 0.0001) to prevent numerical overflow or division by zero.

[0060] S24, Finally, based on the Retinex physics model ( The inverse process (representing pixel-by-pixel multiplication) involves using the original visible light image as the dividend and performing pixel-by-pixel division with the reconstructed pixel-level illumination map. This process removes the illumination effects from the original visible light image and extracts the reflection component r, which characterizes the physical properties of the lining surface. This results in the enhanced image.

[0061] .

[0062] In this way, S20 uses a decoupled operation driven by a physical model and a weight-sharing mechanism to brighten the dark areas of the original image output by S10, completing the self-calibration denoising and brightening of the image within milliseconds. At the same time, it effectively suppresses high-frequency random noise generated by the sensor by utilizing the characteristics of total variational physical constraints. Compared with traditional histogram equalization or deep convolutional networks, this step achieves uniformity of light and shadow distribution through an extremely short inference path (inference time less than 1 ms) and completely preserves the gradient features of the seepage water marks and the edge of the pipe lining. This provides a high-quality feature source for the refined feature encoding in the subsequent step S30, effectively solving the technical problems of poor recognition accuracy under severe lighting interference and insufficient real-time inference performance at the edge.

[0063] Then, S30 is executed to segment the leakage defects using an improved lightweight semantic segmentation network:

[0064] Combination Figure 4 As shown, based on the MobileNetV3-UNet hybrid network architecture, MobileNetV3 is used as a lightweight backbone network to reduce computational overhead; GBC modules are embedded in the jump connection path to dynamically suppress complex background noise in tunnels; and CSMamba modules with four-way selective scanning capability are introduced to reconstruct long-distance global context semantic information, thereby constructing a refined water leakage identification model suitable for the edge operation of quadruped robots. Figure 4 It demonstrates the connection relationships between the encoder, decoder, and intermediate key modules.

[0065] The enhanced image is mapped to the pre-trained feature space of the refined leakage identification model. The model dynamically suppresses background noise and outputs the cleaned features. Global context modeling is performed, and a high-precision pixel-level binary mask of the leakage is output. Specifically, this includes:

[0066] S31, firstly, the enhanced image is normalized online, mapping the image pixel values ​​to the pre-trained feature space of the water leakage fine-grained identification model. To address the limited computing power of the quadruped robot's edge computing terminal, MobileNetV3-Large, based on depthwise separable convolution, is used as the encoder backbone network. The enhanced image output from step S20 is then processed... Online ImageNet normalization is performed to map the image pixel value distribution to the model's pre-trained feature space, as shown in the following formula:

[0067] ,

[0068] In the formula, This is the normalized feature map; The enhanced image; Represents the mean vector ; Represents the standard deviation vector The above subtraction and division operations are performed element-by-element on a channel-by-channel basis.

[0069] S32, subsequently, multi-scale features are extracted through multi-layer downsampling. Multi-scale features are extracted through five cascaded downsampling layers in the backbone network. ,in These correspond to scales ranging from 1 / 2 to 1 / 32 of the original image resolution. Among them, Indicates the first Skip connection feature maps output by each encoding stage .

[0070] S33 utilizes the bottleneck convolution operator to generate feature extraction branches and gated weight branches. Combined with... Figure 5 As shown, to suppress the interference of high-frequency noise such as efflorescence and cable shadowing on the defect features within the tunnel, a GBC module is embedded in each skip connection path. This module adopts a dual-branch bottleneck convolution structure and achieves adaptive feature calibration through a gating mechanism. First, the bottleneck convolution operator is used... Generate feature extraction branches respectively With gated weight branches :

[0071] ,

[0072] In the formula, This represents the original skip connection features at the current scale; This represents the disease characterization features output by the feature extraction branch; This represents the weight characteristics of the output of the gated weighted branch; and Represents the bottleneck convolution operator with different parameter configurations; This indicates a grouping normalization layer. S34, subsequently, gated interaction is achieved through element-wise multiplication, and the purified features are output using residual connections. :

[0073] ,

[0074] In the formula, This indicates the purification characteristics after gated noise suppression and residual compensation;

[0075] S35 will remove the purified characteristics Decomposed into signal branches and gated branches Combining Figure 6As shown, in order to further address the semantic discontinuity problem in long-distance flow trace recognition, this method integrates a CSMamba module with linear complexity at the key scale level of the decoder for global context modeling.

[0076] S36 derives features from four-way scan synthesis based on signal branches and obtains gate weights based on gated branches. The signal branches, after depthwise convolution and SiLU activation, enter the 2D-SSM core in the 2D state space, performing four-way parallel selective scans along the horizontal, vertical, and flip directions to capture long-range spatial dependencies across the entire image; simultaneously, the gated branches... The input is fed into the integrated CS-Attention (channel-spatial attention) module, which sequentially uses a channel attention mechanism guided by global averaging / max pooling and a spatial attention mechanism guided by large-size kernel convolution to generate a refined multi-dimensional weight mask.

[0077] The four-way parallel selective scanning includes four scanning directions: horizontal forward, horizontal reverse, vertical forward, and vertical reverse.

[0078] S37. Finally, the features synthesized by the four-way scan are fused with the attention-calibrated gated weights, and the reconstructed features are output through a linear projection layer. This process utilizes the recursive evolution characteristics of the state-space model to achieve a global receptive field similar to that of a Transformer, effectively solving the problem of identifying fractures caused by seepage marks traveling long distances under gravity. The relevant mathematical logic is as follows:

[0079] ,

[0080] In the formula, For the input of the aforementioned separated gated branches; These are the attention-calibrated gating weights; For channel attention operators; For spatial attention operators; This is a selective scan operation; Indicates the first Distance step length parameter in each scanning direction; These represent the state transition parameters, input mapping parameters, and output mapping parameters in the state-space model, respectively. These correspond to the four scanning directions: horizontal forward, horizontal reverse, vertical forward, and vertical reverse, respectively. This represents the output reconstruction features after four-way global context modeling and gating calibration.

[0081] S38, Finally, the reconstructed features output in step S37 are... The input is fed into the decoder, where it undergoes multiple upsampling operations to restore spatial resolution, and is then compared with the cleaned features at the corresponding scale. Cascaded fusion and double convolution processing are performed to obtain a high-resolution fused feature map; subsequently, this feature map is passed through a set of 1×1 convolutional layers to perform classification mapping, outputting a binarized water leakage mask:

[0082] ,

[0083] In the formula, Represents a pixel-level binary mask for water leakage; when When, it represents a pixel. The area was identified as a leaking area; when When the value is 0, it indicates that the pixel belongs to the background area.

[0084] In this way, S30 achieves pixel-level delineation of the disease outline while ensuring real-time inspection through lightweight coding of MobileNetV3, dynamic background noise filtering of GBC, and global semantic completion of CSMamba, so as to solve the technical problems of false detection of complex underground structure background interference and difficulty in modeling the spatial dependence of leakage water features in the undersea tunnel.

[0085] Finally, S40 is executed, performing asynchronous post-fusion of 3D space based on edge computing task decoupling:

[0086] By utilizing a joint calibration matrix and 3D spatial pose, a pixel-level binary mask of the seepage water is combined with radar point cloud depth information for reverse spatial projection. Through chained coordinate transformation, a 3D real-scene semantic model of the tunnel seepage water with physical size attributes is finally generated. Figure 7 As shown, it specifically includes:

[0087] S41, Construct a sliding time window based on a circular queue to store the robot's six-DOF pose sequence. First, addressing the inference delay issue in the recognition flow of the quadrupedal inspection robot during high-dynamic motion, this method constructs a sliding time window based on a circular queue to store the robot's six-DOF pose sequence:

[0088] ,

[0089] In the formula, This represents the robot pose sequence cached within the sliding time window; This represents the timestamp corresponding to each pose within the sliding time window; Indicates time The pose transformation matrix of the robot's body coordinate system relative to the world coordinate system.

[0090] S42, align the location task flow and the disease identification task flow in time and space under non-equivalent frequency conditions. If the timestamp output by S30 is... Binarization mask The system then searches for matches within the sliding time window. The corresponding pose data; when there is a positional data within the sliding time window, the corresponding positional data is used. When the pose data is the same, directly use that pose data as the robot pose corresponding to the binarized mask; when there is no matching pose data... When the same pose data is used, based on the location Calculation of two adjacent pose data Robot pose at any given moment .

[0091] In some implementations, linear interpolation is performed on the translation components, and quaternion spherical linear interpolation is used to solve for the rotation components, thereby obtaining... The robot pose corresponding to each moment. This mechanism enables spatiotemporal alignment of the localization task flow and the disease identification task flow under non-equi-frequency conditions.

[0092] S43, using a pre-calibrated camera intrinsic matrix and camera-radar extrinsic matrix, projects the 3D depth point cloud acquired by the lidar onto a binarized mask plane in real time. After completing spatiotemporal alignment, the pre-calibrated camera intrinsic matrix... With camera-radar extrinsic matrix The three-dimensional depth point cloud collected by the lidar is projected onto the binary mask plane in real time to achieve pixel-level depth correlation.

[0093] In some implementations, the camera intrinsic parameter matrix It can be represented as:

[0094] ,

[0095] In the formula, Represents the camera intrinsic parameter matrix; This represents the extrinsic parameter matrix that transforms a 3D point in the lidar coordinate system to the camera coordinate system.

[0096] In some implementations, when multiple laser points are projected onto the same pixel, the effective depth value closest to the camera is selected as the depth value corresponding to that pixel; when the leaking pixel has no corresponding laser point, interpolation estimation is performed using the effective depth of the neighborhood, or a local surface fitting method is used to determine the depth value corresponding to that pixel.

[0097] S44, for any defect pixel in the binarized mask marked as water leakage and having a valid associated depth value. The three-dimensional spatial points of the perspective projection model in the camera coordinate system are calculated by inverse process of perspective projection model. :

[0098] ,

[0099] In the formula, d This refers to the depth value of the current pixel along the optical axis in the camera coordinate system, obtained through point cloud association; that is, the depth value of the 3D point in the camera coordinate system. Axis coordinate components; The coordinates of the camera's principal point; This refers to the camera's focal length parameter; These are the coordinate components of a 3D point along the three coordinate axes of the camera coordinate system.

[0100] The three-dimensional spatial points are represented as homogeneous coordinates in the camera coordinate system:

[0101] .

[0102] S45. Finally, the defect points are mapped to the tunnel's three-dimensional global coordinate system through a chain-like coordinate transformation. Below, the three-dimensional absolute physical coordinates corresponding to each defective pixel are calculated, and their mathematical expression is as follows:

[0103] ,

[0104] in, This is the installation extrinsic parameter matrix from the camera coordinate system to the robot body coordinate system; for The pose transformation matrix from the robot's body coordinate system to the world coordinate system at any given moment; These are homogeneous coordinates in the world coordinate system.

[0105] Since the chain coordinate transformation is a homogeneous rigid body transformation, the fourth component of the homogeneous coordinates remains 1 after the transformation. Therefore, we take... The first three components are used as three-dimensional coordinates in the world coordinate system:

[0106] .

[0107] Thus, after completing the asynchronous spatiotemporal alignment, the system will sequentially traverse the binarized seepage mask output in step S38. For each 2D pixel in the mask identified as a water leak, its absolute physical coordinates in the 3D global coordinate system are calculated using a chained coordinate transformation. The semantic label "water leakage" is directly assigned to the 3D spatial point. By performing this coordinate mapping and semantic empowerment operation on all defective pixels, this method completes cross-modal fusion from 2D image features to 3D point cloud data, and finally generates a 3D real-scene semantic model of tunnel water leakage with physical scale information and global coordinate position.

[0108] This disclosure presents a method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical perception. The method transforms the complex task of detecting tunnel water leakage into a joint optimization problem involving heterogeneous computing flow and physical perception feature extraction. By establishing an asynchronous spatiotemporal matching mechanism based on a sliding window, the localization task flow and the defect identification task flow in the dynamic inspection of a quadruped robot are decoupled, resolving data synchronization and resource contention issues in edge computing scenarios. An illumination self-calibration logic based on the Retinex physical model is established using the SCI++ module, transforming the severe lighting interference in the tunnel into decoupled mathematical component operations, improving image discernibility in low-light tunnel environments under low computing power conditions. A gated convolutional module (GBC) and a state-space model (CSMamba) are deeply embedded in the MobileNetV3 lightweight architecture. Background noise filtering is transformed into feature space weight calibration through a gated mask mechanism, and a linear complexity scanning algorithm is used to capture the long-distance spatial dependence of defect features, improving the identification effect of water leakage areas under complex background conditions. Finally, by combining the two-dimensional seepage mask with the depth information of the lidar point cloud for spatial back projection and coordinate transformation, the spatial positioning and three-dimensional semantic representation of the seepage area can be realized.

[0109] This method enables pixel-level automatic delineation and 3D spatial reconstruction of complex underground lining defects. While ensuring recognition accuracy and robustness, it significantly reduces the computational overhead and memory usage on the quadruped robot's onboard terminal, ensuring real-time performance during inspections. This method is suitable for inspecting undersea tunnels under high humidity, low light, and complex background conditions. It helps reduce interference from background factors such as cable shadows and efflorescence on leakage identification, providing quantifiable data on defect area, actual scale, and precise geographical location, thus supporting spatial distribution analysis, extent assessment, and subsequent maintenance decisions. Compared to traditional manual inspections or ordinary target detection algorithms, this method significantly reduces manual verification, avoids missed detections and false alarms, shortens maintenance decision-making cycles, and provides a solid technical guarantee for the construction of digital twins of tunnel structures and intelligent management throughout their entire lifecycle.

[0110] Comprehensive performance analysis:

[0111] To comprehensively verify the performance of the detection method proposed in this application, this embodiment compares it with traditional methods in two dimensions: recognition accuracy and anti-interference capability. Specific experimental verification results are as follows:

[0112] (1) Verification of recognition accuracy:

[0113] To verify the recognition effect of the proposed method in pixel-level segmentation of water leakage in the lining of undersea tunnels, this embodiment compares and analyzes the proposed method with U-Net, DeepLabV3+, PSPNet and YOLOv8n-seg under the same experimental conditions, based on a real-world dataset of water leakage in the lining of undersea tunnels. Figure 8 This image compares the pixel-level segmentation accuracy of different methods for leak water. The horizontal axis represents the evaluation metrics, and the vertical axis represents the metric values. The evaluation metrics include precision, recall, Dice coefficient, and intersection-over-union ratio. Figure 8 As can be seen, the method in this application outperforms the comparative methods in all evaluation metrics, indicating that the method can effectively improve the pixel-level segmentation accuracy of the water seepage defect area, thereby providing a reliable two-dimensional semantic mask for the subsequent construction of the 3D real-scene semantic model. Furthermore, the method in this application has 4.40 M parameters and 4.52 GFLOPs of computation, indicating that it has good lightweight deployment capabilities while ensuring segmentation accuracy.

[0114] To further verify the effect of the SCI++ module introduced in this application on improving the performance of the baseline segmentation network, this embodiment uses U-Net as the baseline network. While keeping other training conditions consistent, the performance of the verification process with and without the SCI++ module is compared and analyzed. The results are as follows: Figure 9 and Figure 10 As shown. Figure 9 To verify the curve of the intersection and union ratio of sets changing with the number of training rounds, Figure 10 To validate the curve of the set loss changing with training epochs. From... Figure 9 and Figure 10 It can be seen that after introducing the SCI++ module into the baseline network, the model's validation set intersection-union ratio improves faster in the early stages of training and remains at a high level in the later stages; simultaneously, its validation set loss decreases faster and stabilizes at a low level. These results indicate that the SCI++ module can effectively enhance the baseline network's ability to represent the characteristics of water leakage damage, improve the model's convergence stability and validation performance during the validation process, thereby providing support for improving the pixel-level segmentation accuracy of water leakage damage areas.

[0115] The above results show that the method of this application can not only learn the characteristics of water leakage more effectively, but also exhibits good convergence stability and performance retention during the verification stage, thereby providing support for improving the pixel-level segmentation accuracy of water leakage areas.

[0116] (2) Verification of anti-interference capability:

[0117] In response to the multiple interferences under the complex working conditions of undersea tunnels, this embodiment verifies the interference from two aspects: environmental physical interference and system operation interference.

[0118] To further verify the ability of the proposed method to characterize details and suppress noise interference in areas of water leakage, this embodiment selects typical samples from the dataset and compares the processing results of the traditional U-Net method with those of the proposed method. The results are as follows: Figure 11 As shown. Figure 11 In the image, from left to right, are the original image, the segmentation result of the traditional U-Net method, the image enhanced by the method of this application, and the segmentation result of the method of this application. As can be seen from the figure, the traditional U-Net method exhibits certain false positives and false negatives in dark areas, and noise appears in localized regions. In contrast, the method of this application effectively improves the discriminability of dark areas, resulting in a more complete extraction of the water seepage damage area, while reducing irrelevant noise interference. This demonstrates that the method of this application can improve the detection capability of water seepage damage under low-light conditions and enhance the stability and reliability of the segmentation results.

[0119] To address the spatial misalignment problem caused by the non-equivalent sampling frequencies of cameras and radars in edge heterogeneous computing devices, this embodiment further verifies the effect of asynchronous spatiotemporal alignment mechanism on improving the accuracy of three-dimensional spatial projection. Figure 12 This is a comparison chart of the absolute errors of 3D spatial projection between the traditional strongly coupled alignment method and the asynchronous sliding window alignment method of this application. The horizontal axis represents the robot's inspection speed, and the vertical axis represents the absolute error of 3D spatial projection. Figure 12 It can be seen that when using the traditional strongly coupled alignment method, as the robot inspection speed increases, the timing deviation caused by non-uniform frequency sampling between the positioning task flow and the defect identification task flow gradually accumulates, leading to a significant increase in the absolute error of 3D spatial projection. This error further increases, especially after the inspection speed reaches approximately 1.5 m / s. However, by using the asynchronous sliding window alignment method of this application, the absolute error of 3D spatial projection remains at a low level at different inspection speeds, and the overall change is more stable. The above results indicate that the method of this application can effectively reduce the spatiotemporal misalignment error caused by asynchronous sampling of multi-source heterogeneous sensors, and improve the stability, accuracy, and reliability of 3D real-scene modeling under dynamic inspection conditions.

[0120] Through the above Figures 8 to 12 The experimental results show that the method proposed in this application is superior to traditional methods in terms of recognition accuracy, anti-interference ability, and stability of three-dimensional physical model generation. It can meet the application requirements of high-precision, robust and quantifiable leakage detection in complex working conditions of undersea tunnels.

[0121] This disclosure provides a device for detecting water leakage in undersea tunnels that integrates heterogeneous computing and physical sensing. The device includes a task flow construction module, an image enhancement module, a defect identification module, and a 3D fusion module. The task flow construction module is configured to acquire lidar point cloud data and visible light image data, construct a positioning task flow and a defect identification task flow, and asynchronously align the positioning task flow and the defect identification task flow based on timestamps. The image enhancement module is configured to enhance the visible light image to obtain an enhanced image. The defect identification module is configured to extract features and perform global context modeling on the enhanced image, outputting a pixel-level binary mask for water leakage. The 3D fusion module is configured to generate a 3D semantic model of tunnel water leakage by combining the binary mask with lidar point cloud depth information for spatial back-projection and coordinate transformation based on camera calibration parameters, extrinsic parameters between the camera and lidar, and robot pose.

[0122] The specific implementation process of the device can be found in the description of the above method embodiments, and will not be repeated here.

[0123] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0124] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing, characterized in that, include: S10: Acquire the lidar point cloud data and visible light image data collected by the inspection robot, construct the positioning task flow and the disease identification task flow, and asynchronously align the positioning task flow and the disease identification task flow based on the timestamp. S20, the visible light image is enhanced by extracting the initial features of the visible light image, performing feature evolution and illumination reconstruction on the initial features to obtain the enhanced image; S30, input the enhanced image into the water leakage identification model, extract image features and perform global context modeling, and output the pixel-level binary mask of the water leakage. S40, based on camera calibration parameters, external parameters between camera and lidar, and robot pose, the binarized mask is combined with lidar point cloud depth information for spatial back projection and coordinate transformation to generate a three-dimensional semantic model of tunnel seepage. Wherein, S30 includes: S31, perform online normalization processing on the enhanced image, and map the image pixel values ​​to the pre-trained feature space of the water leakage fine recognition model; S32 extracts multi-scale features through multi-layer downsampling, and extracts multi-scale features through cascaded downsampling layers in the backbone network; S33, based on the multi-scale features, a feature extraction branch and a gated weight branch are generated using a bottleneck convolution operator. A GBC module is embedded in each skip connection path. The GBC module adopts a dual-branch bottleneck convolution structure and achieves adaptive calibration of features through a gating mechanism, generating a feature extraction branch and a gated weight branch respectively. S34, based on the feature extraction branch and the gate weight branch, gated interaction is achieved through element-level multiplication, and the purified features are output using residual connection; S35 decomposes the purified features into signal branches and gating branches; S36, based on the signal branch, the features synthesized by the four-directional scan are obtained, and based on the gated branch, the gated weights are obtained; the signal branch, after being processed by depthwise convolution and SiLU activation, enters the two-dimensional state space core 2D-SSM, and performs four-directional parallel selective scanning along the horizontal, vertical, and flip directions; simultaneously, the gated branch is input to the integrated CS-Attention module, which sequentially uses a channel attention mechanism guided by global averaging / max pooling and a spatial attention mechanism guided by large-size kernel convolution to generate a refined multi-dimensional weight mask; wherein, the four-directional parallel selective scanning includes four scanning directions: horizontal forward, horizontal reverse, vertical forward, and vertical reverse. S37 fuses the features synthesized by the four-way scan with the attention-calibrated gated weights, and outputs the reconstructed features through a linear projection layer; S38, the reconstructed features are input into the decoder, and the spatial resolution is restored after multi-layer upsampling operations. The features are then cascaded and fused with the cleaned features of the corresponding scale, and double convolution processing is performed to obtain a high-resolution fused feature map. The high-resolution fused feature map is then classified and mapped through a set of 1×1 convolutional layers to output the binarized mask.

2. The method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing according to claim 1, characterized in that, The asynchronous alignment of the positioning task flow and the disease identification task flow based on timestamps includes: A sliding time window based on a circular queue is constructed to store robot pose data, and the timestamps of the acquired visible light image frames are used as indexes; Within the sliding time window, the robot pose data corresponding to the visible light image frame is retrieved and interpolated to complete the asynchronous alignment of the localization task flow and the disease identification task flow.

3. The method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing according to claim 1, characterized in that, The enhancement process for the visible light image, which involves extracting initial features from the visible light image, performing feature evolution and illumination reconstruction on the initial features to obtain the enhanced image, includes: The visible light image is input into a pre-constructed illumination estimation network, and the initial features are extracted through convolutional layers and activation functions; The initial features are subjected to residual feature evolution to obtain deep residual features; Pixel-level illumination maps are reconstructed through a residual illumination learning mechanism; The enhanced image is obtained by reversing the process based on the Retinex physical model and recovering the reflection components of the lining surface from the pixel-level illumination map.

4. The method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing according to claim 1, characterized in that, The process involves using camera calibration parameters, extrinsic parameters between the camera and the lidar, and the robot pose to perform spatial back projection and coordinate transformation on the pixel-level binarized mask combined with the lidar point cloud depth information, generating a three-dimensional semantic model of tunnel water leakage. This includes: Using a pre-calibrated camera intrinsic parameter matrix and camera-radar extrinsic parameter matrix, the three-dimensional depth point cloud acquired by the lidar is projected onto the binarized mask plane, and the depth value corresponding to the water seepage pixel in the binarized mask is determined. For the pixels marked as water leakage and having a valid associated depth value in the binarized mask, their three-dimensional spatial points in the camera coordinate system are calculated by the inverse process of the perspective projection model. By using chain-like coordinate transformation, the defect points are mapped to the tunnel's three-dimensional global coordinate system to obtain the absolute physical coordinates of the defect point cloud; The absolute physical coordinates are assigned the semantic label "water leakage" to generate the three-dimensional semantic model of the tunnel water leakage.

5. The method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing according to claim 2, characterized in that, The step of retrieving and interpolating the robot pose data corresponding to the visible light image frame includes: When S30 outputs a timestamp When performing binarization masking, search for matches within the sliding time window. The corresponding pose data; When there is no timestamp corresponding to the visible light image frame When the same pose data is used, based on the location Calculation of two adjacent pose data The robot pose at any given moment.

6. The method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing according to claim 4, characterized in that, Determining the depth value corresponding to the water seepage pixel in the binarized mask includes: When multiple laser points are projected onto the same pixel, the effective depth value closest to the camera is selected as the depth value corresponding to that pixel. When there is no corresponding laser point for a pixel marked as a water leak, interpolation estimation is performed using the effective depth of the neighborhood, or a local surface fitting method is used to determine the depth value corresponding to the pixel.

7. A water leakage detection system for undersea tunnels based on heterogeneous computing and physical sensing, characterized in that, The method for detecting water leakage in undersea tunnels based on heterogeneous computing and physical sensing, as described in any one of claims 1 to 6, comprises: The task flow construction module is configured to acquire lidar point cloud data and visible light image data, construct a positioning task flow and a disease identification task flow, and asynchronously align the positioning task flow and the disease identification task flow based on timestamps. The image enhancement module is configured to enhance the visible light image to obtain an enhanced image; The disease identification module is configured to perform feature extraction and global context modeling on the enhanced image and output a pixel-level binary mask for the leakage water. The 3D fusion module is configured to generate a 3D semantic model of tunnel water leakage by combining the binarized mask with the depth information of the LiDAR point cloud based on the camera calibration parameters, the extrinsic parameters between the camera and the LiDAR, and the robot pose, and performing spatial back projection and coordinate transformation.

Images