Tunnel lining fine crack detection method and system based on data fusion

Abstract

The invention discloses a tunnel lining fine crack detection method and system based on data fusion, and relates to the technical field of tunnel disease detection. The method comprises the following steps: acquiring three-dimensional point cloud data and images of a tunnel lining, and preprocessing the three-dimensional point cloud data and images; performing preliminary feature screening on the image by using a deep learning model to obtain a suspected crack region image; performing densification processing on the three-dimensional point cloud data to obtain high-density depth information, and obtaining a depth map; analyzing spatial structure perception provided by the depth map, and fusing the suspected crack region image with the depth map by using a data matching algorithm to obtain fused data; and carrying out fusion confidence coefficient calculation on the fused edge data, and judging a crack detection result according to the fusion confidence coefficient. According to the invention, high-precision depth information of laser is combined with abundant visual features of the camera, and detection difficulties such as fine cracks, environmental noise interference and changeable crack forms are fully handled.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel disease detection, and in particular to a method and system for detecting fine cracks in tunnel lining based on data fusion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The main tunnel diseases include: lining cracks, tunnel water damage, tunnel frost damage, earthquake damage and fire, etc. Among them, lining cracks are one of the more typical and common tunnel diseases, which are mostly manifested in lining cracks. Therefore, tunnel lining crack detection is one of the most difficult items in traffic engineering quality inspection.

[0004] In tunnel lining crack detection, accurate identification and positioning of targets are crucial to ensure structural safety and extend service life. However, actual detection work faces multiple challenges. First, cracks are usually very fine and have low contrast with the overall color and texture of the lining, making it difficult to clearly distinguish their edges in the image; this problem is particularly prominent in traditional image processing algorithms. Secondly, the complex environmental conditions in the tunnel, such as low light, high humidity, dust and impurities, may lead to increased image noise collected by the camera, and also have an adverse effect on the signal stability of auxiliary sensors such as laser or ultrasonic waves. Finally, the morphology of cracks is varied, and they may not only present small, straight features, but also have bifurcated, intertwined or irregular edges. This complexity brings great difficulty to automated detection and model fitting.

[0005] Based on the above problems, how to achieve high-precision and robust crack detection through sensor data technology under conditions of low contrast, harsh environment and complex crack morphology has become a key technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for detecting fine cracks in tunnel linings based on data fusion, which combines the high-precision depth information of laser with the rich visual features of the camera to fully address the detection difficulties such as fine cracks, environmental noise interference and changeable crack morphology.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions: A first aspect of the present invention provides a method for detecting fine cracks in tunnel lining based on data fusion, comprising the following steps: Obtain 3D point cloud data and images of tunnel lining and perform preprocessing; Use the deep learning model to perform preliminary feature screening on the image to obtain images of suspected crack areas; Densify the 3D point cloud data to obtain high-density depth information and obtain a depth map; Analyze the spatial structure perception provided by the depth map, and use the data matching algorithm to fuse the image of the suspected crack area with the depth map to obtain fused data; The fusion confidence of the fused edge data is calculated, and the crack detection result is determined according to the fusion confidence.

[0008] Furthermore, a laser scanner is used to collect three-dimensional point cloud data, and a camera is used to obtain images.

[0009] Furthermore, the specific steps of preprocessing include: The intrinsic parameter matrix of the camera and the extrinsic parameter matrix between the laser scanner and the camera are obtained by calibration; The calibrated parameters are used to project the 3D point cloud data collected by the laser scanner onto the camera image plane to achieve spatial alignment of data from different modalities.

[0010] Furthermore, the specific steps of using the deep learning model to perform preliminary feature screening on the image are: The confidence threshold reduction method is adopted to extract the suspicious areas, establish the confidence distribution based on the detection, analyze the suspicious areas according to the confidence values, and apply dynamic filtering strategies to different categories of suspicious areas; At the same time, a two-dimensional bounding box of each suspicious area is obtained to obtain an image of the suspected crack area.

[0011] Furthermore, the specific steps for densifying the 3D point cloud data are as follows: Construct Poisson's equation based on 3D point cloud data; Solve Poisson's equation to reconstruct the surface and obtain a continuous depth map; Based on the spatial alignment of data from different modalities, the areas corresponding to the questionable areas of the image in the generated depth map are extracted to obtain high-density depth information.

[0012] Furthermore, the spatial structure perception provided by the depth map is analyzed, and the specific steps of fusing the suspected crack area image with the depth map using the data matching algorithm are as follows: Using an edge detection algorithm to detect the suspected crack area image, and obtain a first detection result; By calculating the gradient depth between the pixels of the depth map, a set of points with significantly discontinuous depth values ​​is obtained as a second detection result; A data matching algorithm is used to perform cross-modal data matching and fusion on the first detection result and the second detection result to obtain fused data.

[0013] Furthermore, the specific steps of using edge detection algorithm to detect the image of suspected crack area are as follows: Calculate the local contrast of the image of the suspected crack area; Through the spatial structure perception provided by the depth map, combined with the detailed features of the image grayscale distribution, the depth gradient is calculated using the depth map, and the depth gradient is normalized to obtain the enhancement factor; The local contrast of the image is weighted by the enhancement factor to obtain an enhanced image; An edge detection algorithm is used to perform edge detection on the enhanced image to obtain a first detection result.

[0014] The second aspect of the present invention provides a tunnel lining fine crack detection system based on data fusion, comprising: A data acquisition module is configured to acquire three-dimensional point cloud data and images of the tunnel lining and perform preprocessing; A preliminary screening module is configured to perform preliminary feature screening on the image using a deep learning model to obtain an image of a suspected crack area; A depth information extraction module is configured to perform densification processing on the three-dimensional point cloud data to obtain high-density depth information and obtain a depth map; A data fusion module is configured to analyze the spatial structure perception provided by the depth map, and fuse the image of the suspected crack area with the depth map using a data matching algorithm to obtain fused data; The crack detection module is configured to calculate the fusion confidence of the fused edge data and determine the crack detection result according to the fusion confidence.

[0015] The third aspect of the present invention provides a medium having a program stored thereon, which, when executed by a processor, implements the steps of the method for detecting fine cracks in tunnel lining based on data fusion as described in the first aspect of the present invention.

[0016] The fourth aspect of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for detecting fine cracks in tunnel lining based on data fusion as described in the first aspect of the present invention are implemented.

[0017] One or more of the above technical solutions have the following beneficial effects: The present invention discloses a method and system for detecting fine cracks in tunnel lining based on data fusion, including ROI coarse extraction and depth map generation process, extracting data from two different modes, namely three-dimensional point cloud data and camera image, and enhancing crack features by edge and discontinuous point detection methods. Finally, accurate identification of tunnel lining cracks is achieved through multimodal data fusion and confidence calculation strategy, providing a scientific basis for subsequent structural health assessment and maintenance.

[0018] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 This is a flow chart of a method for detecting fine cracks in tunnel lining based on data fusion in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0021] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0022] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or their combinations; Embodiment 1: Embodiment 1 of the present invention provides a method for detecting fine cracks in tunnel lining based on data fusion, such as Figure 1 As shown, the following steps are included: Step 1: Obtain the 3D point cloud data and images of the tunnel lining and perform preprocessing.

[0023] Step 1.1: In a specific implementation, the system first configures a laser scanner and a camera with sensor calibration completed on the tunnel inspection vehicle, collects three-dimensional point cloud data using the laser scanner, and acquires images using the camera.

[0024] Step 1.2: Preprocess the acquired data.

[0025] Step 1.2.1: Obtain the camera's intrinsic parameter matrix K and the extrinsic parameter matrix T between the laser scanner and the camera through calibration, wherein the extrinsic parameter matrix includes a rotation matrix and a translation vector. The calibration method adopts the existing calibration method in the prior art.

[0026] Step 1.2.2: Use the calibration parameters to project the 3D point cloud data collected by the laser scanner onto the camera image plane to achieve spatial alignment of data of different modalities, laying the foundation for subsequent data fusion.

[0027] In a specific implementation, it is assumed that the three-dimensional points collected by the laser scanner are (homogeneous coordinates), the camera's intrinsic parameter matrix is .

[0028] in, represents the horizontal focal length of the image plane (in pixels), represents the focal length perpendicular to the image plane (in pixels), Represents the horizontal coordinate of the center point of the image, Indicates the vertical coordinate of the center point of the principal point image.

[0029] The extrinsic parameter matrix between the laser scanner and the camera is , then the formula for projecting the point cloud data onto the image plane is: .

[0030] get , after conversion, normalized to pixel coordinates .

[0031] in, Represents the horizontal coordinate of a pixel in an image, usually indicating the position in the width direction of the image. Represents the vertical coordinate of a pixel in an image, usually indicating the position in the height direction of the image. Represents the scale factor in homogeneous coordinates and is used for homogeneous coordinate representation. When it is 1, the pixel coordinates are 2D coordinates in the actual sense. .

[0032] Step 2: Use the deep learning model to perform preliminary feature screening on the image to obtain images of suspected crack areas.

[0033] Step 2.1: Use the confidence threshold reduction method to extract the suspicious areas, establish the detection-based confidence distribution, analyze the suspicious areas according to the confidence values, and apply dynamic filtering strategies to different categories of suspicious areas.

[0034] In a specific implementation, the original image of tunnel lining crack detection faces multiple difficulties. Cracks usually have low contrast and little difference from the tunnel background, especially in uneven lighting or low light environments, which are easily masked; secondly, complex background interference (such as stains, seams and light spots) is similar to the crack morphology, which can easily lead to false detection; in addition, crack features are usually subtle and intermittent, and the width may be only a few pixels, which is difficult to capture effectively by traditional methods; at the same time, the non-uniform lighting and dynamic range limitations in the tunnel further weaken the image details.

[0035] In view of these difficulties, this embodiment first uses the non-local mean filtering method to remove surface texture noise based on the sensor data, while retaining the edge features of the cracks to reduce complex background interference. Secondly, the high dynamic range imaging algorithm (HDR) is used to adjust the contrast of light and dark areas to enhance the visibility of the cracks. After the images are pre-processed, they are used through existing deep learning models such as YOLO or Faster R-CNN models for preliminary feature screening.

[0036] A deep learning model is used on the camera image to perform preliminary crack detection on the tunnel lining. As a preliminary feature screening, this embodiment adopts a method of lowering the confidence threshold to extract more suspicious regions (ROIs). This can cover all areas where there may be subtle cracks as much as possible to avoid missed detections due to low contrast of the target. The deep learning model can use the existing YOLO or Faster R-CNN model. At the same time, a confidence distribution based on detection is established, and the suspicious areas are analyzed according to the confidence values, and dynamic filtering strategies are applied to different categories of suspected areas. For example: high confidence : Keep the target directly without strict filtering. Medium confidence :Use strict area and morphology filtering rules to eliminate possible false positives. Low confidence :Only the targets that meet the aspect ratio or shape factor are retained and further processed. In addition, this embodiment detects the target based on the confidence level. , dynamically adjust the threshold of the filtering rules.

[0037] Step 2.2: Simultaneously obtain the two-dimensional bounding box of each suspicious area and perform image processing to obtain an image of the suspected crack area.

[0038] In a specific implementation, the two-dimensional bounding box is: .

[0039] Where B is a two-dimensional bounding box, , , and They are the minimum value of the horizontal axis, the minimum value of the vertical axis, the maximum value of the horizontal axis and the maximum value of the vertical axis respectively.

[0040] According to the position of the two-dimensional bounding box B, the corresponding region of interest (ROI) is cropped from the original image.

[0041] Convert the cropped ROI to a grayscale image I(x,y). The conversion formula is:

[0042] in, , , are the pixel values ​​of the red, green, and blue channels respectively.

[0043] Step 3: Densify the 3D point cloud data to obtain high-density depth information and obtain a depth map.

[0044] Step 3.1: Construct Poisson's equation based on 3D point cloud data.

[0045] In a specific implementation, laser scanners usually output sparse point cloud data, and in order to efficiently fuse with camera images, the point cloud data needs to be densified. A Poisson reconstruction-based depth continuity algorithm is used to convert discrete point clouds into continuous depth images.

[0046] Specifically, it is to find an indicator function χ so that its gradient is close to the vector field given by the point cloud normal vector field V and solve the following minimization problem: .

[0047] in, The gradient of the scalar field χ is a vector field that represents the rate of change of χ in all directions.

[0048] Step 3.2: Solve the Poisson equation to reconstruct the surface and obtain a continuous depth map.

[0049] The above minimization problem is equivalent to solving the Poisson equation: .

[0050] in, The Laplace operator representing a scalar field is the divergence of the gradient, which reflects the local "curvature" of the scalar field. It represents the divergence of the vector field V, which is a scalar value that describes the degree of "divergence" or "convergence" of V at a certain point. Taking the isosurface χ=0.5, the reconstructed surface can be obtained, thus forming a continuous depth map D.

[0051] Through densification, the point cloud data is resampled and interpolated to form a regular two-dimensional depth map. The depth map is a two-dimensional matrix, and each pixel value of the matrix represents the distance (depth value) from a point to the sensor under the corresponding viewing angle, in centimeters. The depth map is usually the same size as the camera image, and each pixel corresponds precisely to a pixel point on the camera image, with pixel-by-pixel depth resolution.

[0052] Step 3.3: Based on the spatial alignment of data from different modalities, the area corresponding to the questionable area of ​​the image in the generated depth map is extracted to obtain high-density depth information.

[0053] In order to map the two-dimensional bounding box B in the image to the BBOX in the depth map, the intrinsic parameters (camera projection parameters) and extrinsic parameters (rotation and translation matrices) between the camera and the laser scanner obtained in the previous steps are used to establish the mapping relationship between the coordinate systems of the two. The two-dimensional pixel coordinates of B in the image are converted to three-dimensional points in the camera coordinate system through the camera intrinsic parameters and depth back projection. Then, the three-dimensional points in the camera coordinate system are mapped to the laser scanner coordinate system using the extrinsic parameter matrix to obtain three-dimensional points consistent with the laser scanner's viewing angle. Next, the three-dimensional points in the laser scanner coordinate system are projected onto the two-dimensional plane of the depth map to obtain the corresponding BBOX area on the depth map. Finally, based on the projection results, a new BBOX is generated in the depth map to indicate the position and range of the corresponding area on the depth map. Among them, the BBOX area refers to a rectangular area, which is usually used to define the boundary of a rectangular area, and it is represented by the coordinates of the lower left corner and the upper right corner.

[0054] Then, using the aforementioned calibration parameter process, based on the spatial alignment of different modal data, the area corresponding to the image ROI in the generated depth map is extracted, thereby obtaining high-density depth information that matches the visual ROI, namely the depth map D. Tunnel lining cracks are usually accompanied by certain depth changes (such as depressions at the edges of cracks). Through the depth map, these feature areas can be accurately located. In low-contrast images, the texture information of cracks may be blurred, but the depth information can provide supplementary information, thereby enhancing the robustness of detection. High-density depth maps can intuitively display the three-dimensional spatial relationship of the scene, providing basic support for subsequent geometric feature extraction (such as normal vector calculation, surface reconstruction) and object recognition. High-density depth information maps play a key role in enhancing detection accuracy, reducing interference, providing supplementary features and supporting subsequent analysis in crack detection methods. It provides more dimensional information for the detection process, enabling the system to complete detection tasks more robustly in complex environments.

[0055] Step 4: Analyze the spatial structure perception provided by the depth map, and use the data matching algorithm to fuse the suspected crack area image with the depth map to obtain fused data. The high-density depth map is combined with the two-dimensional image. The depth map can supplement the deficiency of two-dimensional information and effectively deal with the problem of target detection and segmentation in complex environments.

[0056] The edge detection algorithm is used to extract the edge features of the image. Since the tunnel crack recognition images are usually in low contrast, the algorithm combines the depth information and the local features of the image. The depth information is fused with the grayscale distribution of the image to enhance the visibility of the crack edge.

[0057] Step 4.1: Use an edge detection algorithm to detect the image of the suspected crack area and obtain a first detection result.

[0058] Step 4.1.1: Calculate the local contrast of the image in the suspected crack area.

[0059] In a specific embodiment, the input grayscale image And the corresponding depth map In grayscale images, local contrast is calculated , Laplace transform or local standard deviation is often used. Take local standard deviation as an example: .

[0060] in, So is the local window centered on is the mean value of the pixels in the window, N represents the local window The total number of pixels contained in is the size of the window.

[0061] Step 4.1.2: Based on the spatial structure perception provided by the depth map and the detailed features of the image grayscale distribution, the depth gradient is calculated using the depth map and the depth gradient is normalized to obtain the enhancement factor.

[0062] In a specific implementation, the depth gradient is calculated using the depth map. : .

[0063] Normalize the depth gradient to get the enhancement factor : .

[0064] Step 4.1.3: Use the enhancement factor to weight the local contrast of the image to obtain the enhanced image.

[0065] The fusion process uses the spatial structure perception provided by the depth map and the detailed characteristics of the image grayscale distribution to locally enhance the low-contrast areas, thereby improving the visibility and coherence of the crack edges.

[0066] In a specific implementation, the local contrast of the image is weighted using the enhancement factor to obtain an enhanced image. : .

[0067] in, It is a parameter to adjust the depth enhancement strength, and the default value can be set to 0.5.

[0068] In a tunnel environment, cracks usually appear as a sudden change in local depth or a break in continuity, with more dramatic depth changes compared to the surrounding areas. The depth gradient can highlight this change characteristic, thereby making the cracks stand out from other smooth areas. Furthermore, there may be pseudo-edges in the image caused by texture, illumination changes, etc., but these pseudo-edges are usually not accompanied by significant depth changes. By analyzing the depth gradient, these interferences can be effectively eliminated, and only the areas with significant depth changes are retained as candidate cracks.

[0069] In addition, crack features in grayscale images may become blurred when the contrast is low or there is a complex background, and the depth gradient can also provide additional boundary information for these areas, supplement the deficiencies in the grayscale image, and improve the overall robustness of the crack features. Cracks are usually linear or curved structures, which appear as continuous mutations in depth values ​​on the depth map. The depth gradient can reveal these linear features, thereby significantly enhancing the cracks. After normalizing the depth gradient, the enhancement factor can be further used to fuse the feature information of different modes, making the crack area more prominent in the multimodal data, which is helpful for subsequent accurate detection and analysis.

[0070] Step 4.1.4: Use edge detection algorithm to enhance the image Perform edge detection to obtain a first detection result. In this embodiment, the edge detection algorithm adopts the Canny or Sobel algorithm.

[0071] Step 4.2: By calculating the gradient depth between the pixels of the depth map, a set of points with significantly discontinuous depth values ​​is obtained as the second detection result.

[0072] In a specific implementation, on the densified depth map, by calculating the depth gradient values ​​between the pixels, a set of points where the depth values ​​are significantly discontinuous is detected. These discontinuous points usually reflect the locations of crack edges or faults.

[0073] Densified depth map Calculate the gradient between pixels : .

[0074] when When , the pixel is marked as a depth discontinuity point, recorded as ,in, is the set threshold. G(i,j) calculated by this formula is the gradient value calculation of the specific pixel position, which is used for the analysis and processing of pixel-by-pixel operations, for the detection of gradient changes between specific points, and directly participates in the discontinuity judgment.

[0075] Step 4.3: Use a data matching algorithm to perform cross-modal data matching and fusion on the first detection result and the second detection result to obtain fused data.

[0076] In a specific implementation, in order to suppress noise interference, non-maximum suppression and filtering are performed on the two parts of the detection results, namely the first detection result and the second detection result, respectively, and then cross-modal matching data fusion is performed through a certain matching algorithm. In this embodiment, the matching algorithm adopts Chamfer matching or matching based on the nearest neighbor distance. Taking Chamfer as an example: Assume the image edge point set is , the set of depth discontinuities is , Chamfer matching can be used to calculate the matching distance: .

[0077] The smaller the matching distance is, the more consistent the edges of the two modes are, thus increasing the credibility of the existence of cracks.

[0078] Step 5: Calculate the fusion confidence of the fused edge data and determine the crack detection result based on the fusion confidence.

[0079] In a specific implementation, a confidence index is designed based on the fused edge data, which may include the following elements: 1. The degree of overlap between the image edge and the depth discontinuity edge (matching accuracy).

[0080] 2. The span of the discontinuity in the depth map (representing crack width or depth change).

[0081] 3. Edge continuity (cracks usually appear as continuous lines).

[0082] These indicators are combined into a multi-factor confidence calculation formula. Finally, the preset threshold is calculated by comprehensive analysis of multiple features. After feature consistency check, when the fusion confidence exceeds the preset threshold, the system determines that there is a crack in the ROI area. The calculation of fusion confidence is based on the weighted results of grayscale distribution, depth features and edge features. The threshold setting is adjusted through experiments to balance false detection and missed detection. At the same time, the morphological characteristics of the cracks and the verification of multimodal data further enhance the reliability of the judgment, laying the foundation for subsequent precise analysis.

[0083] Feature consistency refers to the regularity of coordination or matching of various features (such as grayscale, depth, edge, etc.) in space and nature within the same ROI area. This consistency is a key indicator used to verify whether the area meets the target feature (such as cracks). Feature consistency checking is a conventional technical means in this field and will not be described here.

[0084] Specifically, we combine the image and depth information to define the comprehensive confidence It is a weighted combination of multiple indicators, for example: 1. Edge overlap : The inverse of Chamfer matching can be taken and recorded as .

[0085] 2. Depth discontinuity span :For example, calculate the average gradient amplitude of the depth discontinuity point in the ROI, and record it as .

[0086] 3. Edge continuity : It can be expressed by the detected edge length or continuity index, and after normalization, it is recorded as .

[0087] The comprehensive confidence calculation formula can be expressed as: .

[0088] in, , , is the weight coefficient, satisfying ,when Exceeding the preset threshold When , it is determined that there is a crack in the ROI area.

[0089] In the prior art, high-precision sensors are often used to improve the accuracy of collected data. Although high-precision sensors can theoretically provide more accurate single-modal data, their effectiveness is often limited by various factors in practical applications. High-precision cameras are easily affected by reflections, shadows, and contrast changes under complex lighting conditions. Although high-precision laser scanners can provide accurate depth information, their ability to capture texture details is limited, especially when the crack width is small, their resolution may be insufficient. The method of this embodiment makes full use of the texture features of the camera and the geometric information of the laser scanner through multimodal fusion, making up for the limitations of a single sensor under specific conditions. By fusing the information of the two, this embodiment uses depth information to provide geometric constraints for visual features, eliminating errors caused by lighting changes, and at the same time uses visual information to complement the sparsity of depth data in the laser scanner, forming complementary advantages, thereby significantly improving the robustness and accuracy of detection. On this basis, the uniqueness of the method in this embodiment lies in that the advantages of the two sensors are fully integrated through the spatial enhancement strategy based on the depth gradient and the cross-modal discontinuity matching, and the significance of crack detection is enhanced through multi-stage screening and refinement, and a confidence assessment model is generated, which improves the adaptability and detection effect of the system in real scenes from multiple levels. This fusion can not only provide higher robustness in crack detection, but also further improve the overall accuracy through cross-modal matching and confidence assessment, thus outperforming a single high-precision sensor in real scenes.

[0090] Embodiment 2: Embodiment 2 of the present invention provides a tunnel lining fine crack detection system based on data fusion, comprising: A data acquisition module is configured to acquire three-dimensional point cloud data and images of the tunnel lining and perform preprocessing; A preliminary screening module is configured to perform preliminary feature screening on the image using a deep learning model to obtain an image of a suspected crack area; A depth information extraction module is configured to perform densification processing on the three-dimensional point cloud data to obtain high-density depth information and obtain a depth map; A data fusion module is configured to analyze the spatial structure perception provided by the depth map, and fuse the image of the suspected crack area with the depth map using a data matching algorithm to obtain fused data; The crack detection module is configured to calculate the fusion confidence of the fused edge data and determine the crack detection result according to the fusion confidence.

[0091] Embodiment three: Embodiment 3 of the present invention provides a medium on which a program is stored. When the program is executed by a processor, the steps of the method for detecting fine cracks in tunnel lining based on data fusion as described in Embodiment 1 of the present invention are implemented.

[0092] Embodiment 4: Embodiment 4 of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for detecting fine cracks in tunnel lining based on data fusion as described in Embodiment 1 of the present invention are implemented.

[0093] The steps involved in the above embodiments 2, 3 and 4 correspond to the method embodiment 1. For the specific implementation methods, please refer to the relevant description part of embodiment 1.

[0094] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0095] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for detecting fine cracks in tunnel lining based on data fusion, characterized in that: The following steps are involved: Obtain 3D point cloud data and images of tunnel lining and perform preprocessing; Use the deep learning model to perform preliminary feature screening on the image to obtain images of suspected crack areas; Densify the 3D point cloud data to obtain high-density depth information and obtain a depth map; Analyze the spatial structure perception provided by the depth map, and use the data matching algorithm to fuse the image of the suspected crack area with the depth map to obtain fused data; The fusion confidence of the fused edge data is calculated, and the crack detection result is determined according to the fusion confidence.

2. The method for detecting fine cracks in tunnel lining based on data fusion according to claim 1, characterized in that: A laser scanner is used to collect 3D point cloud data, and a camera is used to acquire images.

3. The method for detecting fine cracks in tunnel lining based on data fusion according to claim 1, characterized in that: The specific steps of preprocessing include: The intrinsic parameter matrix of the camera and the extrinsic parameter matrix between the laser scanner and the camera are obtained by calibration; The calibrated parameters are used to project the 3D point cloud data collected by the laser scanner onto the camera image plane to achieve spatial alignment of data from different modalities.

4. The method for detecting fine cracks in tunnel lining based on data fusion according to claim 1, characterized in that: The specific steps for using the deep learning model to perform preliminary feature screening on images are: The confidence threshold reduction method is adopted to extract the suspicious areas, establish the confidence distribution based on the detection, analyze the suspicious areas according to the confidence values, and apply dynamic filtering strategies to different categories of suspicious areas; At the same time, a two-dimensional bounding box of each suspicious area is obtained to obtain an image of the suspected crack area.

5. The method for detecting fine cracks in tunnel lining based on data fusion according to claim 3, characterized in that: The specific steps for densifying 3D point cloud data are as follows: Construct Poisson's equation based on 3D point cloud data; Solve Poisson's equation to reconstruct the surface and obtain a continuous depth map; Based on the spatial alignment of data from different modalities, the areas corresponding to the questionable areas of the image in the generated depth map are extracted to obtain high-density depth information.

6. The method for detecting fine cracks in tunnel lining based on data fusion according to claim 1, characterized in that: The specific steps of fusing the suspected crack area image with the depth map using the data matching algorithm are as follows: Using an edge detection algorithm to detect the suspected crack area image, and obtain a first detection result; By calculating the gradient depth between the pixels of the depth map, a set of points with significantly discontinuous depth values ​​is obtained as a second detection result; A data matching algorithm is used to perform cross-modal data matching and fusion on the first detection result and the second detection result to obtain fused data.

7. The method for detecting fine cracks in tunnel lining based on data fusion according to claim 6, characterized in that: The specific steps of using edge detection algorithm to detect suspected crack area images are as follows: Calculate the local contrast of the image of the suspected crack area; Through the spatial structure perception provided by the depth map, combined with the detailed features of the image grayscale distribution, the depth gradient is calculated using the depth map, and the depth gradient is normalized to obtain the enhancement factor; The local contrast of the image is weighted by the enhancement factor to obtain an enhanced image; An edge detection algorithm is used to perform edge detection on the enhanced image to obtain a first detection result.

8. A tunnel lining fine crack detection system based on data fusion, characterized in that: include: A data acquisition module is configured to acquire three-dimensional point cloud data and images of the tunnel lining and perform preprocessing; A preliminary screening module is configured to perform preliminary feature screening on the image using a deep learning model to obtain an image of a suspected crack area; A depth information extraction module is configured to perform densification processing on the three-dimensional point cloud data to obtain high-density depth information and obtain a depth map; A data fusion module is configured to analyze the spatial structure perception provided by the depth map, and fuse the image of the suspected crack area with the depth map using a data matching algorithm to obtain fused data; The crack detection module is configured to calculate the fusion confidence of the fused edge data and determine the crack detection result according to the fusion confidence.

9. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executing the method for detecting fine cracks in tunnel lining based on data fusion as described in any one of claims 1 to 7.

10. A terminal device, characterized in that: It includes a processor and a computer-readable storage medium, the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the method for detecting fine cracks in tunnel lining based on data fusion as described in any one of claims 1-7.

Images