Tunnel crack identification method and system based on multi-source image processing

Abstract

The application provides a tunnel crack identification method and system based on multi-source image processing, relates to the technical field of tunnel engineering, and comprises the following steps: acquiring multi-source data of a horizontal rock stratum tunnel; performing spatial registration on the multi-source data to generate multi-modal images under the same reference system; extracting multi-modal features according to the multi-modal images, and constructing multi-source feature maps under the same spatial grid; performing initial crack detection on the multi-source feature maps in combination with multi-scale filtering and structure tensor analysis to obtain a crack candidate region mask; performing accurate crack identification through the crack candidate region mask, combining a bedding direction constraint and a differentiable conditional random field to obtain a crack identification result, and the crack identification result comprises a crack segmentation map and a crack category. The application solves the problem that the existing crack identification does not consider the bedding characteristics of the horizontal rock stratum tunnel.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tunnel engineering, in particular to a tunnel crack identification method and system based on multi-source image processing. BACKGROUND

[0002] Compared with general tunnels, the horizontal rock stratum tunnel is disturbed by the layering surface, and the cracks often develop along the layering surface and have a small interlayer spacing. The crack morphology is highly coupled with the layering texture, making it difficult to distinguish the cracks from the background structure. The existing tunnel crack identification methods are mostly aimed at homogeneous rock mass tunnels, but cannot solve the problem of layering interference in the horizontal rock stratum tunnel.

[0003] The cracks in the horizontal rock stratum tunnel are more easily disturbed by the layering surface. The traditional methods based on single-mode images or manual inspection are prone to missed detection, false detection or fracture, and it is difficult to obtain continuous and complete crack information. In the horizontal rock stratum tunnel, the spatial continuity and real morphology of the cracks are difficult to capture by a single sensor, and the traditional methods are prone to fracture, noise and misjudgment. In addition, the crack labeling in the horizontal rock stratum tunnel is difficult and the number of cracks is limited. The traditional supervised learning method is highly dependent on labeled data, and the existing methods often ignore the layering direction constraint and multi-modal information fusion, resulting in low crack identification accuracy and poor spatial structure consistency, which is more obvious in the horizontal rock stratum tunnel. SUMMARY

[0004] The purpose of the present application is to provide a tunnel crack identification method and system based on multi-source image processing to improve the above problems. In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0005] In a first aspect, the present application provides a tunnel crack identification method based on multi-source image processing, comprising:

[0006] Obtaining multi-source data of the horizontal rock stratum tunnel, the multi-source data comprising infrared images, visible light images, three-dimensional point cloud data and GPR profile graphs;

[0007] Spatially registering the multi-source data to generate multi-modal images in the same reference system;

[0008] Extracting multi-modal features from the multi-modal images to construct multi-source feature maps in the same spatial grid;

[0009] Performing initial crack detection on the multi-source feature maps by combining multi-scale filtering and structure tensor analysis to obtain crack candidate region masks;

[0010] Performing accurate crack identification through the crack candidate region masks, combining layering direction constraints and differentiable conditional random fields to obtain crack identification results, the crack identification results comprising crack segmentation maps and crack categories.

[0011] In a second aspect, the application further provides a tunnel crack identification system based on multi-source image processing, comprising:

[0012] An acquisition module is configured to acquire multi-source data of the horizontal rock stratum tunnel, wherein the multi-source data comprises infrared images, visible light images, three-dimensional point cloud data and GPR profile graphs;

[0013] A registration module is configured to perform spatial registration on the multi-source data to generate multi-modal images in the same reference system;

[0014] An extraction module is configured to extract multi-modal features from the multi-modal images to construct multi-source feature maps in the same spatial grid;

[0015] A preliminary detection module is configured to perform preliminary crack detection on the multi-source feature maps in combination with multi-scale filtering and structure tensor analysis to obtain a crack candidate region mask;

[0016] An identification module is configured to perform accurate crack identification through the crack candidate region mask to obtain a crack identification result in combination with a bedding direction constraint and a differentiable conditional random field, wherein the crack identification result comprises a crack segmentation graph and a crack category.

[0017] The application has the following beneficial effects:

[0018] (1) The application fuses surface texture, thermal anomaly, underground reflection and bedding direction information of the horizontal rock stratum tunnel through multi-source feature map fusion to realize all-around expression of crack features. In combination with multi-scale directional filtering and structure tensor analysis, the linear structure features of the cracks can be enhanced, and the interference of pseudo-cracks between the beddings can be suppressed to generate a high-credibility crack candidate region, thereby providing a reliable basis for fine identification. Further, through feature extraction of the candidate voxels and voxel-level graph structure modeling, spatial proximity and bedding direction consistency are introduced to make the feature propagation comply with the law that the horizontal rock stratum cracks extend along the beddings, thereby effectively improving the capture ability for complex crack shapes;

[0019] (2) The application further utilizes semi-supervised joint training of labeled samples and unlabeled candidate voxels to realize efficient propagation and consistency constraint of information in the crack candidate region. In combination with single-point potential, double-point potential and regional potential function iterative optimization, isolated pseudo-cracks and smooth crack boundaries can be removed, and crack categories can be accurately distinguished. By considering the bedding characteristics and crack extendibility of the horizontal rock stratum tunnel, the crack identification accuracy of the horizontal rock stratum tunnel is improved, thereby providing a reliable technical means for structure safety evaluation and later maintenance of the horizontal rock stratum tunnel.

[0020] Other features and advantages of the application will be set forth in the following description, and in part will become apparent to those skilled in the art from the following description, or can be learned by practice of the application as set forth in the claims, and / or disclosed herein. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.BRIEF DESCRIPTION OF DRAWINGS BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0022] Figure 1 The figure shows the tunnel crack identification method based on multi-source image processing described in the embodiments of the present application.

[0023] Figure 2 The figure shows the tunnel crack identification system structure based on multi-source image processing described in the embodiments of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0025] It should be noted that: similar numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present application, the terms "first", "second" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0026] Embodiment 1:

[0027] The present embodiment provides a tunnel crack identification method based on multi-source image processing.

[0028] Referring to Figure 1 , the figure shows that the method includes step S1, step S2, step S3, step S4 and step S5.

[0029] Step S1: acquiring multi-source data of horizontal rock stratum tunnel, the multi-source data including infrared image, visible light image, three-dimensional point cloud data and GPR profile.

[0030] It can be understood that, compared with general tunnels, the horizontal stratum tunnel has obvious bedding structure. The stratum is horizontally distributed, the interval between layers is small, and the bedding texture is dense, so there are problems that the cracks and the bedding texture are easily confused, the crack morphology is significantly controlled by the bedding, and the surface characteristics and internal characteristics of the cracks are inconsistent. And the horizontal bedding will cause light reflection stripes and temperature gradient anomalies, making the crack response of the single modal image distorted. By distinguishing the surface cracks and the bedding texture through visible light and infrared, and identifying the spatial continuity and internal extension through point cloud and GPR, multi-source data is collected, and the four modalities are complementary in space and have differences in physical meaning, which together constitute the characterization panorama of cracks.

[0031] In this step, the multi-sensor installed on the tunnel inspection vehicle or unmanned aerial vehicle platform is used to obtain the basic environmental parameters of the horizontal stratum tunnel, including the tunnel section morphology, stratum dip angle, strike and layer spacing information, which provides a spatial reference for subsequent multi-source sensor data collection.

[0032] A high-resolution visible light camera is arranged on the tunnel wall along the longitudinal direction to collect the texture image of the tunnel inner surface, that is, a line array camera or a panoramic camera system is used to realize continuous scanning to obtain the visible light image of the tunnel inner wall, which is used to extract the texture and morphology information of the surface cracks.

[0033] Infrared thermal imaging data is synchronously collected on the same scanning path. The infrared camera is used to perceive the temperature field distribution of the tunnel wall surface to capture the thermal anomaly area caused by cracks. Through time sequence shooting and combining with the environmental temperature correction model, the radiance of the thermal image data is corrected and the temperature difference is normalized to obtain the infrared thermal anomaly image sequence, that is, the infrared image.

[0034] Laser radar (LiDAR) or structured light scanning equipment is used to obtain high-density three-dimensional point cloud data of the tunnel inner wall. Through multi-frame point cloud splicing and attitude correction, a spatial geometric model of the tunnel inner wall is established, that is, three-dimensional point cloud data, which is used to describe the spatial undulation, crack depth and bedding geometric characteristics of the tunnel surface.

[0035] A geological radar antenna is arranged on the tunnel wall along the longitudinal direction to collect underground bedding reflected wave signals to obtain a GPR profile. Through time domain reflected wave inversion, underground bedding interface, hidden cracks and reflected anomaly area are extracted to provide underground extension characteristic information of the surface cracks.

[0036] Based on the sensor timestamp and pose information, the visible light image, infrared image, three-dimensional point cloud data and GPR profile are time-synchronized and preliminarily aligned in space. A fusion positioning method based on inertial navigation system (INS) and visual odometry (VO) is adopted to ensure the spatial consistency of multi-source data collection.

[0037] In addition, the collected multi-source data has been processed for noise filtering, artifact removal and illumination normalization; the infrared image is corrected for thermal drift, the point cloud is removed for abnormal points and resampled, and the GPR profile is suppressed for background and compensated for gain, so as to obtain high-quality multi-source data for subsequent spatial registration and feature extraction.

[0038] Step S2: The multi-source data is spatially registered to generate multi-modal images in the same reference system;

[0039] The step S2 comprises:

[0040] Step S21: Selecting the coordinate system of the three-dimensional point cloud data as the reference coordinate system;

[0041] In this step, the three-dimensional point cloud data obtained by the laser radar is taken as the spatial reference to establish the reference coordinate system of the horizontal rock stratum tunnel. The coordinate system has high spatial accuracy and can accurately reflect the geometric shape and bedding plane distribution characteristics of the horizontal rock stratum tunnel, thereby providing a unified spatial reference for multi-source data registration.

[0042] Step S22: Projecting the infrared image, the visible light image and the GPR profile onto the reference coordinate system;

[0043] In this step, based on the sensor calibration parameters and the collection pose information, the extrinsic matrix and the projection transformation relationship of the infrared camera, the visible light camera and the laser radar are calculated. The infrared image and the visible light image are mapped onto the reference coordinate system plane through the perspective projection model, and the GPR profile is voxelized and projected along the tunnel axis based on the radar survey line track, so as to unify each modal data under the reference coordinate system.

[0044] Step S23: Preliminary matching the infrared image, the visible light image, the GPR profile and the three-dimensional point cloud data based on the reference coordinate system to obtain a preliminary matching pair;

[0045] In this step, a coordinate transformation matrix is established for each modality (infrared camera, visible light camera and laser radar).

[0046] The infrared image mainly reflects the surface temperature field characteristics, and the point cloud reflects the geometric shape. Direct matching of the two is difficult, and it is necessary to establish a connection through geometric boundaries and thermal gradient boundaries.

[0047] Specifically, the temperature gradient of the infrared image is calculated, and the local extreme points of the gradient are selected as the thermal anomaly feature point set. Then, based on the three-dimensional point cloud data, the normal gradient of the point cloud surface is calculated, and the geometric boundary point set is extracted through the normal gradient.

[0048] The infrared points in the thermal anomaly feature point set are back-projected to the reference coordinate system through a coordinate transformation matrix, and the nearest points in the geometric boundary point set are searched to obtain a plurality of preliminary matching pairs, which constitute a first preliminary matching pair set, and the weight of each matching pair is determined by the consistency of the thermal gradient and the geometric gradient.

[0049] The visible light image reflects the texture feature, which corresponds to the geometric structure of the point cloud, so the feature point and edge correspondence method is used for matching.

[0050] Specifically, visible key points are extracted on the visible light image to generate SIFT descriptors of the visible key points. The point cloud intensity or reflectivity is projected to a two-dimensional plane to form a pseudo image through the three-dimensional point cloud data. The corresponding point cloud feature points are extracted through the pseudo image to generate SIFT descriptors of the point cloud feature points.

[0051] The SIFT descriptors of the visible key points and the point cloud feature points are used to calculate the feature similarity, and the pair with the highest similarity is selected as the first candidate matching pair. The normal difference and local plane angle consistency of the first candidate matching pair are verified, and if the verification is passed, the first candidate matching pair is reserved as an effective matching pair to obtain a second preliminary matching pair set.

[0052] The GPR profile reflects the reflection band of the underground layering and hidden fracture, and the point cloud reflects the surface geometric shape. The two are related through geometric extension of the layering surface.

[0053] Specifically, the envelope signal energy is calculated through the GPR profile, and the energy peak line is extracted through the envelope signal energy. Based on the three-dimensional point cloud data, the point cloud is segmented into planes to obtain layering surface parameters, and then the layering boundary points are extracted through the layering surface parameters.

[0054] The GPR reflection points on the energy peak line are projected to the tunnel cross section, and the nearest layering boundary points are searched as the second candidate matching pairs. The matching confidence of each second candidate matching pair is calculated (obtained by calculating the difference between the reflection band depth and the layering surface depth), and the third preliminary matching pair set is obtained through the verification of the matching confidence.

[0055] Step S24: A conversion matrix is calculated according to the preliminary matching pairs, and the infrared image, the visible light image and the GPR profile are respectively co-registered with the three-dimensional point cloud data through the conversion matrix to obtain a co-registered result.

[0056] In this step, all the preliminary matching pairs include corresponding preliminary matching pairs in different modalities, including the first preliminary matching pair set, the second preliminary matching pair set and the third preliminary matching pair set. The outlier rejection is performed on each matching pair set. The weighted RANSAC algorithm is used to detect outliers, and the preliminary matching pairs with a matching error greater than a matching threshold are rejected, wherein the matching threshold is 3 times the point cloud resolution.

[0057] After removing outliers, the conversion matrix of each modality is estimated independently by the preliminary matching pairs. Specifically, the weighted centroid of the matching points in the preliminary matching pairs is calculated, the corresponding decentralized coordinates are calculated based on the weighted centroid of the matching points, and the weighted covariance matrix is calculated based on the decentralized coordinates of the matching points. Singular value decomposition is performed on the weighted covariance matrix to obtain a rotation matrix. Then the translation vector is calculated by the weighted centroid and the rotation matrix. The conversion matrix is constructed by the translation vector and the rotation matrix.

[0058] The corresponding image coordinates are converted to the reference coordinate system by the conversion matrix under different modalities, and the coarse registration of multi-source data is realized.

[0059] Step S25: extracting a bedding surface three-dimensional model according to the coarse registration result, the bedding surface three-dimensional model including a horizontal bedding surface, a bedding texture line and a bedding reflection line;

[0060] In this step, the coarse registration result is the infrared image, the visible light image, the GPR profile and the three-dimensional point cloud data in the same coordinate system after coarse registration.

[0061] The three-dimensional point cloud data under the coarse registration result is subjected to plane segmentation, and potential horizontal bedding surface segments are identified. For each segment, a least square plane fitting is used to obtain a bedding surface equation, and a fitting error is calculated. The in-plane points are screened by the fitting error, and outliers are removed to obtain a horizontal bedding surface.

[0062] The visible light image and the infrared image under the coarse registration result are subjected to high-pass filtering or edge detection to obtain texture edges. The texture edges are projected onto the three-dimensional point cloud data after coarse registration to obtain a three-dimensional coordinate set of the bedding texture line.

[0063] The GPR profile under the coarse registration result is subjected to envelope signal analysis to extract a high-energy reflection band peak line. The reflection band peak line is projected onto the tunnel cross section in the reference coordinate system. In combination with the horizontal bedding surface obtained by point cloud fitting, the points on the reflection peak line are mapped to the corresponding horizontal bedding surface to obtain bedding reflection line coordinate points. The bedding reflection line coordinate points are subjected to smoothing processing (such as B-spline fitting) to generate a continuous reflection line model, i.e., a bedding reflection line.

[0064] The overall bedding surface three-dimensional model is generated by the horizontal bedding surface, the bedding texture line and the bedding reflection line.

[0065] Step S26: calculating a registration deviation based on the bedding surface three-dimensional model;

[0066] A set of reference points uniformly distributed on the bedding surface three-dimensional model is selected, including bedding surface points, texture line points and reflection line points. The corresponding points of the projection points (infrared pixel points, visible light key points, GPR reflection points) of each modality in the reference coordinate system are determined.

[0067] For each corresponding point, calculate its nearest distance to the 3D model of the bedding plane. For each mode, calculate the registration deviation using the nearest distance.

[0068] Step S27: Optimize the coarse registration result based on the registration deviation to obtain a multimodal image under the same reference frame.

[0069] In this step, a registration optimization function is constructed for each modality's point set to the bedding plane:

[0070] ;

[0071] In the formula, Indicates the number of reference points. Indicates the first Weighting coefficients for each reference point and These represent the rotation matrix and translation vector, respectively. Represents Euclidean distance. Indicates in and The minimum value below, Indicates the first A reference point, express The corresponding point of each reference point.

[0072] Using iterative nearest point or weighted least squares method to... and Solve iteratively until the overall registration bias converges or falls below a threshold. Then, apply the optimized solution... and When applied to the corresponding modes, the optimized infrared image, visible light image, and GPR profile are obtained in the same reference frame, thus generating a multimodal image in the same reference frame.

[0073] Step S3: Extract multimodal features from the multimodal image and construct a multi-source feature map under the same spatial grid;

[0074] Step S3 includes:

[0075] Step S31: Perform three-dimensional voxelization on the multimodal images to establish a spatial mesh model with uniform resolution;

[0076] In this step, based on the tunnel cross-sectional dimensions and crack identification requirements, the following steps are set: direction, direction and Spatial voxel resolution in the direction ( , and ), and a unified spatial grid model is established. The data of each modality after the optimization registration is interpolated or projected to make the data values assigned to the corresponding voxel units, so as to realize the voxelization representation of the data of different modalities in the three-dimensional space.

[0077] Step S32: feature extraction is performed on the multi-modal images based on the spatial grid model to obtain surface texture features, thermal anomaly features, geometric structure features and underground reflection features under the spatial grid;

[0078] In this step, local texture calculation is performed on the voxelized visible light image, including a gray level co-occurrence matrix (GLCM), a local binary pattern (LBP) and a histogram of oriented gradients (HOG), and then surface texture features are extracted, wherein the surface texture features include energy, contrast, homogeneity and entropy.

[0079] Based on the voxelized infrared image, local average temperature, temperature gradient and temperature anomaly difference of adjacent bedding planes are calculated to obtain thermal anomaly features. Through the voxelized three-dimensional point cloud data, normal vector, curvature and density are calculated to obtain geometric structure features. Through the voxelized GPR reflection signal, envelope amplitude, instantaneous phase and reflection energy are calculated to obtain underground reflection features.

[0080] Step S33: a bedding direction reference field is established according to the dip angle and strike information of the bedding planes in the spatial grid model;

[0081] In this step, the normal vector of the bedding plane is calculated based on the bedding plane equation. According to the direction of the normal vector and the direction of the tunnel axis, the dip angle and the strike angle of the bedding plane are calculated. At the voxel center of each spatial grid, a local bedding direction vector is assigned according to the bedding plane partition where it is located, so as to establish the bedding direction reference field.

[0082] Step S34: the surface texture features, the thermal anomaly features, the underground reflection features and the bedding direction reference field are aligned and feature fused according to the spatial grid to obtain a multi-source feature map under the same spatial grid.

[0083] In this step, the multi-source feature map is a three-dimensional spatial feature data structure obtained by aligning and fusing the information of different modalities (visible light, infrared, GPR and point cloud) in a unified space. It uniformly maps the features from multiple sensors to the same spatial grid model. Each voxel contains a multi-dimensional feature vector contributed by different modalities at that spatial position.

[0084] Step S4: crack preliminary detection is performed on the multi-source feature map by combining multi-scale filtering and structure tensor analysis to obtain a crack candidate region mask;

[0085] In this step, the multi-source feature maps are preliminarily enhanced in crack response by using multi-scale filtering and structure tensor analysis to generate a crack candidate region mask. In a horizontal stratum tunnel, since cracks often extend along the bedding plane and have complex shapes, multi-scale directional filtering and structure tensor analysis can effectively enhance linear structure features, suppress background noise and inter-bedding pseudo-cracks, cover the real cracks with the candidate region and reduce false positives.

[0086] The step S4 comprises:

[0087] Step S41: extracting continuous structure candidate regions in the multi-source feature maps by a multi-scale directional filter to obtain a crack response enhancement map;

[0088] It can be understood that a crack is an elongated linear structure with strong directionality but variable scale. Therefore, for the surface texture features, thermal anomaly features and underground reflection features in the multi-source feature maps, a multi-scale directional filter (such as a Hessian filter or a Gabor filter) is used for multi-directional convolution response calculation. The maximum response value of each voxel point at all directions and scales is taken as the crack response intensity. Then, the crack response intensity is normalized and non-maximum suppression is performed to obtain a crack response enhancement map. The linear structure is strengthened and the texture and background are suppressed through the crack response enhancement map.

[0089] Step S42: calculating a structure tensor matrix from the gradient information of the multi-source feature maps;

[0090] Only a response with high intensity may not be a crack, but also noise, illumination changes or bedding lines. The structure tensor can describe the consistency of the local gradient direction.

[0091] The gradient information of the multi-source feature maps is calculated, an initial structure tensor matrix is constructed from the gradient information, and the initial structure tensor matrix is Gaussian smoothed to obtain the structure tensor matrix.

[0092] Step S43: constructing a preliminary crack response map from the crack response enhancement map and the structure tensor matrix;

[0093] In this step, the intensity information of the crack response enhancement map and the direction consistency information of the structure tensor matrix are fused to obtain a crack response with double constraints of intensity and direction.

[0094] Specifically, the structure tensor matrix is subjected to eigenvalue decomposition to obtain three eigenvalues (λ1, λ2, λ3), and each eigenvalue reflects the change degree of the local gradient along the corresponding eigenvector direction. 、 and ), and each eigenvalue reflects the change degree of the local gradient along the corresponding eigenvector direction. For the eigenvalue of the maximum gradient change direction, the corresponding linear structure main direction is λ1, is the eigenvalue of the direction with the second largest gradient variation, reflecting local planar extension. is the eigenvalue of the direction with the smallest gradient variation, reflecting local isotropy (noise or point-like structure).

[0095] And the crack in the horizontal rock stratum tunnel is mainly linear structure, with strong linearity and small thickness, The corresponding direction is very weak, and the contribution to the linear structure judgment is small (more reflecting noise or point-like disturbance), and The corresponding main line direction gradient is large, The gradient strength perpendicular to the main line direction (the smaller the structure is more elongated), in order to determine the linearity, only And If Close to , it indicates that the local structure tends to be planar, not a crack; if , it indicates that the local structure is linear, and it is very likely to be a crack.

[0096] At each voxel, the crack direction consistency index is calculated as:

[0097] ;

[0098] Wherein, represents the crack direction consistency index at voxel , and and both represent the eigenvalues of the structure tensor matrix, The larger the value is, the more linear the structure is, and the more likely it is to be a crack.

[0099] The expression of the preliminary crack response map is:

[0100] ;

[0101] In the formula, represents the fusion weight, represents the crack response strength at voxel .

[0102] Step S44: Based on the bedding direction reference field, the preliminary crack response map is directionally constrained to obtain an enhanced crack response map;

[0103] In the horizontal rock stratum tunnel, the interface between the bedding may be linear or reflect enhanced, which is easy to be misjudged as a crack. But the real crack is usually orthogonal or large-angle intersected with the bedding surface.

[0104] In this step, the local direction vector of the bedding direction reference field is introduced.The crack direction is calculated by the structure tensor main direction eigenvalue angle between the fracture direction and the bedding direction :

[0105] ;

[0106] When approaches 90°, the fracture direction is orthogonal to the bedding plane, and it is more likely to be a real fracture; when approaches 0°, it is likely to be a bedding-plane pseudo-fracture, in which corresponds to the largest eigenvalue , for the elongated linear structure (fracture), the local gradient changes most strongly in the main direction of the fracture, so is the direction vector of the fracture.

[0107] Therefore, the direction constraint factor is defined as

[0108] ;

[0109] The bedding-plane pseudo-fracture is suppressed by the direction constraint factor, and the real fracture line is highlighted, so the enhanced fracture response map is obtained by fusing the preliminary fracture response map with the direction constraint factor.

[0110] ;

[0111] Step S45: preliminary fracture extraction is performed according to the enhanced fracture response map, and a fracture candidate region mask is generated.

[0112] In this step, threshold segmentation and connected region analysis are performed on the enhanced fracture response map. The response threshold is determined by an adaptive threshold method (such as the Otsu method or the local average method), and a binary fracture mask is generated.

[0113] ;

[0114] Three-dimensional morphological closing operation and small voxel filtering are performed on the binary fracture mask to obtain a fracture candidate region mask , which is used for subsequent fine fracture recognition and classification.

[0115] Step S5: accurate fracture recognition is performed through the fracture candidate region mask, and a fracture recognition result is obtained by combining the bedding direction constraint and the differentiable conditional random field, the fracture recognition result including a fracture segmentation map and a fracture class.

[0116] In this step, multi-source feature vectors of each candidate voxel are extracted within the crack candidate region mask, and a voxel-level graph structure is constructed based on spatial proximity and bedding direction similarity. By fully utilizing the distribution pattern of cracks along bedding planes or diagonally in horizontal rock tunnels, spatial continuity and bedding direction consistency are introduced into graph modeling. This ensures more reasonable feature propagation of candidate voxels in semi-supervised learning, improves the ability to identify complex crack morphologies, and maintains local and global consistency.

[0117] Step S5 includes:

[0118] Step S51: Extract the multi-source feature vectors of each candidate voxel within the crack candidate region mask from the multi-source feature map to form a candidate region feature set;

[0119] In this step, the first step is to mask the crack candidate region. Select all candidate voxels. For each candidate voxel, extract its multi-source feature vector in the multi-source feature map:

[0120] ;

[0121] In the formula, Indicates the first Multi-source feature vectors of candidate voxels Indicates the first Surface texture features of candidate voxels Indicates the first Thermal anomaly characteristics of candidate voxels No. Geometric structural features of candidate voxels Indicates the first Subsurface reflectance characteristics of candidate voxels Indicates the first The reference field vector of the bedding direction for each candidate voxel , The number of candidate voxels.

[0122] The multi-source feature vectors of all candidate voxels are used to form a candidate region feature set:

[0123] ;

[0124] In the formula, This represents the feature set of the candidate region.

[0125] Step S52: Construct a voxel-level graph structure based on the spatial proximity relationship and stratification direction similarity of candidate voxels;

[0126] In this step, the voxel-level diagram structure is established. It is actually an undirected weighted graph. , , ,in, Represents a set of nodes. Represents an edge set. This represents the weight matrix between voxels. Indicates the first One candidate voxel, Indicates the first One candidate voxel, Indicates the first The candidate voxels and the first The edges formed by candidate voxels are established through spatial proximity relations (such as k-nearest neighbors), and Determined based on feature similarity and directional consistency:

[0127] ;

[0128] In the formula, Indicates the first The candidate voxels and the first The weights of each candidate voxel and They represent the first The candidate voxels and the first Spatial coordinates of candidate voxels and They represent the first The candidate voxels and the first The reference field vector of the bedding direction for each candidate voxel and These represent the first scale parameter and the second scale parameter, respectively. Represents Euclidean distance. This represents an exponential function.

[0129] Step S53: Construct a joint loss function based on label propagation, graph regularization, and consistency regularization;

[0130] In this step, we define the joint loss function for semi-supervised learning:

[0131] ;

[0132] ;

[0133] ;

[0134] In the formula, Denotes the joint loss function. This represents the supervised loss (cross-entropy) for labeled samples. This represents a graph regularization term that makes similar nodes predict to be close. denotes a consistency regularization term for enhancing the prediction stability of unlabeled samples, and denote regularization weights, denotes the weight of the th candidate voxel and the th candidate voxel, and denote the multi-source feature vector of the th candidate voxel and the th candidate voxel, denotes the updated multi-source feature vector of the th candidate voxel, denotes the Euclidean distance.

[0135] By minimizing , the information propagation between the supervised and unsupervised nodes can be realized.

[0136] Step S54: Based on the voxel-level graph structure and the joint loss function, semi-supervised learning is performed on the candidate region feature set and the labeled sample voxel node set to obtain the preliminary crack class probability distribution of each candidate voxel.

[0137] In this step, the labeled sample voxel node set is obtained through a small amount of high-confidence sample labeling, manual auxiliary sample labeling, and historical sample knowledge transfer . And the corresponding nodes in the labeled sample voxel node set are removed from the candidate region feature set to obtain the unlabeled node set .

[0138] The training input is formed by the labeled sample voxel node set and the unlabeled node set :

[0139] ;

[0140] In the formula, and denote the multi-source feature vector of the th candidate voxel and the th candidate voxel, denotes the crack class label of the th candidate voxel.

[0141] The voxel-level graph structure is constructed , wherein, denotes the inter-voxel weight matrix, denotes the degree matrix. ​

[0142] Based on the idea of ​​graph convolution, the aggregated features are calculated for each candidate voxel:

[0143] ;

[0144] In the formula, Indicates the first The learnable parameter matrix of the layer, Represents a non-linear activation function. Indicates the first The set of neighbor nodes of each candidate voxel and Indicates the first The candidate voxels and the first The degree matrix of candidate voxels and They represent the first Layer and first The corresponding layer Aggregation characteristics of candidate voxels Indicates the first The candidate voxels and the first The weights of each candidate voxel, where... .

[0145] The joint loss function is optimized through backpropagation. Gradient descent updates are performed iteratively until the loss function converges or the maximum number of training epochs is reached. After training, the preliminary crack class probability distribution for each candidate voxel is obtained:

[0146] ;

[0147] In the formula, Indicates the first The first candidate voxel The aggregated features of the layers, i.e., the final output features of the layers. Represents the normalization function. Indicates the first The preliminary crack category probability distribution of the candidate voxels, including the first... The probability that each candidate voxel belongs to a different crack category.

[0148] Step S55: Optimize the initial crack category probability distribution based on the differentiable conditional random field to generate a crack segmentation map and crack categories.

[0149] Based on the voxel-level semi-supervised classification results, this step utilizes differentiable conditional random fields to introduce spatial smoothness constraints, directional consistency constraints, and feature similarity constraints to optimize the preliminary category probability distribution, thereby obtaining the final crack segmentation results and category determination.

[0150] Step S55 includes:

[0151] Step S551: Construct a single-point potential function based on the preliminary crack category probability distribution;

[0152] In this step, the single-point potential represents the uncertainty of the model's prediction for each voxel category. Therefore, a single-point potential function is defined for each candidate voxel to reflect the initial confidence that the candidate voxel belongs to a certain crack category:

[0153] ;

[0154] In the formula, Indicates about The single-point potential function, Indicates the first The initial crack category probability distribution of candidate voxels. The negative logarithmic form ensures that the higher the confidence level, the lower the potential energy.

[0155] Step S552: Construct an attention-guided nonlocal kernel, and construct a two-point potential function through the nonlocal kernel;

[0156] In this step, the bi-point potential adaptively selects important neighborhoods through an attention mechanism, achieving nonlocal feature association and spatial consistency propagation. Therefore, to introduce spatial and feature similarity relationships between voxels, an attention-guided nonlocal kernel function is defined to construct the bi-point potential function:

[0157] ;

[0158] In the formula, Indicates about and The two-point potential function, Indicates about and Tag compatibility functions, if and If the tags are the same, the value is 0; otherwise, the value is 1. and They represent the first The candidate voxels and the first One candidate voxel.

[0159] in, For about and The nonlocal kernel function is defined as:

[0160] ;

[0161] In the formula, and They represent the first The candidate voxels and the first a spatial coordinate of the i-th candidate voxel, and respectively represent a multi-source feature vector of the i-th candidate voxel and the j-th candidate voxel, and respectively represent a first scale parameter and a third scale parameter, represents an Euclidean distance, represents an exponential function, represents an attention weight, which is calculated by feature similarity. Step S553: defining a region potential function based on the multi-source feature vector of the candidate voxel;

[0162] In this step, to prevent the influence of noise and pseudo-cracks, a region potential function is introduced to constrain the consistency of the local region:

[0163]

[0164] ;

[0165] ;

[0166] In the formula, represents a region potential function about the i-th candidate voxel, represents a local region centered on the i-th candidate voxel, represents a spatial weighting coefficient, and respectively represent a spatial coordinate of the i-th candidate voxel and the j-th candidate voxel, represents a fourth scale parameter, represents a region potential balance weight, represents an Euclidean distance, represents an exponential function, and respectively represent a multi-source feature vector of the i-th candidate voxel and the j-th candidate voxel. And through the region potential term, local feature mutations can be inhibited, so that the crack boundary is smoothed but the real boundary position is still maintained. Step S554: performing a message aggregation operation by a two-point potential function to generate an aggregated message; In this step, based on the two-point potential function, a message passing and aggregation operation is performed to propagate information between adjacent voxels:

[0167]

[0168] Step S554: performing a message aggregation operation by a two-point potential function to generate an aggregated message;

[0169] In this step, based on the two-point potential function, a message passing and aggregation operation is performed to propagate information between adjacent voxels:

[0170] ​​​​​​;

[0171] wherein, denotes the th iteration, denotes the aggregated message from neighbor nodes to the th candidate voxel after the th iteration, denotes the two-point potential function for denotes the th iteration, denotes the crack class probability distribution of the th candidate voxel after the

[0172] th iteration.

[0173] Step S555: assigning a regional potential gradient to each candidate voxel by a regional potential function, and calculating a single-point potential by a single-point potential function;

[0174] In this step, the combined response of the regional potential gradient and the single-point potential is calculated:

[0175] ;

[0176] wherein, denotes the th iteration, denotes the combined response of the th candidate voxel after the

[0177] th iteration.

[0178] Step S556: iteratively updating the candidate voxel probability map by combining the aggregated message, the regional potential gradient and the single-point potential, to obtain a converged candidate voxel probability map;

[0179] In this step, the probability is updated by a differentiable mean field inference process:

[0180] ;

[0181] wherein, denotes the th iteration, denotes the crack class probability distribution of the denotes a normalization function.

[0182] The process is repeated for multiple times until convergence or a maximum iteration number is reached, to obtain a converged candidate voxel probability map , denotes a maximum iteration number, and the crack class probability of the candidate voxel is gradually optimized under local constraints and global smoothing in the process, to remove false cracks and refine boundaries.

[0183] Step S557: Post-process the candidate voxel probability map to generate a crack segmentation map and crack categories.

[0184] In this step, the converged probability graph... Post-processing is performed.

[0185] First, classify the voxels to obtain the crack category label for each candidate voxel:

[0186] ;

[0187] In the formula, Indicates the first Crack category label for each candidate voxel Indicates the first In the candidate voxel probability map after iterative convergence, each candidate voxel belongs to a certain category. The probability, This represents the category index corresponding to the highest probability.

[0188] The crack category labels are then mapped to a 3D spatial voxel mesh to generate a crack segmentation map. Isolated pseudo-crack voxels are removed through 3D closure operations, void filling, and connected component analysis to obtain the final crack identification result.

[0189] Example 2:

[0190] like Figure 2 As shown, this embodiment provides a tunnel crack identification system based on multi-source image processing. The system includes:

[0191] The acquisition module is used to acquire multi-source data of horizontal rock tunnels, including infrared images, visible light images, three-dimensional point cloud data, and GPR profile maps.

[0192] The registration module is used to spatially register multi-source data to generate multimodal images under the same reference frame;

[0193] The extraction module is used to extract multimodal features from multimodal images and construct a multi-source feature map under the same spatial grid.

[0194] The initial detection module is used to perform initial crack detection on multi-source feature maps by combining multi-scale filtering and structural tensor analysis to obtain crack candidate region masks.

[0195] The identification module is used to accurately identify cracks through a crack candidate region mask, and to obtain crack identification results by combining bedding direction constraints and differentiable conditional random fields. The crack identification results include crack segmentation maps and crack categories.

[0196] The extraction module includes:

[0197] a first constructing unit, configured to perform three-dimensional voxelization on the multi-modal images to establish a spatial grid model with a unified resolution;

[0198] a first extracting unit, configured to perform feature extraction on the multi-modal images based on the spatial grid model to obtain surface texture features, thermal anomaly features, geometric structure features and underground reflection features under the spatial grid;

[0199] a second constructing unit, configured to establish a bedding direction reference field according to the dip angle and strike information of the bedding surface in the spatial grid model;

[0200] a fusion unit, configured to align and fuse the surface texture features, the thermal anomaly features, the underground reflection features and the bedding direction reference field according to the spatial grid to obtain a multi-source feature map under the same spatial grid.

[0201] The preliminary detection module comprises:

[0202] a second extracting unit, configured to extract a continuous structure candidate region in the multi-source feature map through a multi-scale directional filter to obtain a crack response enhancement map;

[0203] a calculating unit, configured to calculate a structure tensor matrix through gradient information of the multi-source feature map;

[0204] a third constructing unit, configured to construct a preliminary crack response map through the crack response enhancement map and the structure tensor matrix;

[0205] a constraint unit, configured to perform direction constraint on the preliminary crack response map based on the bedding direction reference field to obtain an enhanced crack response map;

[0206] a generating unit, configured to perform preliminary extraction of cracks according to the enhanced crack response map to generate a crack candidate region mask.

[0207] The identification module comprises:

[0208] a third extracting unit, configured to extract a multi-source feature vector of each candidate voxel in the crack candidate region mask from the multi-source feature map to form a candidate region feature set;

[0209] a fourth constructing unit, configured to construct a voxel-level graph structure based on spatial proximity of the candidate voxels and bedding direction similarity;

[0210] a fifth constructing unit, configured to construct a joint loss function based on label propagation, graph regularization and consistency regularization;

[0211] a learning unit, configured to perform semi-supervised learning on the candidate region feature set and a labeled sample voxel node set based on the voxel-level graph structure and the joint loss function to obtain a preliminary crack class probability distribution of each candidate voxel.

[0212] an optimization unit configured to optimize the preliminary fracture class probability distribution according to the differentiable conditional random field to generate a fracture segmentation map and a fracture class.

[0213] It should be noted that the specific manner in which the various modules perform operations in the system of the above embodiments has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0214] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0215] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for identifying tunnel cracks based on multi-source image processing, characterized in that, include: Acquire multi-source data of horizontal rock strata tunnels, including infrared images, visible light images, three-dimensional point cloud data, and GPR profile maps; Spatial registration of multi-source data generates multimodal images under the same reference frame; Multimodal features are extracted from multimodal images, and a multi-source feature map is constructed under the same spatial grid, including: Multimodal images are processed into three-dimensional voxels to establish a spatial mesh model with uniform resolution; Feature extraction is performed on multimodal images based on a spatial grid model to obtain surface texture features, thermal anomaly features, geometric structure features, and underground reflection features under the spatial grid. Establish a bedding direction reference field based on the dip angle and strike information of the bedding planes in the spatial grid model; The surface texture features, thermal anomaly features, subsurface reflection features, and stratification direction reference field are aligned and fused according to the spatial grid to obtain a multi-source feature map under the same spatial grid. After applying directional constraints to the preliminary crack response map constructed by calculating the structural tensor matrix based on the gradient information of the gradient information of the multi-source feature map and the crack response enhancement map extracted by the multi-scale directional filter based on the stratification direction reference field, preliminary crack extraction is performed to generate crack candidate region masks. Accurate crack identification is performed using a crack candidate region mask. The crack identification results are obtained by combining bedding orientation constraints and differentiable conditional random fields. These results include a crack segmentation map and crack categories, including: Extract the multi-source feature vectors of each candidate voxel within the crack candidate region mask from the multi-source feature map to form a candidate region feature set; Based on the spatial proximity relationship and stratification direction similarity of candidate voxels, a voxel-level graph structure is constructed; A joint loss function is constructed based on label propagation, graph regularization, and consistency regularization; Based on the voxel-level graph structure and joint loss function, semi-supervised learning is performed on the candidate region feature set and the labeled sample voxel node set to obtain the preliminary crack category probability distribution for each candidate voxel. The probability distribution of the initial crack category is optimized based on a differentiable conditional random field to generate a crack segmentation map and crack categories.

2. The tunnel crack identification method based on multi-source image processing according to claim 1, characterized in that... The step of spatially registering multi-source data to generate a multimodal image under the same reference frame includes: The coordinate system of the 3D point cloud data is selected as the reference coordinate system; Infrared images, visible light images, and GPR profiles are projected onto a reference coordinate system; Based on the reference coordinate system, infrared images, visible light images, GPR profiles and 3D point cloud data are initially matched to obtain preliminary matching pairs. Based on the initial matching pairs, the transformation matrix is ​​calculated. The infrared image, visible light image, and GPR profile are then coarsely registered with the 3D point cloud data using the transformation matrix to obtain the coarse registration result. The three-dimensional model of the bedding plane is extracted based on the coarse registration results. The three-dimensional model of the bedding plane includes horizontal bedding planes, bedding texture lines, and bedding reflection lines. Calculation of registration deviation based on 3D model of layered surfaces; The coarse registration result is optimized based on the registration deviation to obtain a multimodal image under the same reference frame.

3. The tunnel crack identification method based on multi-source image processing according to claim 1, characterized in that... The process involves constraining the direction of the preliminary crack response map constructed from the crack response enhancement map extracted by the multi-scale directional filter and the preliminary crack response map constructed by calculating the structure tensor matrix based on the gradient information of the multi-source feature map using the layered direction reference field. Then, preliminary crack extraction is performed to generate a crack candidate region mask, including: A crack response enhancement map is obtained by extracting continuous structural candidate regions from the multi-source feature map using a multi-scale directional filter. The structure tensor matrix is ​​calculated using the gradient information from the multi-source feature maps; A preliminary crack response map is constructed using the crack response enhancement map and the structural tensor matrix; An enhanced crack response map is obtained by applying directional constraints to the preliminary crack response map based on the stratification direction reference field. Preliminary crack extraction is performed based on the enhanced crack response map, generating a crack candidate region mask.

4. The tunnel crack identification method based on multi-source image processing according to claim 1, characterized in that... The step of optimizing the initial crack category probability distribution based on a differentiable conditional random field to generate a crack segmentation map and crack categories includes: Construct a single-point potential function based on the preliminary crack category probability distribution; Construct an attention-guided nonlocal kernel, and then construct a two-point potential function using the nonlocal kernel; Define the region potential function based on the multi-source eigenvectors of candidate voxels; Message aggregation is performed using a two-point potential function to generate aggregated messages; The region potential gradient is assigned to each candidate voxel by the region potential function, and the single-point potential is calculated by the single-point potential function. By combining aggregated messages, regional potential gradients, and single-point potentials for iterative updates, a converged candidate voxel probability map is obtained. Post-processing of candidate voxel probability maps generates crack segmentation maps and crack categories.

5. A tunnel crack identification system based on multi-source image processing, characterized in that, include: The acquisition module is used to acquire multi-source data of horizontal rock tunnels, including infrared images, visible light images, three-dimensional point cloud data, and GPR profile maps. The registration module is used to spatially register multi-source data to generate multimodal images under the same reference frame; The extraction module is used to extract multimodal features from multimodal images and construct a multi-source feature map under the same spatial grid. The extraction module includes: The first building unit is used to perform three-dimensional voxelization processing on multimodal images and establish a spatial mesh model with uniform resolution. The first extraction unit is used to extract features from multimodal images based on a spatial grid model, and obtain surface texture features, thermal anomaly features, geometric structure features and underground reflection features under the spatial grid. The second building unit is used to establish a bedding direction reference field based on the dip angle and orientation information of the bedding planes in the spatial grid model. The fusion unit is used to align and fuse surface texture features, thermal anomaly features, subsurface reflection features, and bedding orientation reference fields according to a spatial grid to obtain a multi-source feature map under the same spatial grid. The initial detection module is used to perform initial crack extraction and generate crack candidate region masks after applying directional constraints to the preliminary crack response map constructed by calculating the structural tensor matrix based on the gradient information of the gradient information of the multi-scale directional filter and the crack response enhancement map based on the layering direction reference field. The identification module is used to accurately identify cracks through a crack candidate region mask, and to obtain crack identification results by combining bedding direction constraints and differentiable conditional random fields. The crack identification results include crack segmentation maps and crack categories. The identification module includes: The third extraction unit is used to extract the multi-source feature vectors of each candidate voxel in the crack candidate region mask from the multi-source feature map to form a candidate region feature set. The fourth building unit is used to construct a voxel-level graph structure based on the spatial proximity relationship and stratification direction similarity of candidate voxels; The fifth building unit is used to construct a joint loss function based on label propagation, graph regularization, and consistency regularization; The learning unit is used to perform semi-supervised learning on the candidate region feature set and the labeled sample voxel node set based on the voxel-level graph structure and joint loss function to obtain the preliminary crack category probability distribution of each candidate voxel. The optimization unit is used to optimize the initial crack category probability distribution based on a differentiable conditional random field, and generate crack segmentation maps and crack categories.

6. The tunnel crack identification system based on multi-source image processing according to claim 5, characterized in that, The initial detection module includes: The second extraction unit is used to extract continuous structural candidate regions from the multi-source feature map through a multi-scale directional filter to obtain a crack response enhancement map. The computation unit is used to calculate the structure tensor matrix using gradient information from multi-source feature maps; The third building unit is used to construct a preliminary crack response map using the crack response enhancement map and the structural tensor matrix; Constraint elements are used to constrain the initial crack response map based on the bedding direction reference field to obtain an enhanced crack response map. The generation unit is used to perform preliminary crack extraction based on the enhanced crack response map and generate a crack candidate region mask.

Images