Bridge structure surface crack intelligent monitoring method and system based on deep learning

Abstract

This application discloses a deep learning-based intelligent monitoring method and system for surface cracks in bridge structures. First, it performs spatiotemporal alignment and heterogeneous fusion of multi-source sensor data, then utilizes a dual-stream deep neural network to extract the probability distribution features and pixel-level extension direction fields of the cracks in parallel. Based on this, by introducing a directionally regularized potential energy field and a non-maximum suppression mechanism, high-precision skeleton extraction conforming to hydrodynamic characteristics is achieved. Furthermore, this scheme transforms the visual perception space into a graph theory space, constructing a connection cost matrix based on crack breakpoints and their local tangent directions, and using a minimum spanning tree algorithm to intelligently stitch together visually broken but logically connected paths. Finally, by combining depth information to invert the true physical size and hazard level of the cracks, it achieves a leap from simple image segmentation to physical structural integrity assessment, ensuring the topological authenticity and engineering usability of the monitoring results.

Description

Technical Field

[0001] This application relates to the field of bridge structure monitoring technology, and more specifically, to a method and system for intelligent monitoring of surface cracks in bridge structures based on deep learning. Background Technology

[0002] Bridge structural health monitoring is a core component of ensuring the safe operation of transportation infrastructure. Among key indicators, concrete surface cracks are the most direct reflection of structural stress state and potential defects. Traditional crack detection relies primarily on manual visual inspection with instruments or on patrol using bridge inspection vehicles. This approach is not only inefficient and prone to missed detections, but also poses significant operational safety risks in complex scenarios such as crossing rivers and valleys. With the rapid development of artificial intelligence technology, automated crack recognition technology based on deep learning has gradually become the industry mainstream. This technology can achieve pixel-level segmentation and localization of crack areas by training neural networks.

[0003] However, existing deep learning-based crack monitoring methods still face significant technical challenges in practical engineering applications. First, the complex outdoor environment of bridges, influenced by uneven lighting conditions under the beam, shadows from ancillary facilities, or surface dust and dirt, often causes physically continuous long cracks to appear as discontinuous dashed lines in acquired images. Existing algorithms primarily rely on convolutional kernels to extract local texture features, and the commonly used cross-entropy loss function is based on the pixel independence assumption, lacking the ability to perceive and infer the global geometric topological relationship of cracks. This leads to the network easily misclassifying cracks as background in areas with weak texture or occlusion, resulting in topological breaks in the cracks and an inability to accurately reconstruct their true physical connectivity length and structural hazards. Second, when converting the segmentation mask into the required skeleton geometric parameters for engineering, existing techniques typically employ isotropic morphological thinning algorithms (such as the Zhang-Suen algorithm) and denoising strategies based on a single length threshold. This approach ignores the inherent directional field characteristics of cracks as stressed manifold structures, easily leading to skeleton distortion and node offset at crack intersections or width changes. Meanwhile, simple length filtering cannot effectively distinguish between tiny new cracks with consistent flow direction and irregular linear artifacts (such as water stains and template joints), which can easily lead to the dilemma of missed detection of real cracks and residual artifacts, seriously affecting the accuracy and reliability of the final monitoring report.

[0004] Therefore, an optimized intelligent monitoring scheme for surface cracks in bridge structures is desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method and system for intelligent monitoring of surface cracks in bridge structures based on deep learning.

[0006] According to one aspect of this application, a deep learning-based intelligent monitoring method for surface cracks in bridge structures is provided, comprising:

[0007] S1: Acquire the raw sensor data stream containing the color image array, depth point cloud data, and camera intrinsic parameter matrix;

[0008] S2: Based on the camera intrinsic parameter matrix, spatial alignment and enhancement processing are performed on the depth point cloud data and the color image array, and the processed data is channel-stitched and normalized to obtain a composite input tensor.

[0009] S3: Based on a pre-set dual-stream deep neural network model, dual-stream feature encoding and orientation field prediction are performed on the composite input tensor to obtain a segmentation feature map that predicts the probability of crack existence and an orientation field feature map that predicts the pixel extension trend of crack.

[0010] S4: Based on the orientation field feature map, perform binarization segmentation on the segmentation feature map under topological consistency constraints to obtain a skeleton binary map;

[0011] S5: Based on the orientation field feature map, perform crack breakpoint repair on the skeleton binary map based on the minimum spanning tree to obtain the repaired topology map;

[0012] S6: Perform physical parameter quantization and monitoring report generation on the repaired topology graph and composite input tensor to obtain the final monitoring report data.

[0013] According to another aspect of this application, a deep learning-based intelligent monitoring system for surface cracks in bridge structures is provided, comprising:

[0014] The data acquisition module is used to acquire raw sensor data streams containing color image arrays, depth point cloud data, and camera intrinsic parameter matrices.

[0015] The data preprocessing module is used to perform spatial alignment and enhancement processing on depth point cloud data and color image array based on camera intrinsic parameter matrix, and to perform channel stitching and normalization on the processed data to obtain composite input tensor.

[0016] The dual-stream feature prediction module is used to perform dual-stream feature encoding and orientation field prediction on the composite input tensor based on a preset dual-stream deep neural network model to obtain a segmentation feature map that predicts the probability of crack existence and an orientation field feature map that predicts the pixel extension trend of crack.

[0017] The topology constraint segmentation module is used to perform binarization segmentation of the segmentation feature map under topology consistency constraints based on the orientation field feature map to obtain a skeleton binary map.

[0018] The crack breakpoint repair module is used to repair crack breakpoints in the skeleton binary map based on the orientation field feature map to obtain a repaired topology map.

[0019] The parameter quantization and report generation module is used to perform physical parameter quantization and monitoring report generation on the repair topology graph and composite input tensor to obtain the final monitoring report data.

[0020] Compared with existing technologies, this invention proposes a deep learning-based intelligent monitoring method for surface cracks in bridge structures. First, it utilizes spatial alignment and fusion of deep point clouds and color images to construct a composite input tensor containing geometric and texture information. This tensor is then input into a two-stream neural network to predict the probability distribution of cracks and their pixel-level extension direction fields in parallel. Based on this, the direction field is introduced as a strong topological constraint, and an anisotropic potential field guides skeleton extraction, effectively eliminating the jagged edges and center offsets produced by traditional isotropic thinning. Furthermore, the visually perceived crack skeleton is transformed into a graph structure, and a connection cost matrix is ​​constructed by combining directional consistency and Euclidean distance. A minimum spanning tree algorithm is used to intelligently stitch together logical breakpoints caused by occlusion or weak texture. Finally, by combining depth information to invert pixel physical proportions, accurate quantification of the true connected length and width of the repaired cracks is achieved. Attached Figure Description

[0021] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 This is a flowchart of a deep learning-based intelligent monitoring method for surface cracks in bridge structures according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of data flow in a deep learning-based intelligent monitoring method for surface cracks in bridge structures according to an embodiment of this application.

[0024] Figure 3 This is a flowchart illustrating the process of performing dual-stream feature encoding and orientation field prediction on a composite input tensor to obtain a segmentation feature map predicting the probability of crack existence and an orientation field feature map predicting the pixel extension trend of cracks, based on a pre-set dual-stream deep neural network model of a deep learning-based intelligent monitoring method for surface cracks in bridge structures according to embodiments of this application.

[0025] Figure 4This is a flowchart illustrating the process of performing binarization segmentation of a segmented feature map under topological consistency constraints to obtain a skeleton binary map based on the orientation field feature map of the intelligent monitoring method for surface cracks on bridge structures based on deep learning, according to an embodiment of this application.

[0026] Figure 5 This is a flowchart illustrating the topology-preserving morphological skeletonization of a binary mask to obtain a skeleton binary map, according to the deep learning-based intelligent monitoring method for surface cracks in bridge structures in this application.

[0027] Figure 6 This is a flowchart illustrating the process of repairing crack breakpoints in a skeleton binary map based on a minimum spanning tree to obtain a repair topology map, according to the orientation field feature map of the intelligent monitoring method for surface cracks in bridge structures based on deep learning, as described in the embodiments of this application.

[0028] Figure 7 This is a block diagram of a deep learning-based intelligent monitoring system for surface cracks in bridge structures, according to an embodiment of this application. Detailed Implementation

[0029] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0030] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0031] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on bridge health monitoring center server clusters, UAV / inspection vehicle edge computing terminals, fixed data acquisition gateways on bridges, and / or cloud platforms. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0032] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0033] Existing deep learning-based bridge crack monitoring technologies mainly rely on convolutional kernels to extract local texture features. When faced with complex environments such as uneven lighting, shadow occlusion, or stains on the bridge surface, the weakening of texture features can easily cause physically continuous long cracks to appear as discontinuous fragments. Furthermore, they lack the ability to perceive and logically repair the global topological structure of cracks, which seriously affects the accuracy of structural safety assessment. Therefore, this application proposes a deep learning-based intelligent monitoring method for surface cracks in bridge structures. Specifically, the method first uses camera intrinsic parameters to spatially align and multi-dimensionally fuse depth point clouds and color images, constructing a composite input tensor containing geometric and texture information. This tensor is then input into a two-stream deep neural network to decode the probability distribution map of crack existence and the orientation field feature map of pixel-level extension trends. Based on this, the traditional isotropic thinning strategy is abandoned, and an anisotropic potential field is constructed using orientation field features to guide skeleton extraction, effectively eliminating jagged edges and ensuring the skeleton is located on the physical centerline. The visual skeleton is then transformed into a graph structure, and the Euclidean distance and orientation consistency connection cost between breakpoints are calculated. The minimum spanning tree algorithm is used to intelligently stitch together logical breakpoints caused by occlusion. Finally, the pixel physical ratio is inverted using depth information to achieve accurate quantification of the true connectivity length and width of the repaired crack.

[0034] Figure 1 This is a flowchart of a deep learning-based intelligent monitoring method for surface cracks in bridge structures according to an embodiment of this application. Figure 2 This is a schematic diagram of data flow in a deep learning-based intelligent monitoring method for surface cracks in bridge structures, according to an embodiment of this application. Figure 1 and Figure 2 As shown, the intelligent monitoring method for surface cracks in bridge structures based on deep learning according to an embodiment of this application includes the following steps: S1, acquiring the original sensor data stream containing a color image array, depth point cloud data, and a camera intrinsic parameter matrix; S2, based on the camera intrinsic parameter matrix, spatially aligning and enhancing the depth point cloud data and the color image array, and then performing channel stitching and normalization on the processed data to obtain a composite input tensor; S3, based on a preset two-stream deep neural network model, performing two-stream feature encoding and orientation field prediction on the composite input tensor to obtain a segmentation feature map predicting the probability of crack existence and an orientation field feature map predicting the pixel extension trend of cracks; S4, based on the orientation field feature map, performing binarization segmentation on the segmentation feature map under topological consistency constraints to obtain a skeleton binary map; S5, based on the orientation field feature map, performing crack breakpoint repair on the skeleton binary map based on minimum spanning tree to obtain a repair topology map; S6, performing physical parameter quantization and monitoring report generation on the repair topology map and the composite input tensor to obtain the final monitoring report data.

[0035] Specifically, in step S1, a raw sensor data stream containing a color image array, depth point cloud data, and a camera intrinsic parameter matrix is ​​acquired. It should be noted that single two-dimensional image data lacks crucial spatial depth information when facing complex bridge monitoring scenarios. This makes it impossible to accurately correct perspective distortion caused by tilted shooting angles, and it is also difficult to directly support sub-pixel-level physical quantization of crack width and length, leading to discrepancies between subsequent pixel-based geometric measurements and the actual physical dimensions. Therefore, the technical solution of this application first acquires a raw sensor data stream containing a color image array, depth point cloud data, and a camera intrinsic parameter matrix. Through the above processing, a complete data foundation including texture and geometric features can be provided for subsequent multimodal data fusion, effectively eliminating the uncertainty in physical scale conversion in monocular vision and ensuring the engineering usability of the monitoring results.

[0036] More specifically, in a particular example of this application, this step first involves scanning the bridge target surface using an integrated sensor array mounted on a drone or other mobile monitoring platform. This sensor array integrates a high-resolution industrial camera and a lidar ranging device. During the scanning process, the industrial camera is controlled to acquire a color image array reflecting the texture details of the bridge surface at a set sampling frequency. Each frame of the image records visual information including crack edges, lighting and shadows, and surface stains. Simultaneously, the lidar emits laser pulses and receives echoes, synchronously acquiring depth point cloud data within the corresponding field of view. This point cloud data consists of a massive number of three-dimensional spatial coordinate points, accurately recording the unevenness and geometric distance information of the bridge surface. Furthermore, the camera intrinsic parameter matrix is ​​directly read from a pre-stored calibration file. This matrix contains key optical parameters such as the camera's focal length, principal point coordinates, and distortion coefficients, which are used to subsequently establish a mathematical mapping relationship between the two-dimensional image pixel coordinate system and the three-dimensional spatial coordinate system. Finally, the synchronously acquired color images, depth point clouds, and read intrinsic parameter matrices are packaged and encapsulated to form a raw sensor data stream with timestamp synchronization marks, and transmitted to the computing unit for subsequent processing steps.

[0037] Specifically, in step S2, based on the camera intrinsic parameter matrix, the depth point cloud data and the color image array are spatially aligned and enhanced, and the processed data are then channel-stitched and normalized to obtain a composite input tensor. It should be noted that, given that existing bridge crack monitoring technologies mostly rely solely on the texture features of two-dimensional images, they often struggle to distinguish between real cracks with physical concavity and convexity and flat surface noise when faced with complex water stains, oil pollution, and shadow occlusion caused by uneven lighting on bridge surfaces. Furthermore, sensor data from different sources such as LiDAR and industrial cameras exhibit inherent spatial heterogeneity. Direct input without unified processing can lead to difficulties in effectively fusing features and training convergence for neural networks. Therefore, the technical solution of this application further projects the spatial points in the 3D point cloud set onto the pixel coordinate system of the original color image based on the camera intrinsic parameter matrix, and interpolates and fills the sparse projection matrix to construct an aligned depth map. Simultaneously, the original color image undergoes contrast-limited adaptive histogram equalization to enhance the detail contrast in shadow areas. Subsequently, the aligned depth data and enhanced image data are numerically normalized and stitched together along the channel dimension to construct a composite input tensor. Through the above processing, discrete physical geometric depth information and continuous visual texture information can be accurately aligned and complemented at the pixel level. This not only enhances the feature saliency of small cracks in dark areas and occluded regions through image enhancement, but also provides rich features that integrate visual appearance and physical structure for subsequent models through standardized multidimensional tensor input. This effectively avoids false alarms caused by relying solely on texture and improves the robustness of the algorithm in complex engineering environments.

[0038] More specifically, in a specific example of this application, step S2 includes: parsing the original sensor data stream to obtain an original color image, a 3D point cloud set, and a camera intrinsic parameter matrix; projecting the spatial points in the 3D point cloud set onto the pixel coordinate system of the original color image based on the camera intrinsic parameter matrix, and interpolating and filling the sparse matrix after projection to obtain a depth channel map aligned with the pixels of the original color image; performing CLAHE-based shadow enhancement processing on the original color image to obtain an enhanced color image; performing numerical normalization processing on the enhanced color image and the depth channel map respectively, and concatenating the normalized enhanced color image and the depth channel map in the channel dimension to obtain a composite input tensor.

[0039] Accordingly, the original sensor data stream is parsed to obtain the original color image, the 3D point cloud set, and the camera intrinsic parameter matrix. Then, based on the camera intrinsic parameter matrix, the spatial points in the 3D point cloud set are projected onto the pixel coordinate system of the original color image. The projected sparse matrix is ​​then interpolated and filled to obtain a depth channel map aligned with the pixels of the original color image. It should be noted that, due to the inherent heterogeneity in spatial coordinate system definition, data density, and resolution between the 3D point cloud data acquired by the LiDAR and the 2D image data acquired by the industrial camera, direct superposition will lead to spatial misalignment of geometric and texture features. Furthermore, the sparse point cloud cannot directly cover every pixel of the high-resolution image, making it difficult for subsequent networks to establish a precise correspondence between physical depth and visual appearance. Based on this, the technical solution of this application further parses the original sensor data stream to obtain the original color image, the 3D point cloud set, and the camera intrinsic parameter matrix. Then, based on the camera intrinsic parameter matrix, the spatial points in the 3D point cloud set are projected onto the pixel coordinate system of the original color image. The projected sparse matrix is ​​then interpolated and filled to obtain a depth channel map aligned with the pixels of the original color image. Through the above processing, discrete physical ranging information can be accurately mapped to a continuous image plane using an optical imaging model. The resolution mismatch of heterogeneous sensors can be solved by numerical padding, thereby providing pixel-level strictly aligned geometric depth prior information for crack detection, effectively assisting the model in distinguishing between real structural cracks and surface stains on a plane.

[0040] More specifically, in a concrete example of this application, the data packets with timestamp synchronization markers transmitted back by the UAV or mobile monitoring vehicle are first unpacked to separate the high-resolution color photographs taken at the same acquisition time, the set of spatial point coordinates generated by the LiDAR scan, and the pre-calibrated camera intrinsic parameter data, which includes focal length, principal point coordinates, and lens distortion parameters. Next, each valid spatial point in the 3D point cloud set is traversed, and based on the pinhole camera imaging principle and rigid body transformation matrix, it is transformed from the world coordinate system or radar coordinate system to the camera coordinate system. Then, the intrinsic parameter matrix is ​​used to project it onto the 2D image plane, calculate the row and column coordinate index of the spatial point on the image, and fill the depth distance value of the point into an initial zero matrix of the same size as the color image. To address the issue of numerous unassigned pixel holes in the initial matrix due to the sparse point cloud wiring, a bilinear interpolation or bilateral filtering algorithm is employed. This algorithm uses the depth values ​​of known projection points to perform weighted estimation and numerical filling of empty pixels in the neighborhood, ultimately generating a dense and continuous depth grayscale image. Each pixel in this depth image contains physical distance information that strictly coincides with the spatial location of its corresponding color pixel.

[0041] Accordingly, the original color image undergoes CLAHE-based shadow enhancement processing to obtain an enhanced color image. The enhanced color image and the depth channel image are then numerically normalized. Finally, the normalized enhanced color image and the depth channel image are concatenated along the channel dimension to obtain a composite input tensor. It should be noted that because the monitoring areas, such as the bottom of the bridge beam and the inner side of the box girder, are in a backlit environment year-round, large areas of shadow exist in the acquired images. This results in a slight difference in grayscale between the crack texture and the background concrete. Furthermore, the pixel grayscale values ​​of the original image and the physical distance values ​​of the depth channel differ in magnitude. Directly mixing the inputs would cause instability in the neural network weight updates. Therefore, the technical solution of this application further performs CLAHE-based shadow enhancement processing on the original color image to obtain an enhanced color image. The enhanced color image and the depth channel image are then numerically normalized. Finally, the normalized enhanced color image and the depth channel image are concatenated along the channel dimension to obtain a composite input tensor. Through the above processing, the edge contrast of tiny cracks in the shadow area can be effectively improved by using local histogram equalization technology, and the dimensional differences between heterogeneous data can be eliminated by numerical standardization. This allows for the construction of multi-dimensional feature inputs containing texture details and geometric depth for deep learning models, ensuring the convergence speed and detection accuracy of the model under complex lighting conditions.

[0042] More specifically, in a concrete example of this application, this step first converts the original RGB format color image to the LAB color space to separate the luminance and chrominance components. Then, for the luminance component, an adaptive histogram equalization algorithm with limited contrast is used to divide the image into multiple non-overlapping rectangular sub-blocks, and the grayscale histogram of each sub-block is calculated. Based on the histogram cropping to limit noise amplification, pixel values ​​are redistributed to expand local contrast. The processed luminance component is then merged with the original chrominance component and converted back to the RGB space to obtain an enhanced image with clear texture. Next, a min-max normalization method is used to read the pixel intensity values ​​of the enhanced color image and the distance measurements of the depth channel map, linearly mapping them to a floating-point range of zero to one. Finally, using matrix concatenation, the normalized three-channel color image data and single-channel depth data are stacked in the depth direction to construct a composite tensor data structure with four channel dimensions, which serves as the standard input data for subsequent convolutional neural networks.

[0043] Specifically, in step S3, based on a pre-set two-stream deep neural network model, two-stream feature encoding and orientation field prediction are performed on the composite input tensor to obtain a segmentation feature map predicting the probability of crack existence and an orientation field feature map predicting the pixel extension trend of cracks. It should be noted that, given that traditional deep learning segmentation networks mainly rely on local texture features for pixel-level classification, and their loss functions are mostly based on the pixel independence assumption, they often ignore the inherent geometric topological continuity of cracks as stressed physical structures. This leads to difficulties in extracting significant edge features when weak texture areas such as light and shadow, dust coverage, or partial occlusion appear on the bridge surface, resulting in the network misclassifying physically continuous cracks as background. Consequently, the detection results appear as discontinuous dashed lines, failing to accurately reflect the true extension length and structural hazard of the cracks. Therefore, the technical solution of this application further utilizes a pre-set two-stream deep neural network model to perform two-stream feature encoding and orientation field prediction on the composite input tensor to obtain a segmentation feature map predicting the probability of crack existence and an orientation field feature map predicting the pixel extension trend of cracks. Through the above processing, the probability distribution information reflecting the appearance of the crack and the spatial vector field information reflecting the physical manifold characteristics of the crack can be decoded in parallel under a unified feature extraction framework. This allows the model to learn the extension trend of the crack by utilizing the orientation field features, ensuring that potential connection paths can still be indicated by orientation vectors in areas where visual texture is missing. This provides key geometric prior constraints for subsequent breakpoint repair and high-precision skeleton extraction based on topological consistency, effectively overcoming the technical defect of simple pixel segmentation technology that is prone to topological breakage under complex interference.

[0044] Figure 3 This document describes a flowchart illustrating the process of using a pre-set dual-stream deep neural network model to perform dual-stream feature encoding and orientation field prediction on a composite input tensor to obtain a segmentation feature map predicting the probability of crack presence and an orientation field feature map predicting the pixel extension trend of cracks, according to an embodiment of this application. Figure 3 As shown, step S3 includes: S31, initial encoding of the composite input tensor based on the convolutional layer adapted to four-channel input, and extraction of abstract semantic features of different resolutions through a deep residual network to obtain a multi-scale feature pyramid; S32, decoding of the multi-scale feature pyramid by a segmentation branch based on dilated convolution to obtain a segmentation feature map; S33, inputting the multi-scale feature pyramid into the orientation prediction branch to obtain an orientation field feature map.

[0045] In step S31, the composite input tensor is initially encoded based on a convolutional layer adapted to four-channel input, and abstract semantic features at different resolutions are extracted through a deep residual network to obtain a multi-scale feature pyramid. It should be noted that existing mainstream deep convolutional neural networks are typically designed for three-channel visible light images and cannot be directly compatible with the four-channel composite data containing additional depth information constructed in this application. Furthermore, features at a single resolution are insufficient to simultaneously capture the fine texture of small cracks and the global topological structure of long cracks, leading to dimensionality mismatch or loss of physical geometric information when directly inputting them. Therefore, the technical solution of this application further uses a convolutional layer adapted to four-channel input to initially encode the composite input tensor and extracts abstract semantic features at different resolutions through a deep residual network to obtain a multi-scale feature pyramid. Through the above processing, a deep learning model can establish a low-level perception capability for multimodal heterogeneous data, capturing the rich semantics of crack targets from local details to macroscopic morphology using a multi-level pyramid structure, thereby effectively improving the model's feature representation ability and generalization level for cracks of different widths and lengths.

[0046] More specifically, in a concrete example of this application, the ResNet-101 deep residual network is first selected as the backbone feature extraction architecture. To address the limitation that the first layer of this network only supports three input channels, the structure is reconstructed, building a convolutional layer with four input channels to replace the original first-layer convolutional structure. This new convolutional layer uses a 7x7 kernel size. During the weight initialization stage, a strategy combining channel reuse and random initialization is adopted. The convolutional weights corresponding to the red, green, and blue channels in the pre-trained model are directly copied to the first three channels of the new convolutional layer, and the convolutional weights corresponding to the fourth channel are initialized with a random Gaussian distribution. This preserves the original visual feature extraction capabilities while providing the network with initial values ​​for perceiving depth geometric information. Subsequently, the composite input tensor is input into the reconstructed backbone network, sequentially passing through four stages of residual block processing. As the network depth increases, the resolution of the feature maps gradually decreases while the semantic abstraction level gradually increases. During the inference process, the output feature maps with a step size of 4 in the first stage, 8 in the second stage, 16 in the third stage, and 32 in the fourth stage are extracted respectively. This set of feature tensors with different spatial resolutions and semantic levels is defined as a multi-scale feature pyramid, which serves as a shared feature base for subsequent segmentation and orientation prediction tasks.

[0047] In step S32, the multi-scale feature pyramid is decoded using a segmentation branch based on dilated convolution to obtain a segmentation feature map. It should be noted that bridge cracks typically exhibit extreme aspect ratios, and the width of tiny cracks often occupies only a few pixels. After continuous downsampling operations by a deep convolutional neural network, while high-level features possess strong semantic information, they severely lose spatial details crucial for localization. This makes it difficult to simultaneously cover the fine local edges of cracks and long-distance contextual dependencies using conventional convolutions with a fixed receptive field, easily leading to missed detection of micro-cracks or misclassification of continuous cracks as fractures. Therefore, the technical solution of this application further decodes the multi-scale feature pyramid using a segmentation branch based on dilated convolution to obtain a segmentation feature map. Through the above processing, dilated convolution kernels with different dilation rates can expand the effective receptive field without reducing the feature map resolution, enabling multi-level capture and cross-level fusion of crack features at different scales. This effectively solves the technical problem that small target features are easily submerged by background noise, generating a pixel-level crack existence probability distribution with high spatial resolution and strong semantic discriminative power.

[0048] More specifically, in a concrete example of this application, the fourth-stage feature tensor, which has the highest semantic level in the multi-scale feature pyramid, is first selected as input and fed into the dilated spatial pyramid pooling module for parallel processing. This module contains four parallel convolutional branches with different dilation rates, set to 1, 6, 12, and 18, respectively, corresponding to different field-of-view ranges to simultaneously capture the local texture details and long-range structural context of the cracks. Subsequently, the deep semantic features output by the dilated spatial pyramid pooling module are upsampled to the original model using a bilinear interpolation algorithm. Figure 4 One-third of the size, and then spliced ​​and fused with certain first-stage shallow features that retain rich spatial edge information in the multi-scale feature pyramid along the channel dimension. Next, a convolution kernel with a size of [missing information] is used. The convolutional layer compresses the fused multi-channel feature tensor into two output channels, which correspond to the non-normalized prediction scores of the pixel belonging to the background category and the crack category, respectively. Then, bilinear interpolation is used again to restore the spatial size of the prediction result to be completely consistent with the original input image. Finally, the output is a matrix that records the crack confidence value of each pixel, which is the segmentation feature map.

[0049] In step S33, the multi-scale feature pyramid is input to the orientation prediction branch to obtain the orientation field feature map. It should be noted that, since cracks, as a physical manifold structure generated by force, have significant geometric extension, traditional semantic segmentation networks typically classify based on the pixel independence assumption, focusing only on local texture features while ignoring the global topological orientation. This leads to the network easily predicting physically continuous cracks as broken discrete segments due to loss of directional guidance when encountering weak texture conditions such as lighting shadows, surface stains, or partial occlusion. Therefore, the technical solution of this application further inputs the multi-scale feature pyramid to the orientation prediction branch to obtain the orientation field feature map. Through the above processing, the spatial extension trend of each pixel can be decoded in parallel using a regression task, introducing a strong directional physical constraint for crack detection, thereby effectively compensating for the limitations of simple texture recognition and providing a crucial vector field basis for subsequent breakpoint repair based on flow field consistency.

[0050] More specifically, in a concrete example of this application, a regression decoding branch parallel to the segmentation branch is constructed, which also uses a multi-scale feature pyramid as input data. First, a series of convolutional operations are used to fuse and upsample the abstract features across multiple levels, gradually restoring spatial resolution. However, instead of class probability mapping at the final output layer, a convolutional layer with two output channels predicts the vector components of each pixel in a two-dimensional plane. For each pixel in the image, the network regression output contains the original vector with horizontal and vertical axis components. To eliminate the influence of crack width or contrast differences on the vector magnitude and obtain a pure direction indication, L2 norm normalization is performed on the original vector. This process first calculates the sum of squares of the horizontal and vertical components, adds a very small smoothing constant to this sum, and then takes the square root to obtain the vector magnitude. The smoothing constant is introduced to prevent division by zero errors caused by background regions having a magnitude of 0. Subsequently, the original horizontal and vertical components are divided by the calculated vector magnitude to generate a unit direction vector with a magnitude of 1. The final output orientation field feature map is a matrix with two channels, where each pixel location stores a normalized unit vector that precisely indicates the tangential extension direction of the potential crack at that point.

[0051] Specifically, to ensure that the orientation prediction branch in the dual-stream deep neural network can accurately capture the geometric extension trend of cracks, this embodiment also involves a strategy for constructing ground truth labels for the orientation field and supervising training. When creating the training dataset, the manually labeled binary segmentation mask for cracks is first morphologically thinned to extract the single-pixel center skeleton line of the crack, and the local tangent unit vector of each pixel on the skeleton line is calculated using a first-order difference operator. Subsequently, a nearest neighbor diffusion algorithm based on Euclidean distance is used to project the tangent direction of each point on the skeleton line to all non-skeleton pixels within the crack width range to generate a dense orientation field ground truth map with the same resolution as the original image. Based on this, a composite loss function is defined during the model training phase. .in, A weighted cross-entropy loss function is used to address the class imbalance problem between crack pixels and background pixels. A cosine similarity loss function is used to calculate the angular deviation between the predicted direction vector and the aforementioned true direction field vector. This is the balance coefficient. Minimizing this composite loss function through backpropagation forces the network to explicitly learn the cracks as physical flow characteristics of the manifold structure, thus effectively avoiding the model from producing directionally chaotic predictions in shadowed or blurred regions.

[0052] Specifically, in step S4, based on the orientation field feature map, the segmentation feature map is binarized under topological consistency constraints to obtain a skeleton binary map. It should be noted that traditional single-threshold segmentation strategies often struggle to accommodate crack regions with varying contrasts, leading to continuous cracks in weak texture environments such as shadows or blur being easily truncated due to pixel response values ​​below the threshold. Furthermore, conventional isotropic skeleton thinning algorithms ignore the physical extension direction of cracks as manifold structures, easily causing centerline shifts and topological distortions when processing intersections or regions with abrupt width changes. Therefore, the technical solution of this application further uses the orientation field feature map to perform binarization under topological consistency constraints on the segmentation feature map to obtain a skeleton binary map. Through the above processing, the predicted orientation vector field can be used as topological navigation, calculating the cosine similarity between the direction of neighboring pixels and the geometric connection to weight and enhance the crack confidence at potential breakpoints. Combined with a dual-threshold hysteresis strategy and morphological thinning operations, the connectivity of weak feature regions is effectively restored, outputting a high-fidelity crack skeleton with single-pixel accuracy and conforming to physical flow direction.

[0053] Figure 4 This is a flowchart illustrating the process of performing binarization segmentation of a segmented feature map under topological consistency constraints to obtain a skeleton binary map, based on the orientation field feature map of the intelligent monitoring method for surface cracks on bridge structures based on deep learning, according to embodiments of this application. Figure 4As shown, step S4 includes: S41, traversing the pixels in the segmentation feature map and their neighboring pixels, calculating the cosine similarity between the direction vector of the neighboring pixel in the direction field feature map and the neighboring line vector, and updating the crack confidence of the center pixel based on the cosine similarity to obtain the enhancement probability map; S42, performing double threshold hysteresis binarization on the enhancement probability map to obtain a binarized mask; S43, performing topology-preserving morphological skeletonization on the binarized mask to obtain a skeleton binary map.

[0054] In step S41, the pixels in the segmentation feature map and their neighboring pixels are traversed. The cosine similarity between the direction vector of the neighboring pixel and the vector connecting the neighboring pixels in the orientation field feature map is calculated. Based on the cosine similarity, the crack confidence of the center pixel is weighted and updated to obtain the enhanced probability map. It should be noted that, due to the limitation of the local receptive field, convolutional neural networks often have difficulty extracting significant edge features when processing weakly textured areas such as uneven lighting and oil stains on bridge surfaces. This results in the prediction confidence of crack pixels being suppressed by background noise, causing physically continuous cracks to appear as discontinuous discrete segments in the segmentation feature map. Based on this, the technical solution of this application further traverses the pixels in the segmentation feature map and their neighboring pixels, calculates the cosine similarity between the direction vector of the neighboring pixel and the vector connecting the neighboring pixels in the orientation field feature map, and weights and updates the crack confidence of the center pixel based on the cosine similarity to obtain the enhanced probability map. Through the above processing, the global geometric extension trend can be introduced into the probability estimation process as a topological constraint. The directional consistency of strong texture regions can be used to compensate for the confidence loss of weak texture regions, thereby effectively restoring the connectivity of potential breakpoints and improving the robustness of subsequent segmentation to noise.

[0055] More specifically, in a concrete example of this application, this step first establishes a sliding window traversal mechanism for the segmentation feature map. For any center pixel at specific coordinates, its spatial neighborhood is defined, for example, an eight-neighborhood structure covering surrounding adjacent pixels. During the calculation, neighboring pixels are indexed one by one, and the unit direction vector predicted by the direction regression branch of the neighboring pixel is extracted from the direction field feature map. At the same time, a geometric connection vector from the spatial position of the neighboring pixel to the spatial position of the center pixel is constructed. Subsequently, the cosine similarity between the direction vector of the neighboring pixel and the geometric connection vector is calculated using vector dot product operation. This similarity value quantifies the degree to which the crack extension trend of the neighboring point points towards the center point. If the direction vector of a neighboring pixel with a high probability of crack existence is highly aligned with the direction pointing towards the center pixel, a positive gain value is generated. This gain value is weighted by a preset direction enhancement coefficient and the crack probability intensity of the neighboring pixel itself, and then superimposed on the original log-odds or probability value of the center pixel. After completing the cumulative calculation of all neighborhoods, the updated confidence values ​​are mapped to the standard probability interval using the Sigmoid activation function to generate an enhanced probability map, which selectively amplifies the confidence of pixels located on the crack extension path.

[0056] In step S42, the enhanced probability map is subjected to dual-threshold hysteresis binarization to obtain a binarized mask. It should be noted that bridge cracks in images often exhibit a non-uniform distribution with high contrast in the central region and low contrast in the edge or shadow regions. Furthermore, a simple global single-threshold segmentation strategy struggles to balance suppressing background noise with preserving subtle crack features. Setting a high threshold can lead to topological breakage of the cracks, while setting a low threshold introduces a large amount of discrete artifact noise. Therefore, the technical solution of this application further performs dual-threshold hysteresis binarization on the enhanced probability map to obtain a binarized mask. Through this processing, the connectivity constraint mechanism can be used to distinguish between real weak-texture cracks and random noise, ensuring that only weak signal pixels with spatial connectivity to high-confidence regions are retained. This effectively maintains the topological integrity of the crack structure and improves the signal-to-noise ratio of the binarization result.

[0057] More specifically, in a concrete example of this application, this step first sets two differentiated confidence threshold parameters: a high threshold for determining the core region of a high-confidence crack and a low threshold for defining the edge region of a potential crack. Based on these two thresholds, all pixels in the augmented probability map are divided into three attribute categories: pixels with probability values ​​higher than the high threshold are marked as strong crack points; pixels with probability values ​​between the high and low thresholds are marked as weak crack points; and pixels below the low threshold are directly determined as background. Subsequently, a recursive tracing or queue search operation based on eight-neighborhood connectivity is performed. Starting with all strong crack points as seed points, the pixel states within their spatial neighborhoods are retrieved. If a pixel marked as a weak crack point exists within the neighborhood, it is confirmed that the weak crack point belongs to the real crack and its attribute is upgraded to a strong crack point. This point is then used as a new seed point to continue the search outwards. After the traversal process is complete, all isolated weak crack points that have not established a direct or indirect connection path with any strong crack point are uniformly removed and reset to background values. The final output is a binary mask image containing only the background and the confirmed crack region.

[0058] In step S43, the binarized mask is subjected to topology-preserving morphological skeletonization to obtain a skeleton binary map. It should be noted that, given that traditional morphological thinning algorithms usually follow the principle of isotropic contraction and ignore the inherent manifold extension characteristics of cracks as stressed physical structures, when processing crack intersections or areas with drastic width changes, they are easily affected by edge noise, causing the extracted skeleton nodes to deviate from the real physical centerline, or even produce non-physical topological distortions such as twisting and breaking. Furthermore, the original scheme's strategy of burr trimming based solely on a single length threshold is prone to accidentally deleting short but structurally hazardous new crack branches. Based on this, the technical solution of this application further performs topology-preserving morphological skeletonization on the binarized mask to obtain a skeleton binary map. Specifically, this process constructs a central potential energy field that integrates geometric distance and directional consistency. Non-maximum suppression is applied to the potential energy value using the normal direction in the flow field feature map to accurately locate candidate skeleton points. Subsequently, the candidate skeleton is converted into a graph structure, and the consistency integral energy of each branch path along the tangent direction and the global direction field is calculated. Noise branches with energy below a preset threshold are then eliminated. Through the above processing, the center offset caused by geometric corrosion can be corrected using directional priors, fundamentally eliminating the jaggedness and jitter caused by isotropic corrosion. A noise filtering standard based on flow direction consistency is established, thereby accurately eliminating artifact noise that deviates from the main crack flow field while preserving, to the greatest extent possible, tiny crack bifurcations that conform to the principal stress flow field direction, ensuring that the generated skeleton data strictly conforms to physical fluid dynamics constraints.

[0059] Figure 5This is a flowchart illustrating the topology-preserving morphological skeletonization of a binary mask to obtain a skeleton binary map, according to the deep learning-based intelligent monitoring method for surface cracks in bridge structures as described in this application. Figure 5 As shown, step S43 includes: S431, determining the potential energy field matrix based on the binarized mask and the orientation field feature map; S432, analyzing the normal direction of each pixel in the potential energy field matrix based on the orientation field feature map, and performing non-maximum suppression and competitive screening on the potential energy value in the normal direction to obtain the original skeleton mask; S433, converting the original skeleton mask into a graph structure and identifying branch paths, calculating the flow field consistency energy between the tangent direction of each branch path and the orientation field feature map, and removing noise branches with energy below a preset threshold to obtain the skeleton binary map.

[0060] In step S431, the potential energy field matrix is ​​determined based on the binarized mask and the orientation field feature map. It should be noted that existing thinning algorithms ignore orientation information, causing the skeleton to deviate from the center, and calculations based solely on geometric distance are easily affected by irregular jagged edges or noise at the crack edges, making it difficult to accurately locate the crack trunk that conforms to hydrodynamic characteristics. Therefore, the technical solution of this application further determines the potential energy field matrix based on the binarized mask and the orientation field feature map. Through the above processing, the algorithm is endowed with orientation awareness, and geometric deviations are corrected using orientation priors, enabling the potential energy field to automatically suppress edge noise and irregular protrusions, thereby effectively ensuring the physical authenticity of the skeleton's centerline and providing a high signal-to-noise ratio foundation for subsequent extraction.

[0061] More specifically, in a particular example of this application, the input binarized mask is first subjected to an Euclidean distance transformation to obtain the base distance map. The values ​​in this distance map reflect the distance of each pixel to the nearest background edge, implicitly containing information about the geometric width of the crack. Subsequently, the gradient vector field of the distance field is calculated using a differential operator. Furthermore, the previously generated orientation field feature map is introduced as a strong constraint. Based on this, a special potential energy function is constructed to regularize and modulate the distance gradient. This modulation logic is based on the physical prior that, in an ideal crack's central axis, the gradient direction of the distance field should be strictly perpendicular to the crack's flow direction (tangential direction). Therefore, the potential energy function is designed to reward pixels that are geometrically centered and whose gradient direction is perpendicular to the flow direction. The specific formula for calculating the central axis potential energy value is as follows:

[0062] in, For pixels The central potential energy value, the higher the value, the closer the point is to the actual physical skeleton center. For pixels Euclidean distance transform value, Let be the gradient vector of the range field. The unit flow vector extracted from the direction field feature map. Represents the vector dot product. The term constitutes an orthogonality constraint; it is maximized when the distance gradient is perpendicular to the crack flow direction, significantly enhancing the potential energy, and conversely, inhibiting it. This is the orthogonal reinforcement coefficient (e.g., 2.0), used to adjust the strength of the directional constraints. To prevent the use of minute amounts divided by zero, in practical scenarios involving the monitoring of minute cracks on concrete bridge surfaces, if the crack edges have irregular, jagged edges formed by aggregate spalling or water erosion, conventional distance transformations will disperse local extrema at the geometric centers of these irregular edges, causing the subsequently extracted skeleton to exhibit non-physical jitter. However, the method described in this embodiment, by introducing a predicted smooth direction field as a potential energy gain condition, ensures that even with rough and uneven crack edges, the peak ridge of the potential energy field is still forced to distribute along a path consistent with the overall flow direction. This effectively constructs an energy topography map that integrates the weights of both geometric location and physical flow direction, forcing the skeleton generation process to ignore random disturbances at the edges and focus on the main crack path generated by the stress.

[0063] In step S432, the normal direction of each pixel in the potential energy field matrix is ​​analyzed based on the orientation field feature map, and non-maximum suppression and competitive screening are performed on the potential energy values ​​along the normal direction to obtain the original skeleton mask. It should be noted that, given that traditional skeleton extraction algorithms often employ isotropic erosion strategies, when crack widths change drastically or bifurcations exist, redundant pixels and jagged artifacts are easily generated due to edge noise interference, leading to topological jitter in the extracted skeleton lines and preventing them from being accurately located on the physical center line. Therefore, the technical solution of this application further analyzes the normal direction of each pixel in the potential energy field matrix based on the orientation field feature map, and performs non-maximum suppression and competitive screening on the potential energy values ​​along the normal direction to obtain the original skeleton mask. Through the above processing, the ridge detection process in geography can be simulated. By strictly limiting the extreme value search range to the cross section perpendicular to the crack direction and implementing an anisotropic suppression strategy, the redundant pixels and jagged edges generated by traditional algorithms at crack width drastic changes or intersections are avoided. This ensures that the generated skeleton is strictly located on the ridge line in the sense of hydrodynamics, effectively improving the single pixel accuracy and line smoothness of the skeleton, and ultimately achieving the technical effect of extracting a high-precision, non-redundant skeleton.

[0064] More specifically, in a particular example of this application, for each foreground pixel in the potential energy field First, analyze its local flow vector. Then calculate the normal direction of the vector. The direction of this normal represents the cross-sectional direction of the crack. Subsequently, at the normal... On the defined straight line, the current pixel The potential energy value and its sub-pixel neighborhood and The algorithm compares the potential energy values ​​at a given point. During this process, the algorithm adheres to a strict maximum preservation principle, only comparing the maximum values ​​when... A point is retained as a skeleton candidate only if its potential energy value is strictly greater than that of its two neighbors along the normal direction. The specific decision logic is as follows:

[0065] in, This is the original skeleton mask for output. A value of 1 indicates that the point is retained as a skeleton point, and a value of 0 indicates suppression. For the current pixel The potential energy value in the central potential energy field reflects the point The possibility of it being the center of the crack. The normal vector to the crack flow direction indicates the cross-sectional path perpendicular to the crack propagation direction. This is the interpolation step size (usually 1.0), used to determine the sampling distance of the comparison neighborhood. and Offset along the positive and negative directions of the normal, respectively. The potential energy value at the distance is obtained by bilinear interpolation if the coordinates are not integers.

[0066] In real-world bridge crack monitoring scenarios, for example, a non-uniform crack with a width gradually decreasing from 0.2mm to 0.5mm might be encountered, with irregular serrated edges due to concrete spalling. Conventional refinement methods would result in the skeleton line branching at wider sections or developing burrs due to the serrated edges. However, using the non-maximum suppression strategy described in this embodiment, the algorithm essentially performs a transverse slice scan at each step along the crack's direction. Regardless of the local physical width of the crack, the potential energy field in the cross-section always presents a ridge shape, high in the middle and low on both sides. This step forces the retention of only the highest point on this cross-section (i.e., the ridge vertex), thus filtering out all hillside pixels that, while belonging to the crack area, are not energy extrema. This means that even at a Y-shaped intersection where cracks intersect, the algorithm can accurately characterize the unique intersection point based on the potential energy flow direction when the two branches converge, rather than generating a blurry pixel block, achieving a clear reconstruction of the crack's physical topology.

[0067] In step S433, the original skeleton mask is converted into a graph structure and branch paths are identified. The flow field consistency energy of the tangent direction and orientation field feature map of each branch path is calculated, and noise branches with energy below a preset threshold are removed to obtain a binary skeleton map. It should be noted that, given the difficulty of distinguishing between short true cracks and noise artifacts by the traditional length threshold method, filtering based solely on geometric length often leads to the contradiction of long noise being difficult to remove and small true cracks being easily mistakenly eliminated. This is especially true for template joints or water stain streaks on bridge surfaces, which may be longer in shape than newly formed microcracks but lack the flow direction characteristics generated by physical stress. Based on this, the technical solution of this application further implements graph pruning based on flow field energy, converting the original skeleton mask into a graph structure and identifying branch paths, calculating the flow field consistency energy of the tangent direction and orientation field feature map of each branch path, and removing noise branches with energy below a preset threshold to obtain a binary skeleton map. Through the above processing, the limitations of the traditional view that length equals truth can be broken, and a noise filtering standard with flow direction consistency as the core can be established. This solves the contradiction between the difficulty in removing long noise and the easy false detection of tiny true cracks. Thus, while accurately removing long artifacts, the maximum number of tiny crack bifurcations with engineering value can be preserved, thereby improving the robustness and reliability of monitoring results.

[0068] More specifically, in a particular example of this application, the original skeleton mask generated in the previous step is first converted into a graph structure. All independent branch paths are identified by extracting nodes and edges. Then, abandoning simple length metrics, the consistent integral energy of each branch path along its tangent direction and the global direction field is calculated instead. For the ... Branch paths The energy calculation formula is as follows:

[0069] in, The flow field uniformity energy of this branch characterizes the rationality of its existence in the physical flow field. Let be the geometric arc length of the branch, used to normalize the integral result. For the path at position The unit tangent vector at a given point represents the local geometric extension direction of the skeleton. The flow field direction vector at the corresponding location is the physical prior direction extracted from the direction field feature map predicted by the neural network, and the square operation is performed. This mechanism ignores the sign of direction and focuses only on the degree of collinearity, ensuring high consistency regardless of whether the skeleton traversal direction and the flow field direction are in the same or opposite directions, as long as they are parallel. Finally, an energy threshold is set, and all branches with an average flow field energy below this threshold are removed. This mechanism introduces directional reliability as a criterion, effectively filtering out noise that does not conform to physical laws.

[0070] In a real-world bridge appearance inspection scenario, suppose there is a long, narrow water stain left by rainwater on the concrete surface, with a newly emerging, short structural crack beside it. Using traditional methods, the water stain, due to its length, would be preserved, while the micro-crack, due to its insufficient length, would be filtered out as noise. However, using the method described in this embodiment, because the extension path of the water stain is random, its tangential direction... Global orientation field predicted by neural network based on stress texture There is a significant angular deviation between them, resulting in a consistently low dot product term in the integral formula, ultimately leading to a low calculated flow field uniformity energy. Extremely low energy integration means that even if a noise artifact is long, its energy integral will be extremely low if its direction deviates significantly from the main crack flow field (such as transverse noise), thus allowing it to be precisely removed. Conversely, short, real crack bifurcations are preserved because they are highly consistent with the flow field, achieving precise noise reduction and fidelity preservation of crack morphology.

[0071] Specifically, in step S5, based on the orientation field feature map, the skeleton binary map is repaired for crack breaks using a minimum spanning tree algorithm to obtain a repaired topology map. It should be noted that in actual bridge monitoring scenarios, due to limitations such as lighting and shadows, surface stains, or local physical occlusion, physically continuous long cracks often appear as discontinuous discrete segments in visual imaging. Furthermore, traditional repair methods that rely solely on geometric distances are prone to incorrectly connecting adjacent but unrelated parallel cracks, lacking logical judgment regarding the physical extension trend of the cracks. Therefore, the technical solution of this application further utilizes the orientation field feature map, employing convolutional kernels to detect the number of connected pixels in the skeleton binary map to filter breakpoints. A connection cost matrix is ​​constructed by combining the Euclidean distance between breakpoints and the consistency of the angle between the local tangent direction of the breakpoint and the geometric connection direction. Finally, the minimum spanning tree algorithm is used to select the minimum cost edge to repair crack breaks in the skeleton binary map to obtain a repaired topology map. Through the above processing, the visually perceived image space can be transformed into a graph theory space with logical reasoning capabilities. By introducing strong directional field physical constraints, unstructured fractures caused by environmental interference can be intelligently stitched together, thereby effectively restoring the true topological connectivity of the cracks and laying a solid foundation for subsequent accurate assessment of the overall length of the cracks and the structural hazard level.

[0072] Figure 6 This document describes a flowchart illustrating the process of repairing crack breakpoints in a skeleton binary map based on a minimum spanning tree to obtain a repaired topology map, according to an embodiment of the intelligent monitoring method for surface cracks in bridge structures based on deep learning. Figure 6As shown, step S5 includes: S51, using convolutional kernels to detect the number of pixel connectivity in the skeleton binary graph, and selecting single-neighbor pixels as breakpoints based on the number of pixel connectivity, and then initializing the undirected graph structure based on the breakpoints; S52, calculating the Euclidean distance between breakpoint pairs, and combining the consistency of the angle between the local tangent direction and the geometric connection direction in the direction field feature map of the breakpoint, constructing a connection cost matrix of penalty distance and direction deviation; S53, repairing the skeleton binary graph based on the connection cost matrix to obtain a repaired topology graph.

[0073] In step S51, a convolutional kernel is used to detect the pixel connectivity in the binary skeleton graph, and single-neighbor pixels are selected as breakpoints based on the pixel connectivity, thereby initializing an undirected graph structure based on the breakpoints. It should be noted that although the crack skeleton after morphological refinement visually appears as a single-pixel-width line, it is still represented as a discrete pixel matrix at the data storage level. The computer cannot directly perceive the topological endpoints of the crack and potential fracture relationships. Furthermore, due to local occlusion or sudden changes in illumination on the bridge surface, a complete crack is often divided into several physically related but spatially disconnected independent short lines. If these discrete pixels are not converted into a graph structure with logical attributes, the subsequent repair algorithm will be unable to determine the starting and ending points of the connections. Based on this, the technical solution of this application further utilizes a convolutional kernel to detect the pixel connectivity in the binary skeleton graph, and selects single-neighbor pixels as breakpoints based on the pixel connectivity, thereby initializing an undirected graph structure based on the breakpoints. Through the above processing, the visual fracture features in the image space can be accurately transformed into a set of geometric nodes in the graph theory space. By quantizing the neighborhood topological relationships between pixels, all potential crack repair candidate locations can be automatically identified, providing a clear computational node foundation for the subsequent construction of a connection cost matrix based on orientation field and distance constraints.

[0074] More specifically, in a concrete example of this application, this step first involves defining a convolutional kernel of a specific size, such as a 3x3 kernel with a center weight set to zero and the weights of the remaining eight neighboring positions set to one. This kernel is then used to perform a sliding convolution operation on a single-pixel-width skeleton binary map, calculating the eight-neighbor response value of each foreground pixel along the skeleton path. This response value directly represents the number of spatially connected neighbors of the current pixel. Subsequently, the convolution output is iterated and filtered, with the logical rule set to retain only pixels with a response value of one. This is because in a single-pixel skeleton, a point with two neighbors is a path midpoint, a point with more than two neighbors is a crossover point, and a point with only one neighbor clearly indicates that it is a geometric endpoint or break point of a crack. Finally, the pixel coordinates of all endpoints that meet the filtering criteria are extracted and defined as the set of vertices in the graph theory model. A set of blank edges without any connecting edges is initialized to construct an initial undirected graph structure to be repaired. This structure abstractly represents the distribution of all topological breaks to be stitched in the current crack network.

[0075] In step S52, the Euclidean distance between the breakpoint pairs is calculated, and the connection cost matrix of penalized distance and directional deviation is constructed by combining the consistency of the angle between the local tangent direction of the breakpoint and the geometric connection direction in the orientation field feature map. It should be noted that since the nearest neighbor connection strategy based solely on Euclidean distance lacks the ability to perceive the physical morphology of the crack, it is very easy to mistakenly stitch together two parallel or intersecting cracks that are spatially close but have completely different extension directions, leading to logical confusion in the topology. Based on this, the technical solution of this application further calculates the Euclidean distance between the breakpoint pairs and, by combining the consistency of the angle between the local tangent direction of the breakpoint and the geometric connection direction in the orientation field feature map, constructs a connection cost matrix of penalized distance and directional deviation. Through the above processing, the high-dimensional orientation field features predicted by the deep neural network can be used as strong physical constraints. By penalizing connection operations that deviate from the main crack flow direction, it is ensured that only breakpoint pairs that are spatially adjacent and have collinear extension trends have low connection costs, thereby achieving accurate topology repair that conforms to the laws of fluid mechanics.

[0076] More specifically, in a concrete example of this application, this step first sets a maximum search radius parameter, and only breaks within this radius are subsequently calculated to reduce computational complexity. For any two endpoints that satisfy the distance condition, their image coordinates are read and the Euclidean distance between them is calculated using vector operations. Simultaneously, the orientation field feature map is indexed to obtain the local unit tangent direction vector at the starting endpoint, and a unit geometric connection vector from the starting endpoint to the target endpoint is constructed. Subsequently, the cosine of the angle between these two vectors is calculated based on the dot product operation, and its absolute value is taken to ignore the difference between the positive and negative directions of the crack orientation, thereby constructing a comprehensive cost function that includes a distance penalty term and a direction deviation penalty term. The specific calculation formula of this function is as follows:

[0077] in, Represents endpoints With endpoints The overall connection cost between them; the smaller the value, the higher the connection probability. and These are the weighting coefficients for the distance and direction terms, used to balance the contributions of spatial proximity and directional consistency. Let be the Euclidean distance between the two endpoints. As endpoints The local tangent direction vector at that point. The direction vector of the geometric connection between the endpoints; The term constitutes a direction penalty; it approaches zero when two vectors are collinear and increases sharply otherwise. Finally, all the calculated effective cost values ​​are filled into the sparse matrix to complete the construction of the connection cost matrix.

[0078] In step S53, the skeleton binary graph is repaired based on the connection cost matrix to obtain a repaired topology graph. It should be noted that since the connection cost matrix only quantifies the probabilistic cost of establishing connections between discrete breakpoints and does not yet establish the final topology, and direct straight-line connections easily lead to the repaired crack path exhibiting a rigid, broken-line characteristic in geometry, it cannot truly reflect the natural, smooth extension characteristics of concrete cracks under stress. Therefore, the technical solution of this application further repairs the skeleton binary graph based on the connection cost matrix to obtain a repaired topology graph. Through the above processing, the optimal path conforming to physical laws can be selected from numerous potential connection assumptions using global optimization strategies in graph theory, and a smooth transition curve can be generated using high-order geometric interpolation techniques to fill visual gaps, thereby effectively restoring the true continuous topology and geometric orientation of bridge cracks.

[0079] More specifically, in a specific example of this application, the skeleton binary graph is repaired based on the connection cost matrix to obtain a repaired topology graph, including: selecting the minimum cost edge according to the connection cost matrix; using Bézier curves to geometrically interpolate and connect the breakpoint pairs that meet the threshold conditions to obtain the connection path; and backfilling the generated connection path into the skeleton binary graph to obtain the repaired topology graph.

[0080] More specifically, this step, based on the decision logic of the minimum spanning tree algorithm, sorts all potential connection edges stored in the connection cost matrix in ascending order of their cost values. During the traversal of the sorted edge set, the edge with the smallest current cost is extracted one by one, and its cost value is compared with a preset repair threshold. An edge is considered a valid repair path only if its cost value is below the threshold and the two breakpoints it connects belong to different connected subgraphs, thus avoiding the formation of non-physical closed-loop structures. For breakpoint pairs identified as valid connections, simple linear interpolation is abandoned; instead, Bézier curves are used to geometrically interpolate and connect breakpoint pairs that meet the threshold conditions to obtain the connection path. This process uses the local tangent direction vector of the breakpoint in the direction field feature map as the control point tangent vector of the Bézier curve, generating a smooth curve continuous with the original skeleton tangent at the endpoints through parameterized equations. Finally, the coordinates of all discrete pixels on the generated connection path are backfilled into the corresponding positions in the skeleton binary graph, and the node adjacency table in the graph data structure is updated synchronously, ultimately outputting a geometrically connected and topologically complete repaired topology graph.

[0081] Specifically, in step S6, the repair topology map and composite input tensor are subjected to physical parameter quantization and monitoring report generation to obtain the final monitoring report data. It should be noted that, since the repair topology map and its contained crack skeleton data obtained after the previous steps are essentially still dimensionless representations in the pixel space of a two-dimensional image, while actual bridge engineering inspection and maintenance specifications require precise quantization of the physical width and length of cracks in millimeters, and due to variations in shooting distance and camera perspective effects, the actual physical width of cracks with the same pixel width in the image varies significantly at different depths of field. Directly using pixel statistics instead of physical measurements would lead to serious evaluation errors. Therefore, the technical solution of this application further quantifies the physical parameters of the repair topology map and composite input tensor to obtain the final monitoring report data. Through the above processing, depth perception information can be combined to eliminate imaging perspective distortion, accurately inverting visual features into engineering physical parameters that conform to national inspection standards, thereby achieving automated grading and standardized report output for the health status of bridge structures.

[0082] More specifically, in a specific example of this application, step S6 includes: indexing the depth channel data in the composite input tensor based on the node coordinates in the repair topology map, calculating the pixel-to-physical conversion coefficient of each node in combination with the camera focal length, and converting the pixel width of the node into the actual physical width to obtain physical skeleton data; accumulating and integrating the three-dimensional Euclidean distance between adjacent nodes along the path of the physical skeleton data to calculate the total length of the crack, and filtering out the maximum physical width on the entire path to obtain the crack geometric feature set; comparing the maximum physical width in the crack geometric feature set with a preset bridge technical condition assessment standard to determine the crack hazard level; and encapsulating the crack hazard level, total physical length, and spatial location coordinates to generate the final monitoring report data.

[0083] Further, this step first obtains the physical depth value at the corresponding position of each skeleton node based on the depth channel data in the composite input tensor of the skeleton node coordinate index in the repaired topology graph. According to the pinhole camera imaging model, the pixel-to-physical conversion coefficient at each node is calculated using the known camera focal length parameters. This coefficient characterizes the actual physical size represented by a single pixel in the current depth plane. Subsequently, the original pixel width stored in the skeleton node attributes is multiplied by this conversion coefficient, thereby converting the node's pixel width into the true physical width to obtain the physical skeleton data. The specific physical width calculation follows the following mathematical relationship:

[0084] in, Representing the The actual physical width of the cracks at each skeleton node. Representing the The original pixel width of each skeleton node in the image plane. For the first reading from the depth channel of the composite input tensor The physical depth distance corresponding to each node The focal length parameter of the camera is used. Based on this, the total crack length is calculated by integrating the 3D Euclidean distances between adjacent nodes along the connected paths of the physicalized skeleton data. This process is no longer limited to straight-line distances in a two-dimensional plane, but instead incorporates depth information to construct a 3D spatial vector, calculates and sums the spatial moduli of the path elements, and simultaneously selects the maximum width of all nodes as the maximum physical width of the crack during the traversal of the entire path. This constructs a crack geometric feature set containing length, maximum width, and spatial location. The integral formula for calculating the total length is as follows:

[0085] in, This represents the actual total physical length of the crack. This represents the total number of nodes that constitute the skeleton path of the crack. Adjacent nodes and The three-dimensional spatial physical coordinate components are calculated through depth back projection. Finally, based on industry-standard bridge technical condition assessment criteria, the maximum physical width of the crack's geometric feature set is compared with a preset threshold range to determine the crack's hazard level. For example, a maximum width less than a specific limit is classified as safe, while a width exceeding the limit is classified as requiring repair. Finally, the determined crack hazard level, calculated total physical length, maximum physical width, and the absolute spatial coordinates of the crack on the bridge surface are structured, encapsulated, and serialized to generate a final monitoring report containing complete diagnostic conclusions. This report can be directly accessed by engineering maintenance personnel or stored in a health monitoring database.

[0086] In summary, the intelligent monitoring method for surface cracks in bridge structures based on deep learning, according to embodiments of this application, is explained. First, it utilizes camera intrinsic parameters to spatially align and enhance depth point clouds and color images, constructing a composite tensor containing texture and geometric information. Then, a two-stream neural network is used to decode the crack segmentation map and the orientation field feature map reflecting pixel extension trends in parallel. Subsequently, the orientation field is used to extract the skeleton from the segmentation results under anisotropic constraints, effectively eliminating morphological distortions. Based on this, a graph structure is constructed by identifying skeleton endpoints, and the connection cost is calculated by combining orientation consistency and Euclidean distance. The minimum spanning tree algorithm is then used to intelligently stitch together visually broken but logically connected crack paths. Finally, the physical proportions of pixels are inverted based on depth information to achieve accurate quantification of the true physical width and length of the repaired cracks, thereby generating a monitoring report that meets engineering standards.

[0087] Furthermore, a deep learning-based intelligent monitoring system for surface cracks in bridge structures is also provided.

[0088] Figure 7 This is a block diagram of a deep learning-based intelligent monitoring system for surface cracks in bridge structures, according to an embodiment of this application. Figure 7As shown, the intelligent monitoring system 100 for surface cracks in bridge structures based on deep learning according to an embodiment of this application includes: a data acquisition module 110, used to acquire raw sensor data streams containing a color image array, depth point cloud data, and a camera intrinsic parameter matrix; a data preprocessing module 120, used to perform spatial alignment and enhancement processing on the depth point cloud data and the color image array based on the camera intrinsic parameter matrix, and to perform channel stitching and normalization on the processed data to obtain a composite input tensor; and a two-stream feature prediction module 130, used to perform two-stream feature encoding on the composite input tensor based on a preset two-stream deep neural network model. The system includes a segmentation feature map that predicts the probability of crack existence and a segmentation feature map that predicts the pixel extension trend of cracks, based on the orientation field prediction; a topology constraint segmentation module 140 that performs binarization segmentation of the segmentation feature map under topology consistency constraints based on the orientation field feature map to obtain a skeleton binary map; a crack breakpoint repair module 150 that performs crack breakpoint repair based on minimum spanning tree based on the orientation field feature map to obtain a repaired topology map; and a parameter quantization and report generation module 160 that performs physical parameter quantization and monitoring report generation on the repaired topology map and the composite input tensor to obtain the final monitoring report data.

[0089] As described above, the deep learning-based intelligent monitoring system 100 for bridge surface cracks according to embodiments of this application can be implemented in various types of computing devices or control units. For example, it can be deployed in a server cluster of a large bridge health monitoring center, or installed in an edge computing node of an UAV airborne processing unit. In one possible implementation, the deep learning-based intelligent monitoring system 100 for bridge surface cracks according to embodiments of this application can be integrated into the computing device as a software module and / or hardware module. For example, the system 100 can be a resident data processing service in the operating system of the computing device, the software module being configured to perform spatial alignment and enhancement fusion of color images and deep point clouds, feature encoding and orientation field prediction based on two-stream networks, topological consistency constraints and minimum spanning tree repair for fracture defects, and physical parameter inversion and grading for engineering standards; or it can be a dedicated intelligent bridge defect identification algorithm program developed for the computing device. Of course, the system 100 can also be one of many hardware modules of the computing device or control unit, or it can be embedded in a field-programmable gate array circuit to accelerate high-dimensional tensor convolution operations and graph theory path search in parallel, or it can be a deep neural network inference acceleration integrated circuit for a specific application.

[0090] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for intelligent monitoring of surface cracks in bridge structures based on deep learning, characterized in that, include: S1: Acquire the raw sensor data stream containing the color image array, depth point cloud data, and camera intrinsic parameter matrix; S2: Based on the camera intrinsic parameter matrix, spatial alignment and enhancement processing are performed on the depth point cloud data and the color image array, and the processed data is channel-stitched and normalized to obtain a composite input tensor. S3: Based on a pre-set dual-stream deep neural network model, dual-stream feature encoding and orientation field prediction are performed on the composite input tensor to obtain a segmentation feature map that predicts the probability of crack existence and an orientation field feature map that predicts the pixel extension trend of crack. S4: Based on the orientation field feature map, perform binarization segmentation on the segmentation feature map under topological consistency constraints to obtain a skeleton binary map; S5: Based on the orientation field feature map, perform crack breakpoint repair on the skeleton binary map based on the minimum spanning tree to obtain the repaired topology map; S6: Perform physical parameter quantization and monitoring report generation on the repaired topology graph and composite input tensor to obtain the final monitoring report data.

2. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 1, characterized in that, Step S2 includes: The raw sensor data stream is parsed to obtain the raw color image, 3D point cloud set, and camera intrinsic parameter matrix; Based on the camera intrinsic parameter matrix, the spatial points in the 3D point cloud set are projected onto the pixel coordinate system of the original color image, and the sparse matrix after projection is interpolated and filled to obtain a depth channel map aligned with the pixels of the original color image. The original color image is subjected to CLAHE-based shadow enhancement processing to obtain an enhanced color image; The enhanced color image and the depth channel image are numerically normalized separately, and the normalized enhanced color image and the depth channel image are concatenated along the channel dimension to obtain a composite input tensor.

3. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 1, characterized in that, Step S3 includes: The composite input tensor is initially encoded using a convolutional layer adapted to four-channel input, and abstract semantic features at different resolutions are extracted through a deep residual network to obtain a multi-scale feature pyramid. The segmentation branch decoding based on dilated convolution is performed on the multi-scale feature pyramid to obtain the segmentation feature map; The multi-scale feature pyramid is input into the orientation prediction branch to obtain the orientation field feature map.

4. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 1, characterized in that, Step S4 includes: Traverse the pixels in the segmentation feature map and their neighboring pixels, calculate the cosine similarity between the direction vector of the neighboring pixel and the neighboring line vector in the orientation field feature map, and update the crack confidence of the center pixel based on the cosine similarity to obtain the enhancement probability map. The enhanced probability map is subjected to dual-threshold hysteresis binarization to obtain a binarized mask; The binarized mask is subjected to topology-preserving morphological skeletonization to obtain a skeleton binary map.

5. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 1, characterized in that, Step S5 includes: The convolution kernel is used to detect the number of pixel connectivity in the skeleton binary graph, and single-neighbor pixels are selected as breakpoints based on the number of pixel connectivity. Then, the undirected graph structure is initialized based on the breakpoints. Calculate the Euclidean distance between the breakpoint pairs and combine the consistency of the angle between the local tangent direction of the breakpoint and the geometric connection direction in the direction field feature map to construct the connection cost matrix of penalty distance and direction deviation. Based on the connection cost matrix, the skeleton binary graph is repaired to obtain the repaired topology graph.

6. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 5, characterized in that, Based on the connection cost matrix, the skeleton binary graph is repaired to obtain the repaired topology graph, including: The minimum cost edge is selected based on the connection cost matrix; The connection path is obtained by geometric interpolation connecting the pairs of breakpoints that meet the threshold conditions using Bézier curves. The generated connection paths are backfilled into the skeleton binary graph to obtain the repaired topology graph.

7. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 1, characterized in that, Step S6 includes: Based on the node coordinates in the repaired topology graph, the depth channel data in the composite input tensor is indexed, and the pixel-physical conversion coefficient of each node is calculated in combination with the camera focal length. The pixel width of the node is then converted into the real physical width to obtain the physical skeleton data. The total crack length is calculated by accumulating and integrating the three-dimensional Euclidean distance between adjacent nodes along the path of the physical skeleton data, and the maximum physical width on the entire path is selected to obtain the crack geometric feature set. The maximum physical width of the crack geometric feature set is compared with the preset bridge technical condition assessment standard to determine the crack hazard level; The crack hazard level, total physical length, and spatial coordinates are encapsulated to generate the final monitoring report data.

8. The intelligent monitoring method for surface cracks in bridge structures based on deep learning according to claim 4, characterized in that, The binarized mask is subjected to topology-preserving morphological skeletonization to obtain a skeleton binary map, including: The potential energy field matrix is ​​determined based on the binarized mask and the orientation field feature map; Based on the orientation field feature map, the normal direction of each pixel in the potential energy field matrix is ​​analyzed, and non-maximum suppression and competitive screening are performed on the potential energy value in the normal direction to obtain the original skeleton mask. The original skeleton mask is converted into a graph structure and branch paths are identified. The flow field consistency energy of the tangent direction and orientation field feature map of each branch path is calculated, and noisy branches with energy below a preset threshold are removed to obtain the skeleton binary map.

9. A deep learning-based intelligent monitoring system for surface cracks in bridge structures, characterized in that, include: The data acquisition module is used to acquire raw sensor data streams containing color image arrays, depth point cloud data, and camera intrinsic parameter matrices. The data preprocessing module is used to perform spatial alignment and enhancement processing on depth point cloud data and color image array based on camera intrinsic parameter matrix, and to perform channel stitching and normalization on the processed data to obtain composite input tensor. The dual-stream feature prediction module is used to perform dual-stream feature encoding and orientation field prediction on the composite input tensor based on a preset dual-stream deep neural network model to obtain a segmentation feature map that predicts the probability of crack existence and an orientation field feature map that predicts the pixel extension trend of crack. The topology constraint segmentation module is used to perform binarization segmentation of the segmentation feature map under topology consistency constraints based on the orientation field feature map to obtain a skeleton binary map. The crack breakpoint repair module is used to repair crack breakpoints in the skeleton binary map based on the orientation field feature map to obtain a repaired topology map. The parameter quantization and report generation module is used to perform physical parameter quantization and monitoring report generation on the repair topology graph and composite input tensor to obtain the final monitoring report data.

10. The intelligent monitoring system for surface cracks in bridge structures based on deep learning according to claim 9, characterized in that, The topology constraint partitioning module includes: The probability enhancement unit is used to traverse the pixels in the segmentation feature map and their neighboring pixels, calculate the cosine similarity between the direction vector of the neighboring pixel and the neighboring line vector in the orientation field feature map, and update the crack confidence of the center pixel based on the cosine similarity to obtain the enhanced probability map. The binarization processing unit is used to perform double-threshold hysteresis binarization on the enhanced probability map to obtain a binarization mask; Skeletonization units are used to perform topology-preserving morphological skeletonization of a binary mask to obtain a skeleton binary map.

Images