Bridge deck tiny crack identification and size measurement method based on rough segmentation boundary refinement

Abstract

The application discloses a bridge deck fine crack identification and size measurement method based on rough segmentation boundary refining, and relates to the technical field of bridge structure health detection. In the offline stage, images are collected through multiple distances and multiple inclination angles, and a full-field scale factor database is constructed; RGB and depth maps are synchronously collected through a robot RGB-D camera, and a detection distance is obtained through effective depth point screening; an initial mask is obtained through rough segmentation of a semantic segmentation network containing edges, slender structures and adaptive multi-scale fusion; single-pixel boundary directional correction is completed based on 8-direction offset vectors through a double-branch boundary refining network, and a high-precision refined mask is generated; pixel distance is obtained by extracting crack skeletons and double-side boundaries, local scale factors are called and substituted into a coupling formula to convert crack real width, and multi-dimensional quantitative results such as length, area, disease grade and the like are synchronously output. The application is suitable for stable detection of fine cracks in complex bridge deck scenes, and has wide application value.

Description

Technical Field

[0001] This invention relates to the field of bridge structural health detection technology, specifically to a method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundaries. Background Technology

[0002] With the continuous expansion of China's transportation infrastructure, the demand for monitoring defects in concrete and UHPC bridge decks is rapidly increasing. Manual inspection suffers from drawbacks such as low efficiency, large subjective errors, and high risks associated with high-altitude operations. Bridge deck inspection robots equipped with vision acquisition devices are gradually becoming the mainstream solution for automated inspection. Currently, technologies such as machine vision, deep learning semantic segmentation, and RGB-D depth imaging are widely used in bridge crack identification and size quantification. Existing solutions mostly rely on monocular cameras or simple depth devices to acquire images, combining convolutional networks to segment cracks and convert physical dimensions, which to some extent replaces manual defect localization. However, small cracks in bridge decks are characterized by their width of only a few pixels, low contrast, chaotic direction, and easy confusion with aggregate scratches. At the same time, the camera distance and shooting angle continuously change during robot operation, and perspective distortion and lens distortion cause inconsistencies in the actual length of pixels at different positions in the image. Various existing detection solutions still have significant shortcomings in three aspects: complete detection of small cracks, accurate boundary localization, and accurate size conversion across the entire area. Under these circumstances, the industry has carried out a large amount of related technology research.

[0003] Chinese patent (publication number CN119130986A) discloses a method for detecting cracks in concrete bridges based on computer vision. This method acquires depth maps using LiDAR, extracts crack masks using MaskR-CNN, and converts pixel sizes into physical sizes by combining image registration and fixed conversion coefficients. However, this method only uses globally uniform conversion coefficients and does not distinguish the scale differences caused by different image positions, shooting distances, and tilt angles. The measurement error of small cracks is significant in tilted shooting scenarios. Furthermore, the method directly uses the depth value of the crack area to calculate the size, and the depth voids and noise at the crack will further amplify the measurement deviation.

[0004] Chinese patent (publication number CN119273925A) discloses a bridge crack segmentation method based on YOLOv8n-seg. This method embeds an attention module into the detection network to improve the ability to identify small cracks and outputs crack segmentation images. However, this method only completes pixel-level crack region extraction and does not have an independent boundary correction process. The segmentation mask generally has burrs and outward redundant boundaries. Even a single pixel boundary offset of a small crack can cause distortion in the width measurement. At the same time, it does not have a complete calibration system and cannot output true physical size parameters.

[0005] Chinese patent (publication number CN122336660A) discloses a method for intelligent monitoring of bridge surface cracks based on deep learning. This method integrates color images and depth point cloud data, segments cracks through a U-Net network, and extracts the skeleton to calculate pixel geometric parameters. However, this method only uses a single-stage segmentation network, which cannot simultaneously take into account the complete connectivity and boundary fineness of small cracks. It is easy to miss low-contrast crack fractures. Furthermore, it does not build a multi-condition full-domain scale library, and only completes the size conversion through single-point calibration, which limits the applicable scenarios.

[0006] In summary, existing bridge deck crack detection technologies are insufficient to simultaneously achieve flawless detection of small cracks, high-precision boundary positioning, and accurate physical dimension conversion across the entire area under complex on-site conditions. They cannot meet the engineering requirements for automated and refined bridge deck defect detection and grade assessment. Summary of the Invention

[0007] To address the aforementioned technical problems, this application discloses a method for identifying and measuring the size of fine cracks in bridge decks based on coarse segmentation boundaries, specifically including:

[0008] Calibration images are acquired at different distances and tilt angles. Calibration points are extracted from the calibration images. The scale factor is calculated based on the pixel distance of the calibration points and the corresponding physical distance. A full-field scale factor database containing scale factors corresponding to different detection distances, shooting tilt angles and pixel positions is established.

[0009] The bridge deck inspection robot uses an RGB-D depth camera to acquire RGB and depth images of the bridge deck. Based on the depth image, the detection distance from the current camera to the bridge deck is calculated, and the current shooting tilt angle is obtained.

[0010] With the goal of maintaining the integrity and connectivity of the crack body, the bridge deck RGB image is coarsely segmented to obtain an initial crack mask containing the crack body region by extracting crack region features.

[0011] Based on the initial crack mask, the boundary pixels to be corrected and the offset direction of the boundary pixels pointing into the crack are analyzed. The boundary pixels are then corrected with pixel-level labels according to the offset direction to obtain the crack refinement mask.

[0012] Based on the crack refinement mask, the crack skeleton line and crack boundary line are extracted, and the pixel distance between the two boundary points at the crack measurement location is calculated. Based on the shooting tilt angle and the pixel coordinates of the crack measurement location, the detection distance is corrected, the true size of the crack is calculated, and the crack recognition result and size measurement result are output.

[0013] Preferably, the full-field scale factor database specifically involves performing grayscale conversion, adaptive threshold segmentation, morphological opening and closing operations, contour filtering, and sub-pixel centroid correction on the acquired calibration images in sequence.

[0014] Based on the pixel distance between adjacent calibration points on the calibration board and the known physical spacing, calculate the distance at each calibration point. direction, The directional discrete scale factor is calculated using the following formula:

[0015]

[0016]

[0017] in, Representing an image The scale factor of direction, Representing an image The scale factor of direction, For the first The centroid coordinates of each calibration point The coordinates of adjacent calibration points in the horizontal direction. The coordinates of adjacent calibration points in the vertical direction. This is the actual physical distance between adjacent dots on the calibration board;

[0018] The moving least squares reconstruction algorithm is used to reconstruct the full-image scale field for all discrete scale factors, obtaining the corresponding detection distance and shooting tilt angle. Directional full-field scale field and Directional full-field scale field ,in, The detection distance from the camera to the bridge surface. The camera's tilt angle relative to the bridge surface. These are the pixel coordinates of the image.

[0019] Distance interpolation is performed on the detection distance and shooting tilt angle of non-calibrated working conditions to supplement the full-field scale data, integrate the full-field scale field under all working conditions, and construct a complete full-field scale factor database.

[0020] Preferably, the moving least squares reconstruction algorithm specifically involves defining polynomial basis functions. With the coefficient vector to be determined Construct scale field approximation function ;

[0021] A weighted error function is constructed using a distance-related weighting function, with the following formula:

[0022]

[0023] in, The weighted error function is... The number of nodes represents the discrete scale factor. For distance-related weighting functions, For known Discrete scale factor nodes Directional Scale Field Using the same method, the discrete scale factor reconstruction;

[0024] The coefficient vector is solved by minimizing the weighted error function. Complete single-condition scale field reconstruction. The directional scale field was reconstructed using the same algorithm.

[0025] Preferably, the calculation of the detection distance from the current camera to the bridge surface specifically involves: selecting effective depth points in the neighborhood of the crack area or in the flat bridge surface area of ​​the current image, and removing depth abrupt change points, invalid depth points, and outliers; for the filtered effective depth points, median statistics, mean statistics, or plane fitting methods are used to calculate the accurate detection distance from the camera to the bridge surface.

[0026] Preferably, the coarse segmentation of the cracks specifically involves: segmentation using a semantic segmentation network that includes edge-guided feature enhancement, elongated structure enhancement, and adaptive scale fusion; the encoder of the semantic segmentation network performs multi-level downsampling on the bridge deck RGB image, outputting shallow detail features, mid-level structural features, and high-level semantic features respectively; the edge-guided feature enhancement module optimizes the shallow features to suppress the loss of small crack boundary information caused by downsampling; the elongated structure enhancement process strengthens the elongated shape features of the cracks and suppresses bridge deck texture noise interference; the adaptive scale fusion module dynamically weights and fuses the multi-level features to generate features that fit the crack region; the decoder upsamples and restores the fused features, outputs a crack probability map, and obtains an initial crack mask after thresholding.

[0027] Preferably, the edge-guided feature enhancement specifically involves: the semantic segmentation network consisting of an encoder, an edge-guided feature enhancement module, an adaptive scale fusion module, and a decoder; the edge-guided feature enhancement module is deployed at the shallow feature output end of the network, and enhances the shallow features output by the encoder through the edge prediction branch. The process involves generating an edge logits map, which is then normalized using the sigmoid function to produce an edge attention weight map. Combining the original shallow features with the attention weight map generates edge enhancement shallow features, as shown in the formula:

[0028]

[0029] in, This is the edge prediction branch operation function. It is the Sigmoid activation function. This is an element-wise multiplication operation. This is a shallow feature.

[0030] Preferably, the enhancement of the slender structure specifically involves: employing a combination of strip convolution and coordinate attention to aggregate features bidirectionally along both the horizontal and vertical axes; and enhancing the high-level semantic features of the network. Perform horizontal and vertical strip convolution operations separately, using the following formulas:

[0031]

[0032] in, For high-level semantic features, This is a horizontally elongated feature map obtained after performing a horizontal strip convolution operation on high-level semantic features; This is a vertically elongated feature map obtained after performing a vertical strip convolution operation on high-level semantic features. For horizontal strip convolution, Vertical strip convolution;

[0033] Channel concatenation is performed on the bidirectional convolutional features. Then, through 1×1 convolution, normalization, and activation fusion, elongated structure enhancement features are generated, as shown in the formula:

[0034]

[0035] in, This is the final enhanced feature map of the slender crack structure obtained by concatenating the horizontal and vertical slender feature maps through channel stitching, convolutional fusion, and normalized activation. This is the feature fusion function.

[0036] Preferably, the adaptive scale fusion specifically involves: uniformly compressing the shallow features, mid-level structural features, and high-level semantic features after edge enhancement to the same number of channels to complete feature dimension alignment;

[0037] Channel concatenation is performed on the aligned multi-level features, and spatially variable fusion weights for each level of features are generated using a weight prediction function, as shown in the formula:

[0038]

[0039] Based on spatially variable weights, multi-level features are weighted and fused to obtain adaptive fused features, as shown in the formula:

[0040]

[0041] in, These represent the shallow, middle, and high-level features after dimension alignment. For weighted prediction functions, Pre-assigned fusion weights for features at corresponding levels This is a multi-class probability normalized activation function.

[0042] Preferably, the crack refinement mask is specifically implemented as follows: for a pixel identified as a boundary in the boundary probability map, the two-dimensional offset vector corresponding to the pixel is read according to the direction map, and the mask classification label of the current boundary pixel is replaced with the pixel label after moving one pixel along the two-dimensional offset vector, thereby completing one-pixel-level directional correction and outputting a crack refinement mask with accurate boundary convergence.

[0043] Preferably, the formula for calculating the true size of the crack is as follows: extract the skeleton of the crack by refining the crack mask to obtain the single-pixel skeleton line of the crack; calculate the tangent direction of the crack at each skeleton point; and search for the nearest boundary points on the left and right sides of the crack perpendicularly along the normal of the tangent direction.

[0044] The horizontal component of the pixel distance is obtained by calculating the coordinate difference between the two boundary points. and longitudinal components ;

[0045] Based on the horizontal component of pixel distance and longitudinal components The true width of the crack is calculated using a spatial scale coupling formula. The formula is:

[0046]

[0047] in, Querying the pixel location of the crack measurement Directional local scale factor, Querying the pixel location of the crack measurement Directional local scale factor, The horizontal pixel difference at the boundary points. The vertical pixel difference between boundary points. This represents the final calculated physical width of the crack.

[0048] Compared with the prior art, the technical solution of this application has the following technical effects:

[0049] This invention acquires calibration images under multiple working conditions using a calibration board, solves for the discrete scale factor, and then uses moving least squares to reconstruct the full-field scale field. It also supplements the uncalibrated working condition scale data with linear interpolation, distinguishing the independent scale factors of the image's horizontal and vertical directions. It can dynamically retrieve the local scale based on the real-time detection distance, shooting angle, and crack pixel coordinates, effectively compensating for scale deviations caused by perspective distortion and lens distortion, significantly reducing the measurement error of small crack widths under different shooting conditions, and achieving millimeter-level stable size quantization.

[0050] The coarse segmentation network of this invention integrates edge-guided enhancement, slender structure enhancement, and adaptive multi-scale fusion modules to prioritize the integrity and connectivity of low-contrast, discontinuous, and small cracks, reducing missed detections. An independent dual-branch boundary refinement network is added to perform single-pixel directional correction only on the mask boundary, without destroying the main crack area, thus eliminating segmentation mask burrs and outward redundant boundaries.

[0051] This invention uses depth images to filter effective depth points on a flat bridge surface and fits the camera detection distance. It uses depth data only as an index for a scale library and does not directly use the pixel depth of cracks to calculate dimensions. This avoids measurement distortion caused by crack voids, abrupt depth changes, and noise, and stably obtains the working parameters required for dimension conversion.

[0052] This invention enhances the slender and continuous features of cracks through transverse and longitudinal strip convolution, suppressing interference from bridge deck aggregates, scratches, and lighting textures. The boundary refinement network employs an 8-directional gradient label matching image data enhancement strategy to ensure accurate boundary correction direction after image flipping and rotation, enabling stable identification of narrow and fine cracks in complex bridge deck environments.

[0053] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0054] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0056] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0057] Figure 1 This is a flowchart illustrating the overall process for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundaries.

[0058] Figure 2 A flowchart illustrating the process of establishing a full-field scale factor database;

[0059] Figure 3 This is a schematic diagram of the processing flow for the coarse segmentation model of cracks;

[0060] Figure 4 A schematic diagram of the boundary direction finishing process;

[0061] Figure 5 This is a schematic diagram of the conversion of the actual crack width based on the local scale factor. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0063] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0064] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0065] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0066] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0067] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0068] Example 1 mainly describes a method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundaries. Figure 1 As shown, it specifically includes:

[0069] Calibration images are acquired at different distances and tilt angles. Calibration points are extracted from the calibration images. The scale factor is calculated based on the pixel distance of the calibration points and the corresponding physical distance. A full-field scale factor database containing scale factors corresponding to different detection distances, shooting tilt angles and pixel positions is established.

[0070] The bridge deck inspection robot uses an RGB-D depth camera to acquire RGB and depth images of the bridge deck. Based on the depth image, the detection distance from the current camera to the bridge deck is calculated, and the current shooting tilt angle is obtained.

[0071] With the goal of maintaining the integrity and connectivity of the crack body, the bridge deck RGB image is coarsely segmented to obtain an initial crack mask containing the crack body region by extracting crack region features.

[0072] Based on the initial crack mask, the boundary pixels to be corrected and the offset direction of the boundary pixels pointing into the crack are analyzed. The boundary pixels are then corrected with pixel-level labels according to the offset direction to obtain the crack refinement mask.

[0073] Based on the crack refinement mask, the crack skeleton line and crack boundary line are extracted, and the pixel distance between the two boundary points at the crack measurement location is calculated. Based on the shooting tilt angle and the pixel coordinates of the crack measurement location, the detection distance is corrected, the true size of the crack is calculated, and the crack recognition result and size measurement result are output.

[0074] Furthermore, such as Figure 2 The diagram shows the process of establishing the full-field scale factor database. Specifically, the full-field scale factor database involves performing grayscale conversion, adaptive threshold segmentation, morphological opening and closing operations, contour filtering, and sub-pixel centroid correction on the acquired calibration images in sequence.

[0075] Based on the pixel distance between adjacent calibration points on the calibration board and the known physical spacing, calculate the distance at each calibration point. direction, The directional discrete scale factor is calculated using the following formula:

[0076]

[0077]

[0078] in, Representing an image The scale factor of direction, Representing an image The scale factor of direction, For the first The centroid coordinates of each calibration point The coordinates of adjacent calibration points in the horizontal direction. The coordinates of adjacent calibration points in the vertical direction. This is the actual physical distance between adjacent dots on the calibration board;

[0079] The moving least squares reconstruction algorithm is used to reconstruct the full-image scale field for all discrete scale factors, obtaining the corresponding detection distance and shooting tilt angle. Directional full-field scale field and Directional full-field scale field ,in, The detection distance from the camera to the bridge surface. The camera's tilt angle relative to the bridge surface. These are the pixel coordinates of the image.

[0080] Distance interpolation is performed on the detection distance and shooting tilt angle of non-calibrated working conditions to supplement the full-field scale data, integrate the full-field scale field under all working conditions, and construct a complete full-field scale factor database.

[0081] Furthermore, the formula for the distance interpolation algorithm is:

[0082]

[0083] in, Interpolated under the target working condition Directional local scale factor, The detection distance from the camera to the bridge surface. The standard detection distance between two adjacent sets of data already calibrated in the database. Angle of inclination Calibration distance Corresponding known Directional scale field value, for Calibration distance Corresponding known Directional scale field value, The camera's tilt angle relative to the bridge surface;

[0084] Directional Scale Field Obtained using the same method.

[0085] Furthermore, the moving least squares reconstruction algorithm specifically defines the polynomial basis function. With the coefficient vector to be determined Construct scale field approximation function ;

[0086] A weighted error function is constructed using a distance-related weighting function, with the following formula:

[0087]

[0088] in, The weighted error function is... The number of nodes represents the discrete scale factor. For distance-related weighting functions, For known Discrete scale factor nodes Directional Scale Field Using the same method, the discrete scale factor reconstruction;

[0089] The coefficient vector is solved by minimizing the weighted error function. Complete single-condition scale field reconstruction. The directional scale field was reconstructed using the same algorithm.

[0090] Furthermore, the full-field scale factor database includes data corresponding to different detection distances, different shooting angles, and different pixel coordinates. Orientation Scale Factor and Orientation Scale Factor .

[0091] Furthermore, the depth image is specifically a depth grayscale image. Each pixel value in the depth grayscale image corresponds to the physical distance from the camera to the corresponding point in the bridge scene. The pixel positions of the depth image are aligned with those of the RGB image to estimate the detection distance from the current camera to the bridge surface.

[0092] Furthermore, the specific steps for calculating the detection distance from the current camera to the bridge surface are as follows: select effective depth points in the neighborhood of the crack area or in the flat bridge surface area of ​​the current image, and remove depth abrupt change points, invalid depth points, and outliers; for the selected effective depth points, use median statistics, mean statistics, or plane fitting methods to calculate the accurate detection distance from the camera to the bridge surface.

[0093] Furthermore, a valid depth point is specifically defined as a compliant depth pixel output by an RGB-D depth camera that is located in a flat area of ​​the bridge surface, does not belong to cracks or holes, does not belong to areas of abrupt changes in bridge surface texture, has a depth value within the effective range of the camera's ranging, and has no missing holes.

[0094] Furthermore, such as Figure 3 The diagram illustrates the processing flow of the coarse crack segmentation model. Specifically, coarse crack segmentation involves: using a semantic segmentation network that includes edge-guided feature enhancement, elongated structure enhancement, and adaptive scale fusion; the encoder of the semantic segmentation network performs multi-level downsampling on the bridge surface RGB image, outputting shallow detail features, mid-level structural features, and high-level semantic features respectively; the edge-guided feature enhancement module optimizes the shallow features to suppress the loss of small crack boundary information caused by downsampling; the elongated structure enhancement process strengthens the elongated morphological features of the cracks and suppresses bridge surface texture noise interference; the adaptive scale fusion module dynamically weights and fuses the multi-level features to generate features suitable for the crack region; the decoder upsamples and restores the fused features, outputting a crack probability map, which is then thresholded to obtain the initial crack mask.

[0095] Furthermore, the shallow detail features are the low-magnification downsampled output features of the encoder, which retain the original image texture, edge contours, and fine crack boundary details. The channel dimension is adapted to the input of the edge-guided feature enhancement module, which is responsible for providing fine crack boundary information.

[0096] The mid-level structural features are the output features of the encoder at mid-rate downsampling, which take into account both spatial details and semantic information, and preserve the overall strip structure and direction continuity features of the crack, which are used to balance the fusion and transition of detailed features and high-level semantic features.

[0097] High-level semantic features are the encoder's highest downsampled output features, which have strong semantic discrimination capabilities and weak texture detail characteristics. They are used to distinguish crack targets from complex background interference such as bridge deck aggregate, scratches, and lighting textures.

[0098] After dimensional alignment, the three-level features are fed into the adaptive scale fusion module to complete spatial dynamic weighted fusion. The decoder upsamples step by step to restore the original image resolution and outputs a crack probability feature map of the same size as the input image.

[0099] Furthermore, the edge-guided feature enhancement specifically involves the semantic segmentation network consisting of an encoder, an edge-guided feature enhancement module, an adaptive scale fusion module, and a decoder. The edge-guided feature enhancement module is deployed at the shallow feature output of the network, and enhances the shallow features output by the encoder through the edge prediction branch. The process involves generating an edge logits map, which is then normalized using the sigmoid function to produce an edge attention weight map. Combining the original shallow features with the attention weight map generates edge enhancement shallow features, as shown in the formula:

[0100]

[0101] in, This is the edge prediction branch operation function. It is the Sigmoid activation function. This is an element-wise multiplication operation. This is a shallow feature.

[0102] Furthermore, the edge-guided feature enhancement module strengthens the shallow features of the crack contour region through attention weights, effectively reducing the weakening of fine crack boundary details during the downsampling process.

[0103] Furthermore, the slender structure enhancement specifically involves: employing a combination of strip convolution and coordinate attention to aggregate features bidirectionally along both the horizontal and vertical axes; and enhancing the high-level semantic features of the network. Perform horizontal and vertical strip convolution operations separately, using the following formulas:

[0104]

[0105] in, For high-level semantic features, This is a horizontally elongated feature map obtained after performing a horizontal strip convolution operation on high-level semantic features; This is a vertically elongated feature map obtained after performing a vertical strip convolution operation on high-level semantic features. For horizontal strip convolution, Vertical strip convolution;

[0106] Channel concatenation is performed on the bidirectional convolutional features. Then, through 1×1 convolution, normalization, and activation fusion, elongated structure enhancement features are generated, as shown in the formula:

[0107]

[0108] in, This is the final enhanced feature map of the slender crack structure obtained by concatenating the horizontal and vertical slender feature maps through channel stitching, convolutional fusion, and normalized activation. This is the feature fusion function.

[0109] Furthermore, adaptive scale fusion specifically involves compressing the shallow features, mid-level structural features, and high-level semantic features after edge enhancement to the same number of channels, thereby aligning the feature dimensions.

[0110] Channel concatenation is performed on the aligned multi-level features, and spatially variable fusion weights for each level of features are generated using a weight prediction function, as shown in the formula:

[0111]

[0112] Based on spatially variable weights, multi-level features are weighted and fused to obtain adaptive fused features, as shown in the formula:

[0113]

[0114] in, These represent the shallow, middle, and high-level features after dimension alignment. For weighted prediction functions, Pre-assigned fusion weights for features at corresponding levels This is a multi-class probability normalized activation function.

[0115] Furthermore, the allocation rules for the pre-allocated fusion weights are as follows: the weight of shallow detail features is increased in the crack boundary region to preserve boundary details; the weight of high-level semantic features is increased in the crack body region to enhance crack integrity; and invalid feature responses are reduced in the background region to suppress noise interference.

[0116] Furthermore, such as Figure 4 The schematic diagram of the boundary direction refinement process shows that the crack skeleton line and crack boundary line are extracted through the boundary refinement network. The boundary refinement network includes a shared feature extraction module, a boundary branch, and a direction branch. The shared feature extraction module is used to extract common features of the mask boundary. The boundary branch outputs a boundary probability map, which is used to determine whether the current pixel is a crack boundary pixel to be corrected, and accurately lock the correction area. The direction branch outputs a discrete direction map of the boundary pixels pointing into the crack, which is used to determine the accurate correction offset direction of each boundary pixel to be corrected.

[0117] Furthermore, when training the boundary refinement network, a signed distance transformation is performed on the ground truth mask of the crack, the distance field gradient is calculated, the gradient direction is used to characterize the pointing relationship between the inside and outside of the crack, and the distance field gradient direction is quantized into multiple discrete direction categories to generate boundary direction labels pointing to the inside of the crack, thereby achieving accurate supervision of the boundary pixel correction direction.

[0118] Furthermore, the discrete orientation map includes eight orientation categories, which correspond to two-dimensional offset vectors in the directions of 0°, 45°, 90°, 135°, 180°, 225°, 270° and 315°, respectively, covering the full-dimensional boundary offset scenario. When the training samples are subjected to data augmentation operations such as horizontal flipping, vertical flipping or rotation, the orientation category labels are simultaneously transformed by the corresponding angles to ensure the consistency between the training labels and the image transformation.

[0119] Furthermore, the crack refinement mask is specifically as follows: for a pixel identified as a boundary in the boundary probability map, the two-dimensional offset vector corresponding to the pixel is read according to the direction map, and the mask classification label of the current boundary pixel is replaced with the pixel label after moving one pixel along the two-dimensional offset vector, thus completing one-pixel-level orientation correction and outputting a crack refinement mask with accurate boundary convergence.

[0120] Furthermore, such as Figure 5 The diagram shown illustrates the conversion of the true width of a crack based on a local scale factor. The formula for calculating the true size of a crack is as follows: extract the skeleton of the crack by refining the crack mask to obtain the single-pixel skeleton line of the crack; calculate the tangent direction of the crack at each skeleton point; and search for the nearest boundary points on the left and right sides of the crack perpendicularly along the normal of the tangent direction.

[0121] The horizontal component of the pixel distance is obtained by calculating the coordinate difference between the two boundary points. and longitudinal components ;

[0122] Based on the horizontal component of pixel distance and longitudinal components The true width of the crack is calculated using a spatial scale coupling formula. The formula is:

[0123]

[0124] in, Querying the pixel location of the crack measurement Directional local scale factor, Querying the pixel location of the crack measurement Directional local scale factor, The horizontal pixel difference at the boundary points. The vertical pixel difference between boundary points. This represents the final calculated physical width of the crack.

[0125] Furthermore, the dimensional measurement results include one or more of the following: crack length, maximum width, average width, crack area, crack direction, crack pixel coordinates, and bridge deck crack damage level, enabling multi-dimensional quantitative output and damage assessment of bridge deck cracks.

[0126] This embodiment details a method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement. In the offline stage, images are acquired at multiple distances and tilt angles to construct a full-field scale factor database. RGB and depth maps are simultaneously acquired using a robot's RGB-D camera, and the detection distance is obtained through effective depth point filtering. A semantic segmentation network with edge and slender structures, and adaptive multi-scale fusion, is used for coarse segmentation to obtain an initial mask. A dual-branch boundary refinement network is used to perform single-pixel boundary orientation correction based on an 8-way offset vector, generating a high-precision refined mask. By extracting the crack skeleton and bilateral boundaries, pixel distances are obtained, and local scale factors are retrieved and substituted into a coupling formula to calculate the true crack width, simultaneously outputting multi-dimensional quantitative results such as length, area, and damage level.

[0127] Example 2, based on Example 1, details the implementation process of identifying and measuring the size of small cracks in the bridge deck using this method, as follows:

[0128] The effective ranging range of the equipment camera is 0.5m to 3m. It is equipped with a calibration plate with a standard center spacing of 15mm. The overall operation process is divided into two stages: offline scale factor database construction and on-site automated detection and quantification.

[0129] During the offline scale factor database construction phase, the calibration board was laid flat on the simulated flat bridge surface area, the camera shooting tilt angle was fixed at 10°, and the detection distance between the camera and the bridge surface was set to 0.8m, 1.2m, 1.6m, 2.0m, and 2.4m in sequence. Each detection distance was matched with four sets of shooting tilt angles of 0°, 10°, 20°, and 30° respectively. Twenty calibration images were collected for each working condition to ensure data stability.

[0130] The acquired calibration images were sequentially converted to grayscale, subjected to adaptive thresholding, morphological opening and closing operations, contour filtering, and subpixel centroid correction to accurately extract the pixel coordinates of each dot on the calibration board. Scale conversion was performed using the pixel coordinates of typical calibration points and adjacent calibration points. For example, the horizontal spacing between adjacent pixels at a certain calibration point was 16 pixels, corresponding to a discrete scale of 0.9375 mm / pixel in the x-direction; the vertical spacing between adjacent pixels was 15 pixels, corresponding to a discrete scale of 1.0000 mm / pixel in the y-direction.

[0131] By using the quadratic polynomial moving least squares algorithm, 32 sets of discrete calibration point data from the surrounding area are selected for weighted fitting to complete the reconstruction of the full-image pixel continuous scale field, generating full-field scale fields in the x and y directions under different distances and tilt angles.

[0132] For the non-calibration working condition with an actual detection distance of 1.4m and a shooting tilt angle of 10°, two sets of adjacent calibration working condition data at 1.2m and 1.6m were selected for linear interpolation calculation. Finally, the x-direction scale factor of the typical pixel position of the image under this working condition was obtained as 0.118mm / pixel. The y-direction scale factor interpolation was completed in the same process. All calibration and interpolation working condition data were integrated to build a complete full-field scale factor database covering all distances, all tilt angles, and all pixel positions.

[0133] During the on-site inspection phase, the inspection robot moves along the bridge surface at a constant speed. The RGB-D depth camera simultaneously acquires RGB images of the bridge surface with a resolution of 512×512 and matching depth grayscale images. At the same time, the IMU sensor acquires the camera's shooting tilt angle in real time.

[0134] Effective depth pixels were selected from the flat bridge surface area in the crack neighborhood. Invalid outliers with excessive range, missing values, or large neighborhood depth differences were removed, resulting in 96 sets of effective depth pixel data. The actual detection distance between the current camera and the bridge surface was accurately calculated to be 1.4m through plane fitting. Combined with the sensor data, the current shooting tilt angle was determined to be 10°, providing accurate working condition parameters for subsequent size conversion.

[0135] After obtaining the working condition parameters and bridge deck images, the RGB images are input into a dedicated multi-feature enhanced semantic segmentation network to complete the coarse segmentation of the cracks;

[0136] The network outputs shallow detail features, mid-level structural features, and high-level semantic features through three levels of downsampling. To address the problem of easily losing details at the boundaries of small cracks, the shallow features are optimized and strengthened through an edge-guided feature enhancement module, which effectively improves the response intensity of pixel features at the crack edges and compensates for the loss of details caused by the downsampling operation.

[0137] A slender structure enhancement method is employed, using horizontal 1×7 and vertical 7×1 strip convolutions to enhance high-level features, fully exploiting the continuous morphological characteristics of cracks and suppressing background texture interference such as bridge deck aggregate and scattered scratches. After multi-dimensional feature enhancement, the three-level features are uniformly compressed to the same channel dimension. Dynamic weights are generated through an adaptive scale fusion module to complete feature fusion, emphasizing shallow detail features in the crack boundary region and high-level semantic features in the crack body region, ensuring complete detection of small cracks without any missed detections. Finally, the decoder upsamples the image to the original resolution, and threshold binarization is performed to obtain an initial crack mask that retains the complete crack body and allows for slight redundancy at the boundaries. During network training, multiple sets of weighted loss functions are used for collaborative constraints to effectively avoid defects such as breaks and holes in crack segmentation.

[0138] Pixel-level boundary correction and optimization are achieved through an independent boundary refinement network. The initial crack mask is input into the dual-branch boundary refinement network. Common features of the mask boundary are extracted through a shared feature module. The boundary branch accurately identifies and locks the pixel region to be corrected on the mask boundary, and the direction branch outputs a discrete direction map containing eight types of angles, corresponding to full-dimensional offset vectors of 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°. During the network training phase, a signed distance transformation and gradient calculation are performed on the ground truth crack mask to generate accurate boundary direction supervision labels. The direction labels are updated synchronously during image flipping and rotation data augmentation operations to ensure training accuracy. During inference, for the identified boundary pixels, the corresponding 45° inward offset vector is read, and the boundary pixel mask label is transferred and assigned along the offset direction. The original labels of the pixels inside the crack body remain unchanged throughout. Without destroying the integrity of the crack connectivity, the boundary jaggedness and redundant pixels are eliminated, and a high-precision refined mask with a boundary that fits the real crack contour is output.

[0139] Based on a high-precision crack refinement mask, crack skeleton extraction and boundary localization are completed, thereby achieving accurate conversion of multi-dimensional physical dimensions of the crack. After extracting the single-pixel skeleton line of the crack, the boundary pixels on both sides are searched along the vertical crack direction at the skeleton measurement point, obtaining a horizontal pixel difference of 4 pixels and a vertical pixel difference of 6 pixels. The local scale factors in the x and y directions corresponding to the pixel position are retrieved, and the actual crack width at the measurement point is calculated to be approximately 0.84 mm through a spatial scale coupling conversion method. The physical length of each segment is calculated and accumulated along the crack skeleton segment by segment, and finally the entire crack width is calculated. The total length of the crack is 126.4 mm. After traversing all crack pixels within the refined mask and converting the area pixel by pixel, the total area of ​​the crack is 78.2 mm². The crack width data of all measurement points are statistically analyzed in batches, and the maximum width of the crack is found to be 1.12 mm and the average width is 0.76 mm. Combined with the preset bridge deck defect classification standard, the crack is determined to be a minor defect. Finally, the multi-dimensional quantitative detection results of crack location, outline, length, width, area, and defect level are fully output, completing the fully automatic high-precision identification and measurement of small cracks in the bridge deck.

[0140] This embodiment details the implementation process of identifying and measuring the size of small cracks on bridge decks using the proposed method. The process involves establishing a full-field scale factor database through multi-condition calibration, full-field scale reconstruction, and interpolation completion. Accurate detection parameters are then obtained by combining effective depth points on the smooth bridge deck. Coarse segmentation ensures complete detection of small cracks, while independent boundary refinement networks complete pixel-level boundary correction. Local scale factors are then used to achieve accurate quantitative detection of the crack's true width, length, area, and severity. The results fully demonstrate that this invention can stably and accurately complete the automated identification and size measurement of small cracks in complex bridge deck scenarios, exhibiting good engineering practicality and detection reliability.

[0141] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for identifying and measuring the size of fine cracks in bridge decks based on coarse segmentation boundaries, characterized in that, include: Calibration images are acquired at different distances and tilt angles. Calibration points are extracted from the calibration images. The scale factor is calculated based on the pixel distance of the calibration points and the corresponding physical distance. A full-field scale factor database containing scale factors corresponding to different detection distances, shooting tilt angles and pixel positions is established. The bridge deck inspection robot uses an RGB-D depth camera to acquire RGB and depth images of the bridge deck. Based on the depth image, the detection distance from the current camera to the bridge deck is calculated, and the current shooting tilt angle is obtained. With the goal of maintaining the integrity and connectivity of the crack body, the bridge deck RGB image is coarsely segmented to obtain an initial crack mask containing the crack body region by extracting crack region features. Based on the initial crack mask, the boundary pixels to be corrected and the offset direction of the boundary pixels pointing into the crack are analyzed. The boundary pixels are then corrected with pixel-level labels according to the offset direction to obtain the crack refinement mask. Based on the crack refinement mask, extract the crack skeleton line and crack boundary line, and calculate the pixel distance between the two boundary points at the crack measurement location. Based on the shooting tilt angle and the pixel coordinates of the crack measurement location, the detection distance is corrected, the true size of the crack is calculated, and the crack identification result and size measurement result are output.

2. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 1, characterized in that, The full-field scale factor database specifically involves performing grayscale conversion, adaptive threshold segmentation, morphological opening and closing operations, contour filtering, and sub-pixel centroid correction on the acquired calibration images in sequence. Based on the pixel distance between adjacent calibration points on the calibration board and the known physical spacing, calculate the distance at each calibration point. direction, The directional discrete scale factor is calculated using the following formula: in, Representing an image The scale factor of direction. Representing an image The scale factor of direction. For the first The centroid coordinates of each calibration point The coordinates of adjacent calibration points in the horizontal direction. The coordinates of adjacent calibration points in the vertical direction. This is the actual physical distance between adjacent dots on the calibration board; The moving least squares reconstruction algorithm is used to reconstruct the full-image scale field for all discrete scale factors, obtaining the corresponding detection distance and shooting tilt angle. Directional full-field scale field and Directional full-field scale field ,in, The detection distance from the camera to the bridge surface. The camera's tilt angle relative to the bridge surface. These are the pixel coordinates of the image. Distance interpolation is performed on the detection distance and shooting tilt angle of non-calibrated working conditions to supplement the full-field scale data, integrate the full-field scale field under all working conditions, and construct a complete full-field scale factor database.

3. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 2, characterized in that, The moving least squares reconstruction algorithm specifically involves defining polynomial basis functions. and the coefficient vector to be determined Construct scale field approximation function ; A weighted error function is constructed using a distance-related weighting function, with the following formula: in, The weighted error function, The number of nodes represents the discrete scale factor. For distance-related weighting functions, For known Discrete scale factor nodes Directional Scale Field Using the same method, the discrete scale factor reconstruction; The coefficient vector is solved by minimizing the weighted error function. Complete single-condition scale field reconstruction. The directional scale field was reconstructed using the same algorithm.

4. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 1, characterized in that, The calculation of the detection distance from the current camera to the bridge surface is specifically as follows: select effective depth points in the neighborhood of the crack area or the flat bridge surface area of ​​the current image, and remove depth abrupt change points, invalid depth points and outliers; for the filtered effective depth points, use median statistics, mean statistics or plane fitting methods to calculate the accurate detection distance from the camera to the bridge surface.

5. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 1, characterized in that, The coarse segmentation of the cracks specifically involves: using a semantic segmentation network that includes edge-guided feature enhancement, elongated structure enhancement, and adaptive scale fusion for segmentation; the encoder of the semantic segmentation network performs multi-level downsampling on the bridge surface RGB image, outputting shallow detail features, mid-level structural features, and high-level semantic features respectively; the edge-guided feature enhancement module optimizes the shallow features to suppress the loss of small crack boundary information caused by downsampling; the elongated structure enhancement process strengthens the elongated shape features of the cracks and suppresses bridge surface texture noise interference; and the adaptive scale fusion module dynamically weights and fuses the multi-level features to generate features adapted to the crack region. The decoder upsamples and restores the fused features, outputs a crack probability map, and obtains the initial crack mask after thresholding.

6. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 5, characterized in that, The edge-guided feature enhancement specifically involves the semantic segmentation network consisting of an encoder, an edge-guided feature enhancement module, an adaptive scale fusion module, and a decoder. The edge-guided feature enhancement module is deployed at the shallow feature output of the network, and enhances the shallow features output by the encoder through the edge prediction branch. The process involves generating an edge logits map, which is then normalized using the sigmoid function to produce an edge attention weight map. Combining the original shallow features with the attention weight map generates edge enhancement shallow features, as shown in the formula: in, For edge prediction branch operation function, It is the Sigmoid activation function. This is an element-wise multiplication operation. This is a shallow feature.

7. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 6, characterized in that, The enhancement of the slender structure specifically involves: employing a combination of strip convolution and coordinate attention to aggregate features bidirectionally along both the horizontal and vertical axes; and enhancing the high-level semantic features of the network. Perform horizontal and vertical strip convolution operations separately, using the following formulas: in, For high-level semantic features, This is a horizontally elongated feature map obtained after performing a horizontal strip convolution operation on high-level semantic features; This is a vertically elongated feature map obtained after performing a vertical strip convolution operation on high-level semantic features. For horizontal strip convolution, Vertical strip convolution; Channel concatenation is performed on the bidirectional convolutional features. Then, through 1×1 convolution, normalization, and activation fusion, elongated structure enhancement features are generated, as shown in the formula: in, This is the final enhanced feature map of the slender crack structure obtained by concatenating the horizontal and vertical slender feature maps through channel stitching, convolutional fusion, and normalized activation. This is the feature fusion function.

8. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 7, characterized in that, The adaptive scale fusion specifically involves: compressing the shallow features, mid-level structural features, and high-level semantic features after edge enhancement to the same number of channels to achieve feature dimension alignment; Channel concatenation is performed on the aligned multi-level features, and spatially variable fusion weights for each level of features are generated using a weight prediction function, as shown in the formula: Based on spatially variable weights, multi-level features are weighted and fused to obtain adaptive fused features, as shown in the formula: in, These represent the shallow, middle, and high-level features after dimension alignment. For weighted prediction functions, Pre-assigned fusion weights for features at corresponding levels This is a multi-class probability normalized activation function.

9. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 1, characterized in that, The crack refinement mask is specifically defined as follows: for a pixel identified as a boundary in the boundary probability map, the two-dimensional offset vector corresponding to the pixel is read according to the direction map, and the mask classification label of the current boundary pixel is replaced with the pixel label after moving one pixel along the two-dimensional offset vector, thus completing one-pixel-level orientation correction and outputting a crack refinement mask with accurate boundary convergence.

10. The method for identifying and measuring the size of small cracks in bridge decks based on coarse segmentation boundary refinement according to claim 1, characterized in that, The formula for calculating the true size of the crack is as follows: extract the skeleton of the crack by refining the crack mask to obtain the single-pixel skeleton line of the crack; calculate the tangent direction of the crack at each skeleton point; and search for the nearest boundary points on the left and right sides of the crack perpendicularly along the normal of the tangent direction. The horizontal component of the pixel distance is obtained by calculating the coordinate difference between the two boundary points. and longitudinal components ; Based on the horizontal component of pixel distance and longitudinal components The true width of the crack is calculated using a spatial scale coupling formula. The formula is: in, Querying the pixel location of the crack measurement Directional local scale factor, Querying the pixel location of the crack measurement Directional local scale factor, The horizontal pixel difference at the boundary points. The vertical pixel difference between boundary points. This represents the final calculated physical width of the crack.

Images