Pavement crack identification method based on binocular aerial view and improved YOLOv8

Abstract

The invention discloses a pavement crack identification method based on a binocular aerial view and improved YOLOv8. The method comprises the following steps: S1, converting a front view into an aerial view BEV by adopting an inverse perspective transformation technology; and S2, carrying out pavement crack identification on the aerial view BEV by adopting an improved YOLOv8 model. According to the method, binocular stereoscopic vision and an inverse perspective transformation technology are combined, the BEV image of the road surface is constructed, the problem that target scales are inconsistent under a traditional visual angle is effectively solved, a global visual field is provided, and a model is focused on form and position information of a crack in a global image; according to the invention, the C2f-DRR module enhances the multi-scale feature extraction capability of the model through a region residual error-semantic residual error two-step method. And original feature information is reserved through residual connection, and gradient propagation is effectively enhanced.

Description

Technical Field

[0001] This invention relates to the field of pavement crack identification technology, and more specifically, to a pavement crack identification method based on binocular bird's-eye view and improved YOLOv8. Background Technology

[0002] With the acceleration of urbanization and the growth of transportation demand, highways, as a key infrastructure for regional economic development, are affected by a combination of factors, including traffic load, climate conditions, and material fatigue. Cracks, as the initial manifestation of pavement distress, can easily expand into serious structural damage and shorten the service life of highways if not detected and repaired in a timely manner. Traditional vision-based crack detection methods mainly employ image processing techniques such as threshold segmentation, edge detection, and region growing. However, these methods rely on set threshold parameters, resulting in poor generalization ability in complex backgrounds. Therefore, researching efficient pavement crack detection and segmentation technologies is of significant practical importance for reducing potential economic losses, improving highway quality, and extending service life.

[0003] In recent years, advancements in machine vision and deep learning technologies have significantly propelled the development of non-destructive testing for road cracks. Yang et al. proposed PAF-Net, specifically designed for road crack segmentation. It captures multi-scale contextual information and reduces the loss of detailed features through a PCF (Progressive Context Fusion) module and a dual attention mechanism, while simultaneously achieving efficient feature map fusion through a dynamic weight learning module. Sun et al. introduced a multi-scale attention module, Multi-Scale Attention, into the DeepLabv3+ decoder, generating attention masks and dynamically allocating weights between high- and low-level feature maps, thereby more effectively combining multi-scale features to generate more accurate crack segmentation results. Although improved semantic segmentation networks demonstrate excellent accuracy, their complex structure and high computational cost limit detection speed. Wu et al. optimized multi-scale feature fusion using a Bidirectional Feature Pyramid Network (BiFPN) and introduced a P6 layer and a CA module into the detection head to enhance the transfer of shallow features to deep features, thereby improving small target detection capabilities. Their improved YOLOv5s model achieved an mAP50 of 73% on the open-source dataset UAPD. Qu et al. significantly improved the model's performance in small crack detection by adding a detection layer specifically designed for small target extraction to the original YOLOv7 architecture and replacing traditional convolutional operations with RepConv and MPConv. This approach preserved more feature information and effectively fused multi-scale and channel features. While these two improved models offer enhanced accuracy and meet real-time requirements, they primarily focus on target detection tasks and cannot provide precise crack boundary information.

[0004] Most studies focus on the local morphology and location features of road defects in images, neglecting the global defect information of the road segment to be detected. Therefore, many researchers have begun to use unmanned aerial vehicles (UAVs) to obtain more comprehensive data for a holistic assessment of road defects. Gao et al. introduced Deeply Separable Residual Blocks (DR-Blocks) based on U-Net to reduce feature information loss; then, they used a hybrid loss function of Dice Loss and BCE Loss to effectively solve the imbalance problem in UAV images. Sun et al. combined the advantages of the dense nested structure of Unet++ and the deep nested structure of U2Net, and proposed a deep competitive fusion feature extraction block, which enhanced the feature difference between cracks and the background, and significantly improved the model's ability to extract crack features. Although UAVs have a wide shooting range, they are easily limited by weather conditions and have high equipment deployment costs. With the development of autonomous driving technology, traffic scene understanding based on bird's eye view (BEV) has gradually become a research hotspot. Zhao et al. significantly improved the robustness and accuracy of road elevation reconstruction by employing BEV perception and multi-scale feature fusion techniques. Their proposed RoadBEV-Stereo model based on binocular stereo images achieved an elevation error of 0.50 cm, providing reliable road information support for autonomous driving. Yang et al. used the LSTR (Lane Shape Transformers) algorithm to calculate vanishing points in images and then converted the road image into a bird's-eye view through inverse perspective mapping to obtain the region of interest of the road surface, thereby improving defect detection efficiency.

[0005] In summary, to address the shortcomings of existing pavement crack detection methods, this paper combines binocular stereo vision with inverse perspective mapping technology to acquire high-precision BEV images. This effectively maintains target scale consistency and provides a global field of view to capture global features of long cracks and large-area defects. Compared to BEV generation methods based on monocular images and LiDAR, the binocular vision approach offers advantages such as acquiring dense depth information, lower cost, and flexible deployment. Although its robustness under extreme weather conditions is slightly lower than that of LiDAR, it can better balance recognition accuracy, real-time performance, and system complexity in crack detection, demonstrating significant application value. Furthermore, an improved C2f-DRR feature extraction module is proposed, which enhances the model's ability to extract contextual multi-scale features while reducing the loss of detailed information. A CAA (Context Anchor Attention) mechanism is introduced to dynamically strengthen crack region features. Unlike existing channel attention or spatial attention mechanisms, CAA can model the contextual dependencies between distant pixels, improving sensitivity to slender targets. Summary of the Invention

[0006] The purpose of this invention is to provide a road surface crack identification method based on binocular bird's-eye view and improved YOLOv8, so as to overcome the defects of the existing technology.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The pavement crack identification method based on binocular bird's-eye view and improved YOLOv8 includes the following steps:

[0009] S1. Use inverse perspective transformation technology to convert the front view into a bird's-eye view (BEV).

[0010] S2. Use the improved YOLOv8 model to identify road surface cracks in the bird's-eye view BEV.

[0011] Furthermore, step S1 specifically includes:

[0012] S10. Perform ground plane fitting on the 3D image point cloud reconstructed by binocular vision using the RANSAC algorithm.

[0013] S11. Extract the normal vector n = (A, B, C) of the fitted ground plane, and determine the rotation angle θ and rotation axis r = (rx, ry, rz) between the road plane and the orthophoto view. Align the planar point cloud to the bird's-eye view through rotation transformation, and map the point cloud back to the pixel coordinate system in combination with camera intrinsics to establish a point-to-point mapping relationship between the road front view and the bird's-eye view, so as to obtain the corresponding point set.

[0014] S12, through the corresponding point set The homography matrix H of the ground plane is solved using the least squares method, and the homography matrix H is used to perform inverse perspective transformation to achieve projection mapping from the front view to the bird's-eye view.

[0015] Furthermore, step S10 specifically includes:

[0016] S100. Randomly sample three points and calculate a planar model A1x+B1y+C1z+D1=0;

[0017] S101. Calculate the distance di from all remaining points to the plane, and count the number N of points whose distance is within the set threshold ε.

[0018] S102. Resampling: The number of inliers in the i-th iteration is Ni, and the number of inliers in the (i+1)-th iteration is Ni+1. If Ni+1 is greater than Ni, the model parameters are updated to ensure that the model parameters with the largest number of inliers are always retained.

[0019] S103. After iterating to the set iteration threshold, the interior points are used for re-estimation to finally obtain the optimal model parameters.

[0020] Furthermore, step S11 determines the rotation angle θ between the road plane and the orthographic viewing angle and the rotation axis r = (rx, ry, rz) using the following formula.

[0021]

[0022] r = n × z

[0023] In the formula, n = (0,0,1) represents the direction of the orthographic projection axis;

[0024] The point cloud is mapped back to the pixel coordinate system using the following formula in conjunction with camera intrinsic parameters:

[0025]

[0026] In the formula, (u,v) are pixel coordinates, (u0,v0) are principal point coordinates, (Xc,Yc,Zc) are coordinates in the camera coordinate system, and fx and fy are focal length values ​​in pixels in the two pixel directions.

[0027] Furthermore, in step S12, the homography matrix H is used to perform inverse perspective transformation using the following formula:

[0028]

[0029] In the formula, (us,vs) are the pixel coordinates of the original plane, (ut,vt) are the pixel coordinates of the target plane, and the 3×3 matrix is ​​the homography matrix.

[0030] Furthermore, the architecture of the improved YOLOv8 model in step S2 includes a backbone network, a feature fusion network, and a detection head. The C2f-DRR module is used to replace the C2f module in the backbone network, and a CAA attention perception layer is added at the end of the backbone network.

[0031] Furthermore, the improved YOLOv8 model uses the DWR module to decouple the single-step feature extraction process into two steps: region residualization and semantic residualization. The DRB module is introduced into the second and third branches of the region residualization step. The DWR module as a whole adopts a residual structure design. The first step of the DWR module is region residualization, which consists of a standard 3x3 convolutional layer, a batch normalization layer, and a ReLU activation layer to achieve preliminary region feature extraction. The 3x3 convolutional layer is used to extract local texture and edge features, while the batch normalization layer and ReLU activation layer are used to activate region features and simplify feature representation. The second step of the DWR module is semantic residualization, which uses multi-rate dilated depth convolutions to extract region feature maps at different scales.

[0032] Furthermore, the DRB module includes one large-kernel convolutional main branch and three parallel convolutional branches. The large-kernel convolutional main branch has a kernel size of K=7 and is responsible for capturing the global continuity of the crack. The second branch has a kernel size of K=5 and a dilatancy of r=1, which is used to focus on the fine-grained features of the crack. The third and fourth branches use small-kernel convolutional layers with a kernel size of K=3 and dilatancy of r=2 and 3 respectively. They utilize the characteristics of sparse receptive fields to reduce redundant calculations for irrelevant regions and can adapt to the diverse morphologies of cracks. The feature maps output by each parallel branch are spliced ​​in the channel dimension to form a multi-scale fused feature representation.

[0033] Furthermore, the CAA attention perception layer first extracts global information through global average pooling and extracts local region features using 1×1 convolution. Then, it uses depth strip convolutions with shapes of 1×11 and 11×1 to approximate the standard large kernel depth convolution.

[0034] Furthermore, the formula for extracting global information through global average pooling and extracting local region features using 1×1 convolution is as follows:

[0035]

[0036] In the formula, P avg Represented as a global average pooling operation, X l-1 The input feature map comes from the -1st layer of the network;

[0037] The formula for approximating the standard large-kernel depthwise convolution using depthwise strip convolutions with shapes of 1×11 and 11×1 is as follows:

[0038]

[0039] In the formula, This represents horizontal depth-strip convolution. This represents depth-strip convolution in the vertical direction.

[0040] Compared with existing technologies, the advantages of this invention are as follows: This invention combines binocular stereo vision and inverse perspective transformation technology to construct BEV images of the road surface, effectively solving the problem of inconsistent target scales under traditional perspectives, and providing a global view, enabling the model to focus on the morphology and location information of cracks in the global image; The C2f-DRR module in this invention enhances the multi-scale feature extraction capability of the model through a two-step method of region residualization-semantic residualization, preserves the original feature information through residual connections, and effectively enhances gradient propagation. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 of the present invention.

[0043] Figure 2 This is a flowchart of how the inverse perspective transformation technique is used in this invention to convert the front view into a bird's-eye view (BEV).

[0044] Figure 3 This is a schematic diagram of the fitting plane principle of the RANSAC algorithm in this invention.

[0045] Figure 4 This is a structural diagram of the DWR module in this invention.

[0046] Figure 5 This is a structural diagram of the DRB module in this invention.

[0047] Figure 6 This is a diagram of the CAA attention mechanism in this invention.

[0048] Figure 7 This is a diagram illustrating the binocular camera calibration process in this invention.

[0049] Figure 8 These are the stereoscopic correction results diagrams in this invention. a is the left view, b is the Quality view, c is the Ultra view, and d is the Neural view.

[0050] Figure 9 These are depth maps of the three algorithms used in this invention.

[0051] Figure 10 This is a schematic diagram of the road surface reconstruction point cloud in this invention. a is the original point cloud, and b is the planar fitted point cloud.

[0052] Figure 11 These are comparison images from different perspectives in this invention. a is the front view, and b is the BEV view.

[0053] Figure 12 These are the various indicators in the training process of this invention: a is accuracy P, b is recall R, c is mAP50, and d is mAP50-95.

[0054] Figure 13 This is a comparison chart of the detection results in this invention, where a is the original... Figure 1b is the YOLOv8n result corresponding to a, c is the Ours result corresponding to a, and d is the original result. Figure 2 e is the YOLOv8n result corresponding to d, f is the Ours result corresponding to e, and g is the original result. Figure 3 h represents the YOLOv8n result corresponding to g, and i represents the Ours result corresponding to g. Detailed Implementation

[0055] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0056] See Figure 1 and Figure 2 As shown, this embodiment discloses a method for identifying road surface cracks based on binocular bird's-eye view and improved YOLOv8, including the following steps:

[0057] Step S1: Use inverse perspective transformation technology to convert the front view into a bird's-eye view (BEV).

[0058] Step S2: Use the improved YOLOv8 model to identify road surface cracks in the bird's-eye view BEV.

[0059] In this embodiment, step S1 specifically includes:

[0060] Step S10: Fit the ground plane of the 3D image point cloud reconstructed from binocular vision using the RANSAC algorithm, such as... Figure 3 As shown, the specific steps are as follows:

[0061] Step S100: Randomly sample three points and calculate a planar model A1x+B1y+C1z+D1=0.

[0062] Step S101: Calculate the distance di from all remaining points to the plane, and count the number N of interior points whose distance is within the set threshold ε.

[0063] Step S102: Resampling. The number of interior points in the i-th iteration is Ni, and the number of interior points in the (i+1)-th iteration is Ni+1. If Ni+1 is greater than Ni, the model parameters are updated to ensure that the model parameters with the largest number of interior points are always retained.

[0064] Step S103: After iterating to the set iteration threshold, re-estimate using interior points to finally obtain the optimal model parameters.

[0065] Step S11: Extract the normal vector n=(A,B,C) of the fitted ground plane, and determine the rotation angle θ and rotation axis r=(rx,ry,rz) between the road plane and the orthophoto view. The formulas are as follows (1) and (2). Align the planar point cloud to the bird's-eye view through rotation transformation, and map the point cloud back to the pixel coordinate system in combination with the camera intrinsic formula (3) to establish the point-to-point mapping relationship between the road front view and the bird's-eye view, so as to obtain the corresponding point set.

[0066]

[0067] r = n × z (2)

[0068]

[0069] In the formula, n = (0, 0, 1) is the direction of the orthographic projection axis.

[0070] Step S12: Through the corresponding point set The homography matrix H of the ground plane is solved by the least squares method. The homography matrix H is used to perform inverse perspective transformation through formula (4) to realize the projection mapping from the front view to the bird's-eye view. This method ensures the accurate projection of ground information and the consistency of the viewing angle.

[0071]

[0072] In the formula, (us,vs) are the pixel coordinates of the original plane, (ut,vt) are the pixel coordinates of the target plane, and the 3×3 matrix is ​​the homography matrix.

[0073] The YOLOv8 model inherits the excellent design of its predecessors, achieving further improvements in network architecture and inference efficiency. It adopts a modular design, combining lightweight and high efficiency to meet the needs of real-time object detection and segmentation. Its core architecture consists of three parts: the backbone, the feature fusion network (Neck), and the head. The CSPDarknet53 backbone optimizes the gradient flow in the feature extraction process by introducing a C2f module, reducing redundant computation and enhancing the model's ability to extract features at multiple scales. The Neck uses a multi-scale feature fusion structure combining FPN and PAN, effectively capturing feature information of the target at different scales. The head employs a decoupled head and anchor-free design, which not only optimizes the learning process for different targets but also significantly reduces computational complexity.

[0074] However, due to the high visual similarity between road surface cracks and the road background in bird's-eye view BEVs, crack areas are usually small and unevenly distributed, making detection difficult. To address these issues, this embodiment proposes an improved YOLOv8 model. The core idea is to replace the C2f module in the backbone network with a C2f-DRR module, aiming to capture structural information at multiple scales to enhance the receptive field, detect large-scale cracks while maintaining fine-grained details, and employ a structural reparameterization strategy to reduce computational overhead while preserving multi-scale representations. A CAA attention perception layer is added at the end of the backbone network to dynamically adjust the multi-scale feature weights, enabling the model to focus on the crack center area and suppress background noise such as road stains.

[0075] In this embodiment, the improved YOLOv8 model uses the DWR module to decouple the single-step feature extraction process into two steps: region residualization and semantic residualization. The DRB module is introduced into the second and third branches in the region residualization step to alleviate the problem of uneven sampling of feature information caused by standard dilated convolution.

[0076] like Figure 4 As shown, the DWR module adopts a residual structure design. The first step of the DWR module is region residualization, which consists of a standard 3x3 convolutional layer, a batch normalization layer, and a ReLU activation layer to achieve preliminary region feature extraction. Among them, the 3x3 convolutional layer is used to extract local texture and edge features, while the batch normalization layer and the ReLU activation layer are used to activate region features and simplify feature representation. The second step of the DWR module is semantic residualization. In this stage, multi-rate dilated depth convolution is used to extract region feature maps at different scales. Each channel uses only a fixed receptive field to avoid feature redundancy.

[0077] Specifically, the region feature map generated in the first step is split into three branches and morphological filtering is performed using depthwise convolutions with dilation rates of 1, 3, and 5. Furthermore, since it is difficult to directly establish large-span spatial connections through convolution, and long-span connections rely on the assistance of small-span connections, small receptive fields remain crucial at each stage. Based on this, the output channel capacity of the first branch is expanded to twice that of the other branches to better support small-span connections.

[0078] After acquiring multi-scale contextual information, the feature maps of all branches are concatenated and batch normalized. Then, pointwise convolution is used to merge the features, forming the final residuals. Finally, these residuals are added to the input feature maps to construct a more powerful and comprehensive feature representation. The region residualization-semantic residualization method transforms complex semantic extraction into ordered morphological filtering, effectively capturing the global structure of cracks, more accurately capturing multi-scale features of cracks, and reducing background interference.

[0079] In this embodiment, road surface cracks from the BEV perspective are characterized by their small size and diverse shapes. The original DWR module's second and third branches use dilated convolutions with a fixed void ratio for feature extraction. While this expands the receptive field, the fixed void ratio makes the receptive field adjustment inflexible, resulting in insufficient adaptability to cracks of different scales and shapes. It also lacks sufficient feature capture capability at crack edges and in complex structural areas, failing to balance local details with overall continuity, thus affecting the accuracy of the detection results. Therefore, this embodiment introduces two DRB (Dilated ReparamBlock) modules of different sizes, with the following structures: Figure 5 As shown, replacing the dilated convolutions in the second and third branches of the DWR module, the DRB module effectively integrates local fine-grained features with global structural information by introducing a main branch with a large kernel and multiple branches with small kernels, thereby enhancing the flexibility and precision of feature extraction. The hyperparameters of the DRB module include the kernel size K of the large kernel convolution, the kernel size k of the parallel small kernel convolution, and the dilation rate r.

[0080] Since the crack widths in BEV images are concentrated in the single-digit pixel range, an excessively large receptive field may lead to overly smoothed local features, weakening the detection capability of small cracks. Therefore, without significantly increasing the computational cost and model parameter count while maintaining a sufficient receptive field, DRB modules with main branch kernel sizes of K=5 and K=7 were selected. Figure 5 Taking the example shown, the DRB module contains one large-kernel convolutional main branch and three parallel convolutional branches. The large-kernel convolutional layer in the main branch has a kernel size of K=7 and is responsible for capturing the global continuity of the crack, providing a wide field of view for the overall features. The second branch has a kernel size of K=5 and a dilation rate of r=1, which is used to focus on the fine-grained features of the crack's local area, compensating for the fuzziness at the detail level caused by the large kernel. The third and fourth branches use small-kernel convolutional layers with a kernel size of K=3 and dilation rates of r=2 and 3 respectively, utilizing their sparse receptive field characteristics to reduce redundant computation on irrelevant regions, while also adapting to the diverse morphologies of the crack. Finally, the feature maps output by each parallel branch are concatenated along the channel dimension to form a multi-scale fused feature representation.

[0081] To optimize the inference efficiency of the DRB module, the Batch Normalization (BN) layer is merged into the convolutional layer using the structure reparameterization method. When the input channel is C, the output channel is D, and the convolutional kernel size is K×K, the convolutional kernel can be represented as a fourth-order tensor. Bias term The input to this convolutional layer is a shape of... The feature map outputs a shape of Its formula is expressed as equation (5).

[0082] O=I#F+REP(b)(5)

[0083] Let j be the channel index. During training, the Batch Normalization (BN) layer performs mean normalization and a linear transformation on each channel. Specifically, μ j and σ j These represent the mean and standard deviation accumulated during the training process for that channel, respectively; γ j and β j Let be the learnable scaling factor and bias term of the BN layer. Under the action of the BN layer, the output of channel j is given by equation (6).

[0084]

[0085] Due to the homogeneity of convolution, BN layers can be fused into preceding convolutional layers during the inference phase. In practical applications, a new equivalent convolutional layer can be directly constructed, with its convolution kernel denoted as F′ and its bias term denoted as b′. According to formulas (5) and (6), for each output channel j, they are calculated as shown in formula (7). Thus, only F′ and b′ need to be stored, eliminating the need for BN layer calculations during the inference phase, and an equivalent convolutional layer can be directly used for forward propagation.

[0086]

[0087] Attention mechanisms, due to their simplicity and efficiency, play a crucial role in enhancing the representation capabilities of neural networks. Attention mechanisms such as SimAM are widely used in pavement distress detection tasks to enhance feature representation. However, pavement cracks are easily affected by complex backgrounds such as pavement stains, tree branches, and fallen leaves, and typically extend in a single direction, exhibiting a high aspect ratio and linear structure. This characteristic makes it difficult for traditional channel and spatial attention mechanisms to effectively capture long-range dependencies from a BEV (Browser-Engineer-Driven) perspective.

[0088] To address this, this embodiment introduces a lightweight context anchor attention mechanism (CAA) to model the contextual dependencies between distant pixels, thereby improving the integrity and continuity of crack target segmentation. Its structure is as follows: Figure 6 As shown, global information is first extracted by global average pooling, and local features are extracted by 1×1 convolution. The calculation formula is shown in Equation (8). Then, depth strip convolution with shapes of 1×11 and 11×1 is used to approximate the standard large kernel depth convolution, thereby enhancing the model's ability to capture long-distance features such as cracks while maintaining computational efficiency. The calculation formula is shown in Equation (9).

[0089]

[0090] In formula (8): Π avg Represented as a global average pooling operation, X l-1 The input feature map comes from the (l-1)th layer of the network.

[0091] In equation (9): This represents horizontal depth-strip convolution. This represents depth-strip convolution in the vertical direction.

[0092] Depth strip convolution has low computational cost, using two 1×k... b and k b ×1 One-dimensional depthwise convolution replaces the standard k b ×k b Two-dimensional depthwise convolution can achieve similar results. Its parameter count is reduced from the original C×k... b ×k b Reduced to 2×C×k b It is approximately 2 / k of the standard depthwise convolution. b C is the number of input channels, k b The kernel size is denoted by . Secondly, strip convolution is sensitive to the continuous directional features of slender targets, such as bridges or linearly distributed road cracks in remote sensing images. Finally, the CAA module generates an attention weight using equation (10). Used to enhance the feature output of the backbone network.

[0093]

[0094] In the formula: the Sigmoid function is used to normalize the attention weights, limiting them to the range (0, 1); ⊙ represents element-wise multiplication. This indicates element-wise addition; For the enhanced features; P l-1 These are the output characteristics of the backbone network.

[0095] The following experimental verification and analysis further illustrate this embodiment:

[0096] This embodiment establishes a lightweight binocular camera system for data acquisition, including a camera mount, a ZED2i binocular camera, a computer, and a test vehicle. Compared to traditional testing vehicles, this platform is more portable, easier to deploy, lower in cost, and more compact.

[0097] I. Binocular Camera Calibration and Image Correction: The experiment used a 30mm × 30mm checkerboard calibration board and acquired 20 pairs of binocular images in different poses. Camera calibration was completed using the 2D planar checkerboard calibration method proposed by Zhang. The calibration process is as follows: Figure 7 As shown, the reprojection error of each pair of calibration images is less than 0.2 pixels, and the total average error is 0.15 pixels. The obtained intrinsic and extrinsic parameters of the binocular system provide key parameter support for subsequent point cloud extraction. The final calibration results are shown in Table 1.

[0098] Table 1

[0099]

[0100] The acquired images are then subjected to distortion correction based on the calibrated distortion parameters to eliminate lens-induced geometric distortion and ensure that the images reflect the true geometric structure. Subsequently, stereo correction is performed using an extrinsic parameter matrix to ensure that the two imaging planes are parallel to the baseline, guaranteeing that the same object point in both images is on the same horizontal line. Figure 8 As shown.

[0101] II. Road BEV View Conversion: After stereo correction, the stereo matching algorithm only needs to search for matching points near the same height, significantly improving matching efficiency and obtaining accurate disparity information to calculate depth values. The ZED SDK provides Quality and Ultra modes based on traditional stereo matching, as well as a Neural mode based on deep learning. To evaluate the performance of the three depth estimation modes in different scenarios, this embodiment collected video data in campus, reservoir, and urban road environments, and constructed test samples according to the strategy of extracting one frame every 10 seconds, extracting 42 test images for each scenario. This strategy ensures diversity in covering complete road segments while reducing image content redundancy. Partial results of the depth maps for the three modes are shown below. Figure 9 As shown, it can be observed that the Quality and Ultra modes often exhibit deep holes in weak texture areas and shadow areas of lane lines, resulting in poor structural continuity; while the depth map generated by the Neural mode performs better in terms of density and edge coherence.

[0102] This embodiment selects the Neural mode, using the left view as a reference to obtain the disparity information of each pixel, and then reconstructs a road surface point cloud map containing three-dimensional spatial coordinates based on the disparity and binocular camera parameters. An example result is shown below. Figure 10 (a) shows.

[0103] Instead of using a globally fixed plane assumption, road plane fitting was performed frame-by-frame using the RANSAC algorithm to improve the stability of BEV view generation. In the experiment, when the distance threshold ε was set to 30mm, the fitting conditions were too strict, leading to the incorrect removal of effective point clouds at the far end of the road surface. When the threshold ε was set to 50mm, too many non-ground points were included, resulting in significant noise interference. Since the number of effective road surface point clouds directly affects the accuracy of the homography matrix H, the distance threshold ε was optimized to 40mm. This ensured that as many effective road surface point clouds as possible were retained while effectively suppressing noise interference, thus achieving the optimal accuracy of H. Furthermore, 100 planes were fitted at different iteration numbers, and the root mean square error (RMSE) between the fitted plane and the actual point cloud was calculated to evaluate the fitting effect. The experimental results are shown in Table 2. The results show that the RMSE decreased to 8.0394mm at 1200 iterations, reaching the lowest value, indicating the best plane fitting effect at this point. The corresponding road point cloud segmentation results are shown in Table 2. Figure 10 (b) shows.

[0104] Table 2 Fitting Plane Error

[0105]

[0106]

[0107] After the road surface fitting is completed, the road surface normal vector n = (a, b, c) is extracted using the plane equation ax + by + cz + d = 0, and the angle between it and the Z-axis (0, 0, 1) is calculated using the formula. Then, a rotation matrix R is constructed based on the Rodriguez Formula to rotate the road plane to be parallel to the XY plane. After the rotation operation is completed, the point cloud is projected onto the pixel coordinate system using equation (11), thereby obtaining the matching points between the original front view and the target bird's-eye view. Based on these matching points, a system of linear equations is constructed and the homography matrix H is solved to achieve inverse perspective transformation and generate the required bird's-eye view. Figure 11 The results show that BEV views can eliminate the near-large-far-small effect caused by perspective distortion, keep the scale of pavement damage information consistent, and improve the coverage of the entire road.

[0108]

[0109] The experimental metrics of the improved model in this embodiment will be analyzed below.

[0110] I. Environment and Parameter Settings

[0111] The experiment used the campus and surrounding urban roads as the test area. A total of 1330 BEV (Build-Effective Vehicle) images of the road surface, including various weather conditions and crack morphologies, were collected and processed using the experimental platform. These images were labeled with LabelMe under one category: crack, containing 3568 defect instances. The training, validation, and test sets were randomly divided in an 8:1:1 ratio. The test set was not used for training but was used to evaluate the model's final performance. Data augmentation techniques such as rotation, mirroring, and brightness transformations were employed to increase the scale of the training data, adapting to various changes in real-world application scenarios. As shown in Table 3, all experiments were conducted under the same environment and configuration. Secondly, consistent parameters were used throughout the training process. The model used was YOLOv8n, with model depth and width coefficients of 0.33 and 0.25, respectively. The initial learning rate was 0.01, the learning rate momentum was 0.937, the weight decay coefficient was 0.0005, the stochastic gradient descent optimizer was used, the input image size was 800×800, the batch size was 16, the number of subprocesses during data loading was set to 4, the total number of training epochs was 300, and the loss function used was CIoU.

[0112] Table 3 Experimental Environment Configuration

[0113]

[0114] To quantitatively evaluate the performance of each training model in the pavement crack segmentation task, this embodiment uses the following metrics for analysis: accuracy (P), recall (R), F1 score, and mean precision (mAP50) at an IoU threshold of 0.5. The calculation formulas for each metric are shown in (14) to (18). In the crack segmentation task studied, accuracy P measures the proportion of actual cracks among the crack instances predicted by the model, recall R measures the proportion of all real cracks that are correctly detected, the F1 score is a weighted average of P and R, used to comprehensively measure the detection capability of the model, and mAP50 reflects the overall performance of the model at different confidence thresholds.

[0115]

[0116] In the formula, TP represents the number of samples that the model correctly segmented into cracks; FP represents the number of samples that the model incorrectly predicted as cracks instead of cracks; and FN represents the number of samples that the model predicted as background cracks.

[0117] II. Curve Comparison Analysis

[0118] The original YOLOv8n and the improved network model were trained on the training set respectively. Various metrics during the training process were as follows: Figure 12The results show that the improved model significantly outperforms the original model in all metrics, validating the superiority of the improved algorithm in overall detection performance. Specifically, the accuracy (P) metric of the improved model is more stable during training and significantly surpasses the original YOLOv8n during the convergence phase, indicating that the improved network effectively reduces false detections and improves the accuracy of target prediction. The recall (R) metric is basically on par with the original model, indicating that the improved scheme does not affect the model's target detection ability while improving accuracy, ensuring the stability of the false negative rate. In addition, the improved model also outperforms the original YOLOv8n in both mAP50 and mAP50-95 metrics, indicating that the overall detection accuracy of the model is enhanced under different IoU thresholds. The improved strategy optimizes multi-scale feature fusion, which significantly improves the model's robustness in dealing with cracks of different scales and enhances its adaptability to complex road scenarios.

[0119] III. Comparative Analysis of Different Attention Mechanisms

[0120] To verify the effectiveness of the CAA attention mechanism in BEV crack detection, three mainstream attention mechanisms were introduced into the benchmark model for comparative analysis. The experimental results are shown in Table 4. SimAM improved accuracy by 1.2%, but its recall and mAP50 decreased by 2.8% and 0.3%, respectively, indicating that while suppressing false positives, it also suffers from false negatives. MLCA slightly improved mAP50 by 0.4%, but both accuracy and recall decreased, indicating limited generalization ability. ELA improved accuracy by 1.3%, but the overall detection performance improvement was limited. In contrast, the CAA mechanism introduced in this embodiment improved accuracy by 3.2% and mAP50 by 1.4%. Although the CAA attention mechanism suppressed weak features to some extent, causing a slight decrease in recall, its overall detection performance was superior to other mechanisms. Compared to other attention mechanisms that focus on enhancing channel and spatial information, CAA models long-distance dependencies through strip convolution, better capturing the spatial continuity and directional features of cracks, making it more suitable for slender structure detection tasks from a BEV perspective.

[0121] Table 4 Comparison of different attention mechanisms

[0122]

[0123]

[0124] IV. Ablation Experiment

[0125] To further verify the improved model's performance in detecting pavement cracks in BEV scenarios and to evaluate the effectiveness of the C2f-DRR module and the CAA attention module, this embodiment was tested on a test set, and four ablation experiments with different schemes were designed. The experimental results are shown in Table 5. The first group is the baseline model without the improvement. Due to the fixed receptive field design of the C2f block, the original model's multi-scale detection capability is weak, with an mAP50 of only 0.792, indicating room for improvement. The second group replaced all C2f modules in the original backbone network with the C2f-DRR module. The number of model parameters was reduced to 3.08M, the computational cost was reduced by 4.2%, while the accuracy P increased by 2.6%, the recall R remained the same, the mAP50 increased by 3.0%, and the F1 score increased by 1.3%. This demonstrates that the two-step decoupled multi-scale feature extraction method of "regional residualization-semantic residualization" adopted by this module effectively enhances the ability to capture crack defects and makes up for the shortcomings of the original C2f module in handling crack structures of different scales. Secondly, the synergistic effect of large-kernel convolution and small-kernel convolution enhances the ability to model the global continuity of pavement cracks and extract local fine-grained information, reducing the loss of key information. The third group adds a CAA attention layer at the end of the backbone network, slightly increasing the number of model parameters by 4.3% and computation by 0.8%, while improving detection accuracy by 3.2%, mAP50 by 1.4%, and F1 score by 1.1%. This demonstrates that the CAA attention mechanism, from a BEV perspective, effectively enhances the model's ability to perceive slender linear features and has a significant effect on modeling long-distance dependencies of cracks. However, the recall rate R slightly decreases, possibly due to insufficient coverage of crack edges by strip convolution, leading to missed detection of some microcracks. The fourth group incorporates all modules, resulting in the final optimized model. This model has a parameter count and computational cost close to the baseline model, while its accuracy P is improved by 4%, mAP50 by 4.4%, and F1 score by 1.8%. This demonstrates that the combined effect of the two modules significantly improves the model's detection performance and global-local information fusion capabilities in the BEV pavement crack segmentation task.

[0126] Table 5 Ablation Experiment Results

[0127]

[0128] V. Comparison Experiments of Different Models

[0129] To verify the actual detection performance of the improved model proposed in this embodiment, the algorithm of this embodiment was compared and analyzed with other advanced models. The results are shown in Table 6. It can be seen that the accuracy P, recall R and mAP50 of the model in this embodiment reached 83.9%, 83.2% and 83.7% respectively. Compared with YOLOv5n, YOLOv8n, RTMDet, and the literature [Bai Feng, Ma Qinglu, Zhao Min. AC-YOLO for aerial road surface crack detection[J]. Computer Engineering and Applications, 2025, 61(01):153-164.DOI:10.3778 / j.issn.1002-8331.2407-0139] and the literature [Deng Tianmin, Chen Yuetian, Yu Yang, et al. Road surface disease detection algorithm focusing on shape features], the model is significantly improved. [J]. Computer Engineering and Applications, 2024, 60(24):291-305.DOI:10.3778 / j.issn.1002-8331.2404-0259], the average precision of the model in this embodiment was improved by 13.0, 4.4, 7.0, 3.6, and 5.4 percentage points, respectively, and the accuracy was improved by 10.9, 4.0, 9.7, 2.8, and 5.5 percentage points, respectively. Although RTMDet has a high recall rate, the number of parameters is as high as 5.6×106, which is much higher than that of the model in this embodiment, which is not conducive to lightweight deployment. In terms of computational complexity, the number of parameters of the model in this embodiment is 3.22×106, and the computational cost is 11.6 GFLOPs. Compared to YOLOv8n and the literature [Deng Tianmin, Chen Yuetian, Yu Yang, et al. Pavement distress detection algorithm focusing on shape features [J]. Computer Engineering and Applications, 2024, 60(24):291-305.DOI:10.3778 / j.issn.1002-8331.2404-0259], this model not only reduces the parameter scale, but also slightly reduces its computational cost. This shows that the model proposed in this embodiment achieves a good balance between complexity and computational efficiency.

[0130] Table 6 Comparison of Results from Different Models

[0131]

[0132]

[0133] VI. Generalization Test

[0134] To further verify the actual generalization ability of the optimized model proposed in this embodiment, the publicly available UAV-PDD2023 dataset was used. This dataset consists of road surface crack images captured by a UAV flying at a height of 30 meters and a speed of 0.8 meters per second. The dataset has an image resolution of 2592×1944 and contains 2440 images of road surface defects in complex scenarios including different weather conditions, lighting conditions, and obstructions from trees and vehicles. The experimental results are shown in Table 7. The experimental results show that the accuracy, recall, and mean precision are all superior to the baseline model, improving by 0.6%, 4.5%, and 3.5%, respectively. This demonstrates that the method in this embodiment has stronger robustness and generalization performance.

[0135] Table 7 Generalization Test Results

[0136]

[0137] VII. Visual Analysis

[0138] To more intuitively analyze the detection performance of the model, the visualization results of the original YOLOv8n model and the improved model on the crack segmentation task were compared, such as... Figure 13 As shown in the figure, when the crack is long and spans the entire lane, subfigure (b) shows that the original model failed to detect some crack areas, resulting in missing detections; in subfigure (e), the middle part of the crack was not completely segmented due to low contrast, leading to local missing detections. This may stem from the original model's insufficient ability to extract features of small cracks, resulting in discontinuous segmentation results. In contrast, subfigures (c) and (f) show that the improved model can segment cracks more completely, improving the coherence of detection. Furthermore, in subfigure (h), the original model missed a detection, while the improved model successfully identified the crack, further validating the effectiveness of the optimization method in improving the completeness of crack detection and reducing missed detections.

[0139] Detection of road surface cracks is fundamental to repair work, helping to prevent cracks from worsening into more serious damage, thereby improving road safety and reducing the cost of intelligent maintenance. Addressing existing problems in road surface crack detection, this embodiment proposes a road surface crack identification method based on binocular bird's-eye view and an improved YOLOv8. Through experimental verification and theoretical analysis, the main advantages include:

[0140] (1) By combining binocular stereo vision and inverse perspective transformation technology, a BEV image of the road surface is constructed, which effectively solves the problem of inconsistent target scale under traditional perspective and provides a global view, so that the model can focus on the shape and location information of the crack in the global image.

[0141] (2) The C2f-DRR module enhances the model's multi-scale feature extraction capability through a two-step method of region residualization and semantic residualization. Residual connections preserve the original feature information and effectively enhance gradient propagation. The large-kernel convolutions in the 2nd and 3rd branches of this module capture the overall crack orientation, while the small-kernel hollow branches enhance local details; their synergistic effect achieves refined extraction of multi-scale crack features. The CAA attention mechanism strengthens the feature focusing capability of the crack center region through dynamic weight allocation. Joint optimization enables the final model to achieve an mAP50 of 83.7%, an accuracy of 83.9%, and an F1 score of 83.5%, representing improvements of 4.4%, 4%, and 1.8% respectively compared to the original model.

[0142] (3) Test results on the UAV-PDD2023 dataset show that the improved model improves mAP50 by 3.5%, recall by 4.5%, and accuracy by 0.6% in complex lighting and occlusion scenarios, demonstrating its stronger robustness and generalization learning ability.

[0143] While this embodiment demonstrates superior detection performance, it still has shortcomings in terms of data diversity and adaptability to extreme environments, and the model's performance in detecting small-target cracks in complex terrain still has room for improvement. Current BEV image generation is based on frame-by-frame ground plane fitting, primarily suitable for large-scale, approximately flat road environments; however, projection consistency and accuracy still face challenges in scenarios with significant slope changes or complex terrain. In future, for defect identification, generative adversarial networks (GANs) can be introduced to synthesize road data from extreme environments, enriching the training samples, and combined with deformable convolutional networks to dynamically adapt to changes in crack morphology, further improving detection accuracy and robustness. Regarding BEV generation, end-to-end BEV feature mapping methods based on the Transformer mechanism can be explored, combining multi-sensor information such as IMU with temporal modeling, fusing pose and image features to reduce computational overhead, improve overall reconstruction robustness and accuracy, and enhance adaptability to complex terrain road scenarios.

[0144] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, the patent owner may make various modifications or alterations within the scope of the appended claims, as long as they do not exceed the protection scope described in the claims of the present invention, they shall be within the protection scope of the present invention.

Claims

1. A road surface crack identification method based on binocular bird's-eye view and improved YOLOv8, characterized in that, Includes the following steps: S1. Use inverse perspective transformation technology to convert the front view into a bird's-eye view (BEV); S2. Use the improved YOLOv8 model to identify road surface cracks in the bird's-eye view BEV.

2. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 1, characterized in that, Step S1 specifically includes: S10. Perform ground plane fitting on the 3D image point cloud reconstructed by binocular vision using the RANSAC algorithm. S11. Extract the normal vector n = (A, B, C) of the fitted ground plane, and determine the rotation angle θ and rotation axis r = (rx, ry, rz) between the road plane and the orthophoto view. Align the planar point cloud to the bird's-eye view through rotation transformation, and map the point cloud back to the pixel coordinate system in combination with camera intrinsics to establish a point-to-point mapping relationship between the road front view and the bird's-eye view, so as to obtain the corresponding point set. S12, through the corresponding point set The homography matrix H of the ground plane is solved using the least squares method, and the homography matrix H is used to perform inverse perspective transformation to achieve projection mapping from the front view to the bird's-eye view.

3. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 2, characterized in that, Step S10 specifically includes: S100. Randomly sample three points and calculate a planar model A1x+B1y+C1z+D1=0; S101. Calculate the distance di from all remaining points to the plane, and count the number N of points whose distance is within the set threshold ε. S102. Resampling: The number of inliers in the i-th iteration is Ni, and the number of inliers in the (i+1)-th iteration is Ni+1. If Ni+1 is greater than Ni, the model parameters are updated to ensure that the model parameters with the largest number of inliers are always retained. S103. After iterating to the set iteration threshold, the interior points are used for re-estimation to finally obtain the optimal model parameters.

4. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 2, characterized in that, Step S11 determines the rotation angle θ between the road plane and the orthographic viewing angle and the rotation axis r = (rx, ry, rz) using the following formula. r = n × z In the formula, n = (0,0,1) represents the direction of the orthographic projection axis; The point cloud is mapped back to the pixel coordinate system using the following formula in conjunction with camera intrinsic parameters: In the formula, (u,v) are pixel coordinates, (u0,v0) are principal point coordinates, (Xc,Yc,Zc) are coordinates in the camera coordinate system, and fx and fy are focal length values ​​in pixels in the two pixel directions.

5. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 2, characterized in that, In step S12, the homography matrix H is used to perform inverse perspective transformation using the following formula: In the formula, (us,vs) are the pixel coordinates of the original plane, (ut,vt) are the pixel coordinates of the target plane, and the 3×3 matrix is ​​the homography matrix.

6. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 1, characterized in that, The architecture of the improved YOLOv8 model in step S2 includes a backbone network, a feature fusion network, and a detection head. The C2f-DRR module is used to replace the C2f module in the backbone network, and a CAA attention perception layer is added at the end of the backbone network.

7. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 6, characterized in that, The improved YOLOv8 model uses the DWR module to decouple the single-step feature extraction process into two steps: region residualization and semantic residualization. The DRB module is introduced into the second and third branches of the region residualization step. The DWR module as a whole adopts a residual structure design. The first step of the DWR module is region residualization, which consists of a standard 3x3 convolutional layer, a batch normalization layer, and a ReLU activation layer to achieve preliminary region feature extraction. The 3x3 convolutional layer is used to extract local texture and edge features, while the batch normalization layer and ReLU activation layer are used to activate region features and simplify feature representation. The second step of the DWR module is semantic residualization, which uses multi-level dilated depthwise convolutions to extract region feature maps at different scales.

8. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 7, characterized in that, The DRB module includes one large-kernel convolutional main branch and three parallel convolutional branches. The large-kernel convolutional main branch has a kernel size of K=7 and is responsible for capturing the global continuity of the crack. The second branch has a kernel size of K=5 and a dilatancy of r=1, which is used to focus on the fine-grained features of the crack. The third and fourth branches use small-kernel convolutional layers with a kernel size of K=3 and dilatancy of r=2 and 3 respectively. They utilize the characteristics of sparse receptive fields to reduce redundant computation on irrelevant regions and can adapt to the diverse morphologies of cracks. The feature maps output by each parallel branch are spliced ​​in the channel dimension to form a multi-scale fused feature representation.

9. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 7, characterized in that, The CAA attention perception layer first extracts global information through global average pooling and extracts local region features using 1×1 convolution. Then, it uses depth strip convolutions with shapes of 1×11 and 11×1 to approximate the standard large kernel depth convolution.

10. The road surface crack identification method based on binocular bird's-eye view and improved YOLOv8 according to claim 9, characterized in that, The formula for extracting global information through global average pooling and extracting local region features using 1×1 convolution is as follows: In the formula, P avg Represented as a global average pooling operation, X l-1 The input feature map comes from the -1st layer of the network; The formula for approximating the standard large-kernel depthwise convolution using depthwise strip convolutions with shapes of 1×11 and 11×1 is as follows: In the formula, This represents horizontal depth-strip convolution. This represents depth-strip convolution in the vertical direction.

Images