A lightweight asphalt road defect detection method and system based on partial multi-scale attention

Abstract

This invention discloses a lightweight asphalt road defect detection method and system with partial multi-scale attention, belonging to the field of computer vision and intelligent transportation technology. This invention is applicable to the real-time detection and precise localization of defects such as cracks and potholes. It employs PMA modules, SCConv modules, and the inner-AIoU loss function to construct the required asphalt road detection model, achieving accurate detection of the road image to be detected. The model constructed in this invention, while maintaining its lightweight advantage, achieves significant improvements in speed, substantial increases in defect detection accuracy, and diversification of detected defect categories, fully meeting the needs of real-time detection. Furthermore, the detection model constructed in this invention also possesses good portability and scalability, making its integration with hardware devices more convenient.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to a lightweight asphalt road defect detection method and system with partial multi-scale attention. Background Technology

[0002] In recent years, asphalt pavement defect detection has become a focus of industry research. Due to multiple factors such as severe weather, pavement erosion, high-load operation, and road aging, asphalt pavements are highly susceptible to defects. These defects not only shorten the service life of the road but, in the absence of maintenance and excessive use, may also accelerate pavement deterioration or even collapse, posing serious safety hazards and directly threatening people's lives and property.

[0003] Traditional algorithms excel at identifying obvious defects, but when faced with small defects or images with severe noise interference, they often misidentify noise as defects, resulting in spurious segmentation and significantly impacting detection performance. To address these challenges, convolutional neural networks (especially the Unet network) have been widely used in defect detection. However, the Unet network is somewhat lacking in processing speed, making it difficult to meet the demands of real-time detection.

[0004] In existing technologies, improved YOLO-based algorithms enhance detection accuracy through multi-scale feature fusion, but their high parameter count makes them difficult to deploy on edge devices. While dynamic deformable convolution methods optimize feature extraction, they fail to address noise interference in complex backgrounds. Furthermore, traditional methods lack real-time recording capabilities for defect locations, making it difficult to meet the closed-loop management requirements of road maintenance.

[0005] The YOLOv10 algorithm, as a highly efficient object detection algorithm, can quickly locate and identify objects in images. It divides the input image into multiple feature maps and uses a multi-layer convolutional neural network to learn and extract features to predict the location and class probability of the object detection box, thus quickly generating detection results. However, the YOLOv10 algorithm faces certain challenges in small object detection because small objects occupy a limited number of pixels in the image and are easily overlooked or misclassified as noise or background. The invention with publication number CN115205684A improves the model's generalization ability and detection accuracy by adding MobileNetV3 as the main feature extraction module, incorporating an attention mechanism during feature concatenation, and adding a ConvLSTM module before the feature extraction network. The invention with publication number CN 117437201A introduces the PP-LCNet network and further introduces a coordinate attention mechanism (CA) module deep within PP-LCNet, effectively improving the accuracy of defect detection. In summary, deep learning-based road defect detection methods have the following shortcomings: existing models lack accuracy in detecting small targets, have a large number of parameters, and are difficult to run in real time; they lack collaborative optimization of multi-scale feature fusion and lightweight design; and they lack defect localization functions, making it impossible to link with maintenance systems. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a lightweight asphalt road defect detection method and system with partial multi-scale attention, in order to improve the detection accuracy of road defects.

[0007] On the one hand, the present invention provides a method for detecting defects in lightweight asphalt roads using partial multi-scale attention, which includes the following steps:

[0008] Step S1: Construct a training set for model training;

[0009] Step S2, constructing an asphalt road defect detection model based on the YOLOv10 model, including:

[0010] A self-calibrating convolutional module (SCConv) is added to the third layer of the backbone network of the YOLOv10 model to reduce feature redundancy through spatial reconstruction unit (SRU) and channel reconstruction unit (CRU).

[0011] At the end of the backbone network, a Partial Multi-scale Attention (PMA) module is embedded. The main branches of the PMA module include a standard convolutional layer 1, a partial multi-scale attention mechanism module iPMA, a depthwise separable convolutional layer 1, a depthwise separable convolutional layer 2, a splicing layer, and a standard convolutional layer 2.

[0012] The input feature map of the depthwise separable convolutional layer 1 is the element-wise sum of the output feature map of the standard convolutional layer 1 and the output feature map of the iPMA module; the input feature map of the concatenation layer includes the element-wise sum of the input feature map of the depthwise separable convolutional layer 1 and the output feature map of the depthwise separable convolutional layer 2, as well as the output feature map of the standard convolutional layer 1.

[0013] The iPMA module divides its input feature map into two parts by channel. It performs multi-branch convolution on the front part (high-level channels) to capture multi-scale features and fuses them based on an attention mechanism to obtain the processed front part features. The back part (low-level channels) retains the original features and performs lightweight operation through depthwise separable convolution. Then it is concatenated with the processed front part features and the channel dimensions are adjusted through convolutional layers to achieve feature fusion. The output is a unified feature map, which is the output feature map of the iPMA module.

[0014] Step S3: Based on the constructed training set, the asphalt road defect detection model is trained using the inner-AIoU loss function. When the preset training convergence condition is met, the trained asphalt road defect detection model for the road image to be detected is obtained.

[0015] Furthermore, step S1 includes: acquiring a road defect dataset through a multi-view imaging system, constructing an original dataset by combining it with a public dataset, and performing defect annotation and image preprocessing on it to construct a training set;

[0016] Further image preprocessing includes adaptive histogram equalization and image enhancement.

[0017] Furthermore, image enhancement processing includes: randomly rotating the image, adjusting the scaling ratio, performing horizontal or vertical flipping, and adjusting the image's color and brightness. These enhancement operations can improve the model's generalization ability, enabling it to cope with more diverse input scenarios.

[0018] Furthermore, the iPMA module divides its input feature map by channels: the first 1 / 3 is used as the high-level channel, and the last 2 / 3 is used as the low-level channel.

[0019] Furthermore, the iPMA module utilizes four parallel paths to extract attention weight descriptors from the grouped feature maps. Two of these parallel paths are 1×1 convolutional branches, while the latter two are 3×3 and 5×5 convolutional branches.

[0020] Furthermore, the inner-AIoU loss function is:

[0021]

[0022] Among them, the internal intersection ratio loss `inter` and `union` are the intersection and union areas of the predicted bounding box and the auxiliary bounding box (ground truth box) output by the model, respectively; (The last part, "weighting parameters," appears to be an unrelated fragment and is left untranslated.) Aspect Ratio Consistency Parameters ρ represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, and w and h are the width and height of the predicted bounding box, respectively. gt and h gt b is the width and height of the actual bounding box gt b and b represent the ground truth bounding box and its anchor point, respectively. IOU represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box. t is the current training round.

[0023] Furthermore, the scaling factor ratio is used to control the scaling of the auxiliary bounding box used to calculate the inner-AIoU loss. The scaling factor ratio is dynamically adjusted with each training epoch, and the adjustment formula is as follows:

[0024]

[0025] Where T is the total number of training rounds, and t is the current training round.

[0026] Another aspect of the present invention provides a lightweight asphalt road defect detection system with partial multi-scale attention, which includes: a camera module, a defect detection module and a GPS positioning module;

[0027] The camera module uses multispectral imaging technology to acquire images of the road to be inspected in order to support image acquisition in low-light environments; and transmits the acquired images of the road to be inspected to the defect detection module.

[0028] The defect detection module detects road defects based on the asphalt road defect detection model constructed in this invention. The module first performs image preprocessing on the received road image to be detected to match the model input, and then inputs it into the asphalt road defect detection model to obtain the detection result. The defect detection module sends the detection result to the GPS positioning module, which is used to obtain the corresponding defect location, and generates a defect information report based on the detection result and sends it to the target object.

[0029] Furthermore, the GPS positioning module works in conjunction with a GIS (Geographic Information System) platform to obtain the location of defects. The GPS positioning module also includes generating a heatmap of the defect locations and outputting a maintenance report. In addition, the asphalt road defect detection model of this invention supports real-time video stream processing. The detection system of this invention can also be continuously optimized and iterated based on actual application results to further improve overall detection performance.

[0030] The technical solution provided by this invention brings at least the following beneficial effects:

[0031] 1) Adding PMA to the YOLOv10 backbone network enables the model to extract target features more effectively. This invention adds the PMA attention mechanism to the last layer of the YOLOv10 model backbone network. With the goal of preserving information on each channel and reducing computational overhead, some channels are reshaped into batch dimensions, and the channel dimensions are grouped into multiple sub-features, so that spatial semantic features are evenly distributed in each feature group.

[0032] 2) Add SCConv to the third layer of the YOLOv10 backbone network. SCConv consists of two units: Spatial Reconstruction Unit (SRU) and Channel Reconstruction Unit (CRU). The SRU uses a split reconstruction method to suppress spatial redundancy, while the CRU uses a segment-transform-fusion strategy to reduce channel redundancy. Ultimately, performance is improved by significantly reducing complexity and computational cost by reducing redundant features.

[0033] 3) Set the loss function of the improved YOLOv10 model to inner-AIoU; inner-AIoU introduces a scale factor ratio, which can control the scale of the auxiliary bounding box, thereby overcoming the limitations of existing methods in generalization ability.

[0034] 4) Compared to current road defect detection models, the model trained in this invention achieves significant improvements in speed, greatly enhanced defect detection accuracy, and a greater diversity of detected defect categories while maintaining its lightweight advantage, fully meeting the requirements for real-time detection. Furthermore, the model possesses excellent portability and scalability, making its integration with hardware devices more convenient. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A flowchart illustrating the workflow of a lightweight asphalt road defect detection method based on partial multi-scale attention.

[0037] Figure 2 This is a diagram of the YOLOv10 model structure.

[0038] Figure 3 A diagram showing the structure of a lightweight YOLOv10 model with partial multi-scale attention;

[0039] Figure 4 The structure diagram of the PMA module for improving the YOLOv10 model;

[0040] Figure 5 This is a structural diagram of a portion of the multiscale attention mechanism (iPMA) in the PMA module;

[0041] Figure 6 Here is a diagram of the SCConv convolutional module structure;

[0042] Figure 7 This is a diagram illustrating the road defect detection effect in an example.

[0043] Figure 8 This is a schematic diagram of the implementation architecture of a lightweight asphalt road defect detection system. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described in detail and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Generally, the components of the embodiments of the present invention described and shown in the accompanying drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present invention.

[0045] like Figure 1 As shown in the figure, the lightweight asphalt road defect detection method with partial multi-scale attention provided by the present invention includes the following steps:

[0046] Step 1: Collect road defect datasets using a multi-view imaging system and combine them with publicly available datasets to build a training set; that is, the training set includes: photographed and publicly available road defect datasets.

[0047] In this embodiment, the RDD2022 public road defect dataset and images of road surface defects captured by a camera are used to construct the road surface defect dataset to be detected. The server device is an i7-13700K, with Ubuntu 23.04 operating system, a GeForce RTX 4080 graphics card, and 32GB of memory.

[0048] Step 2: Perform defect annotation and preprocessing operations on the acquired dataset, including:

[0049] (1) Filter the images in the dataset;

[0050] (2) Label the defect categories in the image;

[0051] (3) To expand the dataset size, data augmentation strategies are employed to process the images. These strategies involve the following specific methods: Brightness adjustment: altering the overall brightness of an image by increasing or decreasing its brightness value. Contrast adjustment: adjusting the image's contrast can change the degree of difference between bright and dark parts, making the image more vivid or softer. Combined adjustment: combining color, brightness, and contrast adjustments to generate more diverse training samples or meet specific visual effect requirements. Horizontal flip: flipping the image along the vertical central axis to generate a horizontally flipped image. This helps the model learn the features of objects in the mirror direction. Vertical flip: flipping the image along the horizontal central axis to generate a vertically flipped image. This also helps the model learn the features of objects in the vertical direction. Random scaling: randomly scaling the image during training to change the size of the input image. Proportional scaling: maintaining the aspect ratio of the image during scaling to avoid image distortion. Reduction and enlargement: reducing the image to a smaller size to simulate objects being far away, or enlarging the image to a larger size to simulate objects being close, as needed.

[0052] The dataset after defect annotation and preprocessing can be divided into training set and validation set according to a specified ratio (7:3).

[0053] Step 3: Construct a lightweight asphalt road defect detection model with partial multi-scale attention:

[0054] The network structure of the YOLOv10 model is as follows: Figure 2 As shown, YOLOv10 is a next-generation real-time object detection model. This model integrates an enhanced CSPNet backbone for efficient feature extraction and achieves multi-scale feature fusion in the neck layer through a PAN layer. Its head employs a dual-label allocation strategy and a lightweight classification head, combined with spatial-channel decoupling downsampling technology, to improve detection accuracy while maintaining high efficiency. Figure 2In this framework, Conv is the standard convolutional layer, including convolution, batch normalization, and activation functions; C2f is a cross-stage partial convolutional layer, which reduces computation and speeds up inference while maintaining or even improving performance through more skip connections and segmentation; SCDown is spatial-channel decoupled downsampling, which replaces traditional convolutional downsampling by decoupling the operations of spatial size reduction and channel number increase. It first adjusts the number of channels using 1x1 convolution and then performs downsampling using depthwise convolution; SPPF is spatial pyramid pooling-fast version, which fuses input features through multiple max pooling layers of different sizes; Upsample is an upsampling layer, which typically uses nearest neighbor or bilinear interpolation to enlarge the size (height and width) of the feature map; Concat is the concatenation operation; PSA is the aggregation attention module; CSDown is the cross-stage spatial-channel decoupled downsampling module; the C2fC1B module is a cross-stage partial fusion lightweight convolutional module, i.e., a lightweight standard C2f module; and v10Detect is the YOLOv10 detection head.

[0055] In this embodiment, the improved YOLOv10 model is constructed as follows: Figure 3 As shown, the construction method is as follows:

[0056] 1) Add PMA to the last layer of the YOLOv10 backbone network, that is, add the PMA module after SPPF.

[0057] The structure of the PMA module used in this embodiment is as follows: Figure 4 As shown, the main branch consists of a standard convolutional layer (Conv), a partial multi-scale attention mechanism (iPMA) module, two depthwise separable convolutional layers (DwConv), a concatenation layer (Concat), and a standard convolutional layer (Conv). The input feature map of the first depthwise separable convolutional layer is the element-wise sum of the output of the first Conv and the output feature map of the iPMA. The input feature map of the concatenation layer includes the element-wise sum of the input feature map of the first depthwise separable convolutional layer and the output feature map of the second depthwise separable convolutional layer, as well as the output feature map of the first standard convolutional layer.

[0058] In this embodiment of the invention, the structure of the iPMA module of the PMA module is as follows: Figure 5 As shown:

[0059] Step 1: By dividing the input feature map channels into two parts (first 1 / 3 and last 2 / 3), only the four-dimensional tensor [G,C,H,W] of the feature map X of the higher-level channels (last 2 / 3) is extracted; where G represents the batch size of the images, C represents the number of channels, H represents the height of the image, and W represents the width of the image. Attention weight descriptors for the grouped feature maps are extracted using four parallel paths. Two of these parallel paths are 1x1 convolutional branches, and the last two are 3×3 and 5×5 convolutional branches. To capture the dependencies between all channels and reduce computational cost, cross-channel information interaction in the channel directions is modeled. Specifically, in the 1x1 branch, two one-dimensional global average pooling operations (X Avg Pool and Y Avg Pool) encode the channels along the two spatial directions (X and Y directions), respectively; the encoded features obtained from the two one-dimensional global average pooling operations are concatenated in the image height direction and made to share the same 1x1 convolution. The output of this convolution is decomposed into two vectors, and two non-linear sigmoid functions are used to fit a two-dimensional binary distribution on the linear convolution. To achieve different cross-channel interactive features between the two parallel paths in the 1x1 branch, the attention maps of the two channels within each group are aggregated together by simple multiplication. In addition, multi-scale 3×3 and 5×5 convolutional branch kernel stacking can capture features with different receptive fields to capture multi-scale feature representations.

[0060] The global spatial information in the 1x1 branch output is encoded using a two-dimensional global average pooling method. This is followed by the aggregation of the two spatial attention weights generated from the output feature maps within each pair C, and then a sigmoid function. This captures pixel-level pairwise relationships and highlights the global context of all pixels.

[0061] Step 2: Preserve the original features of the lower-level channels (first 1 / 3) through lightweighting operations;

[0062] The original features of the low-level channels are preserved through depthwise separable convolution. Then, the high-level features processed in the first step are concatenated with the low-level features, and feature fusion is achieved by adjusting the channel dimensions through 1×1 convolution, outputting a unified feature map. This design reduces the computational cost by about 30% through selective computation, while taking into account both global modeling and preservation of local details.

[0063] Figure 5 In this context, Re-weight is the reweighting of features, which involves multiplying the feature maps element-wise based on the obtained attention weight values; GroupNorm is within-group normalization; Matmul is matrix multiplication; and Avg Pool is global average pooling.

[0064] like Figure 5As shown, in this embodiment, the four parallel branches of the iPMA module are as follows: the first parallel branch includes, in sequence: one-dimensional global average pooling in the X direction (horizontal), a concatenation layer and a 1×1 convolutional layer shared with the second branch, and a Sigmoid function; the second parallel branch includes, in sequence: one-dimensional global average pooling in the Y direction (vertical), a concatenation layer and a 1×1 convolutional layer shared with the first branch, and a Sigmoid function; the third branch is a 3×3 convolutional layer, and the fourth branch is a 5×5 convolutional layer; the four parallel branches perform interleaved learning based on cross-space interleaved branches, wherein the cross-space interleaved branches include two branches, the first interleaved branch includes, in sequence: an intra-group normalization layer, two-dimensional global average pooling, a Softmax function, and a matrix multiplication layer; the second interleaved branch includes, in sequence: two-dimensional global average pooling, a Softmax function, and a matrix multiplication layer; its In this process, the input feature map of the first interleaved branch is the output feature map after weighted fusion of the output feature maps of the first and second parallel branches and the original feature map of the high-level channel through a feature reweighting layer; the input feature map of the second interleaved branch is the element-wise sum of the output feature maps of the third and fourth parallel branches; and the input feature map of the intra-group normalization layer is fed into the matrix multiplication layer of the second interleaved branch; the input feature maps of the two interleaved branches are fed into the matrix multiplication layer of the first interleaved branch; the output feature maps of the two interleaved branches are element-wise summed and then fused with the original feature map of the high-level channel through a feature reweighting layer to obtain the processed high-level features; the low-level channel is passed through a 3×3 depthwise separable convolutional layer and then element-wise summed with the processed high-level features, and the sum is passed through a 1×1 convolutional layer to obtain the output feature map of the iPMA module.

[0065] 2) Add SCConv to the third layer of the YOLOv10 backbone network. The overall structure of SCConv is as follows: Figure 6 As shown, it consists of two parts: an SRU (Spatial Reconstruction Unit) and a CRU (Channel Reconstruction Unit). The SRU reduces spatial redundancy through a separation-reconstruction method, while the CRU reduces channel redundancy through a segmentation-transformation-fusion method. These two units work together to reduce redundant information in the features of the CNN.

[0066] The SRU structure comprises two operations: separation and reconstruction. The separation operation aims to separate information-rich feature maps from feature maps with less information corresponding to spatial content. First, the input features X are group-normalized (GN) to obtain the input features Xi. out .

[0067]

[0068] Where u and σ are the mean and standard deviation of the input feature X, respectively, γ and β are trainable affine transformations, ε is a minimal constant used to ensure stability, and a larger γ represents greater variation between pixels, i.e. richer spatial information.

[0069] The channel weight W is obtained by normalizing the randomly initialized parameter γ. γ :

[0070]

[0071] Where C represents the number of channels.

[0072] W γ With X out The multiplied values ​​are mapped to the range (0,1), and a threshold is used for gating. Values ​​greater than the threshold are marked as useful features W1, and values ​​less than the threshold are marked as useless features W2. Weights W1 and W2 are calculated using a formula and denoted as W.

[0073] W = Gate(sigmoid(W) γ (X out )))

[0074] Finally, a cross-reconstruction method is used to enhance useful features in the spatial dimension to suppress redundant features, thereby obtaining spatially refined features X. W That is, for any given intermediate mapping X, weights are applied based on weights W1 and W2 respectively, resulting in two different informational features after weighting: information-rich features and information-rich features. and features with less information Then use cross-reconstruction operation to and Combine to get X w1 and X w2 Connecting them together gives X w The cross-reconstruction can be represented as:

[0075]

[0076] X w1 ∪X w2 =X w

[0077] in, This indicates element-wise multiplication. ∪ represents element-wise addition, and ∪ represents the Concat operation.

[0078] Finally, after passing through the SRU structure, the spatial feature map X is reconstructed. wThere is still redundancy in the channel dimension, so the CRU structure can further reduce the features in the channel dimension. This process mainly includes three steps: segmentation, transformation and fusion, as follows.

[0079] In the segmentation phase, X is first... w The feature map is divided into two parts: one part has αC channels and the other part has (1-α)C channels, where α is set to 0.5. Then, a 1×1 convolution is used to compress the channels of the feature map to improve computational efficiency. A compression ratio r is introduced to control the number of feature channels in the CRU to balance computational cost. After segmentation and compression operations, the spatially refined feature X is... w Divided into upper part X up and the lower part X low The data is input into the upconversion stage as a "rich feature extractor." Efficient, low-computational-cost grouped convolution (GWC) and pointwise convolution (PWC) are used to extract features. With the group size in GWC set to 16, the upconversion stage can be represented as follows:

[0080]

[0081]

[0082] Among them, M G and M P1 Learnable weights for GWC and PWC, respectively; X up Y1 and Y2 are the feature maps of the upper input and output, respectively. low It is output to the bottom conversion stage.

[0083] In the fusion stage, a simplified SKNet method is used to adaptively merge the output features Y1 and Y2 from the up-conversion and down-conversion stages. Global spatial information is collected through global average pooling, and the global channel descriptor S... m The calculation is as follows:

[0084]

[0085] Subsequently, the global channel descriptors S1 and S2 of the upper and lower parts are stacked, and channel attention operations are used to generate importance vectors.

[0086]

[0087] Finally, the channel reconstruction feature can be represented as β1Y1+β2Y2.

[0088] In the SCConv module, the number of parameters can be significantly reduced during the channel conversion stage compared to traditional convolution operations.

[0089] 3) Set the loss function of the improved YOLOv10 model to inner-AIoU; inner-AIoU introduces a scale factor ratio, which can control the scale of the auxiliary bounding box, thereby overcoming the limitations of existing methods in generalization ability.

[0090] The inner-AIoU loss function, the steps are as follows:

[0091] First, calculate the top, bottom, left, and right boundaries of the auxiliary bounding box of the annotation box:

[0092]

[0093] The real bounding box and its anchor point are labeled b. gt and b, This represents the coordinates of the anchor point within the actual bounding box, (x c ,y c The coordinates of the center point of the inner bounding box are represented by ; the width and height of the true bounding box are marked as w. gt and h gt The variable “ratio” represents the scaling factor.

[0094] Then calculate the boundaries of the auxiliary bounding boxes of the predicted bounding boxes:

[0095]

[0096] Among them, (x c ,y c ) represents the center coordinates of the prediction box, w is the width of the prediction box, and h is the height of the prediction box.

[0097] Calculate the intersection area (inter) and union area (union) of the auxiliary boxes of the predicted bounding box and the ground truth bounding box, respectively:

[0098]

[0099] union = (w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter

[0100] Finally, the IoU is calculated using the intersection area (inter) and the union area (union). inner :

[0101]

[0102] Based on IoU inner The part of calculating the bounding box regression loss function of the YOLOv10n network:

[0103]

[0104] Here, v is a positive tradeoff parameter, α measures the consistency of aspect ratio, and ρ represents the Euclidean distance between the center points of the predicted box and the ground truth box. Inner-AIoU compensates for the shortcomings of existing IoU. inner To address the issues of weak generalization and slow convergence speed of the loss function across different detection tasks, a scaling factor ratio is used to control the scaling of the auxiliary bounding box used to calculate the loss, for different crack sizes. Typically, the scaling factor ratio ranges from [0.5, 1.5]. The scaling factor is dynamically adjusted with each training epoch, using the following formula:

[0105]

[0106] Where T represents the total number of training rounds, and t represents the current round. The iterative strategy increases the rate of increase from slow to fast in the first half and from fast to slow in the second half, conforming to the learning pattern of crack detection models from global to local features. This allows the model to focus on the core crack region (such as the centerline) in the early stages of training, filtering out background noise such as asphalt texture; in the middle stages of training, the model begins to focus on crack edges and branches, enhancing its ability to compensate for the continuity of fracture cracks; and in the later stages of training, the detection area is expanded to the complete crack boundary, alleviating the problem of missed detections.

[0107] Step 4: Train and evaluate the performance of the preprocessed road defect dataset using the improved YOLOv10 model. When the performance indicators meet the requirements, it is then constructed as an asphalt road defect detection model.

[0108] In this embodiment, the loss function is set to inner-AIoU for model training. When map@0.5 reaches 92%, the model is deployed to introduce a scale factor ratio to optimize bounding box regression for road defect detection. The model training process includes:

[0109] 1) Set the training hyperparameters: batch size to 32 and number of iterations to 800.

[0110] 2) Start training the model using the training set data. In each training iteration, calculate the loss and update the model weights.

[0111] 3) After each training cycle, use the validation set to evaluate the model's performance and monitor metrics such as loss and accuracy on the validation set so as to stop training in a timely manner to prevent overfitting.

[0112] The performance evaluation of the model includes: calculating the model's accuracy and F1 score on the test set, and plotting a confusion matrix to visualize the evaluation results in order to determine its performance in real-world situations.

[0113] Figure 7The image shows the road defect detection effect of an embodiment of the present invention. The road defect detection model trained by the present invention achieves a significant improvement in speed, a substantial increase in defect detection accuracy, and a diversification of detected defect categories while maintaining the advantage of lightweight design, fully meeting the needs of real-time detection.

[0114] Another aspect of this invention provides a lightweight asphalt road defect detection system with partial multi-scale attention, comprising: a camera module, a defect detection module, and a GPS positioning module. The camera module acquires an image of the road to be detected and sends it to the defect detection module. The defect detection module uses the asphalt road defect detection model (trained) provided in this invention to detect defects in the road image and sends the detection results to the GPS positioning module to obtain and record the defect locations, thereby outputting detection results with defect locations. The implementation architecture of this lightweight asphalt road defect detection system is as follows: Figure 8 As shown, it can be divided into a user layer (road surface crack detection), a core business logic layer (road defect detection model), and a system support layer (software and hardware platform). The core business logic layer processes the following: after preprocessing the input road image to be detected, it is fed into the road defect detection model to realize defect detection and localization, so as to generate a complete defect information report (defect monitoring results with defect location); finally, the generated defect information report is presented to the user.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0116] The above descriptions are merely some embodiments of the present invention. Those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A lightweight asphalt road defect detection method based on partial multi-scale attention, characterized in that, Includes the following steps: Step S1: Construct a training set for model training; Step S2: Construct an asphalt road defect detection model based on the YOLOv10 model: A self-calibrating convolutional module is added to the third layer of the backbone network of the YOLOv10 model to reduce feature redundancy through spatial reconstruction units and channel reconstruction units. At the end of the backbone network, a partial multi-scale attention module (PMA) is embedded. Its main branches sequentially include a standard convolutional layer 1, a partial multi-scale attention mechanism module (iPMA), a depthwise separable convolutional layer 1, a depthwise separable convolutional layer 2, a concatenation layer, and a standard convolutional layer 2. The input feature map of the depthwise separable convolutional layer 1 is the element-wise sum of the output feature map of the standard convolutional layer 1 and the output feature map of the iPMA module. The input feature map of the concatenation layer includes the element-wise sum of the input feature map of the depthwise separable convolutional layer 1 and the output feature map of the depthwise separable convolutional layer 2, as well as the output feature map of the standard convolutional layer 1. The iPMA module divides its input feature map into two parts by channel. Only the first part undergoes multi-branch convolution to capture multi-scale features, which are then fused using an attention mechanism to obtain the processed first part features. The second part undergoes lightweighting through a depthwise separable convolution, is then concatenated with the processed first part features, and the channel dimensions are adjusted through a convolutional layer to obtain the output feature map of the iPMA module. Step S3: Based on the constructed training set, the asphalt road defect detection model is trained using the inner-AIoU loss function. When the preset training convergence condition is met, the trained asphalt road defect detection model for the road image to be detected is obtained.

2. The method as described in claim 1, characterized in that, Step S1 includes: collecting a road defect dataset through a multi-view imaging system, combining it with a public dataset to construct an original dataset, and then labeling the defects and preprocessing the images to construct a training set.

3. The method as described in claim 2, characterized in that, Image preprocessing includes adaptive histogram equalization and image enhancement.

4. The method as described in claim 3, characterized in that, Image enhancement processing includes: randomly rotating the image, adjusting the scaling ratio, performing horizontal or vertical flipping, and adjusting the image's color and brightness.

5. The method as described in claim 1, characterized in that, The iPMA module divides its input feature map by channels: the first 1 / 3 is used as the high-level channel, and the last 2 / 3 is used as the low-level channel.

6. The method as described in claim 4, characterized in that, The iPMA module uses four parallel paths to extract attention weight descriptors from grouped feature maps. Two of these parallel paths are 1×1 convolutional branches, and the latter two are 3×3 and 5×5 convolutional branches.

7. The method as described in claim 1, characterized in that, The inner-AIoU loss function is: Among them, the internal intersection ratio loss `inter` and `union` are the intersection and union areas of the predicted bounding box and the auxiliary bounding box (ground truth box) output by the model, respectively; (The last part, "weighting parameters," appears to be an unrelated fragment and is left untranslated.) Aspect Ratio Consistency Parameters ρ represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, and w and h are the width and height of the predicted bounding box, respectively. gt and h gt b is the width and height of the actual bounding box gt b and b represent the ground truth bounding box and its anchor point, respectively. IOU represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box. t is the current training round.

8. The method as described in claim 7, characterized in that, The scaling factor ratio controls the scaling of the auxiliary bounding box used to calculate the inner-AIoU loss. The scaling factor ratio is dynamically adjusted with each training epoch, and the adjustment formula is as follows: Where T is the total number of training rounds, and t is the current training round.

9. A lightweight asphalt road defect detection system with partial multi-scale attention, characterized in that, include: Camera module, defect detection module, and GPS positioning module; The camera module uses multispectral imaging technology to acquire images of the road to be inspected and transmits the acquired images to the defect detection module. The defect detection module detects road defects based on the trained asphalt road defect detection model as described in any one of claims 1 to 8. The defect detection module first performs image preprocessing on the received road image to be detected to match the input of the asphalt road defect detection model, then inputs the image into the model to obtain the defect detection result, and sends the detection result to the GPS positioning module. The GPS positioning module is used to obtain the corresponding defect location, and then combine the detection results to generate a defect information report and send it to the target object.

10. The system as described in claim 9, characterized in that, The GPS positioning module obtains the location of the defect through a geographic information system platform.

Images