Road anomaly detection method

Abstract

The invention discloses a road anomaly detection method, which is characterized by comprising the following steps of: firstly, constructing a road anomaly detection model, and training the road anomaly detection model to obtain a trained road anomaly detection model; inputting a to-be-detected road scene image into the trained road anomaly detection model to obtain a detection result; wherein the road anomaly detection model is composed of a backbone network, a neck network and a detection head which are connected in sequence; the backbone network comprises a CBS layer, a C3 layer, a position modulation attention module and an active visual follow-up multi-scale feature extraction module; the method has the advantages that a long-distance dependency relationship is effectively captured through the position modulation attention module to perform accurate positioning and enhance the scene adaptability, and the weak feature target detection capability under the complex road background is improved; the active vision follow-up multi-scale feature extraction module actively adjusts the sight distance to perform multi-scale feature extraction in a multi-receptive-field mode under the condition of fixing the position of a target to be detected, and the limitation of a single-scale method is relieved.

Description

Technical Field

[0001] This invention relates to a detection method, and more particularly to a road anomaly detection method. Background Technology

[0002] With the continuous growth of global transportation demand and the surge in motor vehicle ownership, the capacity of existing roads and traffic management capabilities face enormous challenges. Motor vehicle lanes, in particular, are frequently subjected to heavy loads and high traffic volumes, leading to various safety hazards such as intrusion by foreign objects, non-motorized vehicles, pedestrians or animals (e.g., pigs, sheep, cattle), fires, road surface abnormalities, and traffic accidents. Therefore, anomaly detection is crucial for ensuring the efficient operation of the transportation system during road maintenance.

[0003] Existing road anomaly detection methods are mainly divided into two categories: traditional manual monitoring and interpretation, and automated intelligent detection utilizing advanced perception and analysis technologies. Traditional road anomaly detection relies primarily on manual review of video surveillance, which suffers from high labor intensity, low efficiency, and a high risk of missed detections, failing to meet the demands of modern intelligent transportation development. Therefore, computer vision-based traffic anomaly detection technologies have emerged, with detectors such as DETR (DEtection Transformer) and the YOLO (You Only Look Once) series attracting significant attention due to their unique advantages. DETR, as the first framework to introduce the Transformer architecture into object detection, achieves an end-to-end detection paradigm, eliminating the need for anchor box design and non-maximum suppression, thus improving overall scalability. However, this increases model complexity, and DETR and its variants generally face problems such as slow training convergence, high computational resource consumption, strong dependence on large-scale labeled data, and insufficient performance in small object detection, placing high demands on hardware environments. The YOLO series algorithms, with their significant advantages in speed, real-time performance, and adaptability to multiple scenarios, can efficiently support the practical needs of road anomaly detection and traffic management. However, the YOLO series of algorithms still face certain technical challenges and performance bottlenecks in handling weak feature target detection in complex backgrounds, dealing with dynamic size changes caused by target movement and line-of-sight changes, and achieving high-precision positioning. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a road anomaly detection method that can not only detect road anomalies efficiently and accurately, but also improve detection accuracy, robustness and practicality.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: a road anomaly detection method, comprising: constructing a road anomaly detection model composed of a backbone network, a neck network and a detection head connected in sequence; training the road anomaly detection model to obtain a trained road anomaly detection model; inputting the road scene image to be detected into the trained road anomaly detection model to obtain a road anomaly detection result;

[0006] The specific operational process for obtaining road anomaly detection results is as follows:

[0007] Step ①: The backbone network extracts features from the input road scene image to be detected, and obtains a multi-scale fused feature map;

[0008] Step 2: The neck network performs multi-level feature fusion and enhancement on the multi-scale fusion feature map output by the backbone network to obtain a detection feature map with multi-scale perception capability.

[0009] Step 3: The detection head performs target classification and location regression prediction on the detection feature map output by the neck network to obtain road anomaly detection results containing target category, confidence level and bounding box coordinates;

[0010] The backbone network includes a first CBS layer, a second CBS layer, a first C3 layer, a third CBS layer, a second C3 layer, a fourth CBS layer, a third C3 layer, a fifth CBS layer, a position modulation attention module, and an active visual follow-up multi-scale feature extraction module, which are connected in sequence.

[0011] The specific operation process for step ① is as follows:

[0012] Step ①-1: The first CBS layer performs convolutional feature extraction on the input road scene image to be detected to obtain the first basic feature map;

[0013] Step ①-2: The second CBS layer performs convolutional feature extraction on the first basic feature map output by the first CBS layer to obtain the second basic feature map;

[0014] Step ①-3: The first C3 layer performs feature fusion on the second basic feature map output by the second CBS layer to obtain the first hierarchical feature map;

[0015] Steps ①-4: The third CBS layer performs convolutional feature extraction on the first hierarchical feature map output by the first C3 layer to obtain the third basic feature map;

[0016] Steps ①-5: The second C3 layer performs feature fusion on the third basic feature map output by the third CBS layer to obtain the second hierarchical feature map;

[0017] Steps ①-6: The fourth CBS layer performs convolutional feature extraction on the second-level feature map output by the second C3 layer to obtain the fourth basic feature map;

[0018] Steps ①-7: The third C3 layer performs feature fusion on the fourth basic feature map output by the fourth CBS layer to obtain the third-level feature map;

[0019] Steps ①-8: The fifth CBS layer performs convolutional feature extraction on the third-level feature map output by the third C3 layer to obtain the fifth basic feature map;

[0020] Steps ①-9: The position modulation attention module performs attention calculation based on spatial distance modulation on the fifth basic feature map output by the fifth CBS layer to obtain a global association feature map with enhanced spatial perception.

[0021] Step ①-10: The active vision follow-up multi-scale feature extraction module performs adaptive multi-scale feature extraction and fusion operations on the global correlation feature map output by the position modulation attention module through a parallel processing method of multiple branches and multiple receptive fields to obtain a multi-scale fused feature map.

[0022] Compared with existing technologies, the advantages of this invention lie in its effective enhancement of the spatial perception capability of feature maps through spatial distance modulation-based attention computation operations in the position modulation attention module, significantly improving the detection capability for small targets and targets with weak features in complex road backgrounds. Simultaneously, the active vision-guided multi-scale feature extraction module employs a multi-branch and multi-receptive field parallel processing approach, enabling dynamic adjustment of the receptive field for adaptive multi-scale feature extraction and fusion, effectively alleviating the limitations of single-scale methods and enhancing robustness to dynamic size fluctuations caused by target movement and changes in viewing distance. Furthermore, the combination of multi-level feature fusion and enhancement from the neck network improves the multi-scale perception capability of the road anomaly detection model, enabling the detection head to achieve higher accuracy results in target classification and position regression prediction. Experiments on a self-built database show that the proposed method achieves an mAP@0.5 of 98.2%, compared to...

[0023] The accuracy of the proposed method is 96.2%, which is 2.8% higher than that of YOLOv5, while the recall rate is 96.5%, which is 2.9% higher than that of YOLOv5. The F1 score of the proposed method is also 96.3%, which is 2.9% higher than that of YOLOv5. This effectively verifies the advantages of the proposed method in improving the accuracy and adaptability of road anomaly detection, and meets the needs of efficient, stable and accurate detection in practical applications.

[0024] Furthermore, the specific operation process of steps ①-9 is as follows:

[0025] Step ①-9-1: Denote the input fifth basic feature map as X∈R C1×H×W The feature map X is obtained by performing convolutional dimensionality reduction on X through the first Conv layer. in ∈R C′×H×W Where C1 represents the number of channels in the input fifth basic feature map, H represents the height dimension of the input fifth basic feature map, W represents the width dimension of the input fifth basic feature map, and C′ represents the number of channels in the position modulation feature.

[0026] Step ①-9-2: Perform relative position encoding calculation on X using the RelPos2d layer, and generate a spatial attenuation matrix along the width direction based on the Manhattan distance. and the spatial attenuation matrix along the height direction Where γ represents the learnable decay base, 0 < γ < 1, used to control the decay rate, (x n y n (x) represents the two-dimensional coordinates of the nth marker on the feature map. m y m () represents the two-dimensional coordinates of the m-th marker on the feature map, where n, m ∈ {(c, j) | 1 ≤ c ≤ H, 1 ≤ j ≤ W};

[0027] Step ①-9-3, extract the feature map X output from the first Conv layer. in and the spatial decay matrix along the width direction of the RelPos2d layer output. and the spatial attenuation matrix along the height direction The input is fed into the modulation attention layer for distance decay attention calculation to obtain attention-weighted features. The specific operation is as follows:

[0028] Step ①-9-3-1, obtain the position modulation feature X pos ∈R C′×H×W X pos =X in +DWConv pos (X in ), where DWConv pos (·) indicates a depthwise separable convolution operation;

[0029] Step ①-9-3-2: Obtain the query vector Q, key vector K, and value vector V, where Q = X′W Q K = X′W K V = X′W V Where X′ represents X after layer normalization. pos W Q WK W V These represent the learnable linear projection matrices, respectively.

[0030] Step ①-9-3-3, obtain intermediate feature X out ∈R C′×H×W X out =X pos +DropPath(W O (PMAttn(LN1(X pos ))+LEPE(V))), where DropPath(·) represents the regularization operation, W O LN1(·) represents the learnable output projection matrix, LN1(·) represents the layer normalization operation, LEPE(-) represents the local augmentation sensing coding operation, and LEPE(V) = DWConv 5×5 (V), DWConv 5×5 (·) denotes a depthwise separable convolution operation with a kernel size of 5×5, V represents the value vector, and PMAtn(·) denotes a position-modulated attention operation. Softmax represents the normalized exponential function, Q represents the query vector, and K... T d represents the transpose of the key vector. k It represents the dimension of the key vector, and ⊙ indicates element-wise multiplication;

[0031] Step ①-9-3-4: Obtain attention-weighted feature X final ∈R C′×H×W X final =X out +DropPath(FFN(LN2(X out ))), where FFN(·) represents the feedforward neural network operation and LN2(·) represents the layer normalization operation;

[0032] Step ①-9-4: Perform convolutional dimensionality reduction on X through the second Conv layer to obtain the feature map X″∈R. C′×H×W ;

[0033] Step ①-9-5, modulate the attention-weighted feature X output by the attention layer. final The feature map X′” output from the second Conv layer is concatenated to obtain the fused feature;

[0034] Step ①-9-6: Perform feature integration operation on the fused features through the third Conv layer to obtain the global associated feature map X″′∈R. C2×H×WWhere C2 represents the number of channels in the globally associated feature map. By generating a spatial decay matrix based on Manhattan distance through the RelPos2d layer and combining it with distance decay attention calculation in the modulation attention layer, the spatial dependencies of the feature map are accurately modulated, thereby effectively improving the model's spatial perception ability for weak features in small targets and complex backgrounds, reducing computational redundancy, and thus improving detection accuracy.

[0035] Furthermore, the specific operation process of steps ①-10 is as follows:

[0036] Step ①-10-1: The first branch uses 1×1 convolution to extract small features from the input global correlation feature map, and then performs spatial downsampling on the small features through max pooling to obtain local fine-grained features;

[0037] Step ①-10-2: The second branch uses 3×3 convolution to extract the medium features of the input global correlation feature map. The medium features after max pooling and the medium features after average pooling are added element-wise to obtain the medium-scale structural features.

[0038] Step ①-10-3: The third branch uses a 5×5 convolution to extract the large features of the input global correlation feature map. After performing three consecutive average pooling operations on the large features, the global structural features are obtained.

[0039] Step ①-10-4: Concatenate the small features, local fine-grained features, mesoscale structural features, and global structural features along the channel dimension to obtain a multi-scale fused feature. Use the learnable weights of a 1×1 convolution to perform a weighted summation of the multi-scale fused feature to obtain the multi-scale fused feature map Y∈R. 2C′×H×W By extracting features at different scales through parallel processing, complementary fusion of multi-level features is achieved; this not only adaptively enhances the model's robustness to dynamic changes in target size, but also improves the accuracy and generalization ability of multi-scale target detection.

[0040] Furthermore, the specific operations for obtaining the trained road anomaly detection model are as follows:

[0041] Step 1: Collect environmental images from roads, streets and grasslands, and define environmental images with a pixel size lower than 60×25 or 45×30 as small road target images. Randomly select at least 16244 small road target images as a road anomaly detection dataset. Manually annotate all images in the road anomaly detection dataset. Randomly select 80% of the images from the annotated road anomaly detection dataset as the training set.

[0042] Step 2, set the training parameters including: total training cycles set to 200, batch size set to 4, and initial learning rate set to 0.01;

[0043] Step 3, define the total loss function as Loss = λ1·LossBox + λ2·LossObj + λ3·LossCls, where LossBox represents the bounding box regression loss, LossObj represents the target confidence loss, LossCls represents the classification loss, and λ1, λ2, and λ3 represent the weight coefficients that balance the importance of different tasks.

[0044] Step 4: Based on the total loss function, input the training set into the road anomaly detection model for training. Update the road anomaly detection model through the SDG optimizer until the total training cycle is reached, then stop iterative updates to obtain the trained road anomaly detection model. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the road anomaly detection model in this invention;

[0046] Figure 2 This is a schematic diagram of the position modulation attention module in this invention;

[0047] Figure 3 This is a schematic diagram of the active vision-guided multi-scale feature extraction module in this invention;

[0048] Figure 4 This diagram illustrates a comparison of detection results using the present invention and existing technologies on a road anomaly detection dataset. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0050] A road anomaly detection method, comprising:

[0051] Construct a road anomaly detection model, such as Figure 1 As shown, the road anomaly detection model consists of a backbone network, a neck network, and a head network connected in sequence. The backbone network includes a first CBS layer, a second CBS layer, a first C3 layer, a third CBS layer, a second C3 layer, a fourth CBS layer, a third C3 layer, a fifth CBS layer, a positionally modulated attention module (PMA), and an active tracking multi-scale feature extraction module (AT-MSFE).

[0052] The road anomaly detection model is trained to obtain the trained road anomaly detection model. The specific steps are as follows:

[0053] Step 1: Collect environmental images from various environments such as roads, streets, and grasslands. Define environmental images with a pixel size lower than 60×25 or 45×30 as small road target images. Randomly select at least 16,244 small road target images as the MCRAD road anomaly detection dataset. Manually annotate all images in the MCRAD road anomaly detection dataset. Randomly select 80% of the images in the annotated MCRAD road anomaly detection dataset as the training set and 20% as the validation set. Specifically, manually annotate the road anomaly targets in each image, including the target category and precise bounding box location information. The MCRAD road anomaly detection dataset has 9 categories: construction signs, road debris, pedestrians, fire, spills, road conditions, illegal vehicles, animals, and traffic accidents.

[0054] Step 2, set the training parameters including: total training cycles set to 200, batch size set to 4, and initial learning rate set to 0.01;

[0055] Step 3, define the total loss function as Loss = λ1·LossBox + λ2·LossObj + λ3·LossCls, where LossBox represents the bounding box regression loss, LossObj represents the target confidence loss, LossCls represents the classification loss, and λ1, λ2, and λ3 represent the weighting coefficients that balance the importance of different tasks. In this embodiment, λ1 = 0.05, λ2 = 1.0, and λ3 = 0.5.

[0056] Step 4: Based on the total loss function, input the training set into the road anomaly detection model for training. Update the road anomaly detection model through the SDG (Stochastic Gradient Descent) optimizer until the total training cycle is reached and the iterative update stops, resulting in the trained road anomaly detection model. The momentum of the SDG optimizer is set to 0.937 and the weight decay is set to 0.0005.

[0057] The road scene image to be detected is input into the trained road anomaly detection model to obtain the road anomaly detection result. The specific operation process is as follows:

[0058] Step ①: The backbone network extracts features from the input road scene image to be detected, and obtains a multi-scale fused feature map;

[0059] Step ①-1: The first CBS layer performs convolutional feature extraction on the input road scene image to be detected with a size of 3×640×640, and obtains the first basic feature map with a size of 64×320×320. The first CBS layer consists of a Conv (convolutional) layer, a BN layer and a SiLU activation function layer. The Conv layer in the first CBS layer has 64 output channels, a kernel size of 6×6, a stride of 2 and padding of 2.

[0060] Step ①-2: The second CBS layer performs convolutional feature extraction on the first basic feature map output by the first CBS layer to obtain a second basic feature map with a size of 128×160×160. The second CBS layer consists of a Conv (convolutional) layer, a BN layer, and a SiLU activation function layer. The Conv layer in the second CBS layer has 128 output channels, a kernel size of 3×3, a stride of 2, and padding of 1.

[0061] Steps ①-3: The first C3 layer performs feature learning on the second basic feature map output by the second CBS layer through cross-stage feature fusion and deep feature extraction operations, and obtains a first hierarchical feature map with a size of 128×160×160.

[0062] Steps ①-4: The third CBS layer performs convolutional feature extraction on the first hierarchical feature map output by the first C3 layer to obtain a third basic feature map with a size of 256×80×80. The third CBS layer consists of a Conv (convolutional) layer, a BN layer, and a SiLU activation function layer. The Conv layer in the third CBS layer has 256 output channels, a kernel size of 3×3, a stride of 2, and padding of 1.

[0063] Steps ①-5: The second C3 layer performs feature learning on the third basic feature map output by the third CBS layer through cross-stage feature fusion and deep feature extraction operations, and obtains a second-level feature map with a size of 256×80×80.

[0064] Steps ①-6: The fourth CBS layer performs convolutional feature extraction on the second-level feature map output by the second C3 layer to obtain a fourth basic feature map with a size of 512×40×40. The fourth CBS layer consists of a Conv (convolutional) layer, a BN layer, and a SiLU activation function layer. The Conv layer in the fourth CBS layer has 512 output channels, a kernel size of 3×3, a stride of 2, and padding of 1.

[0065] Steps ①-7: The third C3 layer performs feature learning on the fourth basic feature map output by the fourth CBS layer through cross-stage feature fusion and deep feature extraction operations, and obtains a third-level feature map with a size of 512×40×40.

[0066] Steps ①-8: The fifth CBS layer performs convolutional feature extraction on the third-level feature map output by the third C3 layer to obtain the fifth basic feature map with a size of 1024×20×20. The fifth CBS layer consists of a Conv (convolutional) layer, a BN layer, and a SiLU activation function layer. The Conv layer in the fifth CBS layer has 1024 output channels, a kernel size of 3×3, a stride of 2, and padding of 1.

[0067] Steps ①-9: The position modulation attention module performs attention calculation based on spatial distance modulation on the fifth basic feature map output by the fifth CBS layer to obtain a global association feature map with enhanced spatial awareness, which is used to capture and integrate long-distance dependencies and global contextual information between features.

[0068] like Figure 2 As shown, the specific operation process for obtaining a global association feature map with enhanced spatial awareness is as follows:

[0069] Step ①-9-1: Denote the input fifth basic feature map as X∈R C1×H×W The feature map X is obtained by performing convolutional dimensionality reduction on X through the first Conv layer. in ∈R C′×H×W Where C1 represents the number of channels in the input fifth basic feature map, H represents the height dimension of the input fifth basic feature map, and W represents the width dimension of the input fifth basic feature map.

[0070] Step ①-9-2: Perform relative position encoding calculation on X using the RelPos2d layer, and generate a spatial attenuation matrix along the width direction based on the Manhattan distance. and the spatial attenuation matrix along the height direction Where γ represents the learnable decay base, 0 < γ < 1, used to control the decay rate, (x n y n (x) represents the two-dimensional coordinates of the nth marker on the feature map. m ,y m () represents the two-dimensional coordinates of the m-th marker on the feature map, where n, m∈{(i,j)|1≤i≤H, 1≤j≤W};

[0071] Step ①-9-3, extract the feature map X output from the first Conv layer. in and the spatial decay matrix along the width direction of the RelPos2d layer output. and the spatial attenuation matrix along the height direction The input is fed into the modulation attention layer (PM) for distance decay attention calculation to obtain attention-weighted features. The specific operation is as follows:

[0072] Step ①-9-3-1, obtain the position modulation feature X pos ∈R C′×H×W X pos =X in +DWConv pos (X in ), where DWConv pos (·) indicates a depthwise separable convolution operation;

[0073] Step ①-9-3-2: Obtain the query vector Q, key vector K, and value vector V, where Q = X′W Q K = X′W K V = X′W V Where X′ represents X after layer normalization. pos W Q W K W V These represent the learnable linear projection matrices, respectively.

[0074] Step ①-9-3-3, obtain intermediate feature X out ∈R C′×H×W X out =X pos +DropPath(W O (PMAttn(LN1(X pos ))+LEPE(V))), where DropPath(·) represents the regularization operation, W O LN1(·) represents the learnable output projection matrix, LN1(·) represents the layer normalization operation, LEPE(·) represents the local augmentation sensing coding operation, and LEPE(V) = DWConv 5×5 (V), DWConv 5×5 (·) represents a depthwise separable convolution operation with a kernel size of 5×5, V represents the value vector, and PMAtn(·) represents a position-modulated attention operation, responsible for calculating self-attention weights based on spatial decay mechanism and relative position. Softmax represents the normalized exponential function, Q represents the query vector, and K... T d represents the transpose of the key vector. k It represents the dimension of the key vector, and ⊙ indicates element-wise multiplication;

[0075] Step ①-9-3-4: Obtain attention-weighted feature X final ∈R C′×H×W X final =Xout +DropPath(FFN(LN2(X out In this context, FFN(·) represents the feedforward neural network operation, and LN2(·) represents the layer normalization operation. The feedforward neural network consists of two linear transformations and activation functions, used for further nonlinear transformation and enhancement of features.

[0076] Step ①-9-4: Perform convolutional dimensionality reduction on X through the second Conv layer to obtain the feature map X″∈R. C′×H×W ;

[0077] Step ①-9-5, modulate the attention-weighted feature X output by the attention layer. final The feature map X″ output from the second Conv layer is concatenated to obtain the fused feature;

[0078] Step ①-9-6: Perform feature integration operation on the fused features through the third Conv layer to obtain the global associated feature map X″′∈R. C2×H×W Where C2 represents the number of channels in the global correlation feature map;

[0079] In this embodiment, in steps ①-9, the kernel size of the first Conv layer, the second Conv layer, and the third Conv layer is 1×1, and the stride is 1.

[0080] Step ①-10: The active vision follow-up multi-scale feature extraction module performs adaptive multi-scale feature extraction and fusion operations on the global correlation feature map output by the position modulation attention module through a parallel processing method of multiple branches and multiple receptive fields to obtain a multi-scale fused feature map.

[0081] like Figure 3 As shown, the specific operation process for obtaining the multi-scale fused feature map is as follows:

[0082] Step ①-10-1: The first branch uses 1×1 convolution to extract small features from the input global correlation feature map, and then performs spatial downsampling on the small features through max pooling to obtain local fine-grained features.

[0083] Step ①-10-2: The second branch uses 3×3 convolution to extract the medium features of the input global correlation feature map. The medium features after max pooling and the medium features after average pooling (AvgPool) are added element-wise to obtain the medium-scale structural features.

[0084] Step ①-10-3: The third branch uses a 5×5 convolution to extract the large features of the input global correlation feature map. After performing three consecutive average pooling operations on the large features, the global structural features are obtained.

[0085] Step ①-10-4: Concat the small features, local fine-grained features, mesoscale structural features, and global structural features along the channel dimension to obtain multi-scale fused features. Use the learnable weights of a 1×1 convolution to perform a weighted summation of the multi-scale fused features to obtain the multi-scale fused feature map Y∈R. 2C′×H×W .

[0086] Step ②: The neck network employs a bidirectional fusion structure composed of a Feature Pyramid Network (FPN) and a Path Aggregation Network (PANet). This structure transmits high-level semantic information to the lower levels via a top-down FPN path, and supplements the higher levels with localization information from the lower levels via a bottom-up PANet path. Through this bidirectional, multi-level, cross-scale fusion and enhancement of the multi-scale fusion feature maps output by the active vision-guided multi-scale feature extraction module, a set of detection feature maps with rich contextual information and multi-scale perception capabilities is finally generated for subsequent detection head processing.

[0087] Step 3: After each detection feature map output by the neck network, the detection head independently applies a 1x1 convolutional layer to complete the final prediction task. In parallel, the two core tasks of target classification and bounding box regression are performed simultaneously on feature maps at each scale, synchronously predicting the confidence score, bounding box regression parameters (position and size), and category of each potential target. During the inference phase, the road anomaly detection model first decodes the raw regression parameters output by the network, mapping them to actual bounding box coordinates in image space. Then, the non-maximum suppression (NMS) algorithm is applied to filter and merge all candidate detection boxes to eliminate redundant predictions, ultimately outputting road anomaly detection results containing the target category, confidence score, and precise bounding box coordinates.

[0088] To verify the effectiveness of the present invention, an experiment was conducted on a computing server equipped with an Intel Xeon Gold 5218 CPU (2.30GHz) and an NVIDIA A100 GPU (40GB VRAM), with the operating system being Ubuntu 18.04.5LTS.

[0089] This experiment uses standard object detection evaluation metrics to assess model performance, primarily including precision (P), recall (R), F1 score (F1), and mean average precision (mAP). Precision measures the accuracy of model predictions, recall measures the completeness of detections, and the F1 score provides a measure of the balance between the two. This report primarily uses mAP calculated at a commonly used IoU threshold of 0.5 as the core performance indicator. By comprehensively analyzing these metrics, the specific effectiveness of the model in road anomaly detection tasks can be evaluated in depth and objectively from different dimensions.

[0090] To demonstrate the effectiveness of the proposed method, a comparative experiment was conducted on the established MCRAD dataset, comparing it with current mainstream detection network models to evaluate the model's detection performance. The baseline models included in the comparison were YOLOv5, EfficientDet, YOLOv8, YOLOv10, RT-DETR, and YOLO11. A unified training strategy and evaluation criteria were used, with key performance indicators including mean AP, precision, recall, and F1 score. Detailed comparative experimental results are shown in Table 1. Analysis of the data shows that the proposed method exhibits optimal performance across all key evaluation metrics. Specifically, the proposed method achieved an mAP@0.5 of 98.2%, a 1.6% improvement over the best result among all baseline models. The proposed method achieved a precision of 96.2%, 2.8% higher than the highest precision of the baseline models; and a recall of 96.5%, 2.9% higher than the best recall among the baseline models. The proposed method achieves an optimal F1 score of 96.3%, which is 2.9% higher than the highest F1 score of the baseline model. Notably, compared with the representative Transformer-based model RT-DETR, the proposed method improves mAP@0.5 by 5.3%, demonstrating a significant advantage on the current dataset and task. These quantitative results clearly show that on the MCRAD dataset containing nine types of complex road anomalies, the proposed method has significant advantages over existing mainstream algorithms in terms of detection accuracy, target detection completeness, and the balance between the two, fully validating that the proposed method can more effectively handle anomaly detection tasks in complex road scenarios.

[0091] Table 1 Comparison of detection results for each model

[0092]

[0093] To comprehensively evaluate the performance of the proposed method in road anomaly detection, qualitative visualization analysis is used to compare the detection results of the proposed method with those of several representative methods with better performance on the MCRAD dataset, such as... Figure 4 As shown in the figure. This experiment selects representative road anomaly scenarios for in-depth analysis: small target foreign object detection, pedestrian detection in complex environments, traffic accident detection, and multi-target mixed scenarios. These scenarios cover typical challenges in road anomaly detection, including small target size, complex background, changing environmental conditions, and coexistence of multiple target categories.

[0094] Figure 4 Figures a1, a2, a3, and a4 in the diagram show a performance comparison of different algorithms in scenarios where multiple targets coexist, including pedestrians, construction signs, and road debris. In such scenarios, targets are typically small to medium in size and often share similar color characteristics with the roadside environment, increasing the difficulty of detection. Comparative analysis shows that the method proposed in this invention significantly outperforms the compared algorithms in multi-class detection. Figure 4 Figures b1, b2, b3, and b4 illustrate the performance differences of various algorithms in detecting large foreign objects such as fallen trees on roads. In this case, despite the large size of the foreign objects, their irregular shapes and high degree of integration with the natural environment pose challenges to accurate identification. Experimental results show that the method proposed in this invention not only accurately detected tree trunks crossing the road but also successfully identified pedestrians in the scene, demonstrating superior scene understanding capabilities. In contrast, YOLOv5 and YOLO11 exhibit significant instability in foreign object detection, while YOLOv10, although capable of detecting foreign objects, has significantly low confidence levels. Figure 4 The values ​​cl, c2, c3, and c4 in the figure represent the detection results for pedestrian interaction scenarios in a typical road environment. The main characteristics of this type of scenario are a large number of targets and partial occlusion, which places higher demands on the robustness of the detector. Figure 4 The values ​​d1, d2, d3, and d4 in the model represent the differences in performance when encountering an explosion on the road. Based on the detection and recognition results of different algorithms on complex targets such as the core area of ​​the explosion and the resulting flames, the robustness of the model in such extreme, highly dynamic, and visually disturbed scenarios can be evaluated. Figure 4 Figures e1, e2, e3, and e4 demonstrate the anomaly detection performance of each algorithm in a tunnel environment. The main challenges in tunnel scenes are uneven lighting conditions and low contrast, which can easily affect the accuracy of small target recognition. Figure 4 f1, f2, f3, and f4 in the diagram represent scenarios of severely damaged road surfaces. In such large-area, texture-feature-based anomaly recognition tasks, different models exhibit varying performance. Notably, the method proposed in this invention not only delivers the highest detection confidence, significantly outperforming other algorithms, but its generated bounding boxes also more accurately cover the main damaged areas. Figure 4 G1, g2, g3, and g4 in the table present the detection results for a composite scene containing illegal vehicles. In this scene, the target and background have similar color features, and there is partial occlusion, which increases the difficulty of recognition. Comparative analysis shows that the method proposed in this invention not only successfully detects illegal vehicles, but also provides more accurate bounding box localization and higher class confidence. Figure 4 h1, h2, h3, and h4 in the image show a complex scene of a traffic accident, which includes multiple types of abnormal targets. The targets are diverse in type and have complex factors such as large size variations and partial occlusion.

[0095] To deeply analyze the contributions of the AT-MSFE and PMA modules to model performance and their potential synergistic effects in the proposed method, a series of ablation experiments were designed and implemented. Using YOLO as the baseline model, and ensuring consistency in training configuration, the AT-MSFE and PMA modules were sequentially integrated into the baseline architecture. Detailed comparisons of experimental configurations and performance metrics are shown in Table 2, where "√" indicates the modules included in the current experimental configuration.

[0096] Table 2 Ablation Experiment Analysis

[0097]

[0098] The experimental results clearly reveal the effectiveness of each module. First, the introduction of the PMA module alone (A) shows a more significant performance gain, improving mAP@0.5 by 1.3% to 97.9% compared to the baseline model, with improvements in precision, recall, and F1 score. This indicates that PMA effectively improves the model's ability to discriminate targets in complex contexts by strengthening long-distance dependency capture and global context awareness, which may be particularly helpful for detecting small targets or targets with insignificant features. Second, the integration of the AT-MSFE module alone (B) brings improvements across all metrics compared to the baseline model, with mAP@0.5 increasing from 96.6% to 97.5%, and improvements in precision, recall, and F1 score. This preliminarily verifies that AT-MSFE, by actively adjusting the viewing distance under the condition of fixed target location and using a multi-receptive field approach for multi-scale feature extraction to alleviate the limitations of single-scale methods, does indeed enhance the model's ability to extract and fuse features at different scales.

[0099] Explanation of terms in this invention:

[0100] RelPos2d stands for Relative Position 2D, which refers to the concept of two-dimensional relative position encoding. It is frequently used in vision tasks to generate two-dimensional relative position codes, helping models capture the spatial relationships between input features.

[0101] DWConvpos The (·) operation is specifically designed to enhance the spatial location encoding of features. Its purpose is to introduce and incorporate local spatial location information, giving the output feature map a stronger sense of location. It emphasizes the learning and preservation of location information and tends to learn location-related patterns. In contrast, ordinary DWConv is mainly used for feature extraction and transformation, focusing on content features. pos It is typically used to add its own positional encoding to features, while ordinary DWConv is usually used directly for feature transformation.

[0102] References for YOLOv5: Jocher, G. (2020). Ultralytics YOLOv5 (Version 7.0) [Computer software]. GitHub.

[0103] References for EfficientDet: Tan, Mingxing, Ruoming Pang, and Quoc V.Le. "Efficientdet: Scalable and efficient object detection." Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020.

[0104] References for YOLOv8: Jocher, G., Chaurasia, A., & Qiu, J. (2023). Ultralytics YOLO (Version 8.0.0) [Computer software].

[0105] References for YOLOv10: Wang, Ao, et al. "Yolov10: Real-time end-to-end object detection." Advances in Neural Information Processing Systems 37(2025): 107984-108011.

[0106] References for RT-DETR: Srinivasan, Aparajith, et al. "A novel approach forroad accident detection using DETR algorithm." 2020 international conference on intelligent data science technologies and applications (IDSTA). IEEE, 2020.

[0107] References for YOLO11: Jocher, G., & Qiu, J. (2024). Ultralytics YOLO11 (Version 11.0.0) [Computer software]. GitHub.

Claims

1. A method for detecting road anomalies, comprising: Construct a road anomaly detection model consisting of a backbone network, a neck network, and a detection head connected in sequence; The road anomaly detection model is trained to obtain the trained road anomaly detection model; the road scene image to be detected is input into the trained road anomaly detection model to obtain the road anomaly detection result; The specific operational process for obtaining road anomaly detection results is as follows: Step ①: The backbone network extracts features from the input road scene image to be detected, and obtains a multi-scale fused feature map; Step 2: The neck network performs multi-level feature fusion and enhancement on the multi-scale fusion feature map output by the backbone network to obtain a detection feature map with multi-scale perception capability. Step 3: The detection head performs target classification and location regression prediction on the detection feature map output by the neck network to obtain road anomaly detection results containing target category, confidence level and bounding box coordinates; The backbone network is characterized by comprising, in sequence, a first CBS layer, a second CBS layer, a first C3 layer, a third CBS layer, a second C3 layer, a fourth CBS layer, a third C3 layer, a fifth CBS layer, a position modulation attention module, and an active visual follow-up multi-scale feature extraction module; The specific operation process for step ① is as follows: Step ①-1: The first CBS layer performs convolutional feature extraction on the input road scene image to be detected to obtain the first basic feature map; Step ①-2: The second CBS layer performs convolutional feature extraction on the first basic feature map output by the first CBS layer to obtain the second basic feature map; Step ①-3: The first C3 layer performs feature fusion on the second basic feature map output by the second CBS layer to obtain the first hierarchical feature map; Steps ①-4: The third CBS layer performs convolutional feature extraction on the first hierarchical feature map output by the first C3 layer to obtain the third basic feature map; Steps ①-5: The second C3 layer performs feature fusion on the third basic feature map output by the third CBS layer to obtain the second hierarchical feature map; Steps ①-6: The fourth CBS layer performs convolutional feature extraction on the second-level feature map output by the second C3 layer to obtain the fourth basic feature map; Steps ①-7: The third C3 layer performs feature fusion on the fourth basic feature map output by the fourth CBS layer to obtain the third-level feature map; Steps ①-8: The fifth CBS layer performs convolutional feature extraction on the third-level feature map output by the third C3 layer to obtain the fifth basic feature map; Steps ①-9: The position modulation attention module performs attention calculation based on spatial distance modulation on the fifth basic feature map output by the fifth CBS layer to obtain a global association feature map with enhanced spatial perception. Step ①-10: The active vision follow-up multi-scale feature extraction module performs adaptive multi-scale feature extraction and fusion operations on the global correlation feature map output by the position modulation attention module through a parallel processing method of multiple branches and multiple receptive fields to obtain a multi-scale fused feature map.

2. The road anomaly detection method according to claim 1, characterized in that... The specific operation process of steps ①-9 is as follows: Step ①-9-1: Denote the input fifth basic feature map as X∈R C1×H×W The feature map X is obtained by performing convolutional dimensionality reduction on X through the first Conv layer. in ∈R C′×H×W Where C1 represents the number of channels in the input fifth basic feature map, H represents the height dimension of the input fifth basic feature map, W represents the width dimension of the input fifth basic feature map, and C′ represents the number of channels in the position modulation feature. Step ①-9-2: Perform relative position encoding calculation on X using the RelPos2d layer, and generate a spatial attenuation matrix along the width direction based on the Manhattan distance. and the spatial attenuation matrix along the height direction Where γ represents the learnable decay base, 0 < γ < 1, used to control the decay rate, (x n y n (x) represents the two-dimensional coordinates of the nth marker on the feature map. m y m () represents the two-dimensional coordinates of the m-th marker on the feature map, where n, m∈{(i,j)|1≤i≤H, 1≤j≤W}; Step ①-9-3, extract the feature map X output by the first Conv layer. in and the spatial decay matrix along the width direction of the RelPos2d layer output. and the spatial attenuation matrix along the height direction The input is fed into the modulation attention layer for distance decay attention calculation to obtain attention-weighted features. The specific operation is as follows: Step ①-9-3-1, obtain the position modulation feature X pos ∈R C′×H×W X pos =X in +DWConv pos (X in ), where DWConv pos (·) indicates a depthwise separable convolution operation; Step ①-9-3-2: Obtain the query vector Q, key vector K, and value vector V, where Q = X′W Q K = X′W K V = X′W V Where X′ represents X after layer normalization. pos W Q W K W V These represent the learnable linear projection matrices, respectively. Step ①-9-3-3, obtain intermediate feature X out ∈R C′×H×W X out =X pos +DropPath(W O (PMAttn(LN1(X pos ))+LEPE(V))), where DropPath(·) represents the regularization operation, W O LN1(·) represents the learnable output projection matrix, LN1(·) represents the layer normalization operation, LEPE(·) represents the local augmentation sensing coding operation, and LEPE(V) = DWConv 5×5 (V), DWConv 5×5 (·) denotes a depthwise separable convolution operation with a kernel size of 5×5, V represents the value vector, and PMAtn(·) denotes a position-modulated attention operation. Softmax represents the normalized exponential function, Q represents the query vector, and K... T d represents the transpose of the key vector. k It represents the dimension of the key vector, and ⊙ indicates element-wise multiplication; Step ①-9-3-4: Obtain attention-weighted features Xx inal ∈R C′×H×W X final =X out +DropPath(FFN(LN2(X out ))), where FFN(·) represents the feedforward neural network operation and LN2(·) represents the layer normalization operation; Step ①-9-4: Perform convolutional dimensionality reduction on X through the second Conv layer to obtain the feature map X″∈R. C′×H×W ; Step ①-9-5, modulate the attention-weighted feature X output by the attention layer. final The feature map X″ output from the second Conv layer is concatenated to obtain the fused feature; Step ①-9-6: Perform feature integration operation on the fused features through the third Conv layer to obtain the global associated feature map X″′∈R. C2×H×W Where C2 represents the number of channels in the global correlation feature map.

3. The road anomaly detection method according to claim 1, characterized in that... The specific operation process of steps ①-10 is as follows: Step ①-10-1: The first branch uses 1×1 convolution to extract small features from the input global correlation feature map, and then performs spatial downsampling on the small features through max pooling to obtain local fine-grained features; Step ①-10-2: The second branch uses 3×3 convolution to extract the medium features of the input global correlation feature map. The medium features after max pooling and the medium features after average pooling are added element-wise to obtain the medium-scale structural features. Step ①-10-3: The third branch uses a 5×5 convolution to extract the large features of the input global correlation feature map. After performing three consecutive average pooling operations on the large features, the global structural features are obtained. Step ①-10-4: Concatenate the small features, local fine-grained features, mesoscale structural features, and global structural features along the channel dimension to obtain a multi-scale fused feature. Use the learnable weights of a 1×1 convolution to perform a weighted summation of the multi-scale fused feature to obtain the multi-scale fused feature map Y∈R. 2C′×H×W .

4. The road anomaly detection method according to claim 1, characterized in that... The specific steps to obtain the trained road anomaly detection model are as follows: Step 1: Collect environmental images from roads, streets and grasslands, and define environmental images with a pixel size lower than 60×25 or 45×30 as small road target images. Randomly select at least 16244 small road target images as a road anomaly detection dataset. Manually annotate all images in the road anomaly detection dataset. Randomly select 80% of the images from the annotated road anomaly detection dataset as the training set. Step 2, set the training parameters including: total training cycles set to 200, batch size set to 4, and initial learning rate set to 0.01; Step 3, define the total loss function as Loss = λ1·LossBox + λ2·LossObj + λ3·LossCls, where LossBox represents the bounding box regression loss, LossObj represents the target confidence loss, LossCls represents the classification loss, and λ1, λ2, and λ3 represent the weight coefficients that balance the importance of different tasks. Step 4: Based on the total loss function, input the training set into the road anomaly detection model for training. Update the road anomaly detection model through the SDG optimizer until the total training cycle is reached, then stop iterative updates to obtain the trained road anomaly detection model.

Images