Road surface disease detection method, system and equipment based on hybrid architecture, and storage medium

Abstract

The invention relates to a pavement disease detection method, system and device based on a hybrid architecture, and a storage medium. The method comprises the following steps: obtaining pavement image data; the method comprises the steps that data are input into a re-parameterized feature extraction network, the network is based on an HGNetV2 architecture, a RCHGBlock module is formed by embedding a RepConv structure into HGBlock, and a plurality of modules are cascaded and stacked to construct a four-stage progressive feature pyramid structure; based on a high-level semantic layer of a feature pyramid, integrating a space edge perception enhanced attention mechanism, enhancing disease edge features through a dual-path complementary processing framework of an edge extraction path and a standard convolution path, and performing multi-scale feature fusion by using an improved C3K2-SEAM module as a feature fusion unit; and performing end-to-end disease detection based on the fusion features, and outputting disease types, positions and confidence information. Compared with the prior art, the method has the advantage that the disease detection precision and robustness in a complex scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a road disease detection method, system, device and storage medium based on a hybrid architecture. BACKGROUND

[0002] Highway maintenance management has entered a new stage of scale and normalization. Traditional manual inspection is inefficient and highly subjective, making it difficult to meet the demand for processing massive amounts of road condition data.

[0003] Intelligent detection technology based on computer vision has become a key direction to solve this problem. RT-DETR based on Transformer has high detection accuracy, but has high computational complexity and slow processing speed, making it difficult to meet real-time requirements. YOLO has high computational efficiency, but its feature extraction and fusion mechanism has a bottleneck in feature expression when processing complex scenes, leading to missed detection and false detection, especially in low light, shadow and other complex environments. Existing models often focus on single optimization of accuracy or efficiency, making it difficult to achieve simultaneous improvement of computational efficiency and detection accuracy, and unable to meet the demand for high-precision real-time detection in resource-constrained environments.

[0004] A road disease intelligent detection method and system based on instance segmentation disclosed in publication CN118864443A uses a Swin Transformer module to enhance global feature extraction, but still does not fully optimize the frequency domain characteristics of the disease, especially in low light, shadow and other complex environments.

[0005] Therefore, there is an urgent need for a lightweight detection architecture that combines high accuracy, low computational load and strong environmental adaptability. SUMMARY

[0006] To overcome the technical problems of low efficiency of manual inspection in traditional road disease detection, high computational complexity of models in mobile deployment and insufficient feature representation ability, the present application provides a road disease detection method, system, device and storage medium based on a hybrid architecture, which significantly improves the accuracy and robustness of road disease detection in complex scenes by combining the advantages of RT-DETR and YOLOv11.

[0007] The object of the present application can be achieved by the following technical solutions: The first aspect of the present application provides a road disease detection method based on a hybrid architecture, comprising the following steps: Obtaining road image data; inputting the road surface image data into a reparameterization feature extraction network to extract multi-level feature representations, wherein the reparameterization feature extraction network is based on an HGNetV2 architecture, forms RC_HGBlock modules by embedding a RepConv structure into an HGBlock, and constructs a four-stage progressive feature pyramid structure by cascading and stacking multiple RC_HGBlock modules to realize feature extraction from low-level textures to high-level semantics; Based on the high-level semantic layer integration space edge perception enhanced attention mechanism SEAM of the feature pyramid structure, the edge feature is enhanced through the dual-path complementary processing framework of the edge extraction path and the standard convolution path, and the improved C3K2-SEAM module is used as a feature fusion unit for multi-scale feature fusion to obtain enhanced fusion features. Based on the enhanced fusion features, an end-to-end disease detection is performed to output the type, location and confidence information of the road surface disease.

[0008] Further, the specific process of inputting the road surface image data into the reparameterization feature extraction network to extract multi-level feature representations includes: The obtained road surface image data is input into the input end of the RC-HGNetV2 network, and the initial feature map is obtained through preliminary feature extraction by the front feature extraction layer. The initial feature map is input into the four-stage backbone network constructed by cascading and stacking multiple RC_HGBlock modules for deep feature extraction. After the output of each stage of the four-stage backbone network, the dimension reduction and scale scaling of the feature map are performed by using the lightweight down-sampling module constructed based on the depth separable convolution in sequence, to generate four feature maps of different scales P2, P3, P4 and P5, thereby constructing the four-stage progressive feature pyramid structure, so as to complete the multi-level feature representation extraction from low-level texture features to high-level semantic features.

[0009] Further, the RC-HGNetV2 reparameterization feature extraction network is used to establish a feature pyramid structure among the four feature scales P2, P3, P4 and P5, wherein the stages of the feature pyramid structure are connected by lightweight down-sampling modules to realize feature down-sampling, and the lightweight down-sampling module is constructed based on the depth separable convolution DWConv. When processing the input features, the grouped convolution operation is first performed on the spatial dimension of the features to extract spatial features, and the 1x1 point convolution is used to aggregate the spatial features in the channel dimension, so as to reduce the scale of the feature map while reducing the computational load of the model.

[0010] Further, based on the high-level semantic layer of the feature pyramid structure, a spatial edge-aware attention mechanism SEAM is integrated, and the specific process of enhancing the disease edge features through the dual-path complementary processing framework of the edge extraction path and the standard convolution path includes: From the feature pyramid structure output by the reparameterized feature extraction network, a high-level semantic feature map of P5 / 32 scale is extracted as the input of the spatial edge-aware attention mechanism SEAM; The high-level semantic feature map is input into the dual-path complementary processing framework for parallel processing; The edge gradient features output by the edge extraction path are spliced with the overall semantic features output by the standard convolution path in the channel dimension, and then a 1x1 convolution layer is used to reduce and integrate the spliced features to obtain enhanced attention feature maps; The enhanced attention feature maps are added to the original input high-level semantic feature maps through residual connection to complete the enhancement of the disease edge features and linear structure features, and the feature maps processed by the SEAM mechanism are output.

[0011] Further, the dual-path complementary processing framework includes: In the edge extraction path, a parameterized SobelConv convolution variant is used to convolve the input features to explicitly extract the edge gradient features of the disease target; In the standard convolution path, a standard convolution operation is used to transform the input features to retain the overall semantic information and context relationship of the image.

[0012] Further, the improved C3K2-SEAM module is used as a feature fusion unit for multi-scale feature fusion to obtain enhanced fusion features, and the specific process includes: The spatial edge-aware attention mechanism SEAM is integrated into the C3K2 feature fusion module of YOLOv11 to construct a C3K2-SEAM module, and the C3K2-SEAM module is used as the core feature fusion unit of the neck network; The feature maps from different scales extracted by the reparameterized feature extraction network and the high-level semantic feature maps processed by the spatial edge-aware attention mechanism SEAM are input into the C3K2-SEAM module; Inside the C3K2-SEAM module, the multi-scale features input are cross-fused and information-interacted through the built-in multi-branch convolution structure to reconstruct the geometric continuity features of the pavement diseases in the high-level features; The fused features are output by the C3K2-SEAM module to obtain enhanced fusion features that contain spatial details and have semantic representation capabilities.

[0013] Furthermore, the specific process of performing end-to-end pavement defect detection based on the enhanced fusion features and outputting information on the type, location, and confidence level of pavement defects includes: The enhanced fusion features output by the C3K2-SEAM module are input into the detection head module; In the detection head module, two types of calculations are performed in parallel based on the enhanced fusion features: In the first type of calculation, the probability of each type of disease existing at each preset anchor point is calculated through the classification branch; In the second type of calculation, the category confidence is generated, and the precise offset and size parameters of the disease target bounding box at each preset anchor point are calculated through the regression branch to generate position information and obtain preliminary detection results. The preliminary detection results generated by the detection head module are screened and redundancy removed, including: filtering out low-confidence detection boxes according to a preset confidence threshold, applying a non-maximum suppression algorithm to the remaining detection boxes, removing highly overlapping redundant detection boxes, and retaining the optimal detection result for each disease instance. The final detection results, after screening and redundancy removal, are output in a structured format. For each confirmed disease instance, the specific disease type, its location coordinates in the image, and the corresponding classification confidence level are output simultaneously.

[0014] A second aspect of the present invention provides a pavement distress detection system based on a hybrid architecture, comprising: The image acquisition module is used to acquire road surface image data; The reparameterized feature extraction module is used to input the road image data into the reparameterized feature extraction network to extract multi-level feature representations. The reparameterized feature extraction network is based on the HGNetV2 architecture. It forms an RC_HGBlock module by embedding the RepConv structure into HGBlock, and cascades and stacks multiple RC_HGBlock modules to construct a four-stage progressive feature pyramid structure, which is used to realize feature extraction from low-level texture to high-level semantics. The edge perception enhancement and feature fusion module is used to integrate the spatial edge perception enhancement attention mechanism SEAM based on the high-level semantic layer of the feature pyramid structure. It enhances the disease edge features through a dual-path complementary processing framework of edge extraction path and standard convolution path, and uses the improved C3K2-SEAM module as a feature fusion unit to perform multi-scale feature fusion to obtain the enhanced fused features. The pavement defect detection output module is used to perform end-to-end defect detection based on the enhanced fusion features and output the type, location, and confidence information of pavement defects.

[0015] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the processor is used to execute a program in the memory to implement the road surface defect detection method based on the hybrid architecture described above.

[0016] A third aspect of the present invention provides a storage medium containing computer-executable instructions, characterized in that, when executed by a computer processor, the storage medium containing the computer-executable instructions is used to execute the pavement distress detection method based on a hybrid architecture as described above.

[0017] The present invention has at least the following advantages or beneficial effects: This invention first introduces structural reparameterization technology into the backbone feature extraction network. Through the RC_HGBlock module, a multi-branch heterogeneous architecture is adopted during the training phase, and equivalent fusion into a single convolutional kernel during the inference phase, effectively balancing feature representation capability and computational efficiency. Secondly, a SEAM attention mechanism is introduced into the high-level semantic layer of the feature pyramid network. By constructing a dual-path complementary processing framework of edge extraction and standard convolution, the model's ability to perceive the linear structure and subtle texture features of road surface defects is significantly enhanced. Finally, the C3K2-SEAM feature fusion unit achieves efficient information interaction between shallow edge features and deep semantic features of defects, realizing synergistic optimization of computational efficiency and detection accuracy, and providing an efficient and feasible technical solution for the intelligent transformation of highway maintenance. Attached Figure Description

[0018] Figure 1 This is a flowchart of a road surface distress detection method based on a hybrid architecture, as described in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall framework of the model in an embodiment of the present invention; Figure 3 A schematic diagram comparing the structures of HG Block and RC_HG Block; Figure 4 This is a schematic diagram comparing the HGNetV2 backbone network and the RC_HGNetV2 backbone network. Figure 5 A schematic diagram of the SEAM attention mechanism and the C3k2-SEAM module structure; Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0019] Explanation of reference numerals in the attached figures: 101, memory; 102, processor; 103, communication interface. Detailed Implementation

[0020] Overall, this invention integrates the advantages of RT-DETR and YOLOv11 to construct an RC-YOLO detection model. It introduces structural reparameterization technology into the backbone feature extraction network, embedding the RepConv structure into the HG Block to form the RC_HGBlock module, thus constructing a four-stage progressive feature pyramid RC-HGNetV2 network for efficient multi-scale feature fusion. A spatial edge-aware enhanced attention mechanism (SEAM) is designed, enhancing the perception of linear structure and texture features of road defects through a dual-path complementary framework of edge extraction and standard convolution, and is integrated into the high-level semantic layer of the feature pyramid. Based on the improved model, real-time detection of road defects is performed. This invention significantly improves the accuracy and robustness of defect detection in complex scenarios.

[0021] The following is an explanation of the terms used in this invention: RT-DETR: A real-time detection model based on the Transformer architecture, which is referenced and fused in this invention due to its high-precision feature extraction network HGNetV2.

[0022] YOLOv11: An efficient single-stage object detection model. This invention combines its fast inference capabilities with the advantages of an efficient neck network structure.

[0023] RC-YOLO: The core detection model proposed in this invention refers to a hybrid architecture that combines the advantages of HGNetV2 and YOLOv11 with RT-DETR and introduces reparameterized convolution and SEAM attention mechanisms.

[0024] RepConv is a structural reparameterization technique that uses multi-branch heterogeneous convolutions during training to enhance feature representation capabilities, and then effectively merges them into a single convolution kernel during inference to reduce computational complexity.

[0025] RC_HGBlock: The core module of this invention, formed by embedding the RepConv structure into HGBlock, is used to construct the RC-HGNetV2 backbone network. Specifically, it is formed by merging the structural design of RepConv (reparameterized convolution) with the basic architecture of HGBlock (efficient gradient flow module). During the training phase, this module adopts a multi-branch heterogeneous architecture containing 3x3 convolutions, 1x1 convolutions, and skip connections to enhance the model's feature representation ability and generate diverse gradient flows. During the inference phase, it uses structural reparameterization technology to equivalently fuse the multi-branch structure into a single 3×3 convolution kernel, thereby significantly reducing computational complexity and memory usage. This module is cascaded and stacked to construct the RC-HGNetV2 backbone network, aiming to achieve efficient deployment and real-time inference of the model on mobile or embedded devices while maintaining high-precision feature extraction capabilities.

[0026] HGNetV2: An efficient gradient flow network, serving as the feature extraction backbone of RT-DETR, and is the basic architecture for constructing the RC-HGNetV2 network in this invention.

[0027] RC-HGNetV2: The reparameterized feature extraction network proposed in this invention is a four-stage progressive feature pyramid built on HGNetV2 by introducing the RC_HGBlock module. It is constructed on the basic architecture of the RT-DETR feature extraction network HGNetV2 by replacing the original HGBlock module with the innovative RC_HGBlock module. This network adopts a four-stage progressive feature pyramid structure (P2 to P5), achieving efficient multi-level and multi-scale extraction from low-level texture features to high-level semantic features through cascading and stacking multiple RC_HGBlock modules. Furthermore, the network integrates SPPF and C2PSA modules at the end, forming a multi-granularity feature fusion mechanism that combines global context and local details. Ultimately, while maintaining high accuracy, the reparameterization technique significantly reduces computational complexity, making it suitable for real-time road surface defect detection tasks on mobile or embedded devices.

[0028] SEAM: an abbreviation for Spatial Edge Perception Enhanced Attention Mechanism.

[0029] C3K2-SEAM: This improved feature fusion module integrates the SEAM mechanism into the C3K2 module of YOLOv11 to reconstruct the geometric continuity of road defects in high-level features. It is formed by integrating an innovative Spatial Edge Awareness Enhanced Attention (SEAM) mechanism into the C3K2 feature fusion module of YOLOv11. This module is specifically inserted into the P5 / 32 high-level semantic layer of the feature pyramid network, utilizing the dual-path complementary processing framework of the SEAM mechanism (the edge extraction path explicitly enhances the linear edge and contour features of the defects, while the standard convolutional branch preserves the overall semantic context) to strengthen feature representation. Finally, the C3K2-SEAM module aims to reconstruct the geometric continuity features of road defects blurred by deep semantic abstraction through cross-fusion and information interaction of multi-scale features, thereby significantly improving the detection accuracy of slender targets such as cracks, while its lightweight design maintains low computational overhead.

[0030] SobelConv: A parameterized convolutional variant of the Sobel operator for explicitly extracting edge gradient features of a target in the edge extraction path of the SEAM mechanism.

[0031] DWConv: an abbreviation for depthwise separable convolution, used in this invention to build a lightweight downsampling module, reducing the number of parameters and computational load through two-step operations of group convolution and point convolution.

[0032] SPPF: A spatial pyramid pooling fast module integrated at the end of the RC-HGNetV2 backbone network to extract multi-scale global context information.

[0033] C2PSA: An attention module integrated at the end of the RC-HGNetV2 backbone network, which works with SPPF to form a multi-granularity feature fusion mechanism from global to local.

[0034] This invention provides a pavement distress detection method based on a hybrid architecture, comprising the following steps: Based on the RT-DETR feature extraction network HGNetV2, a structural reparameterization technique is introduced into the backbone feature extraction network. The RepConv structure is embedded into the HG Block to form the RC_HGBlock module, and a four-stage progressive feature pyramid RC-HGNetV2 network is constructed to build the initial detection model. An improved C3K2 feature fusion module is used in the neck of the initial detection model, and a spatial edge awareness enhanced attention mechanism (SEAM) is introduced in the high-level semantic layer of the initial detection model to enhance the edge features of the disease in the image in order to construct a target pavement disease detection model. Detection of pavement defects is carried out based on the target pavement defect detection model.

[0035] This invention addresses the challenges of extracting linear structural features such as cracks and severe interference from complex backgrounds in pavement defect detection. First, it introduces structural reparameterization technology into the backbone feature extraction network. The RC_HGBlock module employs a multi-branch heterogeneous architecture during training and effectively fuses these features into a single convolutional kernel during inference, effectively balancing feature representation capability and computational efficiency. Second, it introduces the SEAM attention mechanism into the P5 / 32 high-level semantic layer of the feature pyramid network. By constructing a dual-path complementary processing framework of edge extraction and standard convolution, it significantly enhances the model's ability to perceive linear structures and subtle texture features of pavement defects while maintaining computational efficiency. Finally, by integrating the SEAM mechanism into the C3K2 module to form the C3K2-SEAM feature fusion unit, this invention achieves efficient information interaction between shallow edge features and deep semantic features of defects, significantly improving detection accuracy in complex scenarios.

[0036] In practice, the RC_HGBlock module adopts a multi-branch heterogeneous architecture design during the training phase, which includes a 3×3 standard convolutional branch, a 1×1 convolutional branch, and a skip connection branch. Each branch is equipped with an independent batch normalization layer. During the inference phase, the convolutional kernels of each branch are size-aligned, and the convolutional weights of each branch are fused with the parameters of the batch normalization layer, which is ultimately equivalent to a single 3×3 convolutional kernel.

[0037] In practice, the SEAM attention mechanism extracts edge features by parameterizing the 3×3 Sobel operator, calculates the horizontal gradient response Gx and the vertical gradient response Gy of the input feature map, and generates the gradient magnitude to distinguish between cracks and asphalt texture and the gradient direction to determine the direction of cracks.

[0038] In practice, the SEAM attention mechanism described above constructs a dual-path complementary processing framework of edge extraction and standard convolution. The two complementary features are concatenated through the channel dimension, integrated by dimensionality reduction through a 1×1 convolutional layer, and finally added to the original input features through residual connections.

[0039] In practice, the RC-HGNetV2 network establishes a feature pyramid across four feature scales from P2 to P5. Each stage uses a lightweight downsampling module built on depthwise separable convolution (DWConv) to achieve feature downsampling, effectively reducing the number of model parameters and computational load.

[0040] In practice, the C3K2-SEAM module integrates the SEAM mechanism into the C3K2 feature fusion module of YOLOv11 and inserts it into the P5 / 32 high-level semantic layer of the feature pyramid to reconstruct the geometric continuity features of pavement distress in high-level features, while reducing computational overhead.

[0041] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Component models, material names, connection structures, circuit structures, control methods, algorithms, and other features not explicitly described in this technical solution are considered common technical features disclosed in the prior art.

[0042] Example 1 like Figures 1-5 As shown, this embodiment provides a pavement distress detection method based on a hybrid architecture, including the following steps: S1. In terms of backbone network design, the efficient gradient flow network HGNetV2 is used as the basic architecture. By introducing structural reparameterization technology, the RC_HGBlock module is constructed to form a four-stage progressive feature pyramid (P2-P5) to build a lightweight backbone network. In some embodiments of this invention, based on the HGNetV2 network architecture, a structural reparameterization technique is first introduced into the backbone feature extraction network to construct an RC_HGBlock module with dynamic parameter sharing characteristics. This module maintains the advantages of the multi-branch parallel computing structure of HGNetV2 while effectively reducing runtime computational complexity and memory consumption through the structural reparameterization strategy.

[0043] Specifically, the RC_HGBlock module employs a multi-branch heterogeneous architecture during training, comprising three parallel computation paths: a 3×3 standard convolution branch, a 1×1 convolution branch, and a skip connection (Identity) branch. The 3×3 convolution captures local spatial features, modeling the correlation between adjacent pixels through a larger receptive field; the 1×1 convolution focuses on non-linear transformations along the channel dimension, enhancing the information exchange capability between feature channels; and the skip connection preserves the original input features, ensuring that low-level detail information can be passed to deeper networks. Each branch is equipped with an independent Batch Normalization (BN) layer, dynamically calibrating the feature distribution using learnable parameters γ and β, as shown in the following formula: (1) (2) In the formula: For convolution kernel weights, For bias terms, , These are the scaling and offset parameters for the BN layer, respectively. and The mean and variance of the input data are calculated during training and fixed during inference.

[0044] This multi-branch heterogeneous structure generates diverse gradient flows during training, enabling the network to simultaneously learn feature representations at different scales and levels of abstraction. This is particularly suitable for addressing the diverse morphologies and complex backgrounds of road surface defects in road defect detection. During the inference phase, RepConv first aligns the size of the convolutional kernels in each branch, expanding the 1×1 kernel padding to the size of a 3×3 kernel, while treating skip connections as 1×1 convolutions of the identity matrix. Then, it fuses the convolutional weights of each branch with the parameters of the BN layer, converting them into equivalent convolutional operations. Finally, it algebraically superimposes the equivalent convolutional kernels of all branches, resulting in a single 3×3 convolutional kernel. Figure 3 As shown in (c), this structural reparameterization strategy significantly reduces computational complexity during the inference phase.

[0045] Specifically, multiple RC_HGBlock modules in the backbone network are cascaded and stacked to form a hierarchical feature extraction structure, which can progressively capture multi-level feature representations from low-level texture to high-level semantics. Combined with a lightweight downsampling module built based on depthwise separable convolution (DWConv), the model establishes an efficient feature pyramid network across four feature scales from P2 to P5, achieving efficient extraction and transfer of multi-scale features.

[0046] In its specific implementation, the structure of the HGNetV2 backbone network is as follows: Figure 4As shown in (a), the network front-end uses the HGStem layer as the feature preprocessing unit, responsible for mining low-level features from the original input data. Then, DWConv is used to perform feature downsampling. DWConv first performs grouped convolution operations in the feature space dimension, and then uses 1×1 point convolutions to aggregate feature information in the channel dimension. This design significantly reduces the number of model parameters and computational load. The core computational unit of the network is the HGBlock layer. This module, through the concatenation of multiple LightConv lightweight convolutions, can capture features at different levels while maintaining low computational complexity.

[0047] As the core feature processing unit of the HGNetV2 network, the HG Block constructs a multi-scale feature extraction system by cascading multiple LightConv lightweight convolutional modules. However, the LightConv modules used in this invention employ fixed convolutional kernels for spatial feature extraction, which, while reducing computational cost, suffers from insufficient local feature capture when processing slender targets such as cracks. Therefore, this invention introduces RepConv into the HG Block, forming RC_HGBlock, and uses it to improve HGNetV2, proposing RC_HGNetV2. Figure 4 As shown in (b), compared to HGNetV2, RC_HGNetV2 maintains the multi-scale feature pyramid architecture while integrating SPPF and C2PSA modules at the end of the backbone network, forming a multi-granularity feature fusion mechanism from global context to local details.

[0048] S2. An improved C3k2-SEAM feature fusion module is used in the neck of the initial detection model YOLO11, and the spatial edge perception enhanced attention mechanism SEAM is introduced in the high-level semantic layer to enhance the edge features of the disease in the image in order to construct the target pavement disease detection model. In its implementation, the SEAM attention mechanism extracts edge features through a parameterized 3×3 Sobel operator, responding to the horizontal gradient of the input feature map. and vertical gradient response Calculations are performed to generate gradient magnitudes to distinguish cracks from asphalt texture and gradient directions to determine crack orientation. For the input feature map... ,That and The gradient calculation formula is: (3) (4) In the formula: The gradient response in the horizontal direction, This represents the gradient response in the vertical direction; The gradient amplitude can be used to distinguish between cracks and asphalt texture. The gradient direction can be used to determine the direction of the crack.

[0049] In its specific implementation, the SEAM attention mechanism described above constructs a dual-path complementary processing framework of edge extraction and standard convolution. The two complementary features are concatenated through the channel dimension, integrated through a 1×1 convolutional layer for dimensionality reduction, and finally added to the original input features through residual connections.

[0050] In some embodiments of this invention, an improved C3K2 feature fusion module is employed at the neck of the model to maintain a lightweight structure while preserving effective features and cross-layer connectivity mechanisms. Addressing the challenges of small target localization, blurred defect edges, and complex background interference in pavement defect detection tasks, this invention proposes a novel Spatial Edge Awareness Enhanced Attention (SEAM) mechanism and integrates it into the feature fusion module of YOLOv11.

[0051] Specifically, the core idea of ​​the SEAM attention mechanism is to combine edge feature extraction with convolutional feature extraction, comprehensively capturing multidimensional information of the input features through parallel processing paths. Compared with traditional attention mechanisms, SEAM has three significant advantages: First, it is designed directly for the specific properties of road surface defects, emphasizing the extraction of linear structure and texture features of road surface defects; second, SEAM's parallel processing architecture achieves complementary enhancement of detailed edge information and overall semantic information; and finally, SEAM has relatively low computational overhead, ensuring the model's computational efficiency and real-time performance.

[0052] S3. Detect pavement defects based on the target pavement defect detection model.

[0053] This invention addresses the challenges of extracting linear structural features such as cracks and severe interference from complex backgrounds in pavement defect detection. First, it introduces structural reparameterization technology into the backbone feature extraction network. The RC_HGBlock module employs a multi-branch heterogeneous architecture during training and effectively fuses these features into a single convolutional kernel during inference, effectively balancing feature representation capability and computational efficiency. Second, it introduces the SEAM attention mechanism into the P5 / 32 high-level semantic layer of the feature pyramid network. By constructing a dual-path complementary processing framework of edge extraction and standard convolution, it significantly enhances the model's ability to perceive linear structures and subtle texture features of pavement defects. Finally, by integrating the SEAM mechanism into the C3K2 module to form the C3K2-SEAM feature fusion unit, this invention achieves efficient information interaction between shallow edge features and deep semantic features of defects, significantly improving detection accuracy in complex scenarios.

[0054] Example 2 This embodiment provides a pavement distress detection system based on a hybrid architecture, including a reparameter feature extraction module, an edge perception enhancement module, and a distress detection module. The specific implementation for detecting pavement distress includes: The reparameterized feature extraction module is based on the RT-DETR feature extraction network HGNetV2. It introduces structural reparameterization technology into the backbone feature extraction network, embeds the RepConv structure into HG Block to form the RC_HGBlock module, and constructs a four-stage progressive feature pyramid RC-HGNetV2 network. The collected road surface image is input into the RC-HGNetV2 backbone network to extract feature maps at four scales from P2 to P5. The edge awareness enhancement module is used to apply an improved C3K2 feature fusion module to the neck of the initial detection model, and introduces the spatial edge awareness enhancement attention mechanism SEAM in the high-level semantic layer of the initial detection model. The extracted feature map is fed into the C3K2-SEAM neck network to enhance the edge features of the disease through the SEAM mechanism. The pavement defect detection module is used to detect pavement defects based on the target pavement defect detection model. It outputs the defect type, location and confidence level through the detection head to complete the pavement defect detection.

[0055] This embodiment combines the high-precision feature extraction capability of RT-DETR with the efficient neck structure of YOLOv11, significantly improving detection accuracy and robustness in complex scenarios while maintaining lightweight characteristics. It provides an efficient and feasible technical solution for real-time target detection applications on mobile devices, effectively solving engineering problems such as limited computing resources, insufficient feature representation capabilities, and poor environmental adaptability faced by traditional models when deployed on vehicle edge devices.

[0056] Example 3 This embodiment provides an electronic device, such as... Figure 6 As shown, it includes a storage unit 101 for storing one or more program codes; and a processing unit 102 for executing the program codes stored in the storage unit 101. When the processing unit 102 executes the program codes, it implements the pavement defect detection method as described in any embodiment of the first aspect above.

[0057] The electronic device also includes a communication unit 103. The storage unit 101, processing unit 102, and communication unit 103 are electrically connected to achieve data interaction. Specifically, these components can be interconnected via a communication bus or signal lines. The storage unit 101 is used to store software programs and related modules. The processing unit 102 executes the program code in the storage unit 101 to implement various functional applications and data processing tasks. The communication unit 103 is used for data transmission or signaling interaction with other devices.

[0058] In terms of implementation, the storage unit 101 can adopt various types of storage media, including but not limited to: random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM).

[0059] The processing unit 102 may employ an integrated circuit chip with signal processing capabilities, specifically a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices. It may also employ hardware components such as discrete gate circuits or transistor logic devices.

[0060] The technical solutions in this embodiment are not limited to the specific implementation methods described above. The flowcharts and block diagrams in the accompanying drawings are only used to exemplify the possible implementation architecture, functions, and operation flows of this application. Each block may represent a functional module, program segment, or code portion, containing executable instructions for implementing specific logical functions. In some alternative implementations, the functional order of the blocks may differ from that shown in the accompanying drawings; for example, some operations may be executed in parallel or in reverse order, depending on the functional requirements involved. Furthermore, each block or combination thereof in the flowchart can be implemented by a dedicated hardware system or by a combination of hardware and computer instructions.

[0061] The functional modules in this application embodiment can be flexibly configured according to actual needs: each module can be integrated to form a unified whole, or it can exist independently, or two or more modules can be combined to form an independent functional unit.

[0062] Example 4 This embodiment provides a computer-readable storage medium storing a computer program. When the program is executed by the processing unit 102, it implements the method described in any of the embodiments of the first aspect above. If the technical solution of this embodiment is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. The essence of the technical solution of this invention or its contribution to the prior art can be embodied in the form of a software product. This software product is stored in a storage medium and contains several instructions for causing a computer device (including a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the embodiments of this application. Applicable storage media include, but are not limited to, media capable of storing program code such as USB flash drives, mobile hard drives, ROM, RAM, disks, or optical discs.

[0063] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. A method for detecting road surface diseases based on a hybrid architecture, characterized in that, The method comprises the following steps: obtaining road surface image data; inputting the road surface image data into a reparameterization feature extraction network to extract multi-level feature representations, wherein the reparameterization feature extraction network is based on an HGNetV2 architecture, an RC_HGBlock module is formed by embedding a RepConv structure into an HGBlock, and a four-stage progressive feature pyramid structure is constructed by cascading and stacking multiple RC_HGBlock modules to realize feature extraction from low-level texture to high-level semantics; based on the high-level semantic layer integration of the feature pyramid structure, a spatial edge perception enhanced attention mechanism SEAM is used to enhance disease edge features through a dual-path complementary processing framework of an edge extraction path and a standard convolution path, and an improved C3K2-SEAM module is used as a feature fusion unit for multi-scale feature fusion to obtain enhanced fusion features; based on the enhanced fusion features, an end-to-end disease detection is performed to output the type, location and confidence information of the road surface disease.

2. The method for detecting road surface diseases based on hybrid architecture according to claim 1, characterized in that, The specific process of inputting the road surface image data into the reparameterization feature extraction network to extract multi-level feature representations comprises: inputting the obtained road surface image data into the input end of the RC-HGNetV2 network to perform preliminary feature extraction by a front feature extraction layer to obtain an initial feature map; inputting the initial feature map into a four-stage backbone network constructed by cascading and stacking multiple RC_HGBlock modules to perform deep feature extraction; after outputting each stage of the four-stage backbone network, a lightweight downsampling module constructed based on a depth separable convolution is used to perform dimension reduction and scale scaling processing on the feature map in sequence to generate four feature maps of different scales P2, P3, P4 and P5, thereby constructing the four-stage progressive feature pyramid structure to complete the multi-level feature representation extraction from low-level texture features to high-level semantic features.

3. The method of detecting road surface diseases based on hybrid architecture according to claim 2, wherein, The RC-HGNetV2 reparameterization feature extraction network is used to establish a feature pyramid structure among the four feature scales P2, P3, P4 and P5, wherein the stages of the feature pyramid structure are connected by lightweight downsampling modules to realize feature downsampling, the lightweight downsampling modules are constructed based on a depth separable convolution DWConv, and when processing input features, a grouped convolution operation is first performed on the spatial dimension of the features to extract spatial features, and a 1x1 point convolution is used to aggregate the spatial features on the channel dimension, thereby reducing the scale of the feature map while reducing the computational load of the model.

4. The method of detecting road surface diseases based on hybrid architecture according to claim 1, wherein, The specific process of integrating a spatial edge perception enhanced attention mechanism SEAM based on the high-level semantic layer of the feature pyramid structure to enhance disease edge features through a dual-path complementary processing framework of an edge extraction path and a standard convolution path comprises: extracting a high-level semantic feature map of P5 / 32 scale from the feature pyramid structure output by the reparameterization feature extraction network as the input of the spatial edge perception enhanced attention mechanism SEAM; inputting the high-level semantic feature map into the dual-path complementary processing framework for parallel processing; The edge gradient features output by the edge extraction path are spliced with the overall semantic features output by the standard convolution path in the channel dimension, and then the spliced features are reduced and integrated by a 1x1 convolution layer to obtain enhanced attention feature maps; The enhanced attention feature maps are added to the original input high-level semantic feature maps through residual connection to complete the enhancement of the disease edge features and linear structure features, and the feature maps processed by the SEAM mechanism are output.

5. The method of detecting road surface distress based on hybrid architecture as claimed in claim 4, wherein, The dual-path complementary processing framework includes: In the edge extraction path, a parameterized SobelConv convolution variant is used to perform convolution operation on the input features to explicitly extract the edge gradient features of the disease target; In the standard convolution path, a standard convolution operation is used to transform the input features to retain the overall semantic information and context relationship of the image.

6. The method of detecting road surface diseases based on hybrid architecture according to claim 1, wherein, The specific process of performing multi-scale feature fusion by using the improved C3K2-SEAM module as a feature fusion unit to obtain enhanced fusion features includes: The spatial edge perception enhanced attention mechanism SEAM is integrated into the C3K2 feature fusion module of YOLOv11 to construct a C3K2-SEAM module, and the C3K2-SEAM module is used as a core feature fusion unit of the neck network; The feature maps from different scales extracted by the reparameterized feature extraction network and the high-level semantic feature maps processed by the spatial edge perception enhanced attention mechanism SEAM are jointly input into the C3K2-SEAM module; Inside the C3K2-SEAM module, the multi-scale features input are cross-fused and information-interacted through the built-in multi-branch convolution structure to reconstruct the geometric continuity features of the pavement diseases in the high-level features; The fused features are output by the C3K2-SEAM module to obtain enhanced fusion features containing spatial details and semantic representation capabilities.

7. The method of detecting road surface diseases based on hybrid architecture according to claim 1, wherein, The specific process of performing end-to-end disease detection based on the enhanced fusion features to output the type, location, and confidence information of the pavement diseases includes: The enhanced fusion features output by the C3K2-SEAM module are input into a detection head module; In the detection head module, two types of calculations are performed in parallel based on the enhanced fusion features: in the first type of calculation, the probability of the existence of each type of disease at each preset anchor point is calculated through a classification branch, and in the second type of calculation, the class confidence is generated, the accurate offset and size parameters of the disease target bounding box at each preset anchor point are calculated through a regression branch to generate location information, and preliminary detection results are obtained; The preliminary detection results generated by the detection head module are screened and processed to remove redundancy, including: filtering out detection boxes with low confidence according to a preset confidence threshold, applying a non-maximum suppression algorithm to the remaining detection boxes to remove redundant detection boxes with high overlap, and retaining the optimal detection result for each disease instance. The final detection results after screening and redundancy removal are structured and output, and for each confirmed disease instance, the specific disease type, position coordinates in the image, and corresponding classification confidence are synchronously output.

8. A hybrid architecture based road surface distress detection system characterized in that, The method comprises the following steps: An image acquisition module is configured to acquire road surface image data. A re-parameterized feature extraction module is configured to input the road surface image data into a re-parameterized feature extraction network to extract multi-level feature representations, wherein the re-parameterized feature extraction network is based on an HGNetV2 architecture, forms RC_HGBlock modules by embedding a RepConv structure into an HGBlock, and constructs a four-stage progressive feature pyramid structure by cascading and stacking multiple RC_HGBlock modules to realize feature extraction from low-level textures to high-level semantics. An edge-aware enhancement and feature fusion module is configured to integrate a spatial edge-aware enhancement attention mechanism (SEAM) based on high-level semantic layers of the feature pyramid structure, enhance disease edge features through a dual-path complementary processing framework of an edge extraction path and a standard convolution path, and perform multi-scale feature fusion by using an improved C3K2-SEAM module as a feature fusion unit to obtain enhanced fusion features. A disease detection output module is configured to perform end-to-end disease detection based on the enhanced fusion features and output type, position, and confidence information of road surface diseases.

9. An electronic device comprising a memory, a processor, characterized in that, The processor is configured to execute programs in the memory to implement the road surface disease detection method based on the hybrid architecture according to any one of claims 1 to 7.

10. A storage medium containing computer-executable instructions, wherein: The storage medium of the computer executable instructions, when executed by the computer processor, is configured to execute the road surface disease detection method based on the hybrid architecture according to any one of claims 1 to 7.

Images