Pavement disease detection network model

Abstract

The invention discloses a pavement disease detection network model which comprises an encoder, an APM module, a DSConv module, an HFEM module, a GAM module and a decoder. The encoder is used for enhancing the feature extraction capability of the model so as to process image features under different scales; the APM module is used for enhancing feature representation by using an attention mechanism; the DSConv module is used for providing multi-scale feature representation for subsequent processing steps; the HFEM module is used for aggregating multi-level features of different deep levels, and redistributing attention weights based on an attention mechanism to highlight target features and reduce interference of redundant features on model performance; the GAM module is used for modeling crack characteristics under a long distance; and the decoder is used for recovering the low-resolution feature map subjected to multi-layer feature extraction and enhancement to a high resolution so as to generate a prediction result which is the same as or similar to the input image in size. The model can effectively capture subtle and complex pavement disease characteristics, and can improve the detection accuracy while maintaining the calculation efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of road surface detection, and in particular to a road surface disease detection network model. Background Art

[0002] Road disease detection is an important part of road maintenance work, and is of great significance for ensuring road safety and extending the service life of roads. Road disease detection generally includes the following aspects: road surface crack detection, road surface pothole detection, road surface rutting detection, and road surface looseness and peeling detection.

[0003] In the prior art, manual observation is generally performed by staff, but it cannot capture subtle and complex road disease characteristics, and the detection efficiency and accuracy are relatively low. Summary of the invention

[0004] The main purpose of this application is to provide a road surface disease detection network model, which aims to solve the technical problems in the existing technology that the subtle and complex road surface disease characteristics cannot be captured and the detection efficiency and accuracy are relatively low.

[0005] To achieve the above purpose, refer to Figure 1 The present application provides a pavement disease detection network model, including: an encoder, an APM module, a DSConv module, a HFEM module, a GAM module and a decoder; the encoder is used to enhance the feature extraction capability of the model to process image features at different scales; the APM module is connected to the encoder and is used to enhance feature representation using an attention mechanism; the DSConv module is connected to the APM module and is used to provide multi-scale feature representation for subsequent processing steps; the HFEM module is connected to the DSConv module and is used to aggregate multi-level features at different depths, and reallocate attention weights based on the attention mechanism to highlight target features and reduce the interference of redundant features on model performance; the GAM module is connected to the HFEM module and is used to model crack features at long distances; the decoder is connected to the GAM module and is used to restore low-resolution feature maps after multi-layer feature extraction and enhancement to high resolution, so as to generate prediction results of the same or similar size as the input image.

[0006] In some embodiments, the encoder is a DDRNet architecture, which enhances the feature extraction capability of the model through an FPN structure.

[0007] In some embodiments, the FPN structure includes a backbone network, a feature pyramid network and a skip connection structure; the backbone network consists of multiple convolutional layers for preliminarily extracting image features; the feature pyramid network connects multiple convolutional layers in parallel and applies convolution kernels of different sizes at each level to capture features of different scales; the skip connection structure is used to add high-level feature maps to low-level feature maps to achieve cross-layer transmission and fusion of information.

[0008] In some embodiments, the APM module includes a feature pyramid construction unit, an attention weight calculation unit, a feature weighted summation unit and a feature fusion unit; the feature pyramid construction unit is used to unify feature maps of different scales to the same size through upsampling or downsampling operations for subsequent fusion processing; the attention weight calculation unit is used to calculate the attention weight of each position; the feature weighted summation unit uses the calculated attention weight to perform weighted summation on the original feature map to obtain a new feature map; the feature fusion unit is used to pass the processed feature map through a series of convolutional layers again to further extract features and prepare to pass it to the DSConv module. In some embodiments, the DSConv module includes a dynamic convolution kernel generation unit, a multi-view feature extraction unit, a feature fusion unit and a parameter optimization unit; the dynamic convolution kernel generation unit is used to dynamically generate a convolution kernel according to the characteristics of the input feature map; the multi-view feature extraction unit is used to use the dynamically generated convolution kernel, DSConv extracts features from different spatial positions and scales; the feature fusion unit is used to fuse features extracted from different viewpoints to form a new feature map; the parameter optimization unit is used to optimize parameters during the training process through the back propagation algorithm to ensure that the model can learn the feature representation that best suits the current task. In some embodiments, the HFEM module includes a feature aggregation unit, an attention weight calculation unit, a feature recalibration unit and a feature refinement unit; the feature aggregation unit is used to aggregate feature maps of different levels; the attention weight calculation unit is used to calculate a set of attention weights; the feature recalibration unit is used to weight the aggregated feature maps using the calculated attention weights; the feature refinement unit is used to further refine the feature representation through a series of convolutional layers, and prepare to pass it to the GAM module.

[0009] In some embodiments, the GAM module includes a global average pooling unit, a fully connected layer conversion unit, a function activation unit, a feature reshaping unit, an attention weight calculation unit, a feature recalibration unit and a residual connection unit; the global average pooling unit is used to reduce the spatial dimension of each feature channel to 1x1 through a global average pooling operation; the fully connected layer conversion unit is used to map the reduced features to a new feature space using a fully connected layer; the function activation unit is used to increase the nonlinear capability of the model through ReLU or other nonlinear activation functions, so that the model can learn more complex features; the feature reshaping unit is used to reshape the activated features back to the original spatial dimensions, preparing for subsequent weighted calculations; the attention weight calculation unit is used to calculate the attention weight; the feature recalibration unit is used to use the calculated attention weight to perform weighted summation on the original feature map to obtain a globally optimized feature map H; the residual connection unit is used to pass part of the input directly to the output, combined with the feature map processed by the attention mechanism.

[0010] In some embodiments, the decoder includes an upsampling unit, a feature fusion unit, a convolutional layer stacking unit, a normalization layer unit and a generation unit; the upsampling unit is used to increase the spatial resolution of the feature map through unpooling, transposed convolution or other upsampling techniques; the feature fusion unit is used to combine the original high-resolution features from the encoder with the features of the current layer of the decoder through skip connections or feature fusion techniques; the convolutional layer stacking unit is used to increase the dimension of the feature map and refine the feature representation through multiple convolutional layers; it is used to insert batch normalization layers or instance normalization layers between multiple convolutional layers to stabilize the training process and improve model performance; the generation unit is used to generate the final prediction results through one or more output layers. A pavement disease detection network model proposed in an embodiment of the present application enhances the feature extraction capability of the model through an encoder to process image features at different scales; enhances feature representation through an APM module; provides multi-scale feature representation for subsequent processing steps through a DSConv module; aggregates multi-level features at different depths through an HFEM module, and reallocates attention weights based on an attention mechanism to highlight target features and reduce the interference of redundant features on model performance, so that the model has good generalization ability and can adapt to road surface conditions of different types and degrees, providing strong technical support for automated road maintenance and safety assessment; models crack features at long distances through a GAM module; and restores low-resolution feature maps after multi-layer feature extraction and enhancement to high resolution through a decoder, so that the pavement disease detection network model can effectively capture subtle and complex pavement disease features and improve detection accuracy while maintaining computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Attached Figure 1A schematic diagram of a module of a road surface disease detection network model provided in an embodiment of the present application; Attached Figure 2 A schematic diagram of the structure of a GAM module provided in one embodiment of the present application.

[0012] Attached Figure 3 A schematic diagram of the structure of an APM module provided in one embodiment of the present application.

[0013] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0014] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0015] Figure 1 A schematic diagram of a pavement disease detection network model provided in an embodiment of the present application, referring to Figure 1 The present application provides a pavement disease detection network model, including: an encoder, an APM module, a DSConv module, a HFEM module, a GAM module and a decoder; the encoder is used to enhance the feature extraction capability of the model to process image features at different scales; the APM module is connected to the encoder to enhance feature representation using an attention mechanism; the DSConv module is connected to the APM module to provide multi-scale feature representation for subsequent processing steps; the HFEM module is connected to the DSConv module to aggregate multi-level features at different depths, and reallocate attention weights based on the attention mechanism to highlight target features and reduce the interference of redundant features on model performance; the GAM module is connected to the HFEM module to model crack features at long distances; the decoder is connected to the GAM module to restore low-resolution feature maps after multi-layer feature extraction and enhancement to high resolution, so as to generate prediction results of the same or similar size as the input image.

[0016] A pavement disease detection network model proposed in an embodiment of the present application enhances the feature extraction capability of the model through an encoder to process image features at different scales; enhances feature representation through an APM module; provides multi-scale feature representation for subsequent processing steps through a DSConv module; aggregates multi-level features at different depths through an HFEM module, and reallocates attention weights based on an attention mechanism to highlight target features and reduce the interference of redundant features on model performance, so that the model has good generalization ability and can adapt to road surface conditions of different types and degrees, providing strong technical support for automated road maintenance and safety assessment; models crack features at long distances through a GAM module; and restores low-resolution feature maps after multi-layer feature extraction and enhancement to high resolution through a decoder, so that the pavement disease detection network model can effectively capture subtle and complex pavement disease features and improve detection accuracy while maintaining computational efficiency.

[0017] In an exemplary embodiment, the encoder is a DDRNet architecture, which enhances the feature extraction capability of the model through an FPN structure.

[0018] Specifically, this embodiment uses the initial encoder in the original DDRNet as the feature extraction network of the baseline model. DDRNet (Decoder-Decoder Residual Networks) is a deep learning architecture designed specifically to solve the problem of image scale changes. It enhances the model's feature extraction capability through the FPN (Feature Pyramid Network) structure, so that it can process image features at different scales.

[0019] Furthermore, the functions and advantages of the DDRNet encoder lie in scale invariance and feature enhancement. Among them, scale invariance: through the FPN structure, effective features can be extracted at multiple scales, making the model more robust to scale changes of the image; feature enhancement: jump connections help the model retain important information during the transmission process while reducing information loss. As the encoder part, DDRNet provides rich and multi-scale feature representations for subsequent processing steps, which is particularly important for processing tasks that require fine-scale perception such as road crack detection.

[0020] In an exemplary embodiment, the FPN structure includes a backbone network, a feature pyramid network and a skip connection structure; the backbone network consists of multiple convolutional layers for preliminarily extracting image features; the feature pyramid network connects multiple convolutional layers in parallel and applies convolution kernels of different sizes at each level to capture features of different scales; the skip connection structure is used to add high-level feature maps to low-level feature maps to achieve cross-layer transmission and fusion of information.

[0021] In an exemplary embodiment, the APM module includes a feature pyramid construction unit, an attention weight calculation unit, a feature weighted summation unit and a feature fusion unit; the feature pyramid construction unit is used to unify feature maps of different scales to the same size through upsampling or downsampling operations for subsequent fusion processing; the attention weight calculation unit is used to calculate the attention weight of each position; the feature weighted summation unit uses the calculated attention weight to perform weighted summation on the original feature map to obtain a new feature map; the feature fusion unit is used to pass the processed feature map through a series of convolutional layers again to further extract features and prepare to pass it to the DSConv module. Specifically, refer to Figure 3 The APM module is a multi-branch convolution module, which is used to fuse global and local features and efficiently extract multi-scale information. The dimension of the input feature map is c×h×w, and the dimension of the output feature map is 256×h×w. The module contains four branches to form a feature pyramid structure. The trunk branch first adjusts the number of channels through a 1×1 convolution, and then performs horizontal pooling and vertical pooling to capture the global information in the horizontal and vertical directions, and then calculates the attention weights in the two directions; the pooling results are enhanced by 3×3 convolutions and then fused (addition operation). The multi-scale branch contains three 3×3 convolutions of different depths: the first branch performs a single 3×3 convolution, the second branch performs two stacked 3×3 convolutions, and the third branch performs three stacked 3×3 convolutions to extract small, medium, and large receptive field features respectively. Finally, the outputs of all branches are spliced, fused and weighted in the channel direction, and then the number of channels is uniformly adjusted through a 1×1 convolution. This module combines global pooling and multi-scale convolution in a multi-branch design, which can efficiently extract and fuse global and local features.

[0022] Furthermore, by introducing the APM structure after the encoder, the pavement disease detection network model can better understand and analyze the crack characteristics in the image, laying a solid foundation for subsequent processing steps and making the model suitable for pavement crack detection tasks that require precise positioning and identification.

[0023] In an exemplary embodiment, the DSConv module includes a dynamic convolution kernel generation unit, a multi-view feature extraction unit, a feature fusion unit and a parameter optimization unit; the dynamic convolution kernel generation unit is used to dynamically generate a convolution kernel according to the characteristics of an input feature map; the multi-view feature extraction unit is used to utilize the dynamically generated convolution kernel, DSConv extracts features from different spatial positions and scales; the feature fusion unit is used to fuse features extracted from different viewpoints to form a new feature map; the parameter optimization unit is used to optimize parameters during the training process through a back-propagation algorithm to ensure that the model can learn the feature representation that best suits the current task. Specifically, the DSConv (Dynamic Snake Convolution) module is a module with an innovative convolution operation that can dynamically adjust the shape and size of the convolution kernel according to the characteristics of the input data. This flexibility makes DSConv particularly suitable for processing images with specific geometric structure features, such as tubular structures in road crack detection. By dynamically adjusting the convolution kernel, DSConv can fuse features from multiple perspectives and ensure the continuity of features.

[0024] Furthermore, the introduction of the DSConv module enhances the adaptability of the pavement defect detection network model: DSConv can adaptively adjust its convolution kernel according to the characteristics of the input data, which enables it to flexibly process features of various shapes and sizes; the introduction of the DSConv module enables the pavement defect detection network model to ensure feature continuity: through multi-perspective feature extraction and fusion, DSConv can ensure feature continuity, which is crucial for detecting coherent structures (such as road cracks).

[0025] In an exemplary embodiment, the HFEM module includes a feature aggregation unit, an attention weight calculation unit, a feature recalibration unit and a feature refinement unit; the feature aggregation unit is used to aggregate feature maps of different levels; the attention weight calculation unit is used to calculate a set of attention weights; the feature recalibration unit is used to weight the aggregated feature maps using the calculated attention weights; the feature refinement unit is used to further refine the feature representation through a series of convolutional layers, preparing to pass it to the GAM module.

[0026] Furthermore, the introduction of the HFEM module can make the pavement disease detection network model feature focused: allowing the model to focus on the most relevant features while ignoring redundant information that may cause confusion. In addition, the performance of the pavement disease detection network model can be improved: by optimizing the quality of feature representation, HFEM helps to improve the performance of the model on various tasks, especially in the presence of a large amount of noise or irrelevant information. Connecting the HFEM module after the DSConv module enables the pavement disease detection network model to more effectively integrate and utilize multi-level feature information, providing stronger support for the final task execution. The HFEM module makes the pavement disease detection network model particularly suitable for application scenarios that require extracting key information from complex data, and pavement crack detection tasks that require precise positioning and identification.

[0027] In an exemplary embodiment, the GAM module includes a global average pooling unit, a fully connected layer conversion unit, a function activation unit, a feature reshaping unit, an attention weight calculation unit, a feature recalibration unit and a residual connection unit; the global average pooling unit is used to reduce the spatial dimension of each feature channel to 1x1 through a global average pooling operation; the fully connected layer conversion unit is used to map the reduced features to a new feature space using a fully connected layer; the function activation unit is used to increase the nonlinear ability of the model through ReLU or other nonlinear activation functions, so that the model can learn more complex features; the feature reshaping unit is used to reshape the activated features back to the original spatial dimensions in preparation for subsequent weighted calculations; the attention weight calculation unit is used to calculate the attention weight; the feature recalibration unit is used to use the calculated attention weight to perform weighted summation on the original feature map to obtain a globally optimized feature map H; the residual connection unit is used to pass a part of the input directly to the output and combine it with the feature map processed by the attention mechanism.

[0028] Specifically, refer to Figure 2 , the GAM module combines the channel attention module and spatial attention module Implement input features The global optimization of First, the channel attention module performs global pooling on the input features to compress the spatial dimension and generate a channel description vector; then the channel weights are generated through the fully connected layer and applied to the Next, the spatial attention module further processes the channel-weighted features, first generating a spatial description (compressing the channel dimension) through full-channel average pooling or maximum pooling, then generating spatial weights through convolution, and applying pixel-by-pixel weighting to the channel-weighted features. Finally, through the series of channel attention and spatial attention weighting operations, the GAM module efficiently fuses global and local features and outputs features. The dimensions remain unchanged.

[0029] Furthermore, the introduction of the GAM module structure enables the network model of the invention to enhance global context perception: GAM can effectively capture the global context information in the image, which is crucial for understanding the complex structures and patterns in the image; the introduction of the GAM module structure improves the performance of the network model of the invention: by integrating global information, GAM helps to improve the performance of the model on various visual tasks, especially in tasks that require understanding scene-level context.

[0030] In an exemplary embodiment, the decoder includes an upsampling unit, a feature fusion unit, a convolutional layer stacking unit, a normalization layer unit and a generation unit; the upsampling unit is used to increase the spatial resolution of the feature map through unpooling, transposed convolution or other upsampling techniques; the feature fusion unit is used to combine the original high-resolution features from the encoder with the features of the current layer of the decoder through skip connections or feature fusion techniques; the convolutional layer stacking unit is used to increase the dimension of the feature map and refine the feature representation through multiple convolutional layers; it is used to insert batch normalization layers or instance normalization layers between multiple convolutional layers to stabilize the training process and improve model performance; the generation unit is used to generate the final prediction results through one or more output layers. Furthermore, the introduction of the decoder structure enables the resolution of the pavement disease detection network model to be restored: the decoder can restore low-resolution feature maps to high resolution, which is crucial for pixel-level prediction tasks; the introduction of the decoder structure enables the model to retain details: through feature fusion technology, the decoder can retain important spatial details while restoring resolution; the introduction of the decoder structure enhances the flexibility of the model: the design of the decoder can be adjusted according to different task requirements, and the number of prediction categories can be adapted by changing the settings of the output layer. The decoder structure is introduced at the end of the pavement disease detection network model, so that the model can accurately restore the spatial structure of the input data while maintaining high-level semantic information, thereby achieving high-quality prediction results for pavement disease detection and recognition.

[0031] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

[0032] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

Claims

1. A road surface disease detection network model, characterized in that: include: Encoder, APM module, DSConv module, HFEM module, GAM module and decoder; The encoder is used to enhance the feature extraction capability of the model to process image features at different scales; The APM module is connected to the encoder and is used to enhance feature representation using an attention mechanism; The DSConv module is connected to the APM module to provide multi-scale feature representation for subsequent processing steps; The HFEM module is connected to the DSConv module to aggregate multi-level features at different depths and redistribute attention weights based on the attention mechanism to highlight target features and reduce the interference of redundant features on model performance; The GAM module is connected to the HFEM module and is used to model the crack characteristics at a long distance; The decoder is connected to the GAM module and is used to restore the low-resolution feature map after multi-layer feature extraction and enhancement to a high resolution so as to generate a prediction result of the same or similar size as the input image.

2. The pavement disease detection network model according to claim 1, characterized in that: The encoder is a DDRNet architecture, and the DDRNet architecture enhances the feature extraction capability of the model through the FPN structure.

3. The pavement disease detection network model according to claim 2, characterized in that: The FPN structure includes a backbone network, a feature pyramid network and a skip connection structure; The backbone network consists of multiple convolutional layers for preliminarily extracting image features; The feature pyramid network captures features of different scales by connecting multiple convolutional layers in parallel and applying convolution kernels of different sizes at each level; The jump connection structure is used to add the high-level feature map to the low-level feature map to achieve cross-layer transmission and fusion of information.

4. The pavement disease detection network model according to claim 1, characterized in that: The APM module includes a feature pyramid construction unit, an attention weight calculation unit, a feature weighted summation unit and a feature fusion unit; The feature pyramid construction unit is used to unify feature maps of different scales into the same size through upsampling or downsampling operations, so as to perform subsequent fusion processing; The attention weight calculation unit is used to calculate the attention weight of each position; The feature weighted summation unit uses the calculated attention weight to perform weighted summation on the original feature map to obtain a new feature map; The feature fusion unit is used to pass the processed feature map through a series of convolutional layers again to further extract features and prepare to pass them to the DSConv module.

5. The pavement disease detection network model according to claim 1, characterized in that: The DSConv module includes a dynamic convolution kernel generation unit, a multi-view feature extraction unit, a feature fusion unit and a parameter optimization unit; The dynamic convolution kernel generation unit is used to dynamically generate a convolution kernel according to the characteristics of the input feature map; The multi-view feature extraction unit is used to extract features from different spatial positions and scales using dynamically generated convolution kernels, DSConv; The feature fusion unit is used to fuse the features extracted from different perspectives to form a new feature map; The parameter optimization unit is used to optimize parameters during the training process through a back-propagation algorithm to ensure that the model can learn the feature representation that is most suitable for the current task.

6. The pavement disease detection network model according to claim 1, characterized in that: The HFEM module includes a feature aggregation unit, an attention weight calculation unit, a feature recalibration unit and a feature refinement unit; The feature aggregation unit is used to aggregate feature maps at different levels; The attention weight calculation unit is used to calculate a set of attention weights; The feature recalibration unit is used to weight the aggregated feature map using the calculated attention weight; The feature refinement unit is used to further refine the feature representation through a series of convolutional layers, ready to be passed to the GAM module.

7. The pavement disease detection network model according to claim 1, characterized in that: The GAM module includes a global average pooling unit, a fully connected layer conversion unit, a function activation unit, a feature reshaping unit, an attention weight calculation unit, a feature recalibration unit and a residual connection unit; The global average pooling unit is used to reduce the spatial dimension of each feature channel to 1x1 through a global average pooling operation; The fully connected layer conversion unit is used to map the reduced-dimensional features to a new feature space using a fully connected layer; The function activation unit is used to increase the nonlinear ability of the model through ReLU or other nonlinear activation functions, so that the model can learn more complex features; The feature reshaping unit is used to reshape the activated features back to the original spatial dimensions in preparation for subsequent weighted calculations; The attention weight calculation unit is used to calculate the attention weight; The feature recalibration unit is used to perform weighted summation on the original feature map using the calculated attention weights to obtain a globally optimized feature map H; The residual connection unit is used to pass part of the input directly to the output, combined with the feature map processed by the attention mechanism.

8. The pavement disease detection network model according to claim 1, characterized in that: The decoder includes an upsampling unit, a feature fusion unit, a convolutional layer stacking unit, a normalization layer unit and a generation unit; The upsampling unit is used to increase the spatial resolution of the feature map by unpooling, transposed convolution or other upsampling techniques; The feature fusion unit is used to combine the original high-resolution features from the encoder with the features of the current layer of the decoder through a skip connection or a feature fusion technique; The convolutional layer stacking unit is used to increase the dimension of the feature map and refine the feature representation through multiple convolutional layers; The normalization layer unit is used to insert a batch normalization layer or an instance normalization layer between the plurality of convolutional layers to stabilize the training process and improve model performance; The generating unit is used to generate a final prediction result through one or more output layers.

Images