Rapid detection method for roadbed retaining structure diseases based on YOLOv9

Abstract

The invention discloses a YOLOv9-based roadbed retaining structure disease rapid detection method, which at least comprises a dynamic feature extraction module STU, the dynamic feature extraction module STU receives an image sequence of continuous frames of a data set to be detected at an input layer, and in the dynamic feature extraction process, a sliding window mechanism, a time attention mechanism and dynamic convolution are combined, so that the image sequence of the continuous frames of the data set to be detected is obtained. And finally, the extracted features are fused with the features of the Swin Transform. According to the method, Conv is replaced by an expansion convolution module DC2f, so that the resolution of a feature map is ensured, and the extraction capability of multi-scale features is enhanced; and by designing a dynamic feature extraction module DTU, weighted fusion is carried out on features among different time frames, and the capturing capability of key features is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of railway roadbed support structure disease detection technology, and in particular to a roadbed support structure disease rapid detection method based on YOLOv9. Background Art

[0002] Detecting defects in railway subgrade retaining structures faces numerous challenges. Traditionally, inspections are conducted manually, which is slow, inaccurate, and relies on the subjective judgment of inspectors. Therefore, efficient inspections are typically conducted on railway inspection vehicles. However, due to the high speed of these vehicles, the captured images are often blurred, making it difficult to clearly capture defect features. Feature information is often weak in image processing and easily lost or missed. Existing detection algorithms also struggle to balance accuracy and model size, often forcing a compromise between real-time performance and detection performance. Summary of the Invention

[0003] The purpose of this invention is to provide a method for rapid detection of roadbed retaining structure defects based on YOLOv9 in order to solve the above problems.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] A YOLOv9-based rapid detection method for roadbed retaining structure defects includes at least a dynamic feature extraction module (STU). The STU receives an image sequence of consecutive frames of a test dataset at an input layer, then uses a dynamic feature extraction process combined with a sliding window mechanism, a temporal attention mechanism, and dynamic convolution to fuse the extracted features with features from a Swin Transformer.

[0006] The temporal attention mechanism formula is:

[0007] Among them, Q t =W t W Q , W t =I t [x:x+W s ,y:y+H s ], (x, y) represents the starting coordinates of the window, and the sliding window size is W s ·H s , I t is the input image sequence; K t =W t W k , W Q and W k is the learned weight;

[0008] The feature extraction formula through the time attention mechanism is F t =α t ·V t ;

[0009] Where V t is a value vector, the formula is V t =W t W V ;

[0010] Combining the features of all time frames, the formula is

[0011] Among them, for the dynamic convolution kernel, the generation formula is K t =g(I t ,θ);

[0012] Among them, θ represents the parameters of the dynamic convolution kernel generator, which is responsible for mapping the input image to the convolution kernel space;

[0013] For the adaptive convolution operation, the formula O is generated t =I t *k t ;

[0014] Among them, O t Represents the output feature map of the convolution operation, k t Represents the dynamically generated convolution kernel, size is k×k;

[0015] The dynamic features are fused with the Swin Transformer features, and the formula is F fused =Concat(F dy ,F swin );

[0016] Finally, mapping is performed through a fully connected layer, the formula is F output =FC(F fused ).

[0017] As an improvement to the above technical solution, the YOLOv9-based rapid detection method for roadbed support structure defects as described in claim 1 is characterized in that the module used for the dynamic convolution is a DC2f module, which extracts features from the input feature map through two different expansion rates to obtain two branch outputs, and then fuses the branch outputs to output the fused features.

[0018] As an improvement to the above technical solution, the image sequence of continuous frames comes from a roadbed support structure disease data set; the roadbed support structure disease refers to cracks, spalling, and collapse data at different vehicle speeds.

[0019] As an improvement to the above technical solution, the roadbed retaining structure disease refers to crack, spalling and collapse data collected at vehicle speeds of 15-30km / h, 30-40km / h and 40-60km / h.

[0020] As an improvement to the above technical solution, the roadbed retaining structure disease data set also includes normal sample data, and the number of normal sample data matches the number of crack, spalling and collapse data.

[0021] As an improvement to the above technical solution, the method for rapid detection of roadbed retaining structure defects also includes evaluation, and the evaluation formula is:

[0022] AP=∫P(R)d(R)

[0023]

[0024] Where P represents the ratio of the number of correct positive objects detected to the number of all positive objects detected (including correct and false ones).

[0025] As an improvement to the above technical solution, the correct positive targets and all positive targets come from test data with vehicle speeds of 15-30km / h, 30-40km / h, and 40-60km / h.

[0026] Compared with the prior art, the present invention includes but is not limited to the following advantages and positive effects:

[0027] The present invention provides a method for rapid detection of roadbed retaining structure defects based on YOLOv9. Images of retaining structures at different speeds are collected according to actual detection conditions, which contributes to the robustness of the model. In the auxiliary branch of YOLOv9, Conv is replaced with the dilated convolution module DC2f, which ensures the resolution of the feature map and enhances the ability to extract multi-scale features. When identifying roadbed retaining structure defects, the model shows certain differences depending on the vehicle speed, which is fundamentally due to the feature sparseness, information loss and confusion caused by the blurring of the image frame. The newly designed module STU, by designing the dynamic feature extraction module STU, performs weighted fusion of features between different time frames, thereby enhancing the ability to capture key features. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1It is a schematic diagram of the fusion process of the present invention;

[0030] Figure 2 Schematic diagram of the D2f structure of the present invention;

[0031] Figure 3 Schematic diagram of the fusion process of the Swin Transformer module and the dynamic feature extraction module STU. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts, any modifications, equivalent replacements, improvements, etc., shall be included in the scope of protection of the present invention.

[0033] The YOLO algorithm is an object detection algorithm that makes predictions based on global image information. Different iterations can be selected depending on the application scenario. This work builds a YOLOv9 model and improves its network to enhance the model's performance in detecting small and blurred objects in fast-moving conditions. These improvements include: 1) constructing a new Transformer architecture (STU) and replacing the CBLinear architecture with the STU; and 2) replacing the Conv module (standard convolutional layer, used for channel adjustment and feature refinement) with a DC2f architecture in the multi-level auxiliary branches.

[0034] The YOLOv9 network has a main branch and auxiliary branches. The main branch is the same as the traditional YOLO structure. The reason for the major changes in the auxiliary branch is threefold: 1) In the original auxiliary branch, although the ordinary convolution operation can extract image information in detail, it also has the problem of redundant features. In order to solve the problem of information dilution and feature loss of fuzzy and small diseases during the detection process, it is replaced by dilated convolution (structured as Figure 2 The introduction of dilated convolution enables the convolution kernel to capture features over a larger spatial range without adding additional parameters or computational overhead, while maintaining the resolution of the feature map and enhancing the ability to extract multi-scale features. Furthermore, the application of the DC2f architecture further enhances the model's feature extraction and expression capabilities, achieving greater flexibility and adaptability through modular design.

[0035] 2) The design of the auxiliary structure can effectively avoid the semantic loss caused by deep supervision in the traditional multi-path feature integration process. However, this structure may also have an adverse effect on the training and inference speed of the model. The original YOLOv9 network contains 6 detection heads, while previous algorithms in the YOLO series usually only use 3 detection heads, which may significantly reduce the speed during the inference phase. The original design of YOLOv9 was to solve the problem of feature loss during the layer-by-layer extraction process, so the auxiliary structure must remain lightweight. In this improvement, we reduced the number of detection heads in the auxiliary structure to 3 to optimize the inference speed of the model.

[0036] 3) When the vehicle is moving, the image quality is closely related to the speed: the lower the speed, the higher the input image quality and the more obvious the captured features; the higher the speed, the lower the input quality, resulting in a weakened detection capability. For this reason, an innovative dynamic feature extraction module STU is designed this time, which aims to enhance the ability to capture key features by weighted fusion of features between different time frames. The module first receives an image sequence of continuous frames of the test data set at the input layer, and then through the dynamic feature extraction process, combined with the sliding window mechanism, temporal attention mechanism and dynamic convolution, finally fuses the extracted features with the features of the Swin Transformer. This method can effectively improve the performance of the model in a dynamic environment and enhance the ability to extract multi-scale and key features. Among them, the temporal attention mechanism formula is as follows:

[0037]

[0038] Among them, Q t =W t W Q , W t =I t [x:x+W s ,y:y+H s ], (x, y) represents the starting coordinates of the window, and the sliding window size is W s ·H s , I t is the input image sequence. K t =W t W k , W Q and W k is the learned weight. The feature is extracted through the time attention mechanism, and the formula is as follows:

[0039] F t =α t ·V t ;

[0040] Where V t is a value vector, the formula is:

[0041] V t =W t W V ;

[0042] Then combine the features of all time frames, the formula is as follows:

[0043]

[0044] Among them, for the dynamic convolution kernel, the generation formula is as follows:

[0045] K t =g(I t ,θ);

[0046] Among them, θ represents the parameters of the dynamic convolution kernel generator, which is responsible for mapping the input image to the convolution kernel space. For the adaptive convolution operation, the generation formula is as follows:

[0047] O t =I t *k t ;

[0048] Among them, O t Represents the output feature map of the convolution operation, k t Represents a dynamically generated convolution kernel of size k × k. In summary, the dynamic convolution kernel generation designed in this paper is to generate a convolution kernel adapted to the current input through the input image and learnable parameters, and the adaptive convolution operation uses the dynamically generated convolution kernel to convolve the input image to extract features.

[0049] Finally, feature fusion is performed to fuse the dynamic features with the Swin Transformer features. The formula is as follows:

[0050] F fused =Concat(F dy ,F swin );

[0051] Finally, mapping is performed through a fully connected layer. The formula is as follows:

[0052] F output =FC(F fused );

[0053] At this point, the dynamic feature extraction module STU designed this time is completed, and the feature extraction method of Swin Transformer itself will not be repeated. Figure 3 It represents the fusion process of the Swin Transformer module and the dynamic feature extraction module STU.

[0054] The dynamic feature extraction module (STU) is a structure that adaptively adjusts feature extraction methods. Its core idea is to dynamically change the convolution kernel, weights, or channel selection based on the input image or context to adapt to the scale, position, or texture characteristics of the target. Incorporating this module improves the model's adaptability to multi-scale and multi-category objects, especially in scenes with densely populated objects or complex backgrounds.

[0055] The Swin Transformer is a hierarchical visual transformer widely used in tasks such as image classification, detection, and segmentation. Fusion of the Swin Transformer can enhance global modeling capabilities.

[0056] The dilated convolution module DC2f (Dilated Convolution with Double-scale and Fusion) extracts richer contextual information, expands the receptive field, and takes into account small target details, achieving stronger feature expression through dual-scale fusion.

[0057] Processing process:

[0058] 1. Input feature map X.

[0059] 2. Branch processing (Split+Conv), extract features through two convolution branches with different dilation rates (such as d=1 and d=3).

[0060] 3. Enter two BottleNecks (depth information extraction) and obtain two branch outputs Y1 and Y2.

[0061] 4. Concatenate the two feature maps (Concat) and fuse them (weighted sum, concat, SE attention fusion, etc.).

[0062] 5. Output the fused feature Y (multi-scale context feature map).

[0063] The STU module (Scale-aware Transformer Unit) combines the local modeling capabilities of CNN and the global modeling capabilities of Transformer, with a particular emphasis on the ability to model objects of different scales.

[0064] Processing process:

[0065] 1. Input feature map X.

[0066] 2. Perform multi-scale branching on the input (possibly down-sampled features at different resolutions).

[0067] 3. Each scale branch enters the local Transformer module for modeling (such as window attention).

[0068] 4. Use cross-scale fusion mechanisms (such as attention, weighted summation) to unify the output.

[0069] 3. Evaluate the quality of the adjusted model through AP and FPS

[0070] AP = ∫P(R)d(R);

[0071]

[0072] Where P represents the ratio of the number of correctly detected positive objects to the number of all detected positive objects (including correct and false positives). The two-parameter test was also divided into three parts: the vehicle speed was 15-30, 30-40, and 40-60.

[0073] True Positives (TP): The number of positive targets predicted by the model that have a sufficiently high IoU overlap with the true targets and whose category predictions are correct.

[0074] Ground Truth Positives: refers to all target (positive) instances that actually exist in the dataset, that is, the number of real boxes belonging to the target category in the annotated box.

[0075] IoU (Intersection over Union): The ratio of the intersection and union between the predicted box (Predicted Box) and the true box (GroundTruth Box):

[0076] Preprocessing Time: The time required to convert the raw input data (image / video) to the required format, scale, and normalize it for model input.

[0077] Inference Time: The time it takes to input preprocessed data into the neural network, perform forward propagation, and output the prediction results.

[0078] Postprocessing Time:

[0079] The time it takes to filter and process the raw output of the model to ultimately form usable detection boxes, categories, etc. Post-processing determines the quality of the final model output, especially NMS, which has a significant impact on detection performance.

[0080] This paper collects images of retaining structures at different speeds based on actual detection conditions, which contributes to the robustness of the model. 2. In the auxiliary branch of YOLOv9, the dilated convolution module DC2f replaces Conv with the auxiliary branch, ensuring the resolution of the feature map and enhancing the ability to extract multi-scale features. When identifying roadbed retaining structure defects, the model exhibits certain differences depending on the vehicle speed. The fundamental reason is that the blurring of the image frame leads to feature thinning, information loss, and confusion. The newly designed module STU, through the design of the dynamic feature extraction module STU, performs weighted fusion of features between different time frames, enhancing the ability to capture key features.

[0081] Specific operation examples:

[0082] 1. Experimental environment:

[0083] CPU: Intel 16-core processor @ 3.40GHz

[0084] GPU: NVIDIA GeForce RTX 4060

[0085] Video memory: 16G

[0086] CUDA: 11.6

[0087] Epoch: 400

[0088] Batchsize: 32

[0089] Image input resolution: 1080*960.

[0090] 2. Experimental Results

[0091] Test set: 30 images captured at each of three speed ranges.

[0092] Speed: 15-30 km / h

[0093] crack peeling collapse YOLOv9 95.2 93.5 97.3 Improved algorithm 96.1 93.7 98.0

[0094] Speed: 30-40 km / h

[0095] crack peeling collapse YOLOv9 92.7 89.6 94.3 Improved algorithm 94.3 92.4 95.9

[0096] Speed: 40-60 km / h

[0097] crack peeling collapse YOLOv9 87.5 84.4 88.3 Improved algorithm 93.8 91.1 94.6

[0098] Improved network FPS: 98.

[0099] YOLOv9 network FPS: 88.

[0100] It can be seen that as the vehicle speed increases, the detection effect decreases significantly. Therefore, the algorithm proposed in this invention can effectively reduce the impact of the detection accuracy caused by the increase in vehicle speed.

[0101] This paper designs a new dynamic feature extraction module, STU, which aims to enhance the ability to capture key features by weighted fusion of features across different time frames. By introducing the STU module and the DC2f module and reducing the number of detection heads, a new YOLOv9 algorithm is constructed, specifically optimized for retaining structure detection during driving. The constructed algorithm demonstrates high detection accuracy and speed across a driving speed range of 15 to 60 km / h.

Claims

1. A YOLOv9-based method for rapid detection of roadbed retaining structure defects, characterized by: At least includes a dynamic feature extraction module STU, which receives an image sequence of consecutive frames of the test data set at the input layer, combines the sliding window mechanism, the temporal attention mechanism and the dynamic convolution in the dynamic feature extraction process, and finally fuses the extracted features with the features of the Swin Transformer; The temporal attention mechanism formula is: Among them, Q t =W t W Q , W t =I t [x:x+W s ,y:y+H s ], (x, y) represents the starting coordinates of the window, and the sliding window size is W s ·H s , I t is the input image sequence; K t =W t W k , W Q and W k is the learned weight; Features are extracted through the time attention mechanism, the formula is F t =α t ·V t ; Where V t is a value vector, the formula is V t =W t W V ; Combining the features of all time frames, the formula is Among them, for the dynamic convolution kernel, the generation formula is K t =g(I t ,θ); Among them, θ represents the parameters of the dynamic convolution kernel generator, which is responsible for mapping the input image to the convolution kernel space; For the adaptive convolution operation, the formula O is generated t =I t *k t ; Among them, O t Represents the output feature map of the convolution operation, k t Represents the dynamically generated convolution kernel, size is k×k; The dynamic features are fused with the Swin Transformer features, and the formula is F fused =Concat(F dy ,F swin ); Finally, mapping is performed through a fully connected layer, the formula is F output =FC(F fused ).

2. The YOLOv9-based rapid detection method for roadbed retaining structure defects according to claim 1 is characterized by: The module used in the dynamic convolution is the DC2f module, which extracts features from the input feature map through two different expansion rates to obtain two branch outputs, and then fuses the branch outputs to output the fused features.

3. The YOLOv9-based rapid detection method for roadbed retaining structure defects according to claim 1 is characterized by: The image sequence of continuous frames comes from a roadbed support structure disease dataset; the roadbed support structure disease refers to cracks, spalling, and collapse data at different vehicle speeds.

4. The YOLOv9-based rapid detection method for roadbed retaining structure defects according to claim 3 is characterized by: The roadbed retaining structure diseases refer to cracks, spalling and collapse data collected at vehicle speeds of 15-30km / h, 30-40km / h and 40-60km / h.

5. The YOLOv9-based rapid detection method for roadbed retaining structure defects according to claim 3 is characterized by: The roadbed retaining structure disease data set also includes normal sample data, and the number of normal sample data matches the number of crack, spalling and collapse data.

6. The YOLOv9-based rapid detection method for roadbed retaining structure defects according to claim 1, characterized in that: The method for rapid detection of roadbed retaining structure defects also includes a quality evaluation after model adjustment, and the evaluation formula is: AP = ∫P(R)d(R); Where P represents the ratio of the number of correctly detected positive targets to the number of all positive targets detected.

7. The YOLOv9-based rapid detection method for roadbed retaining structure defects according to claim 6 is characterized by: The correct positive targets and all positive targets come from test data with vehicle speeds of 15-30km / h, 30-40km / h, and 40-60km / h.

Images