A subgrade retaining structure disease rapid detection method based on YOLOv9

Abstract

The application discloses a kind of based on YOLOv9's embankment retaining structure disease rapid detection method, at least including dynamic feature extraction module STU, the dynamic feature extraction module STU receives the image sequence of the continuous frame of data set to be measured in input layer, in dynamic feature extraction process, in combination with sliding window mechanism, time attention mechanism and dynamic convolution, finally the features extracted are fused with the features of Swin Transformer.The application replaces Conv as expansion convolution module DC2f, guarantees the resolution of feature map, enhances the extraction ability of multi-scale feature;And by designing dynamic feature extraction module DTU, the feature is weighted and fused between different time frames, and the capture ability of key feature is enhanced.

Description

Technical Field

[0001] This invention relates to the field of railway subgrade retaining structure defect detection technology, and in particular to a rapid detection method for subgrade retaining structure defects based on YOLOv9. Background Technology

[0002] The detection of defects in railway subgrade retaining structures faces numerous challenges. Detection methods are typically manual, which is slow, inaccurate, and reliant on the subjective judgment of inspectors. Therefore, efficient inspections are usually conducted on railway inspection vehicles. However, due to the high speed of these vehicles, the acquired images are often blurry, making it difficult to clearly capture defect features. Feature information is often weak in image processing, easily lost or missed. Existing detection algorithms also struggle to balance accuracy and model size, often requiring compromises between real-time performance and detection capabilities. Summary of the Invention

[0003] The purpose of this invention is to address the aforementioned problems by providing a rapid detection method for roadbed retaining structure defects based on YOLOv9.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows:

[0005] A method for rapid detection of roadbed retaining structure defects based on YOLOv9 includes at least a dynamic feature extraction module (STU). The dynamic feature extraction module (STU) receives a sequence of consecutive frames of the dataset to be tested at the input layer, and then, through a dynamic feature extraction process, combines a sliding window mechanism, a temporal attention mechanism, and dynamic convolution, and finally fuses the extracted features with the features of the Swing Transformer.

[0006] The formula for the time attention mechanism is:

[0007] Among them, Q t =W t ·W Q W t =I t [x:x+W s ,y:y+H s ], (x,y) represents the starting coordinates of the window, and the sliding window size is W. s ·H s I t The input image sequence; K t =W t ·W k W Q and W k The learned weights;

[0008] The formula for feature extraction using the time attention mechanism is F. t =α t ·V t ;

[0009] Where V t For a value vector, the formula is V t =W t ·W V ;

[0010] Combining the features of all time frames, the formula is:

[0011] For dynamic convolution kernels, the generation formula is K. t =g(I t ,θ);

[0012] Where θ represents the parameters of the dynamic convolution kernel generator, which is responsible for mapping the input image to the convolution kernel space;

[0013] For adaptive convolution operations, the generating formula O is... t =I t *k t ;

[0014] Among them, O t k represents the output feature map of the convolution operation. t This represents a dynamically generated convolutional kernel of size k×k;

[0015] The dynamic features are fused with the Swing Transformer features, and the formula is F. fused =Concat(F dy ,F swin );

[0016] Finally, a fully connected layer is used for mapping, with the formula F. output =FC(F fused ).

[0017] As an improvement to the above technical solution, the method for rapid detection of roadbed retaining structure defects based on YOLOv9 as described in claim 1 is characterized in that: the module used for dynamic convolution is the DC2f module, which extracts features from the input feature map through two different dilation rates to obtain two branch outputs, and then fuses the branch outputs to output the fused features.

[0018] As an improvement to the above technical solution, the continuous frame image sequence comes from the roadbed retaining structure defect dataset; the roadbed retaining structure defects refer to crack, spalling, and collapse data at different vehicle speeds.

[0019] As an improvement to the above technical solution, the roadbed retaining structure defects refer to the crack, spalling, and collapse data collected at vehicle speeds of 15-30km / h, 30-40km / h, and 40-60km / h.

[0020] As an improvement to the above technical solution, the roadbed retaining structure defect data also includes normal sample data, and the number of normal sample data matches the number of crack, spalling, and collapse data.

[0021] As an improvement to the above technical solution, the rapid detection method for roadbed retaining structure defects also includes an evaluation, the evaluation formula of which is:

[0022] AP=∫P(R)d(R)

[0023]

[0024] Where P represents the ratio of the number of correctly detected positive targets to the total number of all detected positive targets (including both correct and incorrect ones).

[0025] As an improvement to the above technical solution, the correct positive target and all positive targets are derived from test data at vehicle speeds of 15-30km / h, 30-40km / h, and 40-60km / h.

[0026] Compared with the prior art, the present invention includes, but is not limited to, the following advantages and positive effects:

[0027] This invention provides a rapid detection method for roadbed retaining structure defects based on YOLOv9. Images of retaining structures at different speeds were collected based on actual detection conditions, which improves the robustness of the model. The auxiliary branch of YOLOv9, Conv, is replaced with a dilated convolution module DC2f, ensuring the resolution of the feature maps and enhancing the ability to extract multi-scale features. When identifying roadbed retaining structure defects, the model exhibits certain differences depending on the vehicle speed, fundamentally due to feature thinning, information loss, and distortion caused by image frame blurring. The newly designed module STU, through its dynamic feature extraction module STU, performs weighted fusion of features across different time frames, enhancing the ability to capture key features. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1This is a simplified schematic diagram of the fusion process of the present invention;

[0030] Figure 2 This is a schematic diagram of the D2f structure of the present invention;

[0031] Figure 3 This is a schematic diagram illustrating the fusion process between the Swing Transformer module and the dynamic feature extraction module STU. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art to all other embodiments obtained without creative effort should be included within the protection scope of the present invention.

[0033] The YOLO algorithm is a target detection algorithm that predicts based on global image information. Different iterative versions can be selected according to different application scenarios. This paper constructs a YOLOv9 model and improves its network to enhance the model's performance in detecting small and blurred targets under fast-moving conditions. The improvements include: 1) constructing a new Transformer structure STU and replacing CBLinear with STU; 2) replacing the Conv module (standard convolutional layer, used for channel adjustment, feature refinement, etc.) with DC2f in the multi-level auxiliary branches.

[0034] The YOLOv9 network has a main branch and auxiliary branches. The main branch is structurally identical to the traditional YOLO network. The major changes to the auxiliary branches are for three reasons: 1) While the original auxiliary branches used ordinary convolution operations to extract image information meticulously, they also suffered from redundant features. To address the information dilution and feature loss issues associated with blurred and small defects during detection, this branch was replaced with dilated convolution (structure as shown in the diagram). Figure 2 (As shown). The introduction of dilated convolution allows the convolution kernel to capture features over a larger spatial range without adding extra parameters or computational overhead, while maintaining the resolution of the feature map and enhancing the ability to extract multi-scale features. Furthermore, the application of the DC2f structure further improves the model's feature extraction and expressive capabilities, achieving greater flexibility and adaptability through modular design.

[0035] 2) The design of the auxiliary structure effectively avoids the semantic loss caused by deep supervision in traditional multi-path feature integration. However, this structure may also adversely affect the training and inference speed of the model. The original YOLOv9 network contains 6 detector heads, while previous algorithms in the YOLO series typically only use 3 detector heads, which may significantly reduce speed during the inference stage. YOLOv9 was designed to address the problem of feature loss during layer-by-layer extraction, therefore the auxiliary structure must remain lightweight. In this improvement, we reduced the number of detector heads in the auxiliary structure to 3 to optimize the model's inference speed.

[0036] 3) During vehicle movement, image quality is closely related to vehicle speed: the lower the speed, the higher the input image quality and the more obvious the captured features; the higher the speed, the lower the input quality, leading to a weakening of detection capability. To address this, we designed an innovative dynamic feature extraction module (STU) to enhance the capture of key features by weighted fusion of features across different time frames. This module first receives a sequence of consecutive frames from the test dataset at the input layer. Then, through a dynamic feature extraction process, it combines a sliding window mechanism, a temporal attention mechanism, and dynamic convolution to finally fuse the extracted features with the features of the Swin Transformer. This method effectively improves the model's performance in dynamic environments and enhances its ability to extract multi-scale and key features. The formula for the temporal attention mechanism is shown below:

[0037]

[0038] Among them, Q t =W t ·W Q W t =I t [x:x+W s ,y:y+H s ], (x,y) represents the starting coordinates of the window, and the sliding window size is W. s ·H s I t The input image sequence is K. t =W t ·W k W Q and W k These are the learned weights. Features are extracted using a time attention mechanism, as shown in the following formula:

[0039] F t =α t ·V t ;

[0040] Where V t For a value vector, the formula is:

[0041] V t =W t ·W V ;

[0042] Then, the features of all time frames are combined, as shown in the following formula:

[0043]

[0044] The formula for generating dynamic convolution kernels is as follows:

[0045] K t =g(I t ,θ);

[0046] Where θ represents the parameters of the dynamic convolution kernel generator, responsible for mapping the input image to the convolution kernel space. For adaptive convolution operations, the generation formula is as follows:

[0047] O t =I t *k t ;

[0048] Among them, O t k represents the output feature map of the convolution operation. t This represents a dynamically generated convolutional kernel of size k×k. In summary, the dynamic convolutional kernel generation in this design aims to generate a kernel adapted to the current input image using the input image and learnable parameters. The adaptive convolution operation then uses this dynamically generated kernel to convolve the input image, thereby extracting features.

[0049] Finally, feature fusion is performed, combining dynamic features with Swin Transformer features, as shown in the following formula:

[0050] F fused =Concat(F dy ,F swin );

[0051] Finally, a fully connected layer is used for mapping, as shown in the following formula:

[0052] F output =FC(F fused );

[0053] This concludes the design of the dynamic feature extraction module STU. The feature extraction method of Swin Transformer itself will not be elaborated upon here. Figure 3 This indicates the fusion process of the Swing Transformer module and the dynamic feature extraction module STU.

[0054] The aforementioned Dynamic Feature Extraction (STU) module is an adaptive feature extraction structure. Its core idea is to dynamically change the convolutional kernel, weights, or channel selection based on the input image or contextual information to adapt to the target's scale, location, or texture features. Adding this module can improve the model's adaptability to multi-scale, multi-class targets, especially in scenes with dense targets or complex backgrounds.

[0055] The Swin Transformer mentioned above is a hierarchical visual Transformer widely used in tasks such as image classification, detection, and segmentation. Integrating the Swin Transformer can enhance global modeling capabilities.

[0056] Among them, the aforementioned dilated convolution module DC2f (Dilated Convolution with Double-scale and Fusion) has the following functions: extracting richer contextual information, expanding the receptive field, and taking into account the details of small targets, thereby achieving stronger feature expressiveness through dual-scale fusion.

[0057] Processing procedure:

[0058] 1. Input feature map X.

[0059] 2. Split+Conv: Features are extracted by using two convolutional branches with different dilation rates (e.g., d=1 and d=3).

[0060] 3. Enter the two BottleNecks (depth information extraction) to obtain two branch outputs Y1 and Y2.

[0061] 4. Concatenate the two feature maps and fuse them (using methods such as weighted sum, concatenation, SE attention fusion, etc.).

[0062] 5. Output the fused feature Y (multi-scale context feature map).

[0063] The STU module (Scale-aware Transformer Unit) combines the local modeling capabilities of CNNs with the global modeling capabilities of Transformers, with a particular emphasis on the ability to model targets at different scales.

[0064] Processing procedure:

[0065] 1. Input feature map X.

[0066] 2. Perform multi-scale branching on the input (which may be downsampled features at different resolutions).

[0067] 3. Each scale branch enters a local Transformer module for modeling (such as window attention).

[0068] 4. Use cross-scale fusion mechanisms (such as attention and weighted summation) to unify the output.

[0069] 3. Evaluate the performance of the adjusted model using AP and FPS.

[0070] AP=∫P(R)d(R);

[0071]

[0072] Where P represents the ratio of the number of correctly detected positive targets to the total number of detected positive targets (including both correct and incorrect ones). The test for the two parameters is also divided into three parts, with vehicle speeds of 15-30, 30-40, and 40-60.

[0073] True Positives (TP): The number of positive targets predicted by the model that have a sufficiently high IoU overlap with the true target and whose class is correctly predicted.

[0074] All positive targets (Ground Truth Positives): refers to all actual target (positive class) instances in the dataset, that is, the number of true boxes in the labeled boxes that belong to the target category.

[0075] IoU (Intersection over Union): The ratio of the intersection to the union of the predicted box and the ground truth box.

[0076] Preprocessing time: The time consumed from the raw input data (image / video) to the processing of format conversion, scaling, normalization, etc., required for model input.

[0077] Inference Time: The time consumed to input preprocessed data into the neural network, perform forward propagation, and output the prediction result.

[0078] Postprocessing Time:

[0079] The process of filtering and processing the raw output of the model to ultimately generate usable detection bounding boxes, categories, etc., is crucial. Post-processing determines the final output quality of the model, especially since NMS (Non-Maximum Search) has a significant impact on detection performance.

[0080] This invention collects images of retaining structures at different speeds based on actual detection conditions, which helps improve the robustness of the model; 2. In YOLOv9, the auxiliary branch Conv is replaced with the dilated convolution module DC2f, ensuring the resolution of the feature map and enhancing the ability to extract multi-scale features; When identifying roadbed retaining structure defects, the model exhibits certain differences depending on the vehicle speed, fundamentally due to the feature thinning, information loss, and distortion caused by the blurring of image frames. The newly designed module STU, through the design of a dynamic feature extraction module STU, performs weighted fusion of features between different time frames, enhancing the ability to capture key features.

[0081] Specific operational examples:

[0082] 1. Experimental environment:

[0083] CPU: Intel 16-core processor @ 3.40GHz

[0084] GPU: NVIDIA GeForce RTX 4060

[0085] Video memory: 16G

[0086] CUDA: 11.6

[0087] Epoch: 400

[0088] Batch size: 32

[0089] Image input resolution: 1080*960.

[0090] 2. Experimental Results

[0091] Test set: 30 images taken at each of the three speed ranges.

[0092] Vehicle speed: 15-30km / h

[0093] YOLOv9 95.2 93.5 97.3 Improved Algorithm 96.1 93.7 98.0

[0094] Vehicle speed: 30-40km / h

[0095] YOLOv9 92.7 89.6 94.3 Improved Algorithm 94.3 92.4 95.9

[0096] Vehicle speed: 40-60km / h

[0097] YOLOv9 87.5 84.4 88.3 Improved Algorithm 93.8 91.1 94.6

[0098] Improved network FPS: 98.

[0099] YOLOv9 network FPS: 88.

[0100] It can be seen that the detection accuracy drops sharply as vehicle speed increases. Therefore, the algorithm proposed in this invention can effectively reduce the impact of increased vehicle speed on detection accuracy.

[0101] This invention designs a novel dynamic feature extraction module (STU) to enhance the capture of key features by weighted fusion of features across different time frames. By introducing the STU and DC2f modules and reducing the number of detection heads, a novel YOLOv9 algorithm is constructed, specifically optimized for detecting support structures during movement. The constructed algorithm exhibits high detection accuracy and speed within a driving speed range of 15 to 60 km / h.

Claims

1. A rapid detection method for roadbed retaining structure defects based on YOLOv9, characterized in that: The Dynamic Feature Extraction Module (STU) replaces the CBLinear in YOLOv9, reducing the number of detector heads in the YOLOv9 auxiliary branch to two. The output of the second RepNCSPELAN4 in the YOLOv9 main branch is input to the first CBFuse in the auxiliary branch through the Dynamic Feature Extraction Module (STU), and the output of the third RepNCSPELAN4 in the YOLOv9 main branch is input to the second CBFuse in the auxiliary branch through the Dynamic Feature Extraction Module (STU). The Dynamic Feature Extraction Module (STU) receives a sequence of consecutive frames of images from the dataset to be tested at the input layer. In the dynamic feature extraction process, it combines a sliding window mechanism, a temporal attention mechanism, and dynamic convolution to finally fuse the extracted features with the features of the Swing Transformer. The formula for the time attention mechanism is: ; in, , , Represents the starting coordinates of the window, and the size of the sliding window is [value missing]. , The input image sequence; , and The learned weights; Features are extracted using a time attention mechanism, as shown in the formula: ; in For a value vector, the formula is: ; Combining the features of all time frames, the formula is: ; For dynamic convolution kernels, the generation formula is as follows: ; in, The parameters representing the dynamic convolution kernel generator are responsible for mapping the input image to the convolution kernel space; For adaptive convolution operations, the formula is generated. ; in, This represents the output feature map of the convolution operation. This represents a dynamically generated convolutional kernel, with a size of [size missing]. ; The dynamic features are fused with the Swing Transformer features, using the following formula: ; Finally, a fully connected layer is used for mapping, as shown in the formula. .

2. The rapid detection method for roadbed retaining structure defects based on YOLOv9 as described in claim 1, characterized in that: In the auxiliary branch of YOLOv9, the dilated convolution module DC2f is used instead of the ordinary convolution module Conv. The dilated convolution module DC2f extracts features from the input feature map through two different dilation rates to obtain two branch outputs, and then merges the branch outputs to output the merged features.

3. The rapid detection method for roadbed retaining structure defects based on YOLOv9 as described in claim 1, characterized in that: The image sequence of consecutive frames comes from the roadbed retaining structure defect dataset; the roadbed retaining structure defects refer to data on cracks, spalling, and collapse at different vehicle speeds.

4. The rapid detection method for roadbed retaining structure defects based on YOLOv9 as described in claim 3, characterized in that: The roadbed retaining structure defects refer to the crack, spalling, and collapse data collected at vehicle speeds of 15-30 km / h, 30-40 km / h, and 40-60 km / h.

5. The rapid detection method for roadbed retaining structure defects based on YOLOv9 as described in claim 3, characterized in that: The data set of roadbed retaining structure defects also includes normal sample data, and the number of normal sample data matches the number of crack, spalling, and collapse data.

6. The rapid detection method for roadbed retaining structure defects based on YOLOv9 as described in claim 1, characterized in that: The rapid detection method for roadbed retaining structure defects also includes a quality assessment after model adjustment, and the assessment formula is as follows: ； ； Where P represents the ratio of the number of correctly detected positive targets to the total number of all detected positive targets.

7. The rapid detection method for roadbed retaining structure defects based on YOLOv9 as described in claim 6, characterized in that: The correct positive class target and all positive class targets are derived from test data at vehicle speeds of 15-30 km / h, 30-40 km / h, and 40-60 km / h.

Images