Railway track foreign matter detection optimization method for unmanned aerial vehicle inspection

Abstract

The application discloses a railway track foreign matter detection optimization method for unmanned aerial vehicle inspection, and improves a YOLOv11s target detection network; an LSF-C3k2 feature extraction module is designed in the backbone network, railway track linear structure information and foreign matter local salient feature are extracted through a track structure branch and a foreign matter feature branch, and the feature extraction capability of long-distance small-scale and multi-scale foreign matters in a complex track scene is improved; an RTG-Guide track prior guidance module is introduced, track guidance weight maps are generated according to track structure features to enhance foreign matter features and inhibit background interference in non-track areas; and an EW-MPDIOU loss function is used in the boundary box regression stage to improve the positioning accuracy of long-distance small targets and accelerate the model convergence speed through a dynamic weight mechanism. The application effectively improves the railway track foreign matter detection precision and robustness while guaranteeing real-time detection performance, and can meet the actual application requirements of unmanned aerial vehicle railway inspection.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection of rail transit and computer vision target detection technology, specifically to an optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection. Background Technology

[0002] Foreign objects on railway tracks directly threaten train safety, and railway line safety is the core guarantee for stable train operation. Traditional manual inspections are inefficient and unreliable, making them unsuitable for the large-scale inspection needs of modern railways. In recent years, unmanned aerial vehicle (UAV) inspection technology has gradually become an important means of intelligent inspection of foreign objects on railway tracks due to its advantages such as mobility, wide coverage, and continuous operation in adverse weather conditions. UAVs equipped with high-definition cameras fly low along the tracks, enabling them to collect track images in real time and transmit them back to the ground processing system, significantly increasing the frequency and coverage of inspections. However, due to the limited computing power resources on UAVs, lightweight detection models that can perform real-time inference on embedded edge devices (such as Jetson Nano and Jetson Xavier NX) have become a core requirement for engineering implementation. Deep learning-based target detection algorithms, with their advantages of automation and high efficiency, are widely used in the field of railway foreign object detection, and the YOLO series of lightweight models are the preferred solution for UAV-based real-time detection scenarios.

[0003] The YOLO series of deep learning detection algorithms are fast and easy to deploy, making them widely used in drone inspections; among them, YOLOv11s' lightweight features make it suitable for embedded edge devices. However, directly using native YOLOv11s for foreign object detection on railway tracks still has significant drawbacks: First, the network does not specifically model the linear structure of the track, resulting in poor feature extraction capabilities for small-scale and multi-scale foreign objects in the distance; second, complex track backgrounds are prone to false positives and false negatives, and traditional loss functions are insufficient for locating small targets in the distance, failing to simultaneously meet the accuracy and real-time requirements of railway inspections. The existing application CN202610094727.3 discloses a rail surface defect detection system based on improved YOLOv11s and RK3588. Based on intelligent rail defect detection, it integrates the RK3588 AI chip, improved YOLOv11s algorithm, ATGM336H GPS module and TCP communication mechanism. It can collect track images in real time during train operation, identify defects, record the location and upload data to the central control room, forming a fully intelligent detection system integrating acquisition, processing, transmission and display. However, the network structure and method disclosed therein are not applicable to track foreign object detection.

[0004] Therefore, it is necessary to propose an optimized method for detecting foreign objects on railway tracks in UAV inspection scenarios, which can improve the detection accuracy and robustness of foreign objects in complex environments while ensuring the model's lightweight nature and real-time performance. Summary of the Invention

[0005] 1. The technical problem to be solved:

[0006] To address the aforementioned technical issues, this invention provides an optimized method for detecting foreign objects on railway tracks in UAV inspection scenarios. The method improves the algorithm network structure to effectively integrate track structure information and foreign object features, thereby enhancing the model's ability to detect foreign objects on railway tracks in multi-scale targets and complex scenarios.

[0007] 2. Technical Solution:

[0008] An optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection, characterized by comprising:

[0009] Step 1: Acquire sample images using an image acquisition device and create a dataset of foreign objects on railway tracks;

[0010] Step 2: Construct an improved YOLOv11s model for detecting foreign objects on railway tracks. The YOLOv11s network structure is improved, specifically by replacing the C3k2 module with the LSF-C3k2 feature extraction module in its backbone feature extraction network. This extracts linear track structure features and local salient features of foreign objects through track structure and foreign object feature branches, respectively. The LSF-C3k2 feature extraction module incorporates an RTG-Guide track prior guidance module, generating a track guidance weight map based on track structure features to weight and enhance foreign object features. In the head detection network, the EW-MPDIoU bounding box regression loss function is used to optimize target localization, and the Focal Loss classification loss function is used for target classification.

[0011] Step 3: Train an improved YOLOv11s model for detecting foreign objects on railway tracks using a foreign object dataset.

[0012] Step 4: Input the image of the foreign object to be detected on the railway track into the trained railway track foreign object detection model to obtain the railway foreign object detection results.

[0013] Further, in step one, the foreign objects in the sample images of railway tracks include four different defect types: plastic bag foreign objects, plant foreign objects, long stick foreign objects, and human foreign objects. Each defect type image includes different weather conditions. The foreign object types are manually or software-annotated to generate corresponding railway track foreign object defect data subsets. The four railway track foreign object defect data subsets constitute the railway track foreign object dataset. The dataset is divided into training set, validation set, and test set according to a preset ratio.

[0014] Furthermore, in step two, the LSF-C3k2 feature extraction module specifically includes a track structure branch, a foreign object feature branch, a track guidance unit, and a feature fusion unit;

[0015] The input features are input in parallel into the track structure branch and the foreign object feature branch; the track structure branch uses an asymmetric convolutional structure to extract continuous linear structural features of the railway track from the input features, so as to enhance the network's ability to perceive track edges, rail extension direction and track area geometric information; the asymmetric convolutional structure is that the input features are sequentially passed through 1×7 convolution and 7×1 convolution;

[0016] The foreign object feature branch uses a 3×3 standard convolution and a 5×5 depthwise separable convolution concatenation structure to extract local texture features, edge features and small-scale foreign object features in the track region of the input features, so as to enhance the model's ability to represent small foreign objects in the distance.

[0017] The track guidance unit generates a track guidance weight map based on the track features output by the track structure branch, and uses the track guidance weight map to weight and enhance the features output by the foreign object feature branch, guiding the network to focus on the target in the track area and suppressing background interference outside the track.

[0018] The feature fusion unit concatenates the output features of the track structure branch with the foreign object features enhanced by the track guidance according to the channel dimension, and completes the feature fusion through 1×1 convolution. The fused features establish residual connections with the input features to retain the original feature information and improve the stability of network training, and finally outputs the enhanced feature map.

[0019] Furthermore, in step two, the LSF-C3k2 feature extraction module introduces the RTG-Guide track prior guidance module, specifically: obtaining the continuous linear structure features of the railway track output from the track structure branch in the LSF-C3k2 module as follows: (1);

[0020] in: Features of the orbital structure; Number of channels; , These represent the height and width of the feature map, respectively.

[0021] Constructing orbital response maps using convolutional mapping: (2);

[0022] in: This represents a 1×1 convolution operation; The orbital response feature map is represented; the orbital structure information is mapped to the spatial dimension through equations (1) and (2) to enhance the feature response intensity of the orbital region.

[0023] The orbital guidance weight map is generated using the Sigmoid activation function: (3);

[0024] in: For the Sigmoid function; This is used to guide the weighting of the track area; the above formula is used to assign higher weight values ​​to track areas and lower weight values ​​to non-track areas.

[0025] Foreign object feature branch output features :

[0026] (4);

[0027] Enhancement of foreign object features using orbital guidance weight maps:

[0028] (5);

[0029] in: This represents element-wise multiplication; The enhanced foreign object features are represented by (1+G) in the formula. The enhanced weights are constructed from the orbital guidance weight map. The constant 1 is a preset unit constant used to construct the residual enhanced weights so that the orbital guidance weight map only modulates the gain of the original foreign object features without changing its basic feature expression.

[0030] The following formula fuses the orbital structure features with the enhanced foreign object features:

[0031] (6);

[0032] in: This indicates a channel-dimensional splicing operation; Indicates fusion characteristics;

[0033] Feature compression is then performed using 1×1 convolution:

[0034] (7);

[0035] Obtain the output feature map after orbital guidance enhancement. ;

[0036] Establish a residual connection between the output features and the input features:

[0037] (8);

[0038] in: Input features for the module; This is the final output feature of the RTG-Guide module.

[0039] Furthermore, the EW-MPDIoU bounding box regression loss function optimizes target localization, specifically as follows:

[0040] A dual-branch loss strategy combining EW-MPDIoU and Focal Loss is constructed, where the regression branch uses the EW-MPDIoU loss function to optimize the bounding box regression process, and the classification branch uses the Focal Loss loss function to optimize the target classification process. By jointly optimizing the regression and classification branches, the model's accuracy in locating and classifying foreign objects on railway tracks is improved, thereby enhancing the overall detection performance.

[0041] Furthermore, in step four, the image information of foreign objects on the railway track collected by the UAV carrying a high-definition camera flying at low altitude along the railway track is transmitted back to the ground processing unit in real time via a wireless data link or directly run on the UAV's onboard embedded processor.

[0042] Furthermore, step four also includes evaluating the model; specifically, the model is evaluated using mean precision, accuracy, recall, and detection speed.

[0043] Furthermore, it also includes model deployment steps for embedded edge hardware; after training, the model weights are exported to ONNX or TensorRT format and deployed on the UAV-borne embedded platform; in TensorRT FP16 precision inference mode, the number of parameters of the improved YOLOv11s railway track foreign object detection model does not exceed 10M, the computational load does not exceed 35, and the inference speed on the Jetson Xavier NX platform is not less than 25 frames / second, meeting the engineering deployment requirements of UAV low-altitude inspection for lightweight and real-time performance.

[0044] 3. Beneficial effects:

[0045] (1) The present invention discloses an optimized method for detecting foreign objects on railway tracks for UAV inspection, which is based on the YOLOv11s algorithm and improved to make it more suitable for detecting small foreign objects on railway tracks. In the backbone feature extraction network, the original C3k2 module is replaced with the LSF-C3k2 feature extraction module; the LSF-C3k2 feature extraction module adopts a dual-branch collaborative modeling of track structure and foreign object features to model the linear structure of the track and the local details of the foreign object, thereby improving the network's feature extraction capability for distant small targets and multi-scale foreign objects and alleviating the problem of feature loss of small foreign objects.

[0046] (2) The present invention discloses an optimization method for railway track foreign object detection for UAV inspection. It improves the stability of the model based on the YOLOv11s algorithm. The LSF-C3k2 feature extraction module introduces the RTG-Guide track prior guidance module, generates a track guidance weight map based on the track structure features, enhances the foreign object features and suppresses background interference in non-track areas, thereby improving the target perception ability and detection stability of the model in complex railway scenarios.

[0047] (3) The present invention discloses an optimization method for foreign object detection on railway tracks for UAV inspection. It improves the overall performance of YOLOv11s algorithm for foreign object inspection by improving it. In its head detection network, an EW-MPDIoU joint Focal Loss loss optimization strategy is constructed to optimize the model from the aspects of target localization and category classification, thereby improving the localization accuracy of small distant targets and reducing the influence of background samples on model training, thereby improving the overall detection performance.

[0048] (4) The present invention discloses an optimized method for detecting foreign objects on railway tracks for UAV inspection. Experimental results show that the improved model achieves an average accuracy of 89.4% on the railway track foreign object dataset, which is 6.9% higher than the average accuracy of the unimproved YOLOv11s network model; the accuracy reaches 90.4%, which is 3.6% higher than the unimproved model; and the recall rate reaches 95.0%, which is 2.0% higher than the unimproved model. This verifies the effectiveness of the present invention in railway foreign object detection tasks.

[0049] This invention is based on the YOLOv11s lightweight network. While ensuring improved detection accuracy, it also takes into account the model's computational efficiency. It can be adapted to embedded edge computing platforms such as Jetson Nano and Jetson Xavier NX to realize real-time detection of foreign objects on railway tracks and meet the engineering application needs of UAV inspection scenarios. Attached Figure Description

[0050] Figure 1 This is a diagram showing the overall structure of the improved YOLOv11s railway track foreign object detection model in this invention.

[0051] Figure 2 This is a schematic diagram of the LSF-C3k2 feature extraction module in this invention;

[0052] Figure 3 This is a schematic diagram of the RTG-Guide orbit prior guidance module in this invention;

[0053] Figure 4 This is a specific implementation example showing the mAP curve before algorithm optimization.

[0054] Figure 5 The image shows the mAP curve after optimizing the algorithm in a specific embodiment.

[0055] Figure 6 This is a diagram illustrating the railway foreign object detection effect of YOLOv11s in a specific embodiment of the present invention.

[0056] Figure 7 This is a flowchart of the UAV airborne deployment and inference process in this invention. Detailed Implementation

[0057] The present invention will now be described in detail with reference to the accompanying drawings.

[0058] As attached Figure 1 As shown, an optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection is characterized by including:

[0059] Step 1: Acquire sample images using an image acquisition device and create a dataset of foreign objects on railway tracks;

[0060] Step 2: Construct an improved YOLOv11s model for detecting foreign objects on railway tracks. The YOLOv11s network structure is improved, specifically by replacing the C3k2 module with the LSF-C3k2 feature extraction module in its backbone feature extraction network. This extracts linear track structure features and local salient features of foreign objects through track structure and foreign object feature branches, respectively. The LSF-C3k2 feature extraction module incorporates an RTG-Guide track prior guidance module, generating a track guidance weight map based on track structure features to weight and enhance foreign object features. In the head detection network, the EW-MPDIoU bounding box regression loss function is used to optimize target localization, and the Focal Loss classification loss function is used for target classification.

[0061] Step 3: Train an improved YOLOv11s model for detecting foreign objects on railway tracks using a foreign object dataset.

[0062] Step 4: Input the image of the foreign object to be detected on the railway track into the trained railway track foreign object detection model to obtain the railway foreign object detection results.

[0063] Further, in step one, the foreign objects in the sample images of railway tracks include four different defect types: plastic bag foreign objects, plant foreign objects, long stick foreign objects, and human foreign objects. Each defect type image includes different weather conditions. The foreign object types are manually or software-annotated to generate corresponding railway track foreign object defect data subsets. The four railway track foreign object defect data subsets constitute the railway track foreign object dataset. The dataset is divided into training set, validation set, and test set according to a preset ratio.

[0064] Furthermore, in step two, as shown in the appendix... Figure 2 As shown, the LSF-C3k2 feature extraction module specifically includes a track structure branch, a foreign object feature branch, a track guidance unit, and a feature fusion unit;

[0065] The input features are input in parallel into the track structure branch and the foreign object feature branch; the track structure branch uses an asymmetric convolutional structure to extract continuous linear structural features of the railway track from the input features, so as to enhance the network's ability to perceive track edges, rail extension direction and track area geometric information; the asymmetric convolutional structure is that the input features are sequentially passed through 1×7 convolution and 7×1 convolution;

[0066] The foreign object feature branch uses a 3×3 standard convolution and a 5×5 depthwise separable convolution concatenation structure to extract local texture features, edge features and small-scale foreign object features in the track region of the input features, so as to enhance the model's ability to represent small foreign objects in the distance.

[0067] The track guidance unit generates a track guidance weight map based on the track features output by the track structure branch, and uses the track guidance weight map to weight and enhance the features output by the foreign object feature branch, guiding the network to focus on the target in the track area and suppressing background interference outside the track.

[0068] The feature fusion unit concatenates the output features of the track structure branch with the foreign object features enhanced by the track guidance according to the channel dimension, and completes the feature fusion through 1×1 convolution. The fused features establish residual connections with the input features to retain the original feature information and improve the stability of network training, and finally outputs the enhanced feature map.

[0069] As attached Figure 2 As shown, the LSF-C3k2 (Linear Structure Fusion-C3k2) feature extraction module adopts a dual-branch fusion structure of track branch and object branch. The input features are first processed by 1×1 convolution to adjust the channels, and then fed into the track branch and object branch respectively.

[0070] The track structure branch uses asymmetric convolution to extract the linear structure information of the railway track. The calculation process is as follows:

[0071]

[0072] in, Indicates the input feature map, This represents the characteristics of the track structure. Since railway tracks have obvious longitudinal extension features in UAV top-down images, using a combination of 1×7 and 7×1 convolutions can expand the longitudinal receptive field and improve the model's ability to extract features of track edges, gauge regions, and track orientation.

[0073] The foreign object feature branch uses a combination of standard convolution and depthwise separable convolution to extract local detail information of the foreign object. The calculation process is as follows:

[0074]

[0075] in, It represents the features of foreign objects. Depthwise separable convolution can expand the receptive field while maintaining a low number of parameters and computational cost, thus improving the ability to perceive small-scale foreign objects such as plastic bags, tree branches, and people.

[0076] Furthermore, in step two, such as Figure 3 As shown, the LSF-C3k2 feature extraction module introduces the RTG-Guide track prior guidance module, specifically: obtaining the continuous linear structure features of the railway track as follows from the track structure branch output of the LSF-C3k2 module:

[0077] (1);

[0078] in: Features of the orbital structure; Number of channels; , These represent the height and width of the feature map, respectively.

[0079] Constructing orbital response maps using convolutional mapping:

[0080] (2);

[0081] in: This represents a 1×1 convolution operation; The orbital response feature map is represented; the orbital structure information is mapped to the spatial dimension through equations (1) and (2) to enhance the feature response intensity of the orbital region.

[0082] The orbital guidance weight map is generated using the Sigmoid activation function:

[0083] (3);

[0084] in: For the Sigmoid function; This is used to guide the weighting of the track area; the above formula is used to assign higher weight values ​​to track areas and lower weight values ​​to non-track areas.

[0085] Foreign object feature branch output features :

[0086] (4);

[0087] Enhancement of foreign object features using orbital guidance weight maps:

[0088] (5);

[0089] in: This represents element-wise multiplication; The enhanced foreign object features are represented by (1+G) in the formula. The enhanced weights are constructed from the orbital guidance weight map. The constant 1 is a preset unit constant used to construct the residual enhanced weights so that the orbital guidance weight map only modulates the gain of the original foreign object features without changing its basic feature expression.

[0090] The following formula fuses the orbital structure features with the enhanced foreign object features:

[0091] (6);

[0092] in: This indicates a channel-dimensional splicing operation; Indicates fusion characteristics;

[0093] Feature compression is then performed using 1×1 convolution:

[0094] (7);

[0095] Obtain the output feature map after orbital guidance enhancement. ;

[0096] Establish a residual connection between the output features and the input features:

[0097] (8);

[0098] in: Input features for the module; This is the final output feature of the RTG-Guide module.

[0099] Through the synergistic effect of track structure modeling, foreign object detail enhancement, and track prior guidance mechanism, LSF-C3k2 can effectively improve the model's ability to detect distant small foreign objects, multi-scale foreign objects, and complex background targets in railway scenarios.

[0100] Furthermore, the EW-MPDIoU bounding box regression loss function optimizes target localization, specifically as follows:

[0101] A dual-branch loss strategy combining EW-MPDIoU and Focal Loss is constructed, where the regression branch uses the EW-MPDIoU loss function to optimize the bounding box regression process, and the classification branch uses the Focal Loss loss function to optimize the target classification process. By jointly optimizing the regression and classification branches, the model's accuracy in locating and classifying foreign objects on railway tracks is improved, thereby enhancing the overall detection performance.

[0102] This method constructs a dual-branch loss strategy combining EW-MPDIoU and Focal Loss. The regression branch uses EW-MPDIoU to address the issues of inaccurate localization of small targets and slow bounding box convergence. The classification branch relies on Focal Loss to suppress massive negative background samples, alleviating the sample imbalance problem. The dual-branch loss strategy of EW-MPDIoU and Focal Loss optimizes the division of labor between regression and classification tasks to more comprehensively adapt to the railway foreign object detection scenario. The regression branch uses EW-MPDIoU to introduce three factors—scale, occlusion, and boundary—to dynamically adjust the bounding box constraint strength. The classification branch uses Focal Loss to introduce a balance coefficient and a focusing factor to adjust the sample weight distribution, enabling the model to accurately fit multi-scale targets and effectively distinguish between foreign objects and complex backgrounds. In specific implementation, the Focal Loss balance coefficient can be set to 0.25, and the focusing factor can be set to 2.0. This scheme compensates for the shortcomings of a single loss function by synergistically optimizing localization and classification losses, thus comprehensively improving the model's detection performance.

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] in, This represents the EW-MPDIoU bounding box similarity after introducing edge weights, which is used to comprehensively evaluate the degree of positional matching between the predicted box and the ground truth box. The closer its value is to 1, the more accurate the localization result. It is the intersection-union ratio; S is the area of ​​the true target bounding box; This represents the squared Euclidean distance between the midpoints of the four edges of the ground truth bounding box and the predicted bounding box. These correspond to the left, top, right, and bottom edges, respectively. It is the weight coefficient of the midpoint of the i-th edge, and is calculated by normalizing the boundary offset; This represents the bounding box regression loss; For category classification loss, For confidence loss, For modulation probability, The weighting coefficients for easy and difficult samples. This represents the focusing parameter, which is used to adjust the loss weights for samples of different difficulty levels. Balance the weights for each category; This represents the total loss of the model; , , and These are the classification loss weights, target confidence loss weights, bounding box regression loss weights, and distribution focus loss weights, used to adjust the proportion of each loss term in the total loss function, achieving synergistic optimization between the classification task, target confidence prediction, and bounding box regression task. After the improvement, even if rain or fog causes edge blurring, the model can accurately capture edge offsets through multi-keypoint weighted penalties, improving localization stability and detection accuracy.

[0110] Furthermore, in step four, images of foreign objects on the railway track are collected by a drone equipped with a high-definition camera flying at low altitude along the railway track. These images are then transmitted in real-time via a wireless data link to a ground processing unit or run directly on the drone's onboard embedded processor. The data is then input into the improved YOLOv11s track defect detection model for processing and output of the identification results. During implementation, the drone's data collection altitude can be controlled between 20 and 50 meters, its flight speed no higher than 10 m / s, and its image acquisition frame rate no less than 25 frames / second, ensuring full coverage and real-time detection of foreign objects on the track.

[0111] Furthermore, step four also includes evaluating the model; specifically, the model is evaluated using mean precision, accuracy, recall, and detection speed.

[0112] Furthermore, the feature is that it also includes a model deployment step for embedded edge hardware; after training, the model weights are exported to ONNX or TensorRT format and deployed on the UAV-borne embedded platform; in TensorRT FP16 precision inference mode, the number of parameters of the improved YOLOv11s railway track foreign object detection model does not exceed 10M, the computational load does not exceed 35, and the inference speed on the Jetson Xavier NX platform is not less than 25 frames / second, meeting the engineering deployment requirements of UAV low-altitude inspection for lightweight and real-time performance.

[0113] Example:

[0114] This embodiment was conducted in an Ubuntu 16.04.7 LTS and CUDA 12.4 environment, using the PyTorch deep learning framework, with PyTorch version torch-2.6.0+cu124; Ultralytics version 8.3.99; Python version 3.10.16; GPU configuration: two NVIDIA GeForce RTX 2080 Ti cards with a total of 22GB of video memory; CPU model: Intel(R) Xeon(R) CPU E5-2678 v3 @ 2.50GHz; YOLOv11s input image specifications: 640*640 resolution per image, 400 epochs, 8 workers, and 16 training batches.

[0115] The dataset used in this embodiment has an image size of 2592*1944, obtained by segmenting 4K camera images taken at a height of 30 meters. The dataset includes four typical foreign objects found on railways under different weather conditions: plastic bags (poset), plants (bitki), long sticks (cubuk), and people (insan). The training set, validation set, and test set are divided in a 7:2:1 ratio, with the test set not participating in the training process. The image acquisition method of this dataset closely matches the low-altitude UAV inspection scenario, featuring a top-down perspective, small target size, and complex background, which helps verify the effectiveness of this invention in real-world UAV inspection applications.

[0116] In this embodiment, the training set contains 896 images, the validation set contains 256 images, and the test set contains 128 images. The annotation information of the dataset is converted from XML format to TXT format so that it can be run in the YOLOv11s model based on the PyTorch framework. The YOLOv11s input image specification is that each image has a resolution of 640*640.

[0117] The improved YOLOv11s orbital defect detection model was trained and validated using training and validation sets, and the model was evaluated using the following metrics: mean average precision (mAP), precision, and recall.

[0118] like Figure 7 During training, the model processes the input image information in the manner described above. After training, the trained weights and parameter files will be obtained for model deployment and testing, such as... Figure 7As shown. To adapt to the deployment requirements of UAV-borne embedded hardware, after training, the model weights were further exported to ONNX format, and TensorRT was used for FP16 precision quantization acceleration. Inference deployment verification was completed on the NVIDIA Jetson Xavier NX platform (15W power consumption mode). Testing showed that the optimized YOLOv11s model has approximately 9.4M parameters and approximately 21.4 GFLOPs. After TensorRT FP16 acceleration, the inference speed on Jetson Xavier NX reached approximately 28 frames per second, with mAP@0.5 maintained at 89.4% (less than 0.5% accuracy loss relative to the GPU server side), verifying the feasibility and practical value of the method in UAV-borne embedded edge computing.

[0119] Verification Example 1:

[0120] To verify the contributions of the improved LSF-C3k2 feature extraction module, RTG-Guide orbit prior guidance module, and EW-MPDIoU combined with Focal Loss dual-branch loss strategy to the model performance improvement, the verification experiments were conducted on the same dataset, with the same training parameters, and under the same hardware environment. Evaluation metrics included mAP@0.5, Precision, Recall, and detection speed (FPS). In the table, A represents the LSF-C3k2 feature extraction module, B represents the RTG-Guide orbit prior guidance module, and C represents the EW-MPDIoU combined with Focal Loss dual-branch loss strategy. "√" in the table indicates the introduction of the corresponding improved module, and "×" indicates that the corresponding improved module was not introduced. The experimental results are shown in the table below. The table shows that the improved modules using this method significantly improve the two evaluation metrics: mAP@0.5, Precision, and Recall.

[0121]

[0122] Verification Example 2:

[0123] To verify the effectiveness of the improved algorithm for railway track foreign object datasets, experiments were conducted comparing the original YOLOv11s model with the improved model of this invention. Figure 4 , 5As shown, the mAP curves before and after the improvement are plotted. Experimental results show that the average accuracy (mAP) of the improved algorithm model is 89.4%, which is 6.9% higher than the original YOLOv11s' 82.5%. Furthermore, except for a slight fluctuation in the accuracy of long sticks, the detection accuracy of other categories is significantly improved. Specifically, the accuracy for plastic bags (poset) reaches 90.4%, plants (bitki) reaches 71.7%, long sticks (cubuk) reaches 99.5%, and people (insan) reaches 96.1%. The overall detection accuracy shows an upward trend, indicating that the algorithm can meet the accuracy requirements for foreign object detection on railway tracks in actual inspection scenarios.

[0124] As attached Figure 6 As shown, this image is an example of detecting orbital images containing four different defects using the improved model.

[0125] Although the present invention has been disclosed above with reference to preferred embodiments, these are not intended to limit the invention. Any person skilled in the art can make various changes or modifications without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be defined by the scope of the claims of this application.

Claims

1. An optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection, characterized in that: include: Step 1: Acquire sample images using an image acquisition device and create a dataset of foreign objects on railway tracks; Step 2: Construct an improved YOLOv11s model for detecting foreign objects on railway tracks. The YOLOv11s network structure is improved, specifically by replacing the C3k2 module with the LSF-C3k2 feature extraction module in its backbone feature extraction network. This extracts linear track structure features and local salient features of foreign objects through track structure and foreign object feature branches, respectively. The LSF-C3k2 feature extraction module incorporates an RTG-Guide track prior guidance module, generating a track guidance weight map based on track structure features to weight and enhance foreign object features. In the head detection network, the EW-MPDIoU bounding box regression loss function is used to optimize target localization, and the Focal Loss classification loss function is used for target classification. Step 3: Train an improved YOLOv11s model for detecting foreign objects on railway tracks using a foreign object dataset. Step 4: Input the image of the foreign object to be detected on the railway track into the trained railway track foreign object detection model to obtain the railway foreign object detection results.

2. The optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection as described in claim 1, characterized in that: In step one, the foreign objects on the railway tracks in the sample images include four different defect types: plastic bag foreign objects, plant foreign objects, long stick foreign objects, and human foreign objects. Each defect type image includes different weather conditions. The foreign object types are manually or software-annotated to generate corresponding railway track foreign object defect data subsets. The four railway track foreign object defect data subsets constitute the railway track foreign object dataset. The dataset is then divided into training set, validation set, and test set according to a preset ratio.

3. The optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection as described in claim 1, characterized in that: In step two, the LSF-C3k2 feature extraction module specifically includes a track structure branch, a foreign object feature branch, a track guidance unit, and a feature fusion unit; Input features are combined into parallel input track structure branches and foreign object feature branches; The track structure branch uses an asymmetric convolutional structure to extract continuous linear structural features of the railway track from the input features, thereby enhancing the network's ability to perceive track edges, rail extension directions, and geometric information of the track area; the asymmetric convolutional structure is that the input features are sequentially passed through 1×7 convolution and 7×1 convolution; The foreign object feature branch uses a 3×3 standard convolution and a 5×5 depthwise separable convolution concatenation structure to extract local texture features, edge features and small-scale foreign object features in the track region of the input features, so as to enhance the model's ability to represent small foreign objects in the distance. The track guidance unit generates a track guidance weight map based on the track features output by the track structure branch, and uses the track guidance weight map to weight and enhance the features output by the foreign object feature branch, guiding the network to focus on the target in the track area and suppressing background interference outside the track. The feature fusion unit concatenates the output features of the track structure branch with the foreign object features enhanced by the track guidance according to the channel dimension, and completes the feature fusion through 1×1 convolution. The fused features establish residual connections with the input features to retain the original feature information and improve the stability of network training, and finally outputs the enhanced feature map.

4. The optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection according to claim 3, characterized in that: In step two, the LSF-C3k2 feature extraction module introduces the RTG-Guide track prior guidance module, specifically: obtaining the continuous linear structure features of the railway track output from the track structure branch of the LSF-C3k2 module as follows: （1）； in: Features of the orbital structure; Number of channels; , These represent the height and width of the feature map, respectively. Constructing orbital response maps using convolutional mapping: （2）； in: This represents a 1×1 convolution operation; The orbital response feature map is represented; the orbital structure information is mapped to the spatial dimension through equations (1) and (2) to enhance the feature response intensity of the orbital region. The orbital guidance weight map is generated using the Sigmoid activation function: （3）； in: For the Sigmoid function; This is used to guide the weighting of the track area; the above formula is used to assign higher weight values ​​to track areas and lower weight values ​​to non-track areas. Foreign object feature branch output features : （4）； Enhancement of foreign object features using orbital guidance weight maps: （5）； in: This represents element-wise multiplication; The enhanced foreign object features are represented by (1+G) in the formula. The enhanced weights are constructed from the orbital guidance weight map. The constant 1 is a preset unit constant used to construct the residual enhanced weights so that the orbital guidance weight map only modulates the gain of the original foreign object features without changing its basic feature expression. The following formula fuses the orbital structure features with the enhanced foreign object features: （6）； in: This indicates a channel-dimensional splicing operation; Indicates fusion characteristics; Feature compression is then performed using 1×1 convolution: （7）； Obtain the output feature map after orbital guidance enhancement. ; Establish a residual connection between the output features and the input features: （8）； in: Input features for the module; This is the final output feature of the RTG-Guide module.

5. An optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection, as described in claim 4, is characterized in that: The EW-MPDIoU bounding box regression loss function optimizes target localization as follows: A dual-branch loss strategy combining EW-MPDIoU and Focal Loss is constructed, where the regression branch uses the EW-MPDIoU loss function to optimize the bounding box regression process, and the classification branch uses the Focal Loss loss function to optimize the target classification process. By jointly optimizing the regression and classification branches, the model's accuracy in locating and classifying foreign objects on railway tracks is improved, thereby enhancing the overall detection performance.

6. An optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection, as described in claim 1, is characterized in that: In step four, images of foreign objects on the railway track are collected by a drone carrying a high-definition camera flying at low altitude along the railway track. These images are then transmitted back to the ground processing unit in real time via a wireless data link or run directly on the drone's onboard embedded processor.

7. An optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection, as described in claim 1, is characterized in that: Step four also includes evaluating the model; specifically, the model is evaluated using mean precision, accuracy, recall, and detection speed.

8. An optimized method for detecting foreign objects on railway tracks for unmanned aerial vehicle (UAV) inspection, as described in claim 1, is characterized in that: It also includes model deployment steps for embedded edge hardware; after training, the model weights are exported in ONNX or TensorRT format and deployed on the UAV-borne embedded platform; In TensorRT FP16 precision inference mode, the improved YOLOv11s railway track foreign object detection model has no more than 10M parameters and no more than 35 computational resources. On the Jetson Xavier NX platform, the inference speed is no less than 25 frames / second, which meets the engineering deployment requirements of UAV low-altitude inspection for lightweight and real-time performance.

Images