Method for detecting foreign matter and rail damage of catenary based on unmanned aerial vehicle binocular vision

Abstract

The method for detecting overhead line foreign matters and rail damage based on unmanned aerial vehicle binocular vision comprises the following steps: (1) intelligent flight route planning and safety confirmation based on digital twinning and airspace cooperation; (2) real-time environment perception and high-robustness flight control based on multi-source sensor fusion; (3) collecting images of trackside equipment; (4) image data preprocessing; (5) constructing a high-quality labeled data set for model training; (6) constructing an improved YOLOv11 detection model; (7) primary positioning of key components and regions based on the improved YOLOv11 model; (8) refined defect identification and classification based on the positioned regions; and (9) reporting of detection results and early warning. The present application can stably and reliably operate in the complex environment of high-speed rail, such as high pressure and strong electromagnetism, thereby ensuring the continuity of the detection task and the consistency of the data.

Description

Technical Field

[0001] This invention relates to a method for detecting foreign objects and rail damage in overhead contact lines based on binocular vision from unmanned aerial vehicles (UAVs), belonging to the field of UAV application technology. Background Technology

[0002] The safe operation of railway lines is crucial to the national economy and people's livelihood, and is a key focus for the railway sector. The railway system includes trains, overhead contact lines, tracks, substructures, and other ancillary systems. The overhead contact lines are used to transmit electrical energy to electric locomotives, and the continuity and reliability of this transmission must be ensured. As the core component bearing the train load, the condition of the rails directly affects the operational safety of the train.

[0003] Currently, the inspection technology for overhead contact lines in China is mostly based on manual visual inspection, which is inflexible in terms of time planning, time-consuming, and inefficient. Rail inspection is primarily conducted manually or using rail inspection vehicles. In recent years, with the rapid development of my country's rail transit, the cost of inspection labor has risen rapidly, significantly increasing the pressure on rail transit maintenance.

[0004] Drone-assisted machine vision technology has emerged with the development of science and technology, combining big data technology to replace traditional manual inspections. Drones equipped with binocular machine vision can both monitor foreign objects in the overhead contact system and intelligently identify surface defects on the rails. The principle is to acquire images of the overhead contact system and rail surfaces using machine vision technology, process the images using a cloud server, and ultimately obtain information on the status of foreign objects in the overhead contact system and the detected surface defects on the rails during the drone's flight path. Summary of the Invention

[0005] The purpose of this invention is to address the current problems of low efficiency and high cost in the field of rail transit inspection, to achieve intelligent identification of defects on the surface of railway catenary and rails by drones, and to replace traditional manual inspection with big data technology. This invention proposes a method for detecting foreign objects in catenary and rail damage based on binocular vision of drones.

[0006] The technical solution implemented by this invention is as follows: a method for detecting foreign objects in overhead contact lines and rail damage based on binocular vision of unmanned aerial vehicles, comprising the following steps:

[0007] (1) Intelligent route planning and safety verification based on digital twin and airspace collaboration

[0008] The system calls a pre-stored digital twin model of the high-speed rail line, which integrates the three-dimensional geographic information of the line, the spatial coordinates of the catenary and rails, the height restriction area, and the known fixed obstacle data. According to the requirements of this inspection task, the system automatically generates the optimal flight path, converts it into a kmz file format, and presets flight parameters, including flight altitude, speed, waypoint coordinates, and the shooting angle and frequency of the binocular camera.

[0009] (2) Real-time environmental perception and robust flight control based on multi-source sensor fusion

[0010] The real-time environmental perception based on multi-source sensor fusion is achieved through tight-coupled fusion perception using a forward-looking binocular vision camera and a lidar.

[0011] LiDAR provides high-precision, texture-independent distance information, while binocular vision provides rich texture and color data. The two work together to construct a real-time 3D occupancy grid map within a 30-meter range in front of the flight by directly fusing point cloud and depth map. The 3D occupancy grid map divides the environment into uniform cubic grids and marks each grid as "idle", "occupied" or "unknown" in real time, providing an accurate environmental model for real-time obstacle avoidance.

[0012] When the flight control system detects dynamic or static obstacles based on the 3D occupancy grid map, it prioritizes calculating and executing a smooth detour path; if detour is not possible, it controls the UAV to perform a fixed-point hover; when an obstacle approaches rapidly or the system experiences a serious anomaly, it ultimately triggers an emergency ascent to a safe altitude and automatic return command, achieving highly robust flight control.

[0013] The system adopts a multi-source fusion positioning scheme of GNSS + RTK + IMU + visual sensing positioning system to cope with the complex high-voltage and strong electromagnetic environment along the high-speed railway; GNSS is the Global Navigation Satellite System, RTK is the real-time dynamic carrier phase differential technology, and IMU is the inertial measurement unit.

[0014] The system positioning module hardware is equipped with an electromagnetic shielding layer, and the software adopts a robust adaptive Kalman filter algorithm to ensure that the horizontal positioning accuracy is ≤0.1 meters and the vertical positioning accuracy is ≤0.05 meters when the GNSS signal is lost for a short period of time.

[0015] The system monitors battery power and motor load in real time. When the remaining power is detected to be below 25%, or the communication link signal strength is below -95dBm for 5 consecutive seconds (in wireless communication, -95dBm is usually at the edge or critical point of usable signal), the system automatically triggers a level one alarm and immediately executes the preset emergency return procedure to fly to the nearest emergency alternate airport.

[0016] (3) Acquire images of equipment along the track.

[0017] The binocular vision camera assembly includes two cameras used to capture images of the same scene from different angles to achieve depth perception; the binocular vision camera assembly mounted on the drone's built-in three-axis stabilization gimbal is activated synchronously with the semi-circular LED combined light source; the LED light source dynamically adjusts its brightness according to the ambient light intensity to ensure uniform illumination under any lighting conditions.

[0018] (4) Image data preprocessing

[0019] The collected drone monitoring trajectory image data is preprocessed to improve image quality and receive image data. The preprocessed drone monitoring trajectory data will form a corresponding dataset and be classified and stored in the corresponding database.

[0020] (5) Construct a high-quality labeled dataset for model training

[0021] (6) Construction of the improved YOLOv11 detection model

[0022] (a) Model basic structure and mathematical expression

[0023] Let the input image be Where H and W are the height and width of the image, respectively; the parameter of model F is θ, and its forward propagation process can be represented as:

[0024] ;

[0025] in: Here, represents the center coordinates and width / height (normalized values) of the predicted i-th bounding box. This represents the center coordinates (x, y) of the prediction box; These represent the width and height values ​​of the prediction box, respectively.

[0026] It is a K-dimensional vector representing the confidence score of the target within the bounding box belonging to K preset categories (such as "insulator", "bird's nest", "crack").

[0027] It is the overall confidence score of the presence of an object within the bounding box.

[0028] N is the total number of bounding boxes predicted by the model;

[0029] The model is trained by minimizing a multi-task loss function. To achieve:

[0030] ;

[0031] The bounding box regression loss uses CIOU Loss, and its calculation formula is as follows:

[0032] ;

[0033] in, For intersection, union, and comparison, Let represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively; р is the Euclidean distance; c is the diagonal length of the minimum closure region covering the predicted and ground truth bounding boxes; α is the balancing parameter; and v measures aspect ratio consistency. and h and w represent the height and width of the ground truth bounding box, respectively, while h and w represent the height and width of the predicted bounding box.

[0034] .

[0035] It is the target confidence loss, which measures the model's ability to determine whether a target exists within the bounding box;

[0036] It is the classification loss, which measures the model's ability to distinguish the target category;

[0037] It is a hyperparameter that balances different loss terms.

[0038] (b) Improved YOLOv11 detection model architecture for UAV inspection of small targets

[0039] One of the core improvements of the improved YOLOv11 detection model is the integration of a coordinated attention module into the Feature Pyramid Network (FPN). A (Coordinate Attention Module). The module... A By analyzing the input feature map Global average pooling is performed along both the height and width directions to encode precise spatial location information, enabling the model to more effectively focus on distant, small-sized defect targets in the image while modeling channel dependencies.

[0040] The calculation process can be simplified as follows:

[0041] ;

[0042] Here, X is the input feature map, and Concat (concatenation) is used to fuse the global information extracted from the height and width dimensions respectively. and These represent global average pooling along the height and width directions, respectively. This is a 1×1 convolution transformation, where σ is the Sigmoid activation function. This represents element-wise multiplication, and Y represents the enhanced output feature map. Important features are enhanced, while irrelevant features are suppressed.

[0043] The second core improvement of the improved YOLOv11 detection model is the replacement of the standard feature pyramid with the Dynamic Texture Attention Feature Fusion Network (BiFPN) designed in this invention. This network introduces a Dynamic Texture Attention (DTA) module in the bidirectional fusion path, which can effectively suppress redundant semantic information of high-level features while enhancing the texture details of small targets in low-level features.

[0044] To further enhance the perception of pixel-level subtle defects (such as small cracks and hanging objects), this model employs high-resolution shallow feature enhancement. Specifically, a 160×160 resolution feature map output layer is explicitly added early in the backbone network, injecting higher-resolution shallow features into the feature fusion process in advance. This provides the model with richer texture and detail information, significantly improving its ability to perceive small, pixel-level targets. Furthermore, it integrates with a fast self-attention mechanism (F_SA), specifically an improved F_SA mechanism integrated at the end of the backbone network. By optimizing the normalization function, it significantly reduces computational overhead while maintaining global feature modeling capabilities.

[0045] (c) The improved YOLOv11 detection model is based on the YOLO architecture and uses fast spatial pyramid pooling (SPPF-CBS) and attention mechanism (CA module) to enhance the detection capability of small targets. The model is optimized by training through loss function (CIOU Loss), which includes classification loss, localization loss and confidence loss. The optimal weight file obtained in step (5) is deployed to the UAV onboard computing platform to complete the loading and initialization of the model and prepare for real-time image inference.

[0046] (d) Lightweight model, edge deployment, and real-time performance guarantee

[0047] To efficiently deploy the trained improved YOLOv11 detection model to the UAV onboard computing platform and meet real-time inspection requirements, this method adopts the following technical solution:

[0048] a. Model lightweighting and optimization

[0049] After the model training is complete, backend processing is performed to adapt it to edge devices:

[0050] Pruning and distillation: Channel pruning is used to remove redundant feature channels, and knowledge distillation is combined to reduce the number of model parameters by about 35% without losing accuracy (mAP decrease <0.02).

[0051] Weight quantization: Post-trained static quantization (PTQ) is used to convert the model weights from FP32 to INT8 precision. This reduces the model size by approximately 75% and significantly improves inference speed.

[0052] Computation graph optimization: By using techniques such as layer fusion and operator replacement, the model's computation graph is optimized to reduce memory access and kernel startup overhead.

[0053] b. Edge Inference Platform and Deployment

[0054] Hardware platform: The NVIDIA Jetson AGX Orin is selected as the onboard computing core, with a GPU computing power of up to 200 TOPS (INT8), which is sufficient to support real-time inference of complex visual models.

[0055] Inference Engine: Utilizes the TensorRT deployment framework. PyTorch-trained models are converted to ONNX format, and then TensorRT generates highly optimized serialized engine files (.engine), fully leveraging the Jetson platform's TensorCore and DLA for mixed-precision accelerated inference.

[0056] Memory management: Implements dynamic memory pool and pipeline scheduling to ensure stable memory usage when continuously processing high-definition image streams, avoiding system lag caused by memory jitter.

[0057] c. Real-time performance and reliability indicators

[0058] After the above optimizations and deployment, the system achieves the following performance on the airborne end:

[0059] Inference speed: For a single high-definition input image of 4096×3072 pixels, the time to complete a full-process inference (including preprocessing, model forward propagation, and postprocessing) is ≤50 milliseconds, that is, the frame rate (FPS) is ≥20 frames / second, which meets the real-time processing requirements of UAVs under normal inspection speed.

[0060] Resource consumption: During model operation, GPU utilization remained stable at 60%-80%, CPU load was <40%, and total system memory usage was <2GB, leaving sufficient resources for other flight control tasks.

[0061] Power consumption and stability: Under typical inspection conditions, the power consumption of the entire computing module is controlled within the range of 15W-25W. The system has undergone high and low temperature tests from -10℃ to 55℃ and vibration tests, with inference performance fluctuations of less than ±5%, and a watchdog mechanism ensures stability during long-term operation.

[0062] Through the above collaborative design from algorithm to system, the improved YOLOv11 detection model can be successfully deployed in the laboratory environment, achieving high-precision and robust real-time detection capabilities at the edge where drone resources are limited.

[0063] (7) Primary localization of key components and regions based on the improved YOLOv11 detection model

[0064] This step is the first stage of the defect detection process. Its core task is to quickly and accurately locate all regions of interest to be detected from a complex background, laying the foundation for fine-grained identification.

[0065] (a) Model inference and coarse localization: The preprocessed real-time image is input into the trained improved YOLOv11 detection model for forward inference.

[0066] (b) Spatial logical relationship constraints: The original output of the model is optimized by using predefined spatial logical knowledge of railway scenarios to improve the accuracy and robustness of positioning.

[0067] (c) Generate standardized region proposals; after optimization, the system outputs a set of standardized region proposals; each proposal includes: component / region type, precise bounding box coordinates.

[0068] (8) Refined defect identification and classification based on location area

[0069] This step is the second stage of the defect detection process. Its core task is to perform detailed scanning and feature analysis inside the located critical component area and rail area to ultimately determine whether there are foreign objects or damage and to identify their specific type.

[0070] (9) Reporting and early warning of test results

[0071] When foreign objects or damage are detected, an early warning message containing the defect type, location, and image is automatically generated and sent to the ground monitoring center.

[0072] The steps for constructing a high-quality labeled dataset for model training are as follows:

[0073] (1) Data preparation and preprocessing: The preprocessed UAV monitoring trajectory image data is used as the original material and divided into training set, validation set and test set according to the preset ratio (e.g. 7:2:1); In order to ensure the consistency of model input and take into account details, the image data in the training set is uniformly adjusted to the standard size of high-definition image (4096×3072) pixels.

[0074] (2) Defect labeling and specification formulation: Use the professional image labeling tool LabelImg to manually label the images in the dataset; during labeling, according to the predefined "Defect Labeling Specification", professionals use rectangles to accurately select two types of targets in the image: rail damage: including cracks, spalling, and falling pieces, and contact wire foreign objects: including bird nests, plastic bags, and hanging objects; each rectangle must be associated with a specific defect type label.

[0075] (3) Annotation data generation: The annotation tool will generate a corresponding XML file for each annotated image; the file stores the following key information in a structured manner: Image name: corresponding to the original image file; Defect coordinates: the coordinates of the rectangle with the upper left corner of the image as the origin (xmin, ymin, xmax, ymax); Defect type: the specific category name of the target within the rectangle.

[0076] (4) Format Conversion: To meet the training requirements of the YOLOv11 detection model, the generated XML format annotation files were batch converted into YOLO format TXT text files using a script tool; in this format, the target information was standardized as follows: <object-class><x_center> <y_center> <width> <height>The coordinates and width and height are normalized values ​​relative to the total width and height of the image.

[0077] (5) Quality verification: Random sampling and cross-validation are performed on the converted labeled data to ensure the accuracy and consistency of the labels, and finally a structured rectangular box image dataset that can be directly used for model training is formed.

[0078] The implementation steps for the refined defect identification and classification are as follows:

[0079] (a) Region Focusing and Sub-image Extraction: Based on the generated standardized region proposal, the system crops out the corresponding component sub-images (such as "insulator sub-image" and "positioning pipe sub-image") and rail sub-images from the original high-definition image;

[0080] (b) Refined feature extraction and scanning: The cropped sub-images are input again into the same improved YOLOv11 detection model; thanks to the enhancement of shallow features in the Feature Pyramid Network (FPN) and the introduction of the attention mechanism, it can effectively extract and identify smaller and more subtle defect features in these high-resolution sub-images.

[0081] (c) Defect Presence Determination and Type Identification: The model infers from the sub-images and outputs the final detection result; its determination logic is based on the model's classification confidence and a preset determination threshold.

[0082] Existence determination: When the confidence level of a target (such as "bird's nest" or "crack") identified by the model in a sub-image is higher than a preset threshold (e.g., 0.7), the system determines that there is a foreign object or damage in that area.

[0083] Type recognition: The model synchronously outputs the specific type of the target;

[0084] The system's default defect type library includes:

[0085] Foreign objects in the overhead contact line include: bird nests, plastic bags, kite strings, or other hanging objects.

[0086] Rail damage includes: cracks, spalling, chipping, corrugation, or crushing;

[0087] Normal determination: If no defect target with a confidence level higher than the threshold is detected in the sub-image, the component or rail area is determined to be "normal".

[0088] (d) Result fusion and output:

[0089] The system merges the location information from step (7) (e.g., on "location tube #X") with the identification result from step (8) (e.g., "Bird's nest exists") to generate a complete and readable defect record, for example: {Component: "location tube #X", Status: "abnormal", Defect type: "bird's nest", Confidence: 0.92, Coordinates: [x1, y1, x2, y2]}. This structured data provides accurate basis for the early warning and reporting in step (9), and the results are merged and output.

[0090] The contact wire foreign object and rail damage detection system based on UAV binocular vision includes a flight and load-bearing platform, an environmental perception module, a positioning module, a data acquisition module, an imaging module, a computing and control module, a communication module, and an energy module. These modules are mounted on the UAV, which wirelessly connects to a computer at the ground control center via the communication module to acquire the data and detection results collected by the UAV.

[0091] The environmental perception module is based on real-time environmental perception through multi-source sensor fusion; the localization module is used to quickly and accurately locate all areas of interest to be detected from complex backgrounds; the data acquisition module is used to acquire data on dynamic or static obstacles detected by the 3D occupancy grid map and texture and color data provided by binocular vision; the imaging module is used to crop corresponding component sub-images and rail sub-images from the original high-definition images; the computing and control module is used for the computing and control of the detection system; the communication module is used for communication between the UAV and the ground; and the energy module provides the power required for UAV detection.

[0092] The beneficial effects of this invention are that, through airspace cooperation and multi-source perception adaptive flight control, it constructs a comprehensive safety defense line from mission planning to real-time obstacle avoidance, anti-interference positioning and emergency return, ensuring the safe operation of UAVs in the complex railway environment of high voltage and strong electromagnetic interference, fundamentally avoiding the risk of conflict with train equipment and environmental obstacles; and the flight operation safety is high.

[0093] This invention acquires high-quality images through a dedicated imaging system and combines them with an improved YOLOv11 detection model (optimized for small targets and railway scenarios) to achieve high-precision and high-efficiency simultaneous identification and location of foreign objects in the overhead contact line and rail damage. Its accuracy and speed far exceed those of manual inspection, and its ability to detect small defects is superior to that of traditional rail inspection vehicles; the detection accuracy and efficiency of UAVs are significantly improved.

[0094] This invention achieves full-process automation from task planning, data acquisition, intelligent recognition to result reporting, forming a complete "detection-diagnosis-early warning" closed loop, which greatly reduces manual intervention, lowers operation and maintenance costs and human error, and the detection system has a high degree of automation and intelligence.

[0095] Through the design of multi-source sensor fusion, hardware electromagnetic shielding and anti-interference algorithms, this invention enables the system to operate stably and reliably in the complex environment of high-voltage and strong electromagnetic fields unique to high-speed rail, ensuring the continuity of detection tasks and the consistency of data. The detection system has strong environmental adaptability and reliability. Attached Figure Description

[0096] Figure 1 A schematic diagram of a drone-based binocular vision system for detecting foreign objects and rail damage on overhead contact lines.

[0097] Figure 2 Flowchart of UAV binocular vision method for foreign object detection in overhead contact lines and rail damage detection;

[0098] Figure 3 Schematic diagram of foreign object detection and rail damage detection using a drone's binocular vision system for overhead contact lines;

[0099] Figure 4 Schematic diagram of a semi-circular LTD combined light source;

[0100] Figure 5 This is an enlarged schematic diagram of a semi-circular LTD combined light source;

[0101] In the picture, 1 is a drone; 2 is an overhead contact line; 3 is a steel rail; and 4 is an LED light. Detailed Implementation

[0102] Figure 1 The diagram shows the working operation of a UAV binocular vision-based contact wire foreign object and rail damage detection system.

[0103] like Figure 1 As shown, the detection system in this embodiment includes: a flight and carrier platform, an environmental perception module, a positioning module, a data acquisition module, an imaging module, a computing and control module, a communication module, and an energy module.

[0104] The core process of the overhead contact line foreign object and rail damage detection system in this embodiment includes data acquisition, defect detection, and result reporting. In the specific implementation steps, data acquisition calls the digital twin model of the high-speed railway line, integrates multi-source environmental perception, acquires images and performs preprocessing; defect detection detects defects in the railway overhead contact line and track based on the annotation database and model improvement; and the detection results are reported and closed-loop operation and maintenance are carried out.

[0105] like Figure 2 The diagram shown is a flowchart of the method for detecting foreign objects in overhead contact lines and detecting rail damage based on binocular vision of unmanned aerial vehicles (UAVs) according to an embodiment of the present invention.

[0106] Figure 3 The diagram shows a UAV's binocular vision for foreign object detection and rail damage detection on the overhead contact line. The UAV 1 flies 40 centimeters above the rail surface. Its high-precision industrial-grade gimbal camera collects images of various types of damage on the surface of the rail 3 at a downward angle of 60° relative to the horizontal plane. These damages include, but are not limited to, chipping, corrugation, and pitting. At the same time, a high-precision camera mounted on the UAV collects images of the overhead contact line 2 at an upward angle of 30° relative to the horizontal plane to detect foreign objects.

[0107] Figure 4 and Figure 5 This is a schematic diagram of a semi-circular LTD combined light source. Figure 5 This is an enlarged schematic diagram.

[0108] The semi-circular LED combination light source is a highly uniform active illumination device specifically designed for binocular vision inspection systems of unmanned aerial vehicles (UAVs). Multiple LEDs are installed in the semi-circular grooves of the semi-circular LED combination light source, forming a single unit. Figure 4 As shown in the enlarged schematic diagram of the semi-circular LED combined light source, see below. Figure 5 As shown, the semi-circular LED combined light source is used to provide a stable and controllable lighting environment during the visual inspection of the high-speed railway contact network and rails. This overcomes the impact of complex lighting conditions such as uneven natural lighting, insufficient light in tunnels, and nighttime operations on image quality, thereby ensuring the reliability and accuracy of subsequent defect and foreign object detection algorithms.

[0109] The system supports remote adjustment of brightness, flicker mode, and illumination angle of the semi-circular LED combination light source via a ground control station, ensuring stability and reliability during operation. Structurally, the light source employs a symmetrical semi-circular cross-section design. By optimizing the installation angle of the semi-circular LED combination light source and the brightness of the LED array, precise control over the direction and intensity distribution of light emission is achieved. Its core design goal is to create uniform light coverage on the target surface, avoiding localized overexposure or shadows. Simultaneously, through the combination of lenses and diffuse reflection materials, it enhances luminous efficiency and reduces stray light interference.

[0110] The installation angle of the semi-circular LED combined light source is usually adjusted and optimized online based on the geometric features of the target being detected and the flight attitude of the UAV, ensuring that the light can be concentrated on the surface of the contact wire and rail within the typical working distance, reducing the illumination of the background environment, thereby enhancing the contrast and detail recognition of the target area in the image.

[0111] In use, semi-circular LED combination light sources are typically installed in pairs on either side of the UAV's binocular vision camera, triggered synchronously with the camera shutter. This effectively freezes the image during high-speed flight, reducing motion blur. In practical deployment, the semi-circular LED combination light sources significantly improve image uniformity and detail, laying a reliable optical foundation for accurate identification of damage and foreign objects. This not only enhances data quality during image acquisition but also provides high-quality input for subsequent deep learning-based visual inspection models, making it a key auxiliary component for achieving all-weather, full-section intelligent inspection of overhead contact lines and rails.

[0112] Based on the UAV binocular vision-based contact wire foreign object detection and rail damage detection system, this embodiment presents a method for UAV binocular vision-based contact wire foreign object detection and rail damage detection, the steps of which are as follows:

[0113] (1) Intelligent route planning and airspace coordination

[0114] The operators first initiate the detection task in the ground station system. The system calls upon a pre-built digital twin model of the high-speed rail line, which integrates the line's three-dimensional geographic information and the equipment's spatial coordinates, and automatically generates the optimal flight path and parameters accordingly. Subsequently, the ground station uploads the flight plan to the railway dispatch center via a dedicated data link, applies for and obtains permission to execute the task within a specific "maintenance window," completes airspace safety coordination, and ensures flight safety from the source.

[0115] (2) Autonomous flight and real-time environmental perception

[0116] This embodiment uses a DJI Air 3S drone as the flight platform. After takeoff, the drone enters autonomous flight mode. The forward-looking binocular camera and LiDAR on the nose work synchronously, and through sensor fusion algorithms, they jointly construct a real-time 3D occupancy grid map within a 30-meter range ahead of the drone. When the fusion perception system detects an obstacle ahead, the Flight Control System (FCS) immediately performs autonomous avoidance according to a preset multi-level safety strategy of "fly around -> hover -> emergency return to home," without any manual intervention throughout the entire process.

[0117] (3) Highly robust positioning and high-quality image acquisition

[0118] In this embodiment, the GNSS signal is severely interfered with when passing under a high-voltage overhead contact line. At this point, the system seamlessly switches to a multi-source fusion positioning mode primarily based on IMU and visual odometry. Due to the addition of a copper-aluminum alloy electromagnetic shielding cover to the positioning module hardware and the implementation of a robust adaptive Kalman filter algorithm, the horizontal positioning error of the UAV in the strong magnetic interference section is controlled within 0.1 meters, and the vertical positioning error within 0.05 meters. Simultaneously, the binocular visual camera group mounted on the three-axis stabilization gimbal and the semi-circular LED combined light source are activated synchronously. The LED light source dynamically adjusts its brightness according to the ambient light intensity, ensuring uniform illumination under any lighting conditions. The camera synchronously acquires high-resolution (e.g., 4096×3072 pixels) images of the contact line and rails at a preset frequency.

[0119] (4) Image preprocessing and model inference

[0120] The acquired image data is transmitted in real time to the onboard computer (NVIDIA Jetson AGX Orin). Preprocessing is then performed, including Gaussian filtering (σ=1.5) for noise reduction and the CLAHE algorithm for contrast enhancement.

[0121] Subsequently, the preprocessed image is input into the deployed improved YOLOv11 detection model for two-stage detection inference:

[0122] First, the model infers from the input panoramic image and outputs precise positioning information of key components of the overhead contact system (such as insulators and locators) and the rail area.

[0123] Subsequently, based on the aforementioned positioning results, the system crops out the corresponding component sub-images and rail sub-images from the original image and inputs them back into the model for fine feature extraction and scanning. When the confidence level of the defect target identified by the model (such as "bird's nest" or "fallen block") is higher than a preset threshold (such as 0.7), it is determined that a defect exists and its specific type is identified.

[0124] (5) Reporting and early warning of test results.

[0125] Finally, the system reports and issues warnings for the detection results. When defects such as "missing blocks" are detected, the system immediately triggers an early warning mechanism. The airborne system transmits complete information, including the defect type, location, and image, to the ground monitoring center via a 5G module.

[0126] The monitoring center's large screen system pops up an alarm and automatically generates a maintenance work order, which is then pushed to the relevant personnel, thus forming a rapid closed loop from "discovery" to "handling".< / height> < / width>

Claims

1. A method for detecting foreign objects and rail damage of a catenary based on binocular vision of a UAV, characterized in that, The method steps are as follows: (1) Intelligent route planning and safety verification based on digital twin and airspace collaboration; (2) Real-time environmental perception and robust flight control based on multi-source sensor fusion; (3) Images of equipment along the track are acquired using a binocular vision camera system; (4) Preprocess the collected UAV monitoring trajectory image data; (5) Construct a high-quality labeled dataset for model training; (6) By integrating a coordinated attention module into the feature pyramid network, a dynamic texture attention feature fusion network is used to replace the standard feature pyramid, and a dynamic texture attention module is introduced into the bidirectional fusion path, thereby constructing an improved YOLOv11 detection model; (7) Defect and foreign object detection: The preprocessed image is input into the deployed improved YOLOv11 detection model for two-level detection inference: The improved YOLOv11 detection model infers the input panoramic image and outputs the precise positioning information of the key components of the catenary and the rail area; then, based on the above positioning results, the corresponding component sub-images and rail sub-images are cropped from the original image and input into the model again for fine scanning and feature analysis to determine the specific types of defects and foreign objects. (8) Uploading and early warning of test results; The improved YOLOv11 detection model is constructed as follows: (a) Basic structure and mathematical expression of the model: Let the input image be where H and W are the height and width of the image, respectively; the parameters of the model F are θ, and its forward propagation process is represented as: ； in: This represents the center coordinates and width and height of the predicted i-th bounding box; It is a K-dimensional vector representing the confidence score of the target in the i-th bounding box belonging to K preset categories; is the overall confidence that a target exists within the i-th bounding box; N is the total number of bounding boxes predicted by the model; The model is trained by minimizing a multi-task loss function. To achieve: ； It is a bounding box regression loss, using CIOU Loss, which can better measure the overlap between the predicted box and the ground truth box, the distance between the center points, and the aspect ratio; It is the target confidence loss, which measures the model's ability to determine whether a target exists within the bounding box; It is the classification loss, which measures the model's ability to distinguish the target category; It is a hyperparameter that balances different loss terms; (b) Improved YOLOv11 detection model: A coordinated attention module is introduced into the feature pyramid network of the model. A The module A It not only considers information between channels but also encodes precise location information, enabling the model to more accurately locate distant, small-sized defect targets; its operation is simplified as follows: ； Where X is the input feature map, Its function is to fuse the global information extracted from the two dimensions of height and width respectively. and These represent global average pooling along the height and width directions, respectively. This is a 1×1 convolution transformation, where σ is the Sigmoid activation function. Y represents element-wise multiplication; important features are enhanced, while irrelevant features are suppressed. Shallow feature enhancement for small targets: The connection between the backbone network and the neck network of the model was adjusted to integrate shallower high-resolution feature maps into the feature pyramid earlier; this provides the model with richer detailed information and significantly improves the feature extraction capability for pixel-level small cracks and minute foreign objects; (c) The improved YOLOv11 detection model is based on the YOLO architecture, fast spatial pyramid pooling and attention mechanism to enhance the detection capability of small targets; the model is trained by optimizing the loss function, which includes classification loss, localization loss and confidence loss; the optimal weight file obtained in step (5) is deployed to the UAV onboard computing platform to complete the loading and initialization of the model and prepare for real-time image inference.

2. The method for detecting foreign objects and rail damage in overhead contact lines based on UAV binocular vision according to claim 1, characterized in that, The intelligent route planning and safety verification based on digital twins and airspace collaboration includes: The system calls upon a pre-stored digital twin model of the high-speed rail line, integrating the line's three-dimensional geographic information, the spatial coordinates of the overhead contact line and rails, height-restricted areas, and known fixed obstacle data. Based on the requirements of this inspection task, the system automatically generates the optimal flight path, converts it into a kmz file format, and presets flight parameters, including flight altitude, speed, waypoint coordinates, and known fixed obstacle data.

3. The method for detecting foreign objects and rail damage in overhead contact lines based on UAV binocular vision according to claim 1, characterized in that, The real-time environmental perception and robust flight control based on multi-source sensor fusion includes: The real-time environmental perception based on multi-source sensor fusion is achieved through tight-coupled fusion perception using a forward-looking binocular vision camera and a lidar. LiDAR provides high-precision, texture-independent distance information, while binocular vision cameras provide rich texture and color data. The two work together to construct a real-time 3D occupancy grid map within a 30-meter range in front of the flight by directly fusing point clouds and depth maps. The 3D occupancy grid map divides the environment into uniform cubic grids and marks each grid as "idle," "occupied," or "unknown" in real time, providing an accurate environmental model for real-time obstacle avoidance. When the flight control system detects dynamic or static obstacles based on the 3D occupancy grid map, it prioritizes calculating and executing a smooth detour path; if detour is not possible, it controls the UAV to perform a fixed-point hover; when an obstacle approaches rapidly or the system experiences a serious anomaly, it ultimately triggers an emergency ascent to a safe altitude and automatic return command, achieving highly robust flight control. The system adopts a multi-source fusion positioning scheme of GNSS + RTK + IMU + visual sensing positioning system to cope with the complex high-voltage and strong electromagnetic environment along the high-speed railway line. The system positioning module hardware is equipped with an electromagnetic shielding layer, and the software adopts a robust adaptive Kalman filter algorithm to ensure that the horizontal positioning accuracy is ≤0.1 meters and the vertical positioning accuracy is ≤0.05 meters when the GNSS signal is lost for a short period of time. The system monitors battery power and motor load in real time. When the remaining power is detected to be below 25% or the communication link signal strength is below -95dBm for 5 consecutive seconds, the system automatically triggers a level one alarm and immediately executes the preset emergency return procedure to fly to the nearest emergency alternate airport.

4. The method for detecting foreign objects and rail damage in overhead contact lines based on UAV binocular vision according to claim 1, characterized in that, The images of the equipment along the track are acquired using a binocular vision camera. The binocular vision camera group includes two cameras, which are used to capture images of the same scene from different angles to achieve depth perception. The binocular vision camera group mounted on the three-axis stabilization gimbal of the drone is activated synchronously with the semi-circular LTD combined light source; the LED light source dynamically adjusts its brightness according to the ambient light intensity to ensure uniform illumination under any lighting conditions. The collected UAV monitoring trajectory image data is preprocessed to improve image quality and facilitate image data reception. The preprocessed UAV monitoring trajectory data will form a corresponding dataset and be classified and stored in the corresponding database, thereby improving image quality and facilitating image data reception.

5. The method for detecting foreign objects and rail damage in overhead contact lines based on UAV binocular vision according to claim 1, characterized in that, The defect and foreign object detection is a two-level detection, which includes primary localization of key components and areas, and refined defect identification and classification. The initial localization of key components and regions is the first stage of the defect detection process. Its core task is to quickly and accurately locate all regions of interest to be detected from a complex background, laying the foundation for fine identification. include: (a) Model inference and coarse localization: The preprocessed real-time image is input into the trained improved YOLOv11 detection model for forward inference; (b) Spatial logical relationship constraints; the original output of the model is optimized by using predefined spatial logical knowledge of the railway scene to improve the accuracy and robustness of positioning; (c) Generate standardized region proposals; After optimization, the system outputs a set of standardized region proposals; Each region proposal includes: component type, region type, and precise bounding box coordinates; The refined defect identification and classification is the second stage of the defect detection process. Its core task is to perform fine scanning and feature analysis inside the located key component area and rail area to ultimately determine whether there are foreign objects or damage and to identify their specific type.

6. The method for detecting foreign objects and rail damage in overhead contact lines based on UAV binocular vision according to claim 5, characterized in that, The implementation steps for the refined defect identification and classification are as follows: (a) Region focusing and sub-image extraction: Based on the generated standardized region proposal, the system crops out the corresponding component sub-images and rail sub-images from the original high-definition image; (b) Refined feature extraction and scanning: The cropped sub-images are input again into the same improved YOLOv11 detection model; thanks to the enhancement of shallow features in the feature pyramid network and the introduction of the attention mechanism, it can effectively extract and identify smaller and more subtle defect features in these high-resolution sub-images. (c) Defect Presence Determination and Type Identification: The model infers from the sub-images and outputs the final detection result; its determination logic is based on the model's classification confidence and a preset determination threshold. Existence determination: When the confidence level of a target identified by the model in a sub-image is higher than a preset threshold, the system determines that there is a foreign object or damage in that area. Type recognition: The model synchronously outputs the specific type of the target; The system's default defect type library includes: Foreign objects in the overhead contact line include: bird nests, plastic bags, or hanging objects. Rail damage includes: cracks, spalling, corrugation, or crushing; Normal determination: If no defect target with a confidence level higher than the threshold is detected in the sub-image, the component or rail area is determined to be "normal". (d) Result fusion and output: The system fuses the location information with the identification results to generate a complete and readable record of defects and foreign objects.

7. A detection system for detecting foreign objects and rail damage in overhead contact lines based on binocular vision of unmanned aerial vehicles as described in any one of claims 1-6, characterized in that, The system includes a flight and carrying platform, an environmental perception module, a positioning module, a data acquisition module, an imaging module, a computing and control module, a communication module, and an energy module. These modules are installed on the UAV, which wirelessly connects to a computer at the ground control center via the communication module to obtain the data and detection results collected by the UAV. The environmental perception module is based on real-time environmental perception through multi-source sensor fusion; the localization module is used to quickly and accurately locate all regions of interest to be detected from complex backgrounds. The data acquisition module is used to acquire data on dynamic or static obstacles detected by the 3D occupancy grid map and texture and color data provided by the binocular vision camera; the imaging module is used to crop out the corresponding component sub-images and rail sub-images from the original high-definition images; The calculation and control module is used for the calculation and control of the detection system; The communication module is used for communication between the UAV and the ground. The energy module provides the power needed for drone inspection.

Images