Railway perimeter foreign matter detection method paying attention to small target foreign matter

Abstract

The invention discloses a railway perimeter foreign matter detection method, and relates to the technical field of rail traffic safety monitoring. The method comprises the following steps: acquiring an image collected along a railway and performing enhancement processing to improve image contrast and detail visibility; the processed image is input into an improved target detection network, the network fuses a lightweight pyramid fusion module and an adaptive boundary compensation module on the basis of a YOLOV11 architecture, and foreign matter types and positions are output by capturing multi-scale context information and optimizing feature boundaries; generating a track perimeter region mask by using a semantic segmentation network, and mapping a foreign matter bounding box into the mask to calculate a pixel coverage ratio; and in combination with multi-frame detection tracking logic, judging whether intrusion is formed according to the dynamic and static states of the foreign matters and the pixel coverage ratio. According to the method, the image quality in a complex environment can be effectively improved, the feature extraction and positioning performance of small target foreign matters is remarkably improved, and accurate railway perimeter safety assessment is realized.

Description

A method for detecting foreign objects at railway perimeters that focuses on small-sized objects. Technical Field

[0001] This invention belongs to the field of rail transit safety monitoring and computer vision technology, specifically involving a method for detecting foreign object intrusion into railway perimeters that combines image enhancement, lightweight target detection, and semantic segmentation. Background Technology

[0002] Foreign object intrusion into the railway perimeter is a significant factor affecting train operation safety. These objects may include pedestrians, livestock, vehicles, falling rocks, and other debris. Once an object intrudes into the track perimeter, it can easily cause collisions, derailments, or operational disruptions. Therefore, continuous and reliable monitoring of the railway perimeter is essential.

[0003] Existing methods for detecting foreign objects at railway perimeters are mostly based on computer vision technology. However, at night or under low light conditions, due to low image brightness, insufficient contrast, significant noise, and loss of detail information, the detection accuracy of traditional detection methods and existing deep learning models decreases significantly, especially in terms of the ability to identify distant, small-sized foreign objects.

[0004] In image enhancement, Limiting Contrast Adaptive Histogram Equalization (CLAHE) is suitable for edge deployment due to its low computational complexity and lack of training requirements, but it suffers from noise amplification, color distortion, and unstable enhancement effects in complex scenes. On the other hand, while deep learning object detection models have high accuracy under normal conditions, their large model size and computational cost make them unsuitable for real-time deployment of edge devices along railway lines.

[0005] In addition, existing detection methods mostly only output the foreign object boundary box, lacking spatial relationship analysis with the railway track safety perimeter, making it difficult to accurately determine whether the foreign object has actually invaded the dangerous area of ​​the track, and easily causing false alarms or missed alarms.

[0006] Therefore, it is necessary to propose a railway perimeter foreign object detection technology that can simultaneously improve image quality, enhance the detection capability of small target foreign objects under low illumination conditions, and combine the determination of intrusion risk in the track area. Summary of the Invention

[0007] To address the aforementioned problems, the present invention aims to provide a method for detecting foreign objects at railway perimeters under low-light conditions. By integrating image enhancement, lightweight target detection, and semantic segmentation for boundary intrusion determination, this method achieves high-precision identification, location, and boundary intrusion determination of foreign objects at railway perimeters in low-light environments. The invention includes the following steps: Step 1. Acquiring railway perimeter images captured by fixed cameras along the railway line, and performing image enhancement preprocessing on the acquired images. This preprocessing includes sequentially performing image denoising, brightness and contrast enhancement, secondary denoising, and optimization based on evaluation results to improve image contrast and detail visibility; Step 2. Inputting the enhanced image into a target detection network to detect foreign objects at the railway perimeter, and outputting the foreign object category, confidence level, and bounding box location information; Step 3. Performing pixel-level segmentation of the railway track perimeter area based on a semantic segmentation network to generate a track perimeter area mask, and mapping the detected foreign object bounding boxes onto this mask. The pixel coverage ratio of the foreign object within the track perimeter area is calculated, and a preset boundary intrusion determination process is used to determine whether the foreign object constitutes a railway perimeter intrusion.

[0008] Compared with the prior art, the present invention has the following beneficial effects: (1) The integrated image enhancement preprocessing step can effectively cope with complex lighting environments such as low illumination, significantly improve image contrast and suppress noise, and provide high-quality input for back-end detection.

[0009] (2) By using a lightweight multi-scale feature fusion and boundary compensation mechanism, the detection accuracy of small target foreign objects is significantly improved while maintaining a low computational cost.

[0010] (3) By integrating the pixel-level intrusion determination rules of semantic segmentation, an interpretable railway perimeter safety assessment with low false alarms can be achieved. Attached Figure Description

[0011] Figure 1 is a schematic diagram of the overall process of the railway perimeter foreign object detection method of the present invention.

[0012] Figure 2 is a schematic diagram of the low-light image enhancement process.

[0013] Figure 3 is a schematic diagram of the target detection network structure.

[0014] Figure 4 is a schematic diagram of the lightweight pyramid fusion module structure.

[0015] Figure 5 is a schematic diagram of the adaptive boundary compensation module.

[0016] Figure 6 is a schematic diagram of the calculation process for the pixel coverage ratio of foreign object intrusion.

[0017] Figure 7 shows the image of the foreign object captured on the railway. Detailed Implementation

[0018] To further understand the content of this invention, the specific embodiments of this invention will be further described below with reference to the accompanying drawings.

[0019] A method for detecting foreign objects at railway perimeter under low illumination conditions, the overall process of which is shown in Figure 1. The method mainly includes: image enhancement processing (step S1), target foreign object detection (step S2), foreign object intrusion judgment and early warning (step S3), target detection test (step S4), and railway perimeter foreign object judgment verification (S5). The specific implementation steps are as follows: Step S1: Image enhancement processing.

[0020] The original images captured by cameras along the railway line are input into the Feature Enhancement AMF-CLAHE fusion algorithm. First, AMF is used to filter out salt-and-pepper noise in the image while effectively preserving image edges and details. Then, CLAHE is applied to significantly improve the overall brightness and local contrast of the image, outputting a high-quality enhanced image. As shown in Figure 2, the algorithm consists of five stages: image input, image preprocessing, image enhancement, image postprocessing, and image output. The specific details of each stage are as follows: Step S1.1 Image Input Stage: Input the low-light image to be enhanced. .

[0021] Step S1.2 Image preprocessing stage: The AMF algorithm is used to preprocess low-light images. Denoising is performed to eliminate noise generated during image acquisition, and the image is recorded. The image obtained after denoising using the AMF algorithm is an image .

[0022] Step S1.3 Image Enhancement Stage: Based on the improved CLAHE algorithm, the denoised low-light image is enhanced. Image enhancement is performed to improve the brightness and contrast of the image. (The image is then recorded.) The enhanced image is an image .

[0023] Step S1.4 Image post-processing stage: The AMF algorithm is used to process the enhanced image. Denoising is performed to eliminate noise generated during image processing, and the image is recorded. The image obtained after denoising using the AMF algorithm is an image .

[0024] Step S1.5 Image Output Stage: As a low-light image The image enhancement results are output as an image. .

[0025] Step S2: Detection of foreign objects.

[0026] To meet the requirements of real-time detection of foreign objects at railway perimeters, this invention proposes a lightweight and efficient AF-YOLO target detection model. Based on the advanced YOLOv11 architecture, this model innovatively integrates a Lightweight Pyramid Fusion (LPF) module and an Adaptive Boundary Compensation (ABC) module, aiming to significantly enhance the feature extraction and localization accuracy for small foreign objects. The overall structure of the model is shown in Figure 3.

[0027] AF-YOLO mainly consists of three parts: the backbone, the neck, and the detection head. In the backbone, a two-level 3×3 convolutional layer is first used for efficient downsampling. Then, a multi-stage cascaded C3k2 module is used to gradually improve the feature abstraction capability. Downsampling operations are embedded in each stage to optimize the feature space distribution. At the end of the backbone, a lightweight pyramid fusion module is introduced to provide cross-level context information, and a C2PSA attention mechanism is combined to enhance the expression of key features.

[0028] In the neck network, based on the bidirectional feature pyramid architecture, cross-resolution feature fusion is achieved through an adaptive boundary compensation module: the top-down path transmits high-level semantic information to mid-to-low-level features, while the bottom-up path enhances the detail perception capability of high-level features, ultimately forming an enhanced feature map covering different resolutions. High-resolution features focus on small target detection, while mid-to-low-resolution features are responsible for the localization of medium-to-large targets.

[0029] In the head portion, the enhanced feature map is directly connected to the efficient detection head to complete the multi-scale target detection task.

[0030] The lightweight pyramid fusion module structure is shown in Figure 4. The specific implementation includes: applying a 1×1 convolutional layer Conv to the input feature map. 1×1 After channel compression and dimensionality reduction, the baseline feature y is obtained. 0： ; where x is the input feature map from the first layer of the backbone network.

[0031] The baseline features are input into a 3×3 convolution with shared weights. By setting different dilation rates, multi-scale contextual information is obtained, generating feature maps y1, y2, and y3 with different receptive fields. Among them, W share This represents a set of 3×3 depthwise separable convolutional kernels with shared weights. Dilation refers to the hole ratio of the convolutional kernel.

[0032] Finally, the baseline feature y0 is concatenated with the scale features y1, y2, and y3 along the channel dimension, and the concatenated features are fused and their dimensions adjusted using a 1×1 convolution to obtain the final output y of the module: The output y of this module provides the deep features from the adaptive boundary compensation module. It contains multi-scale contextual information.

[0033] The LPF module employs a progressive dilation strategy, avoiding the heavy computational burden of directly using large convolutional kernels. At the same time, it ensures the integrity of multi-scale information by explicitly stitching feature maps of branches at different scales.

[0034] The structure of the adaptive boundary compensation module is shown in Figure 5. Its specific implementation includes: firstly, introducing two independent feature interaction attention units (FIAU) blocks.

[0035] The FIAU block uses two independent 1×1 convolutions with a linear mapping process to compress the number of channels in T1 and T2 to 32, generating intermediate features and deep paths. Then, it uses a Sigmoid activation function to generate channel attention weights. ; In the formula , After inputting the linear mapping results corresponding to T1 and T2 into the sigmoid function, two sets of attention weight maps with a value range of (0,1) are generated. The input x specifically refers to the feature vector after projection through the linear layer. , The input consists of shallow semantic features and deep semantic features. and This is to incorporate output features that integrate cross-information.

[0036] The attention weights generated for enhancing shallow semantics and calibrating deep boundaries are optimized bidirectionally: In the formula This represents pointwise multiplication of feature maps. It is the inverse operation of subtracting the feature map.

[0037] Finally, a 3×3 convolutional layer is used to perform final fusion and refinement of the concatenated features, integrating multi-scale and cross-layer enhancement information into a unified feature map Z to optimize the prediction performance of subsequent detection heads. This process is described as follows: ;in It is a 3×3 convolutional layer, followed by a normalization layer and a ReLU activation layer in sequence. and These refer to the detection network fusing deep semantic information and feature boundary details from layers two, three, four, and five. `Concat(·)` represents a concatenation operation along the channel dimension. This represents the output of the adaptive boundary compensation module, where H represents the height of the original image and W represents the width of the original image.

[0038] This aggregation strategy aims to effectively achieve robust combination of cross-level features and optimization of coarse features, thereby providing more accurate and reliable feature representations for subsequent tasks.

[0039] Step S3: Determination of foreign object intrusion. The process is shown in Figure 4. The specific steps are as follows: Step S3.1: Delineation of railway perimeter area.

[0040] In order to accurately identify and locate foreign objects (such as stones, animals, or abandoned objects) that have intruded into the perimeter, pixel-level classification of the perimeter image is required, i.e., semantic segmentation.

[0041] Semantic segmentation is based on high-level semantic information of images, assigning precise category labels to each pixel (such as rails, stones, animals, etc., with the background as an independent category). U-Net is an efficient deep learning semantic segmentation network that can fuse low-level spatial details (rail edges, textures) with high-level semantic features (such as object categories, background differentiation). In the task of detecting foreign objects (such as stones, animals, and abandoned objects) along railway lines, it can also achieve high-precision pixel-level segmentation and localization. Therefore, the U-Net semantic segmentation network is used to obtain pixel-level masks of the track area.

[0042] Step S3.2: Foreign object intrusion determination. The determination rules are as follows: Starting from the first detection of a foreign object at time t=T0, the coordinate changes of the foreign object are tracked through multi-frame detection to determine whether the foreign object is moving or stationary. Specifically, if the detected foreign object is a stationary foreign object, it is directly determined to be an intruding foreign object.

[0043] If the detected foreign object is a moving object, first calculate the area around the railway that the foreign object occupies in the current frame (Kth frame) of the image. The pixel coverage ratio change is shown in Figure 7, and the calculation process is as follows: In the formula, N K Let be the total number of pixels in the railway perimeter area of ​​the Kth frame image, and m and n be the number of pixels covered by the Kth frame image in terms of length and width, respectively.

[0044] Next, determine whether the foreign object leaves the track perimeter area within a preset time. If the pixel coverage ratio of the foreign object after the maximum allowed dwell time meets the following conditions, it is determined to be an intrusion and an alarm is triggered; otherwise, no alarm is triggered: ;in for Pixel coverage ratio at time 10:00 The maximum acceptable error threshold for semantic segmentation.

[0045] Step S4: Target detection test.

[0046] Multiple evaluation metrics are used to assess the performance of the trained object detection model on the test set, and the model is compared and verified with several mainstream object detection algorithms.

[0047] Step S4.1: Performance Evaluation Metrics. To verify the effectiveness of the proposed method in object detection, this invention employs several commonly used object detection evaluation metrics. These metrics include Intersection over Union (IoU), Accuracy (AP) and Average Precision (mAP), Floating-Point Operations (FLOPs), and Inference Speed. For simplicity, mAP50 and mAP are used to represent mAP@0.5 and mAP@0.5:0.95, respectively. Step S4.2: Comparison of Object Detection Results.

[0048] To verify the effectiveness of the multi-scale feature fusion strategy, ablation experiments were conducted on the ABC and LPF modules using YOLOv11-N as the baseline model. The performance differences of the models after removing each module were compared, and the performance of different network structures is shown in Table 1.

[0049] Table 1 Network table of ablation results In the field of object detection, CIoU and PIoU are both loss functions used to optimize bounding box regression. They are improvements on the basic IoU metric and aim to predict the location and size of the target more accurately. This invention uses PIoU instead of CIoU, which can improve mAP by 0.8% and mAP50 by 1.4% without increasing computation.

[0050] Replacing the backbone network with the model constructed using ABC improved mAP by 7.5% and mAP50 by 4.9% compared to the baseline, demonstrating that ABC is beneficial for extracting multi-scale features. Integrating the LPF module in the neck region improved mAP by 7.1% and mAP50 by 5.7%, demonstrating the feature reuse capability of the LPF module.

[0051] Finally, after applying all modules to the baseline model simultaneously, mAP improved by 9.3%, mAP50 improved by 7.8%, and FPS improved by 72 frames / s. This meets the model's requirements for real-time small target detection in railway foreign object detection scenarios.

[0052] Step S5: Verification of foreign objects at the railway perimeter.

[0053] Intrusion incidents are low-frequency, high-consequence events, resulting in limited real-world video data. Therefore, to demonstrate the effectiveness of the proposed intrusion risk assessment method, this invention generates synthetic acquisition data based on normal railway data and a test set.

[0054] Let the moment when the image acquisition device first detects the foreign object be defined, the maximum allowable dwell time of the foreign object within the railway perimeter be defined, and the maximum allowable error of semantic segmentation be defined. For three different types of foreign objects—pedestrians, livestock, and vehicles—with a sampling interval of 1 second, the partial images acquired by the image acquisition device under low-light conditions and under natural light conditions are shown in Figure 7, the percentage of the railway perimeter area at each time point is shown in Table 2, and the final foreign object detection results are shown in Table 3.

[0055] Table 2. Percentage of railway perimeter area corresponding to images at each time point (%) Table 3 Foreign body detection results for G1 and G2 As shown in Table 2, the percentage of the railway perimeter area in the image decreases significantly after a foreign object enters the railway perimeter; however, the percentage returns to its initial value once the foreign object has completely left the railway perimeter. Therefore, by calculating the percentage of the railway perimeter area in the images acquired by the image acquisition device at each time point, it is possible to determine whether a foreign object has left the track area, thereby improving the accuracy of detecting moving foreign objects.

[0056] As shown in Table 3, for foreign objects with poor mobility, such as pedestrians and livestock, which fail to leave the railway perimeter area within a given time period, the railway perimeter foreign object detection system will output an alarm signal at all times; for foreign objects with strong mobility, such as vehicles, which are able to leave the railway perimeter area within a given time period, the railway perimeter foreign object detection system will not output an alarm signal.

[0057] Compared with existing railway perimeter foreign object detection systems, the moving foreign object detection strategy proposed in this invention has good robustness and effectively avoids false alarms of foreign object intrusion hazards on railways.

Claims

1. A method for detecting foreign objects at railway perimeters, focusing on small-sized foreign objects, characterized in that, The method includes the following steps: Step 1. Acquire railway perimeter images collected by fixed camera equipment along the railway line, and perform image enhancement preprocessing on the collected railway perimeter images. The preprocessing includes sequentially performing image denoising, brightness and contrast enhancement, secondary denoising, and optimization steps based on evaluation results to improve image contrast and detail visibility. Step 2. Input the enhanced image into the target detection network to detect foreign objects at the railway perimeter and output the foreign object category, confidence level, and bounding box location information; Step 3. Perform pixel-level segmentation of the railway track perimeter area based on the semantic segmentation network to generate a track perimeter area mask, and map the detected foreign object bounding boxes onto the mask. Calculate the pixel coverage ratio of the foreign object within the track perimeter area and determine whether the foreign object constitutes a violation of the railway perimeter limit according to the preset intrusion determination process.

2. The method for detecting foreign objects at railway perimeters according to claim 1, characterized in that, The object detection network, based on the YOLOv11 architecture, innovatively integrates a lightweight pyramid fusion module and an adaptive boundary compensation module to significantly improve the feature extraction and localization performance of small objects. The specific implementation of the lightweight pyramid fusion module includes: firstly, applying a 1×1 convolutional layer Conv... 1×1 After channel compression and dimensionality reduction, the baseline feature map y0 is obtained: ; where x is the input feature map from the first layer of the backbone network; then, the baseline feature map y0 is input in parallel to a set of 3×3 convolutional layers with the same structure. By setting different dilatation rates for each convolutional layer, feature maps y1, y2, and y3 with different receptive fields are obtained while maintaining independent learning of weights, thereby capturing rich multi-scale contextual information. Among them, W share This represents a set of 3×3 depthwise separable convolutional kernels with shared weights; dilation refers to the dilation rate of the convolutional kernel. Finally, the baseline feature map y0 is concatenated with the multi-scale feature maps y1, y2, and y3 along the channel dimension, and the concatenated features are fused and the number of channels is adjusted through a 1×1 convolutional layer to obtain the final output feature map y of this module. The output feature map y of this module provides the deep features in the adaptive boundary compensation module. The adaptive boundary compensation module contains multi-scale contextual information. Its specific implementation includes: the module accepts features from two sources: semantic features T1 from the shallow layers of the backbone network, containing rich details, and deep semantic features T2 from the lightweight pyramid fusion module, containing multi-scale contextual information. First, the feature pair (T1, T2) is input into two independent feature interaction attention units (FIAU blocks). The FIAU blocks use two independent 1×1 convolutional layers to linearly project T1 and T2, compressing their channel count to 32, generating intermediate features and deep paths, and then using a Sigmoid activation function to generate channel attention weights. ； In the formula 、 After inputting the linear mapping results corresponding to T1 and T2 into the sigmoid function, two sets of attention weight maps with a value range of (0,1) are generated. The input x specifically refers to the feature vector after projection through the linear layer, and T1 and T2 are the shallow semantic features and deep semantic features of the input. and To integrate the output features with cross-information, the generated attention weights are then used to perform bidirectional optimization and information fusion on the original input feature map. In the formula This represents the pointwise multiplication of two feature maps. It is the inverse operation of subtracting the feature map; finally, the optimized feature map is... and The features are concatenated along the channel dimension and then fused and refined using a 3×3 convolutional layer. This integrates multi-scale, cross-level enhancement information into a unified feature map Z to optimize the prediction performance of subsequent detection heads. This process is described as follows: ;in It is a 3×3 convolutional layer, followed by a normalization layer and a ReLU activation layer in sequence. and These refer to the detection network fusing deep semantic information and feature boundary details from layers two, three, four, and five. `Concat(·)` represents a concatenation operation along the channel dimension. This represents the output of the adaptive boundary compensation module, where H represents the height of the original image and W represents the width of the original image.

3. The method for detecting foreign objects at railway perimeters according to claim 1, characterized in that, The intrusion limit determination process is as follows: The moment a foreign object is first detected... Starting from this point, the system tracks the coordinate changes of foreign objects through multi-frame detection to determine whether the foreign object is moving or stationary. Specifically: if the detected foreign object is stationary, it is directly identified as an intrusion into the boundary; if the detected foreign object is moving, the pixel coverage ratio of the foreign object in the railway perimeter area in the current frame (Kth frame) image is first calculated. The calculation formula is as follows: In the formula, Let m be the total number of pixels in the railway perimeter area of ​​the Kth frame image, and m and n be the number of pixels covered by the Kth frame image in terms of length and width, respectively. Continue monitoring is then conducted to determine whether the foreign object leaves the track perimeter area within a preset time. If the foreign object eventually passes the maximum allowed dwell time limit... If the pixel coverage ratio meets the following conditions, it will be judged as an infringement and an alarm will be triggered; otherwise, no alarm will be triggered: ;in for Pixel coverage at time 10:00 The maximum acceptable error threshold for semantic segmentation.

Images