Railway rail foreign object detection method, device, medium and program product
By using pre-trained model screening and context-amplified classification, the accuracy and recall issues of foreign object detection on railway tracks from the perspective of UAVs were solved, achieving high-precision and high-efficiency foreign object detection on railway tracks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING JIAXUN FEIHONG ELECTRIC CO LTD
- Filing Date
- 2026-03-23
- Publication Date
- 2026-06-12
Smart Images
Figure CN122200447A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, device, medium, and program product for detecting foreign objects on railway tracks. Background Technology
[0002] In the railway transportation sector, foreign objects (such as gravel, branches, and scrap metal) on railway tracks are a significant hazard that can cause train derailments, equipment malfunctions, and other safety accidents. Timely and accurate detection of these objects is crucial for ensuring railway transportation safety. With the rapid development of drone technology, using drones equipped with high-definition cameras to inspect railway tracks has gradually become an important method for detecting foreign objects due to its advantages such as wide coverage, high inspection efficiency, and lack of terrain limitations. However, drone-based foreign object detection still faces two major technical challenges: traditional detection methods are insufficient to meet the accuracy and efficiency requirements of practical applications. On the one hand, current mainstream detection algorithms mainly rely on the extraction and recognition of the object's own appearance features (such as shape, texture, color, and outline), lacking effective utilization of the object's contextual environment. In UAV-view images, the visual features of the railway track area and its surrounding environment (such as piles of gravel beside the tracks, green belts, and ballast edges) are somewhat similar. Traditional algorithms only focus on "whether there is a foreign object" and cannot determine "whether the foreign object is located on the tracks." This easily leads to the misdetection of non-threatening objects beside the tracks (such as fallen leaves on the roadside or gravel outside the ballast) as foreign objects, resulting in a large number of false positives. This not only increases the workload of manual verification but may also interfere with normal railway operation scheduling due to false alarms, reducing the practical value of the detection system. On the other hand, UAV inspections usually require acquiring large-scale railway track images at high flight altitudes. This results in extremely low resolution for small targets on the tracks, blurry target features, and even confusion with background features, leading to a significant decrease in the score of small targets, placing them in the same range as the classification scores of false detection boxes. Traditional detection algorithms can only filter out detection boxes in low-scoring ranges by setting high score thresholds, resulting in a significant drop in the detection rate of small targets. How to filter out true small targets from low-scoring ranges while simultaneously eliminating false positives is a challenge in UAV foreign object detection.
[0003] In summary, current methods for detecting foreign objects on railway tracks from the perspective of UAVs suffer from problems such as low detection accuracy (prone to false detection) and low recall rate for small targets. There is an urgent need for a technical solution that can integrate contextual information to improve the accuracy of foreign object location and efficiently solve the problem of small target detection, so as to meet the core requirements of railway inspection for "high precision and high real-time performance" in foreign object detection. Summary of the Invention
[0004] This invention provides a method, equipment, medium, and program for detecting foreign objects on railway tracks, which can improve the accuracy and real-time performance of foreign object detection on railway tracks, increase the recall rate of small targets, and reduce the probability of false detection.
[0005] According to one aspect of the present invention, a method for detecting foreign objects on railway tracks is provided, comprising: The detection image captured by the drone is acquired, and the full-image features, initial detection boxes, and corresponding first classification scores are obtained based on the detection image using a pre-trained detection model. Based on the preset score range and the first classification score, a regular target detection box and a candidate small target detection box are obtained in the initial detection box, and the candidate small target detection boxes are subjected to small target enhancement processing to obtain an enhanced small target detection box; Contextual augmentation classification is performed based on the conventional target detection box, the enhanced small target detection box, and the full-image features to obtain the foreign object detection result on the railway track.
[0006] According to another aspect of the present invention, a detection device for foreign objects on railway tracks is provided, comprising: The initial detection box acquisition module is used to acquire the detection image captured by the drone, and obtain the full-image features, initial detection boxes and corresponding first classification scores based on the detection image using a pre-trained detection model; The enhanced small target detection box acquisition module is used to acquire regular target detection boxes and candidate small target detection boxes in the initial detection box according to a preset score range and the first classification score, and to perform small target enhancement processing on the candidate small target detection boxes to acquire enhanced small target detection boxes; The rail foreign object detection result acquisition module is used to perform contextual augmentation classification based on the conventional target detection box, the enhanced small target detection box, and the full image features to acquire the rail foreign object detection result.
[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the rail foreign object detection method according to any embodiment of the present invention.
[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, the computer program being configured to cause a processor to execute and implement the method for detecting foreign objects on railway tracks according to any embodiment of the present invention.
[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method for detecting foreign objects on railway tracks as described in any embodiment of the present invention.
[0010] The technical solution of this invention acquires detection images captured by a drone, and uses a pre-trained detection model to obtain full-image features, initial detection boxes, and corresponding first classification scores based on the detection images. Based on a preset score range and the first classification score, regular target detection boxes and candidate small target detection boxes are obtained from the initial detection boxes, and small target enhancement processing is performed on the candidate small target detection boxes to obtain enhanced small target detection boxes. Contextual augmentation classification is performed based on the regular target detection boxes, enhanced small target detection boxes, and full-image features to obtain the railway track foreign object detection result. By selecting candidate small target detection boxes based on a preset score range, performing small target enhancement processing on the candidate small target detection boxes, and using contextual augmentation classification to obtain the final railway track foreign object detection result, the accuracy and real-time performance of railway track foreign object detection can be improved, the small target recall rate can be increased, and the false detection probability can be reduced.
[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 This is a flowchart of a method for detecting foreign objects on railway tracks according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the yolov11-p2 model provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of small target detection results provided in Embodiment 1 of the present invention; Figure 4 This is a flowchart of the context augmentation classification method provided in Embodiment 1 of the present invention; Figure 5 This is a flowchart of the context augmentation classification method based on bilinear interpolation provided in Embodiment 1 of the present invention; Figure 6This is a flowchart of a context augmentation classification method based on cross-attention according to Embodiment 1 of the present invention; Figure 7 This is a flowchart of a method for detecting foreign objects on railway tracks according to Embodiment 2 of the present invention; Figure 8 This is a schematic diagram of a small target enhancement image provided according to Embodiment 2 of the present invention; Figure 9 This is a flowchart of another method for detecting foreign objects on railway tracks according to Embodiment 2 of the present invention; Figure 10 This is a schematic diagram of the structure of a foreign object detection device for railway tracks provided in Embodiment 3 of the present invention; Figure 11 This is a schematic diagram of the structure of an electronic device that implements the method for detecting foreign objects on railway tracks according to an embodiment of the present invention. Detailed Implementation
[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0015] It should be noted that the terms "first," "second," "modification," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0016] Example 1 Figure 1 This is a flowchart of a method for detecting foreign objects on railway tracks according to Embodiment 1 of the present invention. This embodiment is applicable to situations where foreign object detection on railway tracks is performed from the perspective of an unmanned aerial vehicle (UAV). The method can be executed by a foreign object detection device, which can be implemented in hardware and / or software. Typically, the foreign object detection device can be configured in electronic equipment, such as computer equipment or servers. Figure 1As shown, the method includes: S110. Acquire the detection image captured by the drone, and obtain the full-image features, initial detection boxes, and corresponding first classification scores based on the detection image using a pre-trained detection model.
[0017] In this embodiment, a drone can be controlled to inspect railway tracks and capture a series of large-scale railway track images using its onboard camera, which are then used as detection images. These detection images can then be input into a pre-trained detection model to obtain the full-image features, multiple initial detection boxes, and the classification score corresponding to each initial detection box.
[0018] The detection model can be a YOLOv11-P2 model to enhance small target recall. The structure of the YOLOv11-P2 model can be as follows: Figure 2 As shown, a C2 layer is added to the backbone network based on the standard YOLOv11 model. The standard YOLOv11 model fuses C3, C4, and C5 features with downsampling ratios of 8, 16, and 32 to generate P3, P4, and P5, and then performs detection on them. At this point, the lowest downsampling ratio is 8, meaning that information about objects smaller than 8 pixels is likely to be lost. To enhance the small target detection performance, this embodiment adds a P2 layer with higher feature resolution to handle the detection of even smaller targets. By simultaneously feeding the features from the C2, C3, C4, and C5 layers of the backbone network into the multi-scale feature fusion (PAN) module, P2 layer features are generated. The corresponding feature downsampling ratio is 4, meaning that the features of smaller targets are preserved, and small targets can be matched with positive samples, increasing the small target recall rate.
[0019] Optionally, obtaining an initial detection box and its corresponding first classification score based on the detected image using a pre-trained detection model may include: Using a pre-trained detection model, combined with a task alignment allocator method and a center point matching allocator method, an initial detection box and the corresponding first classification score are obtained based on the detection image.
[0020] Small object detection is not only limited by feature resolution, but also severely affected by the number of positive samples matching small objects in label matching. Therefore, based on the introduction of the P2 layer, this embodiment modifies the label matching method of the YOLOv11 model to enhance the detection performance of small objects by increasing the number of positive samples matching small objects.
[0021] It's worth noting that early YOLO models used the Max-IOU-Assigner label matching method, suitable for anchor-based detection algorithms. This method calculates the Intersection over Union (IOU) between the anchor and all target objects, and identifies the target with the highest IOU. If the IOU between the anchor and the target exceeds a positive sample threshold (typically 0.5), the current anchor is used as a positive sample for the target, responsible for predicting the target's class and bounding box. However, when none of the IOUs exceed the positive sample threshold of 0.5, to prevent small targets from being unrecoverable due to a lack of positive samples, the Max-IOU-Assigner uses the anchor with the highest IOU as its positive sample, regardless of whether the IOU exceeds the positive sample threshold.
[0022] The Task-Aligned Assigner (TAL-Assigner) method determines the TOPK candidate points of the target as positive samples by using a joint quality score of classification score and localization quality. These final sample points have the advantages of both high classification score output and high localization quality, enabling the model output to have both high score and high localization quality, significantly improving the model's detection performance. The TAL-Assigner method uses all feature points located within the target bounding box as candidate positive sample points and the corresponding detection boxes as positive samples. It is worth noting that the TAL-Assigner method does not take any remedial measures when the target does not match any positive sample points, meaning small targets may not be recalled at all.
[0023] Furthermore, the center-point matching assigner method (Yolov5-Assigner) is a center-point-based matching approach. It uses the three feature points closest to the target's center point in the image features as positive sample points for the target. This approach avoids the inability to recall small targets due to a lack of matching positive samples, thereby improving the recall rate of small targets and preventing missed detections. Therefore, to combine the advantages of TAL-Assigner in obtaining high-quality target scores and high-precision positive samples with the strengths of Yolov5-Assigner and Max-IOU-Assigner in forcibly matching positive sample points for small targets to enhance their recall, this embodiment combines Yolov5-Assigner and TAL-Assigner. First, it uses the TAL-Assigner method to obtain all feature points within the target bounding box, then uses the Yolov5-Assigner method to obtain the three feature points closest to the target's center point. Finally, it merges the obtained feature points to obtain candidate positive sample points for the target. This ensures that the target has at least three candidate positive sample points in any situation. Finally, the anchors corresponding to the candidate positive sample points are obtained as initial detection boxes, and the first classification score corresponding to each initial detection box is obtained.
[0024] It is worth noting that when the candidate positive sample points of a small target are located within the target box of a large target, in the case of such a conflict, the conflicting feature points are assigned to the smaller target, thereby ensuring that the small target has enough candidate positive sample points.
[0025] S120. Based on the preset score range and the first classification score, obtain a regular target detection box and a candidate small target detection box in the initial detection box, and perform small target enhancement processing on the candidate small target detection boxes to obtain an enhanced small target detection box.
[0026] It should be noted that the current standardized post-processing workflow of the detection model first filters out duplicate detection boxes of the same object from all detection boxes. Then, it uses a score threshold of 0.25 to extract object detection boxes with a classification score higher than 0.25, discarding unreliable detection boxes with scores lower than 0.25 (the lower the score, the less reliable). However, analysis revealed that the score range below 0.25 contains a large number of small objects because the original image is scaled to 640p for the YOLO model input. At a resolution of 640, the resolution of small targets decreases, leading to feature loss or gradual blurring of features with background areas, resulting in a lower classification score. For example... Figure 3As shown, taking a vehicle scene with dense small targets as an example, the YOLOv11-P2 model is used for detection. The left part of the image shows detection boxes with a classification score higher than 0.25, and the right part shows detection boxes with a classification score in the range of [0.005, 0.2]. It can be observed that there are a large number of real small targets in the low score range, as well as some false detection boxes. Since there is no effective means to distinguish between real small targets and false detection boxes in the low score range, the traditional post-processing process uses a high score threshold such as 0.25 to suppress unreliable detection boxes with lower classification scores.
[0027] Traditional post-processing workflows use high score thresholds to suppress unreliable detection boxes with low classification scores. This is effective for detection tasks where most targets are medium to large, avoiding false detections. However, for scenarios with many small targets, setting a high score threshold leads to a large number of missed detections, while lowering the threshold results in numerous false detections. Therefore, how to filter out small targets in the low-score range without causing false detections is a pressing issue. To address this problem, this embodiment proposes a small target mining post-processing scheme. This scheme mines small targets in the model output through two stages: small target recall and secondary detection at the original small target resolution, while simultaneously eliminating false detections.
[0028] The preset score range can include a first score range for regular targets (e.g., greater than or equal to 0.25) and a second score range for small targets (e.g., [0.005, 0.2]). Specifically, among all initial detection boxes, detection boxes with a first-class score greater than or equal to 0.25 are selected as regular target detection boxes. Simultaneously, detection boxes with a first-class score in the [0.005, 0.2] range are selected as candidate small target detection boxes. It can be understood that detection boxes with a first-class score in the [0.005, 0.2] range include both real small targets and false positives of background areas; these are collectively referred to as candidate small target detection boxes, thereby achieving recall of all potential small targets.
[0029] Next, a second detection can be performed on the recalled potential small targets at the original resolution. This second detection method distinguishes between true small targets and false positives, retaining only the true small targets. It should be noted that the core reason for the low classification score of small targets is that the original image is scaled to 640 pixels by the YOLO model. After inputting 640, the resolution of small targets decreases, resulting in the loss of their features or confusion between their features and background features. Therefore, restoring the resolution of small targets, restoring the discriminative features of small targets, and distinguishing the features of small targets from background features are the core means to distinguish low-scoring small targets from false detections.
[0030] To enhance the resolution of small targets, this embodiment preserves the resolution of each potential small target in the original detection image and re-detects it at a higher resolution. Specifically, the candidate small target detection boxes are cropped in the detection image to obtain sub-images at the original resolution, and these sub-images are then stitched together to form a 640-pixel array. A stitched image of 640 pixels is generated, with missing parts filled with blank images. Next, the stitched image can be input into the YOLOv11-P2 model for secondary detection, obtaining each current detection box and its corresponding classification score. Detection boxes with a classification score greater than or equal to 0.25 are then selected as augmented small object detection boxes.
[0031] S130. Perform context augmentation classification based on the conventional target detection box, the enhanced small target detection box, and the full-image features to obtain the foreign object detection result on the railway track.
[0032] Specifically, the process of context augmentation classification can be as follows: Figure 4 As shown, candidate foreign object detection boxes (box1~boxn) are composed of regular object detection boxes and enhanced small object detection boxes. These candidate boxes are then augmented and cropped to obtain sub-images (crop_img1~crop_imgn) centered on each candidate foreign object. Finally, each sub-image is input into a pre-trained ResNet50 classification model for secondary classification, yielding the target foreign object detection box and its corresponding classification score, which serves as the result of the railway track foreign object detection. It is understandable that because the cropped sub-images contain background information from the candidate foreign object's context, they can effectively distinguish between foreign objects on the railway track and those in the background. The YOLOv11-P2 model and the ResNet50 classification model can be trained on open-source datasets.
[0033] Optionally, performing context-based augmentation classification based on the conventional target detection box, the enhanced small target detection box, and the full-image features to obtain the foreign object detection result on the railway track may include: Based on the conventional target detection box and the enhanced small target detection box, the current target detection box is obtained, and based on the full image features, the image features corresponding to the current target detection box are obtained; A bilinear interpolation method is used to interpolate the image features based on the full-image features to obtain the foreign object context features, and based on the foreign object context features, the third classification score corresponding to the current target detection box is obtained.
[0034] In an optional example, bilinear interpolation can be used for context augmentation classification. For instance, the process of a context augmentation classification method based on bilinear interpolation can be as follows: Figure 5As shown in the diagram, the small detection box in the left image is the current target detection box. It is uniformly sampled to obtain 9 sampling points, and the image features corresponding to each sampling point are extracted from the full image features to obtain the image features corresponding to the current target detection box. The large detection box in the left image is a context augmentation box obtained by enlarging the current target detection box to 150 pixels, which is sufficient to cover the area of the railway track near the foreign object. Sixteen sampling points are evenly distributed in the augmented region to extract the candidate foreign object context information, for a total of 25 sampling points in both parts.
[0035] Next, image features corresponding to the amplified sampling points are extracted from the full image features, and the image features of the 25 extracted sampling points are combined to form the foreign object context features. Furthermore, the shape of the foreign object context features is converted to 1... C 5 5 (C is the number of channels of the image features), and the extracted features are further modeled using stacked 3-layer convolutions, and finally the third classification score is output, which represents the probability that the current target detection box represents an object located on the railway track.
[0036] It should be noted that the augmented bounding box may result in incomplete or inaccurate extraction of the contextual information about the railway tracks. However, the bilinear interpolation method still uses the same augmented bounding box but does not face the above problem. This is because bilinear interpolation directly interpolates from the image features extracted by the YOLOv11-P2 model. The image features of the YOLOv11-P2 model are extracted from the entire image and have already modeled the complete information about the railway tracks. Specifically, all feature points corresponding to the railway track locations in the image features contain the features of the railway tracks. Therefore, as long as there is even one point in the bilinear interpolation that is located in any region of the railway tracks, the complete features of the railway tracks can be obtained.
[0037] Optionally, performing context-based augmentation classification based on the conventional target detection box, the enhanced small target detection box, and the full-image features to obtain the foreign object detection result on the railway track may include: Based on the full-image features, multi-scale fusion features are obtained, and based on the multi-scale fusion features, image feature location codes are obtained. Based on the conventional target detection box and the enhanced small target detection box, obtain the current target detection box, and based on the current target detection box and the multi-scale fusion feature, obtain query information; The fourth classification score corresponding to the current target detection box is obtained by using a cross-attention network based on the multi-scale fusion features, the image feature location encoding, and the query information.
[0038] In another alternative example, the bilinear interpolation-based scheme suffers from sparse sampling points; 25 sampling points may not be enough to extract complete contextual information. Therefore, a cross-attention mechanism is proposed to densely extract features from both the candidate object itself and its context. For example, the flow of the cross-attention-based context augmentation classification method can be as follows: Figure 6 As shown in the diagram, firstly, image features p2, p3, p4, and p5 at different scales are scaled to a uniform scale, and then feature fusion is performed on the unified scale image features to obtain the multi-scale fused feature F_fusion. Subsequently, position encoding is calculated on F_fusion to obtain the image feature position encoding pos_embedding.
[0039] Next, the current target detection bounding box (box) is obtained, and its coordinates are converted to query_pos. Simultaneously, features corresponding to the box are extracted using F_fusion as query_content. Then, query_pos and query_content are combined to obtain the query information. Finally, F_fusion, pos_embedding, and query are input into a three-layer cross-attention layer. The query continuously extracts candidate object features and their contextual information from the image features within the box. Ultimately, a classification head outputs the probability that each candidate object belongs to the object on the railway track, which is the fourth classification score corresponding to the current target detection bounding box.
[0040] In this embodiment, the context augmentation classification method based on bilinear interpolation and cross-attention is implemented by adding new modules to the YOLOv11-P2 model. Therefore, as a whole model, it does not require multiple feedings of cropped boxes of multiple candidate targets into the classification model for inference, as is required in the detection model plus classification model approach. Both schemes in this embodiment only require the model to run once to obtain the result. The increased inference time is 0.5 to 1.5 times that of the original YOLOv11-P2 model (the more boxes output by YOLOv11-P2, the longer the subsequent classification branch based on bilinear interpolation and cross-attention will run), which is much less than n times that of the detection plus classification approach (n is the number of candidate boxes, which can be 1-100 or even higher).
[0041] In a specific example, the number of false detections was compared on a test set of 1000 images of railway tracks with foreign objects (most of which were without foreign objects and were considered interference samples). The results show that the pure YOLOv11-P2 detection model, the YOLOv11-P2+ResNet50 detection model combined with a classification model, the YOLOv11-P2+bilinear interpolation model, and the YOLOv11-P2+cross-attention model all had the highest false detection rate. Using ResNet50 to perform secondary classification on the cropped images corresponding to the YOLOv11-P2 output boxes significantly reduced false detections, but it also significantly increased runtime. The YOLOv11-P2+bilinear interpolation model overcame the insufficient contextual information problem of the YOLOv11-P2+ResNet50 model, further reducing false detections. The YOLOv11-P2+cross-attention model achieved the highest performance by extracting the contextual information of candidate foreign objects from the global image through queries.
[0042] Table 1 Comparison of False Detection Counts The technical solution of this invention acquires detection images captured by a drone, and uses a pre-trained detection model to obtain full-image features, initial detection boxes, and corresponding first classification scores based on the detection images. Based on a preset score range and the first classification score, regular target detection boxes and candidate small target detection boxes are obtained from the initial detection boxes, and small target enhancement processing is performed on the candidate small target detection boxes to obtain enhanced small target detection boxes. Contextual augmentation classification is performed based on the regular target detection boxes, enhanced small target detection boxes, and full-image features to obtain the railway track foreign object detection result. By selecting candidate small target detection boxes based on a preset score range, performing small target enhancement processing on the candidate small target detection boxes, and using contextual augmentation classification to obtain the final railway track foreign object detection result, the accuracy and real-time performance of railway track foreign object detection can be improved, the small target recall rate can be increased, and the false detection probability can be reduced.
[0043] Example 2 Figure 7 This is a flowchart of a method for detecting foreign objects on railway tracks according to Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with one or more of the above implementation methods. Figure 7 As shown, the method includes: S210. Acquire the detection image captured by the drone, and obtain the full-image features, initial detection boxes, and corresponding first classification scores based on the detection image using a pre-trained detection model.
[0044] S220. Based on the preset score range and the first classification score, obtain the regular target detection box and the candidate small target detection box in the initial detection box.
[0045] S230. Based on the detected image, obtain the candidate small target image corresponding to the candidate small target detection box, and based on the candidate small target image, obtain the small target enhancement image.
[0046] Specifically, the coordinate information corresponding to the candidate small target detection box can be obtained, and based on the coordinate information, the region corresponding to the candidate small target detection box can be cropped from the detection image to obtain the corresponding candidate small target image; then, the candidate small target images can be stitched together according to a preset size, and blank images can be used to fill the missing positions to generate a small target enhancement image.
[0047] Optionally, obtaining the candidate small target image corresponding to the candidate small target detection box based on the detected image may include: The candidate small target detection box is amplified to obtain an amplified small target detection box, and the detection image is cropped according to the amplified small target detection box to obtain the candidate small target image corresponding to the candidate small target detection box.
[0048] In one optional example, when generating candidate small target images, the candidate small target detection boxes are first amplified based on a preset amplification ratio to obtain amplified small target detection boxes. Then, the amplified small target detection boxes are cropped from the original detection image to obtain the corresponding candidate small target images.
[0049] Optionally, obtaining an enhanced image of a small target based on the candidate small target image may include: Obtain a blank image template, and based on the blank image template and the candidate small target image, obtain the small target enhancement image.
[0050] In an optional example, a small target augmentation image can be as follows: Figure 8 As shown, 640 is preset. A 640-pixel blank image is used as a blank image template; then, the cropped candidate small target images are pasted into the blank image template to generate... Figure 8 The image on the left shows an enhanced image of a small target. Furthermore, after secondary detection on the enhanced image of the small target, only detection boxes with a classification score greater than 0.25 are displayed. The detection results are as follows: Figure 8 As shown on the right.
[0051] Because the small targets maintain the resolution of the original large image during the secondary detection process, there is no feature loss or confusion caused by the resolution reduction due to image scaling. The classification score of the small target detection boxes is significantly enhanced, increasing from the original 0.005-0.2 to between 0.37 and 0.88. Simultaneously, false positives are suppressed (the classification scores of false positives remain in the 0.005-0.2 range, and may even decrease considerably due to the use of the original resolution for secondary detection). Through this enhanced post-processing stage, consisting of candidate small target recall and secondary detection at the original resolution, a significant improvement in small target recall is achieved without introducing false positives.
[0052] S240. The small target enhancement image is re-detected using a pre-trained detection model to obtain each intermediate small target detection box and its corresponding second classification score. Based on the preset score range and the second classification score, the enhanced small target detection box is obtained from each intermediate small target detection box.
[0053] Specifically, the enhanced image of the small target is input into the YOLOv11-p2 model for secondary detection, resulting in multiple intermediate small target detection boxes output by the YOLOv11-p2 model, along with the corresponding secondary classification score for each intermediate small target detection box. Finally, among all the intermediate small target detection boxes, those with a secondary classification score greater than or equal to 0.25 are selected as the final enhanced small target detection boxes.
[0054] S250. Perform contextual augmentation classification based on the conventional target detection box, the enhanced small target detection box, and the full-image features to obtain the foreign object detection result on the railway track.
[0055] The technical solution of this invention involves obtaining candidate small target images corresponding to candidate small target detection boxes based on the detected image, and obtaining small target enhanced images based on the candidate small target images; performing secondary detection on the small target enhanced images using a pre-trained detection model to obtain each intermediate small target detection box and its corresponding second classification score, and obtaining enhanced small target detection boxes in each intermediate small target detection box based on a preset score range and the second classification score; by using secondary detection of small target detection boxes with unchanged resolution, the recall rate and detection accuracy of small target foreign objects can be improved.
[0056] In this invention, addressing the mismatch between traditional detection algorithms and the UAV perspective in terms of architecture, pre-training objectives, and training tasks for detecting foreign objects on railway tracks, a decoupling of the railway track foreign object detection task into two tasks: foreign object recall and foreign object classification. In the first task, foreign object recall, the goal is to use the detection model to recall as many objects as possible that resemble railway track foreign objects (potential foreign objects), regardless of whether the objects are actually on the tracks. Therefore, no contextual information needs to be modeled, which is no different from traditional detection tasks; thus, the high-performance YOLOv11 detection model is directly used. In the second task, foreign object classification, both the intrinsic features and contextual features of the potential foreign objects detected in the first task are extracted and classified simultaneously, thereby excluding objects not on the tracks and leaving only those on the tracks as output.
[0057] In addition, to address the small object detection problem, the label matching method of YOLOv11 was modified to improve the recall rate of small objects. Furthermore, a two-stage approach was proposed in the post-processing: potential small object recall and precise small object detection. In the first stage, low-scoring detection boxes with scores in the range of 0.005 to 0.20 were selected from the model output. Their bounding boxes were then augmented and cropped onto a larger image at the original resolution. The cropped image patches were then stitched onto a larger image, which was then fed into the detection model for secondary detection.
[0058] In one specific embodiment of this example, the procedure for detecting foreign objects on railway tracks can be as follows: Figure 9 As shown, firstly, the detected image is input into the improved YOLOv11 model. The YOLOv11 detection head outputs potential foreign object detection boxes and full-image features, extracting low-classification-score detection boxes within a specific score range as candidate small object detection boxes, while the remaining areas are regular object detection boxes. Then, the candidate small object detection boxes are fed into the small object enhancement module to extract enhanced small object detection boxes (true small object detection boxes), while suppressing false detection boxes. Finally, the enhanced small object detection boxes and regular object detection boxes are merged and fed together with the full-image features into the context augmentation classification branch to perform secondary classification on all input object detection boxes, outputting the railway foreign object detection box and the probability that it belongs to a foreign object located on the railway track.
[0059] It should be noted that YOLO models generally use the TAL label matching strategy. While this strategy enhances the detection performance of medium and large targets, it requires that any positive sample matching any target must be inside the target bounding box. This results in very few positive samples for some small targets, or even no sample points within their bounding boxes, thus failing to match any positive samples, being unable to participate in training, and consequently failing to recall. This embodiment significantly increases the recall rate of small targets by providing effective positive sample matching for small targets, effectively solving the problem of missed detections of small targets.
[0060] Furthermore, because drone-view images have a large field of view, the target occupies a small portion of the image, and are scaled down to YOLO 640, they are more accurate to view the target. After reaching a resolution of 640, the resolution of small objects like railway tracks is low, resulting in blurred target features that are even confused with background features. This leads to a significant drop in the score of small objects, placing them in the same range as the classification scores of false positives. Traditional detection algorithms can only filter out these low-score detection boxes by setting a high score threshold, causing a significant decrease in the detection rate of small objects. This embodiment significantly improves the recall rate of small objects and suppresses false positives through a two-stage post-processing approach: small object recall followed by secondary detection at the original resolution.
[0061] Finally, traditional detection models cannot extract object context information, and therefore cannot complete the task of detecting foreign objects on railway tracks. This embodiment proposes using bilinear interpolation and a cross-attention-based method to extract object features and rich context information within the candidate foreign object boxes output by YOLO, providing an efficient solution for category classification that requires context extraction.
[0062] Example 3 Figure 10 This is a schematic diagram of a foreign object detection device for railway tracks provided in Embodiment 3 of the present invention. Figure 10 As shown, the device includes: an initial detection box acquisition module 310, an enhanced small target detection box acquisition module 320, and a rail foreign object detection result acquisition module 330; wherein, The initial detection box acquisition module 310 is used to acquire the detection image captured by the drone, and obtain the full-image features, initial detection boxes and corresponding first classification scores based on the detection image using a pre-trained detection model. The enhanced small target detection box acquisition module 320 is used to acquire regular target detection boxes and candidate small target detection boxes in the initial detection box according to a preset score range and the first classification score, and to perform small target enhancement processing on the candidate small target detection boxes to acquire enhanced small target detection boxes; The rail foreign object detection result acquisition module 330 is used to perform context augmentation classification based on the conventional target detection box, the enhanced small target detection box and the full image features to acquire the rail foreign object detection result.
[0063] The technical solution of this invention acquires detection images captured by a drone, and uses a pre-trained detection model to obtain full-image features, initial detection boxes, and corresponding first classification scores based on the detection images. Based on a preset score range and the first classification score, regular target detection boxes and candidate small target detection boxes are obtained from the initial detection boxes, and small target enhancement processing is performed on the candidate small target detection boxes to obtain enhanced small target detection boxes. Contextual augmentation classification is performed based on the regular target detection boxes, enhanced small target detection boxes, and full-image features to obtain the railway track foreign object detection result. By selecting candidate small target detection boxes based on a preset score range, performing small target enhancement processing on the candidate small target detection boxes, and using contextual augmentation classification to obtain the final railway track foreign object detection result, the accuracy and real-time performance of railway track foreign object detection can be improved, the small target recall rate can be increased, and the false detection probability can be reduced.
[0064] Optionally, the initial detection box acquisition module 310 is specifically used to acquire the initial detection box and the corresponding first classification score based on the detection image by using a pre-trained detection model, combined with a task alignment allocator method and a center point matching allocator method.
[0065] Optionally, the enhanced small target detection box acquisition module 320 includes: The small target enhancement image acquisition unit is used to acquire a candidate small target image corresponding to the candidate small target detection box based on the detection image, and to acquire a small target enhancement image based on the candidate small target image; The enhanced small target detection box acquisition unit is used to re-detect the small target enhanced image through a pre-trained detection model, obtain each intermediate small target detection box and the corresponding second classification score, and obtain the enhanced small target detection box in each intermediate small target detection box according to the preset score range and the second classification score.
[0066] Optionally, the small target enhancement image acquisition unit is specifically used to amplify the candidate small target detection box, acquire the amplified small target detection box, and crop the detection image according to the amplified small target detection box to acquire the candidate small target image corresponding to the candidate small target detection box.
[0067] Optionally, the small target enhancement image acquisition unit is further configured to acquire a blank image template, and acquire the small target enhancement image based on the blank image template and the candidate small target image.
[0068] Optionally, the rail foreign object detection result acquisition module 330 is specifically used to acquire the current target detection box based on the conventional target detection box and the enhanced small target detection box, and to acquire the image features corresponding to the current target detection box based on the full image features; A bilinear interpolation method is used to interpolate the image features based on the full-image features to obtain the foreign object context features, and based on the foreign object context features, the third classification score corresponding to the current target detection box is obtained.
[0069] Optionally, the rail foreign object detection result acquisition module 330 is also used to acquire multi-scale fusion features based on the full-image features, and to acquire image feature location codes based on the multi-scale fusion features; Based on the conventional target detection box and the enhanced small target detection box, obtain the current target detection box, and based on the current target detection box and the multi-scale fusion feature, obtain query information; By using a cross-attention network, the fourth classification score corresponding to the current target detection box is obtained based on the multi-scale fusion features, the image feature location encoding, and the query information.
[0070] The rail foreign object detection device provided in this embodiment of the invention can execute the rail foreign object detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.
[0071] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0072] Example 4 Figure 11 A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device 40 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 40 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0073] like Figure 11As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 42 or loaded from the storage unit 48 into the random access memory 43. The RAM 43 can also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.
[0074] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0075] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as methods for detecting foreign objects on railway tracks.
[0076] In some embodiments, the method for detecting foreign objects on the rails can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the method for detecting foreign objects on the rails described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to perform the method for detecting foreign objects on the rails by any other suitable means (e.g., by means of firmware).
[0077] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), system-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0078] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0079] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0080] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 40, which includes: a display device (e.g., a cathode ray tube or liquid crystal display) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device 40. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0081] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0082] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact via a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server.
[0083] This embodiment may also include a computer program product, which includes a computer program that, when executed by a processor, implements the method for detecting foreign objects on railway tracks provided in any embodiment of the present invention.
[0084] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0085] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for detecting foreign objects on railway tracks, characterized in that, include: The detection image captured by the drone is acquired, and the full-image features, initial detection boxes, and corresponding first classification scores are obtained based on the detection image using a pre-trained detection model. Based on the preset score range and the first classification score, a regular target detection box and a candidate small target detection box are obtained in the initial detection box, and the candidate small target detection boxes are subjected to small target enhancement processing to obtain an enhanced small target detection box; Contextual augmentation classification is performed based on the conventional target detection box, the enhanced small target detection box, and the full-image features to obtain the foreign object detection result on the railway track.
2. The method according to claim 1, characterized in that, Using a pre-trained detection model, based on the detected image, an initial detection box and the corresponding first classification score are obtained, including: Using a pre-trained detection model, combined with a task alignment allocator method and a center point matching allocator method, an initial detection box and the corresponding first classification score are obtained based on the detection image.
3. The method according to claim 1, characterized in that, The candidate small target detection boxes are subjected to small target enhancement processing to obtain enhanced small target detection boxes, including: Based on the detected image, obtain the candidate small target image corresponding to the candidate small target detection box, and obtain the small target enhancement image based on the candidate small target image; The small target enhancement image is re-detected using a pre-trained detection model to obtain each intermediate small target detection box and its corresponding second classification score. Based on a preset score range and the second classification score, an enhanced small target detection box is obtained from each intermediate small target detection box.
4. The method according to claim 3, characterized in that, Based on the detected image, obtain the candidate small target image corresponding to the candidate small target detection box, including: The candidate small target detection box is amplified to obtain an amplified small target detection box, and the detection image is cropped according to the amplified small target detection box to obtain the candidate small target image corresponding to the candidate small target detection box.
5. The method according to claim 3, characterized in that, Based on the candidate small target images, an enhanced image of the small target is obtained, including: Obtain a blank image template, and based on the blank image template and the candidate small target image, obtain the small target enhancement image.
6. The method according to claim 1, characterized in that, Based on the conventional target detection box, the enhanced small target detection box, and the full-image features, context-amplified classification is performed to obtain the foreign object detection results on the railway tracks, including: Based on the conventional target detection box and the enhanced small target detection box, the current target detection box is obtained, and based on the full image features, the image features corresponding to the current target detection box are obtained; A bilinear interpolation method is used to interpolate the image features based on the full-image features to obtain the foreign object context features, and based on the foreign object context features, the third classification score corresponding to the current target detection box is obtained.
7. The method according to claim 1, characterized in that, Based on the conventional target detection box, the enhanced small target detection box, and the full-image features, context-amplified classification is performed to obtain the foreign object detection results on the railway tracks, including: Based on the full-image features, multi-scale fusion features are obtained, and based on the multi-scale fusion features, image feature location codes are obtained. Based on the conventional target detection box and the enhanced small target detection box, obtain the current target detection box, and based on the current target detection box and the multi-scale fusion feature, obtain query information; By using a cross-attention network, the fourth classification score corresponding to the current target detection box is obtained based on the multi-scale fusion features, the image feature location encoding, and the query information.
8. An electronic device, characterized in that, The electronic device includes: At least one processor, and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for detecting foreign objects on the railway track as described in any one of claims 1-7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for detecting foreign objects on railway tracks as described in any one of claims 1-7.
10. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the method for detecting foreign objects on railway tracks as described in any one of claims 1-7.