Track foreign matter detection method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of railway intelligent safety monitoring and image recognition, provides a rail foreign matter detection method, device and equipment, a storage medium and a program product, and can avoid unnecessary calculation redundancy and save processing resources. The method comprises the steps of receiving a current track image in a track video stream; extracting a track area mask from the current track image to obtain a current track area mask image; according to the area change condition of the current track area mask image compared with the previous frame track area mask image, whether the track has abnormal change or not is determined; the upper frame track area mask image corresponds to the upper frame track image; and if the track has abnormal change, starting the target detection model to perform track foreign matter identification.

Description

Technical Field

[0001] This application relates to the field of intelligent safety monitoring and image recognition technology for railways, and in particular to a method, device, computer equipment, storage medium, and computer program product for detecting foreign objects on the track. Background Technology

[0002] With the high-speed and intelligent development of railway transportation, foreign object intrusion has become a significant hidden danger affecting the safe operation of tracks. Foreign objects on tracks include pedestrians, fallen objects, obstacles, and wild animals. Once they intrude into the track area, they can cause serious consequences such as derailment, collisions, and train delays. Therefore, building a reliable and efficient foreign object detection system for tracks has become a key task in ensuring the safe operation of trains.

[0003] When performing foreign object detection on the track, the relevant technologies call the target detection model at a high frequency, which requires a lot of processing resources and affects the real-time response of the system. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, storage medium, and computer program product for detecting foreign objects on tracks, in order to address the aforementioned technical problems.

[0005] This application provides a method for detecting foreign objects on a track, the method comprising:

[0006] Receive the current track image from the track video stream;

[0007] Extract the track region mask from the current track image to obtain the current track region mask image;

[0008] Based on the area change of the current track region mask image compared to the previous frame track region mask image, it is determined whether there are any abnormal changes in the track; the previous frame track region mask image corresponds to the previous frame track image;

[0009] If there are abnormal changes in the track, the target detection model will be activated to identify foreign objects on the track.

[0010] This application provides a track foreign object detection device, the device comprising:

[0011] The current image acquisition module is used to receive the current track image from the track video stream;

[0012] The mask extraction module is used to extract the track region mask from the current track image to obtain the current track region mask image;

[0013] The track anomaly detection module is used to determine whether there is an abnormal change in the track based on the area change of the current track region mask image compared to the previous frame track region mask image; the previous frame track region mask image corresponds to the previous frame track image.

[0014] The foreign object identification module is used to activate the target detection model to identify foreign objects on the track if there are abnormal changes in the track.

[0015] This application provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor executes the above-described method.

[0016] This application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0017] This application provides a computer program product having a computer program stored thereon, the computer program being executed by a processor using the above-described method.

[0018] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for detecting foreign objects in a track receive a current track image from a track video stream; extract a track region mask from the current track image to obtain a current track region mask image; determine whether there is an abnormal change in the track based on the area change of the current track region mask image compared to the previous frame track region mask image; the previous frame track region mask image corresponds to the previous frame track image; if there is an abnormal change in the track, a target detection model is activated to identify foreign objects in the track. This application utilizes the difference between the track region mask images of two consecutive frames to determine whether there is an anomaly in the track. When an anomaly is detected, a target detection model is activated to identify foreign objects in the track; when no anomaly is detected, the target detection model can be deactivated, avoiding unnecessary computational redundancy, saving processing resources, and thus improving the real-time response of the system. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a foreign object detection method for tracks in one embodiment;

[0021] Figure 2 This is a schematic diagram of the judgment process for abnormal track changes in one embodiment;

[0022] Figure 3 This is another flowchart illustrating the foreign object detection method for tracks in one embodiment;

[0023] Figure 4 This is a structural block diagram of a foreign object detection device for a track in one embodiment;

[0024] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various objects, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the second object. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0027] The orbital foreign object detection method provided in this application can be executed by computer equipment, and the method may include... Figure 1 The steps are shown.

[0028] Step S101: Receive the current track image from the track video stream.

[0029] Track video streams can be acquired using fixed or mobile monitoring equipment, and these streams consist of multiple consecutive frames. Fixed monitoring equipment includes trackside pillar cameras, while mobile monitoring equipment includes vehicle-mounted cameras. Computer equipment can then extract the current track image from the track video stream acquired by the monitoring equipment.

[0030] Step S102: Extract the track region mask from the current track image to obtain the current track region mask image.

[0031] After obtaining the current orbit image, the computer device extracts the orbit region mask from the current orbit image, and obtains the current orbit region mask image based on the extraction result.

[0032] Step S103: Determine whether there are any abnormal changes in the track based on the area change of the current track region mask image compared to the previous frame track region mask image.

[0033] The mask image of the track region in the previous frame corresponds to the track image in the previous frame.

[0034] Computer equipment can extract the track region mask from the previous frame track image. Based on the extraction result, the previous frame track region mask image can be obtained. The area change of the current track region mask image compared to the previous frame track region mask image can be determined, thereby determining whether there are any abnormal changes in the track.

[0035] Step S104: If there are abnormal changes in the track, the target detection model is activated to identify foreign objects on the track.

[0036] If there are abnormal changes in the track, the computer device can activate the target detection model to identify foreign objects on the track. If there are no abnormal changes in the track, the computer device can choose not to activate the target detection model and receive the next track image, processing the next track image according to steps S102 to S104. Abnormal changes in the track may include track occlusion, foreign object intrusion, or sudden scene changes.

[0037] The above-mentioned track foreign object detection method uses the difference between the track region mask images of two consecutive frames to determine whether there is an anomaly in the track. When an anomaly is detected, the target detection model is activated to identify the track foreign object. When there is no anomaly, the target detection model can be deactivated, which can avoid unnecessary computational redundancy, save processing resources, and thus improve the real-time response of the system.

[0038] In one embodiment, extracting the orbit region mask from the current orbit image to obtain the current orbit region mask image includes:

[0039] The current orbit image is input into the semantic segmentation model; the semantic segmentation model is used to perform pixel-level analysis on the current orbit image to determine the orbit region in the current orbit image and output the current orbit region mask image.

[0040] Semantic segmentation models can be based on lightweight segmentation networks to perform pixel-level semantic interpretation on input images and output binary orbital region mask images of consistent size, thereby providing structural priors for subsequent region change discrimination and object detection.

[0041] The semantic segmentation model's network structure can be based on the LiteSeg framework, with MobileNetV3-Small (a lightweight convolutional neural network, part of the MobileNet series) providing lightweight backbone feature extraction. It combines a Lite-ASPP (Atrous Spatial Pyramid Pooling) segmentation head with a boundary refinement module to enhance edge response capabilities. Processing is performed, and the output size is Binary mask image Each pixel Each pixel corresponds to a value. When a pixel belongs to the track region, its corresponding value is 1; otherwise, the corresponding value is 0. This binary mask image can be called a track region mask image.

[0042] The semantic segmentation model can be trained using image datasets with track region annotations (such as a self-built dataset or RailSem19). The loss function can be a weighted combination of cross-entropy loss and Dice loss, which can be expressed as follows: .in, The loss is the cross-entropy loss. For Dice Loss. and The weights are used for the semantic segmentation model. This model can be deployed on the edge, supporting fast processing and stable generation of prior masks for track region structures. RailSem19 is a publicly available large-scale semantic segmentation dataset for railway scenarios.

[0043] After obtaining the current orbit image, the computer device can first perform standardization preprocessing on the image to ensure it meets the input requirements of subsequent depth models. Specifically, the computer device can resize the current orbit image to a preset network input resolution (e.g., 640×360) to fit the downstream model structure. Subsequently, the computer device can normalize the color channels of the current orbit image, such as converting the RGB three-channel pixel values ​​to the [0,1] range, thereby enhancing the network's adaptability to different lighting conditions.

[0044] After standardized preprocessing, the computer device can input the current orbit image into the semantic segmentation model; the semantic segmentation model can be used to perform pixel-level analysis on the current orbit image to determine the orbit region in the current orbit image, thereby outputting a mask image of the current orbit region.

[0045] This embodiment can achieve pixel-level accurate recognition through a semantic segmentation model, separating the orbital region from the complex background to obtain a more accurate mask image of the current orbital region, which can lay the foundation for subsequent orbital anomaly judgment.

[0046] In one embodiment, the presence of abnormal changes in the track is determined based on the area change of the current track region mask image compared to the previous frame's track region mask image, including... Figure 2 The steps shown are as follows:

[0047] Step S201: Determine the mask area of ​​the current track region mask image to obtain the first mask area; Step S202: Determine the mask area of ​​the previous frame track region mask image to obtain the second mask area; Step S203: Obtain the mask change rate based on the first and second mask areas; Step S204: Determine whether there are any abnormal changes in the track based on the magnitude of the mask change rate.

[0048] Obtain the mask image of the current orbital region. Then, the total number of pixels in the track region of the current track region mask image (i.e., the number of pixels with a mask value of 1) can be calculated, thereby obtaining the mask area of ​​the track region, and subsequently determining the first mask area S. t , can be represented as: , H and W represent the length and width of the image.

[0049] Obtain the track region mask image from the previous frame. Then, the total number of pixels in the track region mask image of the previous frame (i.e., the number of pixels with a mask value of 1) can be calculated, thereby obtaining the mask area of ​​the track region, and subsequently determining the second mask area S. t-1 , can be represented as: , .

[0050] The size difference between the first and second mask areas reflects the area change of the track region between frames, thus yielding the mask change rate. Based on the relative magnitude of the mask change rate and the mask change rate threshold T, it can be determined whether there are any abnormal changes in the track. If the mask change rate is greater than the threshold T, it can be determined that there is a change in the track, and the track region in the current track image may be obstructed or have foreign object interference. The target detection model can then be activated for further analysis of the current track image. If the mask change rate is less than or equal to the threshold T, it can be determined that there are no abnormal changes in the track, and target detection can be skipped, allowing the current frame to proceed to the next frame's processing flow.

[0051] This embodiment transforms the complex visual anomaly detection into a simple numerical comparison problem by quantifying the rate of change of the orbital region between two frames, thus achieving efficient initial screening of anomalies with low computational cost.

[0052] In one embodiment, the mask change rate is obtained based on the first mask area and the second mask area, including:

[0053] The maximum mask area is determined between the first mask area and the second mask area; the mask change rate is obtained based on the ratio of the difference between the first mask area and the second mask area to the maximum mask area.

[0054] The computer device can determine the maximum mask area among the first mask area and the second mask area, which can be expressed as max(S). t ,S t-1 The computer device can calculate the difference between the area of ​​the first mask and the area of ​​the second mask, and obtain the absolute value of the difference, which can be expressed as |S|. t -S t-1 Computer equipment can determine the absolute value of the difference |S. t -S t-1 | and the maximum mask area max(S) t ,S t-1 The ratio of ) is used to obtain the mask change rate, which is the normalized difference between the areas of the track regions in the two consecutive frames, and can be expressed as .in, S is the rate of change of orbit. t S represents the area of ​​the first mask corresponding to the current orbit image. t-1 This represents the area of ​​the second mask corresponding to the previous frame's track image.

[0055] In this embodiment, the maximum mask area is determined from the first mask area and the second mask area; the mask change rate is obtained based on the ratio of the difference between the first mask area and the second mask area to the maximum mask area. This processing method can make the mask change rate scale invariant as much as possible through normalization, thus decoupling the judgment criteria from the specific size of the track in the image.

[0056] In one embodiment, the method provided by this application further includes: dividing the current orbital image according to depth of field to obtain a near-field region and a far-field region; wherein, the first mask area includes a near-field first mask area and a far-field first mask area, and the second mask area includes a near-field second mask area and a far-field second mask area;

[0057] The mask change rate is obtained based on the first mask area and the second mask area, including: obtaining the near-view mask change rate based on the near-view first mask area and the near-view second mask area; and obtaining the far-view mask change rate based on the far-view first mask area and the far-view second mask area.

[0058] Based on the magnitude of the mask change rate, determine whether there is an abnormal change in the orbit, including: if the near-view mask change rate is greater than the near-view mask change rate threshold, or the far-view mask change rate is greater than the far-view mask change rate threshold, then the orbit is determined to have an abnormal change; if the near-view mask change rate is less than or equal to the near-view mask change rate threshold, and the far-view mask change rate is less than or equal to the far-view mask change rate threshold, then the orbit is determined not to have an abnormal change; the near-view mask change rate threshold is greater than the far-view mask change rate threshold.

[0059] The current orbit image and the previous orbit image are two consecutive frames. The depth of field in these two images is relatively similar in various regions. For example, the lower right corner is the foreground region, and the upper left corner is the background region. The computer device can divide the current orbit image according to the depth of field to obtain the foreground and background regions.

[0060] The current track region mask image, the current track image, the previous frame track image, and the previous frame track region mask image can be kept in the same size.

[0061] After obtaining the mask image of the current orbital region, the image region corresponding to the foreground region can be determined within the mask image. The number of pixels with a mask value of 1 in this image region can then be determined, thus obtaining the first mask area of ​​the foreground. Similarly, the image region corresponding to the background region can be determined within the mask image of the current orbital region. The number of pixels with a mask value of 1 in this image region can then be determined, thus obtaining the first mask area of ​​the background.

[0062] After obtaining the track region mask image from the previous frame, the image region corresponding to the foreground area can be determined within the track region mask image. The number of pixels with a mask value of 1 in this image region can then be determined, thus obtaining the area of ​​the second foreground mask. Similarly, the image region corresponding to the background area can be determined within the track region mask image from the previous frame. The number of pixels with a mask value of 1 in this image region can then be determined, thus obtaining the area of ​​the second background mask.

[0063] The computer equipment can determine the maximum near-field mask area from the first and second near-field mask areas. The computer equipment can calculate the difference between the first and second near-field mask areas, obtaining the absolute value of the difference. The computer equipment can then determine the near-field mask change rate based on the ratio of the absolute value of the difference to the maximum near-field mask area.

[0064] The computer equipment can determine the maximum distant mask area from the first and second distant mask areas. The computer equipment can calculate the difference between the first and second distant mask areas, obtaining the absolute value of the difference. The computer equipment can then determine the distant mask change rate based on the ratio of the absolute value of the difference to the maximum distant mask area.

[0065] Considering that foreign objects have a small pixel area in the distance, the segmentation results are more susceptible to image noise, compression artifacts, and small fluctuations in the model. Therefore, the threshold for the change rate of the distance mask can be relatively strict (i.e., the threshold for the change rate of the distance mask is less than the threshold for the change rate of the near-distance mask), which can suppress these noises and improve sensitivity.

[0066] If the near-view mask change rate is greater than the near-view mask change rate threshold, or if the far-view mask change rate is greater than the far-view mask change rate threshold, then an abnormal change in the orbit can be determined. If the near-view mask change rate is less than or equal to the near-view mask change rate threshold, and the far-view mask change rate is less than or equal to the far-view mask change rate threshold, then no abnormal change in the orbit can be determined.

[0067] In one embodiment, activating a target detection model for orbital foreign object identification includes:

[0068] The current orbit image is input into the target detection model; the target detection model is used to analyze and identify the current orbit image to determine whether there are foreign objects on the orbit and the type of foreign objects.

[0069] If there are abnormal changes in the track, the computer equipment can activate the target detection model by inputting the current track image into the target detection model. The target detection model can be used to analyze and identify the current track image to determine whether there are foreign objects on the track and what type of foreign objects they are.

[0070] Considering the challenges of small-scale, heavily occluded, and complex backgrounds of foreign objects in orbital scenarios, a structural optimization can be performed on the YOLOv11 (a version of the YOLO series models) object detection architecture to obtain the YOLOv11-SM architecture. By introducing two techniques—Self-Ensemble Attention Mechanism (SEAM) and Multi-scale Discrepancy Fusion Module (MDFM)—the detection accuracy and robustness of the model in real-world environments can be improved. The YOLOv11-SM architecture can be used to build object detection models.

[0071] Based on the SEAM unit introduced in the backbone network and the C2PSA unit (an enhanced attention unit) embedded in the YOLOv11 backbone network, C2PSA_SEAM can be constructed. Through hierarchical perception and channel-space joint enhancement strategies, the model's ability to represent fine-grained targets (such as people, tools, animals, etc.) is enhanced. Specifically, the image input to the object detection model can first be encoded using a multi-scale patch embedding mechanism, thus dividing the image into image patches of various granularities such as 6×6 and 7×7. Local visual units are extracted through a non-overlapping sliding window, which can be formally represented as: ,in For the input image, This indicates that the size is Convolution operation, The extracted local embedding features can then be fed into a CSMM (Channel and Spatial Mixed Module) unit, whose core is a spatial-channel hybrid modeling structure that uses depthwise separable convolutions to improve the efficiency of information exchange in the spatial and channel dimensions. Feature Map It can be processed via CSMM: To enhance feature generality and stabilize network training, global average pooling and normalized activation mechanisms (such as GELU) can be used to unify feature response amplitudes across different scales, ultimately outputting a low-redundancy, high-sensitivity foreign object representation map. GELU stands for Gaussian Error Linear Unit.

[0072] In the feature fusion stage, an MDFM (Multi-scale Difference Fusion Module) unit can be introduced to replace the FPN (Feature Pyramid Network) structure, enabling multi-scale fusion and enhancement of features from different receptive fields. The MDFM unit can perform initial difference extraction on the multi-scale feature maps using an IDFE (Initial Difference Feature Extraction) unit, capturing significant foreign object regions by utilizing pixel-level differences and absolute value changes. Subsequent MSFF (Multi-Scale Feature Fusion) units can use heterogeneous convolutional kernels such as 1×1, 3×3, 5×5, and 7×7 to perform independent convolution operations on feature maps of each scale, and then concatenate the features. Finally, the channel dimension is weighted and compressed by CWFB (Channel Weighted Fusion Block), which enables the target detection model to retain the response information of key small targets when fusing multi-layer features, effectively alleviating the feature loss problem of foreign objects during scale transformation.

[0073] The target detection model is based on the YOLOv11-SM structure and has a strong ability to perceive foreign objects of different categories and sizes in track images. It can stably identify typical targets including people, cattle, sheep, obstacles, construction tools, and vehicles, reduce the false detection rate in complex backgrounds (such as misidentifying ballast as foreign objects), and maintain a high detection confidence under conditions of changing lighting or severe occlusion.

[0074] During deployment, the object detection model can accept inputs of a complete image or a region of interest (ROI) cropped by the semantic segmentation module, and output detection results including location coordinates, object category, and confidence score. ,in This represents the confidence score. A confidence threshold can be configured. The confidence threshold can be set to The range is used to determine alarm conditions and output alarm signals.

[0075] To better understand the above method, an application example of the orbital foreign object detection method of this application is described in detail below. Traditional orbital foreign object detection methods include: traditional methods based on physical sensing and intelligent methods based on visual perception.

[0076] Traditional physical sensing-based methods rely on peripheral devices such as infrared beam detectors, lidar, vibration cables, and geomagnetic sensors for fixed-point deployment. While these methods are effective in fixed locations, they suffer from significant drawbacks, including high deployment costs, limited monitoring range, poor anti-interference capabilities, and inability to identify the specific type of foreign object. Furthermore, physical sensors cannot visually confirm targets, leading to false alarms or missed detections, and their stability is difficult to guarantee in complex environments with frequent winds, sandstorms, rain, snow, and shadows.

[0077] With the development of computer vision and deep learning, image recognition technology based on video surveillance has gradually become a mainstream trend. Common methods include target detection models based on CNN (Convolutional Neural Network) (such as YOLO, Faster R-CNN, SSD, etc.), as well as preprocessing algorithms such as background modeling, edge detection, and motion analysis. These methods achieve automatic identification and alarm of foreign objects by analyzing track images in real time. However, methods that rely purely on target detection networks still face many challenges: 1) Sample scarcity: The probability of foreign object events on the track is extremely low, and the cost of obtaining real labeled samples is high, making it difficult to train deep models or prone to overfitting; 2) Complex interference factors: External disturbances such as weather changes, uneven lighting, occlusion, and camera shake seriously affect the detection accuracy; 3) Inconsistent target scale: Foreign objects vary greatly in size, and small targets (such as abandoned objects) are easily overlooked, greatly affecting detection performance; 4) Low efficiency of full-image detection: Full-image detection is performed for each frame of the image, which consumes a lot of computing resources and is not conducive to the deployment of edge devices. YOLO is a single-stage detector, Faster R-CNN is a two-stage detector, and SSD is a single-pass multi-box detector.

[0078] This application example fully integrates semantic segmentation results with object detection, enabling dynamic analysis of "track region changes" and forming a reasonable multi-stage judgment process. Semantic segmentation is used to extract the track region first, and detection of this region improves efficiency and accuracy. Furthermore, this application example makes full use of the dynamic information between consecutive frames, leveraging the spatiotemporal continuity of the track region. For example, object detection is not performed when there are no significant changes in the track region, saving resources and reducing the false detection rate.

[0079] In real-world engineering scenarios, foreign objects in track images often obscure track areas. This application example leverages this characteristic to pre-judge the difference in track mask area between consecutive frames, reducing redundant detection and improving system response efficiency. This application example combines semantic segmentation results with a mask difference judgment mechanism to form a complete detection architecture of "pre-screening + phased detection + anomaly identification." The track foreign object detection technology provided in this application example integrates semantic perception, dynamic change perception, and intelligent target recognition mechanisms, offering advantages in multi-scenario adaptability, resource scheduling efficiency, and accuracy in anomaly state discrimination.

[0080] This application example provides a method for identifying foreign objects (FOOs) on orbits based on semantic segmentation and region mask change pre-screening. It can construct a multi-stage FEO detection system with the orbital region as the perception core, dynamic masking as the triggering mechanism, and target detection as the accurate identification method. The main aspects include:

[0081] (1) Construct a lightweight semantic segmentation model to perform pixel-level segmentation of the image and extract the semantic mask of the track region to provide accurate structural prior information for subsequent judgment and detection.

[0082] (2) Introduce a mechanism to judge the ratio of changes in the mask area of ​​the track region between the previous and next frames. When the track region undergoes abnormal changes (such as occlusion or foreign object intrusion), the target detection process is triggered. When the change is within the normal threshold, the detection is skipped to save computing resources.

[0083] (3) Provide a target detection model that is linked with the track mask. When the pre-screening determines that the track is "abnormal", it will be activated, which will significantly improve the overall efficiency of the system and maintain a high response sensitivity to foreign object events.

[0084] (4) Construct a complete multi-stage track foreign object detection logic process with real-time response, high reliability and engineering deployment adaptability, and applicable to various application scenarios such as vehicle-mounted detection equipment, fixed camera monitoring, and unmanned inspection platform.

[0085] This application example combines prior structural information (track region) from track images with temporal dynamic features (mask changes) to form a track foreign object detection framework, which can provide technical support for railway intelligent safety systems.

[0086] This application example provides a track foreign object intrusion detection system based on semantic segmentation, region change judgment, and a multi-stage detection strategy. The system's process is as follows: Figure 3 As shown, the system mainly includes a track region semantic segmentation unit, a track state change pre-screening unit, a foreign object detection and identification unit, and an alarm output unit. The overall method employs a step-by-step processing mechanism. First, the semantic segmentation model of the track region semantic segmentation unit performs pixel-level analysis on the input image, accurately extracting the track region from the image and generating a corresponding semantic mask. Then, using the area difference between the track masks of two consecutive frames, it determines whether the track region has experienced occlusion, coverage, or other abnormal changes, thus achieving preliminary screening and state awareness. If abnormal changes exist in the track region, the target detection model in the foreign object detection and identification unit can be activated to classify and locate any foreign objects that may exist in the track region. The target detection model can adopt an improved YOLOv11 structure, integrating a self-integrated attention mechanism (SEAM) and a multi-scale difference fusion unit (MDFM), which can effectively improve the recognition accuracy and robustness for multi-scale foreign objects, occluded objects, and complex track backgrounds. Finally, the recognition results can be output by the alarm output unit, which can link with the safety system to issue warnings or upload the recognition results to the monitoring platform. The entire system features a clear structure, accurate detection, and real-time response, making it suitable for various practical application scenarios such as fixed monitoring along railway lines, vehicle-mounted video analysis, and unmanned inspection.

[0087] In addition, to ensure the normal operation of the orbital state change pre-screening unit, an inter-frame buffering and time synchronization mechanism can be introduced to ensure that the current orbital image (denoted as ) can be acquired simultaneously during each processing. ) and the previous frame's track image ( This provides continuous frame data support for calculating the mask change rate. Through the aforementioned standardized preprocessing procedure, the system can stably receive multi-source image data and provide a unified and standardized data input interface for subsequent semantic segmentation and object detection modules.

[0088] The following section describes the overall operation process and key technical details of the system, using a deployment scheme in a fixed monitoring scenario along a railway line, to verify the feasibility and engineering adaptability of this application example. This embodiment is merely a recommended form of technical implementation.

[0089] In actual deployment, the image acquisition equipment (belonging to the category of monitoring equipment) can be installed on a fixed trackside post approximately 3.5 meters from the railway centerline and 2.2 meters above the ground. The selected equipment is a high-definition industrial camera with a 5-megapixel resolution and supporting continuous output at 30 frames per second. The monitoring field of view of the image acquisition equipment can cover the double rails and the working area of ​​approximately 2 meters on both sides, providing good track surface visibility and foreign object identification capabilities.

[0090] The system can be integrated into an all-in-one edge computing terminal. The computing unit can use an embedded GPU module and pre-load the model weights of the trained semantic segmentation model and object detection model. The system can execute according to the following process:

[0091] (1) Image input and preprocessing:

[0092] The system can receive orbital images at a frequency of 5 frames per second, with the original image resolution being 1280×720 pixels. The orbital images can be uniformly scaled to 640×360 pixels and normalized and format-converted (e.g., converting RGB images to tensor form) to adapt to the standard input format of subsequent models. This image processing can be performed using relevant tools. Simultaneously, the system can enable a frame buffering mechanism to ensure... With the previous frame It can also be used for dynamic comparison of mask regions.

[0093] (2) Semantic segmentation model operation:

[0094] Current orbit image You can input a semantic segmentation model based on a lightweight structure (such as LiteSeg) to obtain the mask image of the current orbital region. The size of the mask image is consistent with that of the input image. In the mask image, a value of 1 indicates that a pixel is identified as a track region, and a value of 0 indicates a non-track region. This semantic segmentation model can be trained on a pre-built semantic annotation dataset of railway track images and optimized using a joint loss such as the Dice coefficient and cross-entropy. In actual experiments, the model achieves high segmentation accuracy, with an mIoU (Mean Intersection over Union) of approximately 0.895.

[0095] (3) Judgment of changes in orbital region:

[0096] During operation, the track region mask image of the previous frame can be cached simultaneously. And calculate the mask change rate accordingly. mask change rate It can be used as an indicator to determine occlusion or abnormal changes in area. In this embodiment, a mask change rate threshold is set. ,when If a significant structural change occurs in the orbital region, the system can determine that a substantial change has taken place, triggering the subsequent target detection model. Conversely, if no significant change is observed, the detection process can be skipped, and the system can proceed directly to the next frame. It is estimated that under this mechanism, approximately 70% of the frames are in an idle state, effectively reducing the overall inference computation load.

[0097] (4) Execution of the target detection model:

[0098] When the detection process is triggered, the current orbital image This can be passed as input to the object detection model. The object detection model integrates the aforementioned SEAM and MDFM units, possessing good small target recognition capabilities and noise suppression capabilities in complex backgrounds. The output of the object detection model can include bounding box information for multiple targets, in the format of... ,in Indicates the category of foreign object. This indicates the confidence score.

[0099] Object detection categories may include, but are not limited to, the following:

[0100] class_1: Pedestrian;

[0101] class_2: Obstacles;

[0102] class_3: Animals;

[0103] class_4: Automobile.

[0104] When the confidence level of a certain detection box If the target's center point is located within the area covered by the track area mask, it can be determined that there is a valid foreign object intrusion event.

[0105] (5) Alarm and result output:

[0106] When alarm conditions are met, alarm logic can be triggered, and alarm information can be uploaded to the rail transit control system via serial port, network, or other communication interfaces. Simultaneously, the current track image, the current track area mask image, and the detection box results can be stored synchronously for purposes such as log recording, post-event review, and event reproduction.

[0107] If the application platform connected to the system supports a graphical user interface or intelligent prompting module, an alarm prompt window can pop up in real time on the client terminal, supplemented by voice reminders, to guide the on-duty personnel to respond in a timely manner.

[0108] Semantic segmentation models can employ other lightweight or accuracy-priority segmentation networks, such as BiSeNet (Bidirectional Segmentation Network), Fast-SCNN (Fast Segmentation Convolutional Neural Network), and ENet (Efficient Network). The models can also be accelerated and optimized through pruning or distillation to adapt to the computing power of different platforms.

[0109] The mask change judgment strategy can be adjusted according to actual needs, such as adding alternative mechanisms such as track edge boundary change detection and track centerline offset judgment; the change index can also be expanded from area ratio to mask intersection-union ratio and other forms.

[0110] The network architecture of object detection models can adopt YOLOv5, YOLOv8, YOLOv10, etc., and can integrate different types of attention mechanisms (such as CBAM, ECA) or fusion strategies to achieve multi-scale object perception. CBAM stands for Convolutional Block Attention Module. ECA stands for Efficient Channel Attention.

[0111] Image input sources can be from fixed surveillance cameras, or can be expanded to include vehicle vision systems, track robots, drone aerial images, and other forms; the input frame rate and resolution can be flexibly configured according to different scenarios.

[0112] This application example eliminates the need for target detection in a large number of normal frames where there are no foreign objects on the track, saving processing resources on edge devices and improving system real-time response. Furthermore, considering that targets typically appear in or near the track area during foreign object events, this application example utilizes this structural prior to use changes in suspicious areas as a triggering mechanism, avoiding unnecessary computational redundancy and improving system accuracy and stability. Additionally, considering that track occlusion manifests as local disappearance or deformation of the track area, meaning that the masked area differs significantly between adjacent frames, this application example constructs a dynamic mask judgment mechanism across frames, providing highly efficient track state change recognition capabilities.

[0113] This application example integrates image semantic understanding and dynamic perception mechanisms, which can improve the system's operating efficiency and deployment adaptability while ensuring detection accuracy. It has the following advantages and positive effects:

[0114] (1) Introducing prior structural information about the track region to improve detection accuracy: The semantic segmentation model is used to accurately extract the track region in the image, ensuring that subsequent recognition tasks focus on key areas with potential security risks and avoiding false detections and missed detections caused by background noise interference. By constraining the detection range through structural prior information, the detection model's ability to perceive foreign objects and its response efficiency are improved.

[0115] (2) Rapid pre-screening based on mask area change effectively reduces computational burden: The difference judgment mechanism of track mask area change is introduced. When abnormal states such as occlusion or shape change occur in the track area, the target detection process is started, and otherwise it is not started. This can avoid invalid detection of a large number of normal frame images, and the overall computational load of the system is expected to be reduced, which has good edge deployment feasibility.

[0116] (3) A joint optimization target detection model is adopted to enhance the ability to identify small targets and adapt to complex scenes: The target detection model can be based on the YOLOv11 structure and introduces a self-integrated attention mechanism (SEAM) and a multi-scale difference fusion unit (MDFM) to optimize the model's capabilities from the two levels of feature extraction and feature fusion. It is expected to improve the detection capabilities of small foreign objects (such as gravel and tools), occluded targets, and distant objects. The mAP@0.5 is improved to 0.981, which is better than mainstream models such as YOLOv8 and YOLOv10. mAP@0.5 represents the mean of the average accuracy of all detection categories under the standard of IoU threshold of 0.5.

[0117] (4) The multi-stage structure design is flexible and has wide deployment adaptability: The system adopts a step-by-step response strategy of "semantic guidance → track dynamic judgment → condition detection". The overall structure is clear, the calculation path can be tailored, and the response process can be configured. It is suitable for various operating platforms such as fixed-point cameras along railway lines, vehicle-mounted forward vision systems, and mobile track inspection robots, which facilitates industrial system integration and distributed deployment.

[0118] (5) Good engineering feasibility and system stability: The models are all built on standard deep learning architectures (such as PyTorch) and are compatible with mainstream GPU (Graphics Processing Unit) platforms; the models have good training convergence and stable inference, making them suitable for deployment in actual orbital safety systems. PyTorch is an open-source machine learning framework.

[0119] This application example achieves a synergistic improvement in detection accuracy, response efficiency, and deployment adaptability by organically combining a deep learning detection model that integrates structural perception, dynamic judgment, and structural optimization. It provides a high-performance solution that can be engineering-implemented for intelligent identification systems for foreign object intrusion in rail transit, and is suitable for application scenarios such as safety monitoring along the line, early warning of foreign object intrusion, and intelligent detection of track occupancy in rail transit systems.

[0120] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0121] Based on the same inventive concept, this application also provides a track foreign object detection device for implementing the track foreign object detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of the one or more track foreign object detection device embodiments provided below can be found in the limitations of the track foreign object detection method described above, and will not be repeated here.

[0122] In one embodiment, such as Figure 4 As shown, a foreign object detection device for tracks is provided, comprising:

[0123] The current image acquisition module 401 is used to receive the current track image from the track video stream;

[0124] The mask extraction module 402 is used to extract the track region mask from the current track image to obtain the current track region mask image;

[0125] The track anomaly judgment module 403 is used to determine whether there is an abnormal change in the track based on the area change of the current track region mask image compared with the previous frame track region mask image; the previous frame track region mask image corresponds to the previous frame track image.

[0126] The foreign object identification module 404 is used to activate the target detection model to identify foreign objects in the track if there are abnormal changes in the track.

[0127] In one embodiment, the mask extraction module 402 is used for:

[0128] The current orbit image is input into the semantic segmentation model; the semantic segmentation model is used to perform pixel-level analysis on the current orbit image to determine the orbit region in the current orbit image and output the current orbit region mask image.

[0129] In one embodiment, the track anomaly detection module 403 is used for:

[0130] The mask area of ​​the current track region mask image is determined to obtain a first mask area; the mask area of ​​the previous frame track region mask image is determined to obtain a second mask area; the mask change rate is obtained based on the first mask area and the second mask area; and the abnormal changes in the track are determined based on the magnitude of the mask change rate.

[0131] In one embodiment, the track anomaly detection module 403 is used for:

[0132] The maximum mask area is determined between the first mask area and the second mask area; the mask change rate is obtained based on the ratio of the difference between the first mask area and the second mask area to the maximum mask area.

[0133] In one embodiment, the device further includes a region determination module, configured to: divide the current orbital image according to depth of field to obtain a near-field region and a far-field region;

[0134] Wherein, the first mask area includes a near-view first mask area and a far-view first mask area, and the second mask area includes a near-view second mask area and a far-view second mask area;

[0135] Track anomaly detection module 403 is used for:

[0136] The near-field mask change rate is obtained based on the near-field first mask area and the near-field second mask area; the far-field mask change rate is obtained based on the far-field first mask area and the far-field second mask area; if the near-field mask change rate is greater than the near-field mask change rate threshold, or if the far-field mask change rate is greater than the far-field mask change rate threshold, then it is determined that there is an abnormal change in the orbit; if the near-field mask change rate is less than or equal to the near-field mask change rate threshold, and the far-field mask change rate is less than or equal to the far-field mask change rate threshold, then it is determined that there is no abnormal change in the orbit; the near-field mask change rate threshold is greater than the far-field mask change rate threshold.

[0137] In one embodiment, the foreign object detection module 404 is used for:

[0138] The current track image is input into the target detection model; the target detection model is used to analyze and identify the current track image to determine whether there are foreign objects on the track and the type of foreign objects.

[0139] Each module in the aforementioned foreign object detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0140] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to the methods described above. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for detecting foreign objects in orbit.

[0141] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the various method embodiments described above.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0144] In one embodiment, a computer program product is provided having a computer program stored thereon, the computer program being executed by a processor of the steps described in the various method embodiments above.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting foreign objects on a track, characterized in that, The method includes: Receive the current track image from the track video stream; Extract the track region mask from the current track image to obtain the current track region mask image; Based on the area change of the current track region mask image compared to the previous frame track region mask image, it is determined whether there are any abnormal changes in the track; the previous frame track region mask image corresponds to the previous frame track image; If there are abnormal changes in the track, the target detection model will be activated to identify foreign objects on the track.

2. The method according to claim 1, characterized in that, Extracting the orbit region mask from the current orbit image to obtain the current orbit region mask image includes: The current orbit image is input into the semantic segmentation model; the semantic segmentation model is used to perform pixel-level analysis on the current orbit image to determine the orbit region in the current orbit image and output the current orbit region mask image.

3. The method according to claim 1, characterized in that, Based on the area change of the current orbit region mask image compared to the previous frame's orbit region mask image, determine whether there are any abnormal changes in the orbit, including: Determine the mask area of ​​the current orbit region mask image to obtain the first mask area; Determine the mask area of ​​the previous frame track region mask image to obtain the second mask area; The mask change rate is obtained based on the first mask area and the second mask area; Based on the magnitude of the mask change rate, it is determined whether there are any abnormal changes in the orbit.

4. The method according to claim 3, characterized in that, Based on the first mask area and the second mask area, the mask change rate is obtained, including: Determine the maximum mask area between the first mask area and the second mask area; The mask change rate is obtained by the ratio of the difference between the first mask area and the second mask area to the maximum mask area.

5. The method according to claim 4, characterized in that, The method further includes: The current orbital image is divided according to depth of field to obtain the near-field region and the far-field region; Wherein, the first mask area includes a near-view first mask area and a far-view first mask area, and the second mask area includes a near-view second mask area and a far-view second mask area; Based on the first mask area and the second mask area, the mask change rate is obtained, including: The near-field mask change rate is obtained based on the area of ​​the first near-field mask and the area of ​​the second near-field mask. The change rate of the distant mask is obtained based on the area of ​​the first distant mask and the area of ​​the second distant mask. Based on the magnitude of the mask change rate, determine whether there are any abnormal changes in the orbit, including: If the near-view mask change rate is greater than the near-view mask change rate threshold, or if the far-view mask change rate is greater than the far-view mask change rate threshold, then it is determined that there is an abnormal change in the orbit. If the near-view mask change rate is less than or equal to the near-view mask change rate threshold, and the far-view mask change rate is less than or equal to the far-view mask change rate threshold, then it is determined that there is no abnormal change in the orbit; the near-view mask change rate threshold is greater than the far-view mask change rate threshold.

6. The method according to any one of claims 1 to 5, characterized in that, The target detection model is activated to identify foreign objects on the track, including: The current track image is input into the target detection model; the target detection model is used to analyze and identify the current track image to determine whether there are foreign objects on the track and the type of foreign objects.

7. A track foreign object detection device, characterized in that, The device includes: The current image acquisition module is used to receive the current track image from the track video stream; The mask extraction module is used to extract the track region mask from the current track image to obtain the current track region mask image; The track anomaly detection module is used to determine whether there is an abnormal change in the track based on the area change of the current track region mask image compared to the previous frame track region mask image; the previous frame track region mask image corresponds to the previous frame track image. The foreign object identification module is used to activate the target detection model to identify foreign objects on the track if there are abnormal changes in the track.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images