Railway foreign object detection method and system

Abstract

This invention discloses a method and system for detecting foreign objects on railway tracks, relating to the field of track detection technology. First, an unknown foreign object model is constructed using Gaussian distribution and convex hull algorithm, and a known foreign object model is established by combining prior data. Flat areas of the track bed are extracted, and the foreign object model is projected as a depth map and shadows are generated. These are then superimposed to obtain a labeled synthetic image. A segmentation network is trained using the synthetic image and the real image. During inference, heatmap binarization, connected component analysis, and size filtering are performed to obtain a potential foreign object mask. A secondary classification network is then used to eliminate false alarms from existing facilities, finally outputting the actual foreign object detection result. An alarm is triggered based on the detection threshold, achieving efficient and accurate detection. This invention achieves efficient, accurate, and real-time foreign object detection on ballastless tracks under the premise of scarce real foreign object samples and complex track bed scenarios, while reducing the false alarm rate and improving the model's generalization ability to unknown foreign objects.

Description

Technical Field

[0001] This invention relates to the field of track inspection technology, and in particular to a method and system for detecting foreign objects on tracks. Background Technology

[0002] Railway track safety is crucial for ensuring the reliability of transportation systems. Ballastless track is widely used in high-speed railways due to its structural stability; however, the track bed area is susceptible to intrusion by foreign objects (such as broken parts or foreign objects), threatening train operation safety. Traditional inspection methods rely on manual inspections or fixed sensors, which suffer from low efficiency, high false alarm rates, and difficulty in covering complex scenarios. Existing inspection methods can be mainly divided into supervised inspection methods and unsupervised inspection methods.

[0003] Unsupervised methods, which reconstruct grayscale images for anomaly detection, are susceptible to interference from track bed textures. Their reconstruction patterns are limited, typically only detecting larger foreign objects, and performing poorly on smaller objects or areas sensitive to texture changes. Supervised methods, which train models directly on grayscale images, are limited by the types and quantity of training data. They exhibit poor adaptability to unknown foreign objects and complex track bed backgrounds, resulting in high false detection rates. Their detection performance heavily relies on a large amount of manually labeled real-world foreign object data. Furthermore, in real-world scenarios, they suffer from the following problems: real-world foreign objects have diverse forms and low occurrence probabilities, leading to difficult and scarce sample collection, making it difficult to cover all foreign object types; sample labeling is costly and prone to imbalanced sample distribution, directly limiting model training effectiveness and resulting in extremely poor generalization ability for unknown foreign objects.

[0004] In summary, for supervised detection methods, it is urgent to solve the problems of how to achieve efficient, accurate, and real-time foreign object detection on ballastless tracks under the premise of scarce real foreign object samples and complex track bed scenarios, while reducing false alarm rates and improving the generalization ability to unknown foreign objects. Summary of the Invention

[0005] In view of this, this application provides a method and system for detecting foreign objects on a track to address the shortcomings of the existing technology.

[0006] The first aspect of this application provides a method for detecting foreign objects in orbit, comprising:

[0007] Given the maximum and minimum thresholds for the three dimensions of the foreign object, the Gaussian probability distribution function is used to randomly generate the dimensions of the foreign object. Within the given threshold range of the dimensions of the foreign object, random three-dimensional points of a specified length are generated. An irregular polyhedron network is constructed using the convex hull algorithm and noise is added to the surface to complete the modeling of the unknown foreign object.

[0008] Based on prior data, a 3D model of a known irregular object is established and noise is added to the surface to complete the modeling of the known irregular object; the modeling of the unknown irregular object and the modeling of the known irregular object together constitute the model of the irregular object.

[0009] The track bed depth map is subjected to gradient segmentation and curvature filtering, and the flat area of ​​the track bed is obtained by binarization and morphological calculation.

[0010] In the track bed depth map, a suitable location for the superposition of foreign objects is randomly selected in the flat area of ​​the track bed. Based on the 3D camera imaging model and the shooting position and angle of the 3D camera, the foreign object model is projected into a foreign object depth map.

[0011] Based on the foreign object depth map, the position of the foreign object in the track bed depth map is determined. Combined with the installation position of the 3D camera and the 3D camera imaging model, the shadow area formed on the track bed surface after the foreign object blocks the optical path of the 3D camera is calculated. The shadow area corresponds to the position of the foreign object in the foreign object depth map.

[0012] The foreign object depth map and the shadow area are superimposed on the track bed image without foreign objects, and the foreign object depth map is filtered to obtain a track bed image containing foreign objects, which is denoted as the track bed foreign object composite image; based on the foreign object superposition position and the shadow area, the track bed foreign object composite image is automatically labeled to obtain the corresponding mask image;

[0013] The labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed are used as datasets to train the foreign object segmentation network.

[0014] The trained foreign object segmentation network is used to perform forward reasoning on the ballastless track depth map to predict and segment potential foreign object regions, generating a probability distribution heatmap. The probability distribution heatmap is then binarized based on a set threshold to obtain a binarized image. Connected component analysis is performed on the binarized image, and regions with actual sizes smaller than the detection threshold are removed in conjunction with the ballastless track depth map to obtain the mask region of potential foreign objects.

[0015] Extract grayscale images of the inherent facilities of the track as a secondary classification dataset to train a secondary classification network;

[0016] The trained secondary classification network is used to perform secondary reasoning on the masked region of potential foreign objects, eliminating the region corresponding to the inherent facilities of the track, to obtain the masked region of the real foreign objects, and finally output the detection result of the real foreign objects, and output alarm information based on the detection threshold.

[0017] In one possible implementation of the first aspect, the foreign object segmentation network adopts an encoder-decoder architecture, specifically:

[0018] The encoder uses a ResNet50 deep convolutional neural network for high-dimensional feature extraction, and an attention mechanism network is used to connect the layers.

[0019] The decoder uses a feature pyramid (FPN) structure for multi-scale feature extraction and outputs a pixel-level foreign object probability map.

[0020] The loss function of the heterogeneous object segmentation network for:

[0021]

[0022] , and These are the weighting coefficients. To quantify the difference between small target predictions and reality, To quantify the difference between the predicted edge and the actual edge, To quantify the difference between deep feature predictions and reality.

[0023] In one possible implementation of the first aspect, the backbone of the secondary classification network is based on the YOLOv5 model, and the last layer of the YOLOv5 model is split into two parallel fully connected classification heads, using a multi-class mixture loss function, specifically:

[0024]

[0025] and These are the weighting coefficients. To quantify whether the prediction is due to the inherent facilities and the true label, To quantify the error between the predicted facility type and the actual type.

[0026] In one possible implementation of the first aspect, projecting the foreign object model into a foreign object depth map includes:

[0027] By setting the occlusion ratio, the foreign object model is projected into a foreign object depth map with different occlusion states, thus obtaining diverse foreign object samples.

[0028] In one possible implementation of the first aspect, performing connected component analysis on the binarized image includes:

[0029] Eight-neighborhood connectivity analysis is used to cluster white pixels in the binarized image, specifically as follows:

[0030] Determine whether two white pixels are adjacent in the horizontal, vertical, and diagonal directions. If they are, they are considered a connected region; otherwise, they are considered a disconnected region and are removed.

[0031] In one possible implementation of the first aspect, the mask region for obtaining the potential foreign object includes:

[0032] Based on the three-dimensional information of the ballastless track depth map and combined with the calibration parameters of the 3D camera, the pixel range of each connected region is converted into the actual physical size.

[0033] The actual physical dimensions are compared with the preset detection thresholds. If the length, width, and height of the connected region are all smaller than the corresponding detection thresholds, it is determined to be a type of connected region and is removed. At the same time, the remaining connected regions are used as mask regions for potential foreign objects.

[0034] In one possible implementation of the first aspect, excluding the area corresponding to the inherent facilities of the track includes:

[0035] Calculate the minimum bounding rectangle of any connected region in the mask region, and then cut out the corresponding grayscale image after multiplying the length and width of the minimum bounding rectangle by 1.

[0036] The trained secondary classification network is used to identify the grayscale image and determine whether it is an inherent facility of the track. If it is, and the confidence level is greater than a set threshold and the corresponding size matches the prior data, the inherent facility of the track corresponding to the grayscale image is removed. If not, no action is taken.

[0037] In one possible implementation of the first aspect, outputting alarm information based on the detection threshold includes:

[0038] Based on the mask area of ​​the real foreign object, locate the connected region corresponding to the foreign object, and simultaneously obtain the minimum bounding rectangle of the corresponding connected region;

[0039] Based on the minimum bounding rectangle of the corresponding connected region of the foreign object, determine the pixel length of the foreign object and convert it into the length and width corresponding to the actual physical size;

[0040] The track bed area without foreign objects around the foreign object mask area in the ballastless track depth map is taken as the background area, and the average depth of the corresponding pixels in the background area is calculated as the reference depth of the track bed.

[0041] Extract the maximum depth value in the connected region corresponding to the foreign object, and calculate the difference between the maximum depth value and the reference depth of the track bed as the height of the foreign object;

[0042] An alarm message is output only when the length, width, and height of the foreign object are all greater than or equal to the detection threshold; otherwise, the foreign object information is recorded.

[0043] One possible implementation of the first aspect also includes:

[0044] In the dataset consisting of labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed, the synthetic images of foreign objects account for more than 90% of the dataset.

[0045] A second aspect of this application provides a foreign object detection system for orbits, comprising:

[0046] The unknown foreign object modeling unit generates the length, width, and height values ​​of the foreign object randomly using a Gaussian probability distribution function, given the maximum and minimum thresholds for the length, width, and height of the foreign object. Within the given threshold range, it generates random 3D points of a specified length, constructs an irregular polyhedron network using a convex hull algorithm, and adds noise to the surface to complete the modeling of the unknown foreign object.

[0047] The known foreign object modeling unit, based on prior data, establishes a 3D model of a known regular foreign object and adds noise to the surface to complete the known foreign object modeling; the unknown foreign object modeling and the known foreign object modeling together constitute the foreign object model;

[0048] The track bed flat area acquisition unit performs gradient segmentation and curvature filtering on the track bed depth map, and obtains the track bed flat area through binarization and morphological calculation.

[0049] The foreign object depth map unit randomly selects an appropriate foreign object superposition position in the flat area of ​​the track bed in the track bed depth map, and projects the foreign object model into a foreign object depth map based on the 3D camera imaging model and the shooting position and angle of the 3D camera.

[0050] The shadow generation unit determines the position of the foreign object in the track bed depth map based on the foreign object depth map, and calculates the shadow area formed on the track bed surface after the foreign object blocks the optical path of the 3D camera, in combination with the installation position of the 3D camera and the 3D camera imaging model. The shadow area corresponds to the position of the foreign object in the foreign object depth map.

[0051] The foreign object synthesis unit overlays the foreign object depth map and the shadow area onto the track bed image without foreign objects, and filters the foreign object depth map to obtain a track bed image containing foreign objects, denoted as the track bed foreign object synthesis image; based on the foreign object overlay position and the shadow area, the track bed foreign object synthesis image is automatically labeled to obtain a corresponding mask image;

[0052] The foreign object training unit uses labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed as datasets to train the foreign object segmentation network.

[0053] The foreign object inference unit uses the trained foreign object segmentation network to perform forward inference on the ballastless track depth map, predicts and segments potential foreign object regions, and generates a probability distribution heatmap; it performs binarization processing on the probability distribution heatmap based on a set threshold to obtain a binarized image; it performs connected component analysis on the binarized image, and combines it with the ballastless track depth map to remove regions whose actual size is smaller than the detection threshold, thereby obtaining the mask region of potential foreign objects;

[0054] The inherent facility training unit extracts grayscale images of the inherent facilities of the track as a secondary classification dataset to train the secondary classification network.

[0055] The inherent facility reasoning unit uses the trained secondary classification network to perform secondary reasoning on the mask region of potential foreign objects, eliminates the region corresponding to the inherent facilities of the track, obtains the mask region of the real foreign objects, and finally outputs the detection result of the real foreign objects, and outputs alarm information based on the detection threshold.

[0056] Its beneficial effects are as follows: This invention discloses a method and system for detecting foreign objects on a track. First, an unknown foreign object model is constructed using Gaussian distribution and convex hull algorithm. A known foreign object model is then established using prior data (both with added noise). Flat areas of the track bed are extracted, and the foreign object model is projected as a depth map and a shadow is generated. These are then superimposed to obtain a labeled synthetic image. A segmentation network is trained using the synthetic image and the real image. During inference, thermal imaging is used to detect the foreign object. Figure 2 Value-based analysis, connected component analysis, and size screening are used to obtain potential foreign object masks. Then, a secondary classification network is used to eliminate false alarms from existing facilities, ultimately outputting the true foreign object detection results. An alarm is triggered based on the detection threshold, achieving efficient and accurate detection. This invention achieves efficient, accurate, and real-time foreign object detection on ballastless tracks under the premise of scarce real foreign object samples and complex track bed scenarios, while reducing false alarm rates and improving the ability to generalize to unknown foreign objects. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0058] Figure 1 This is a schematic flowchart of a foreign object detection method for tracks provided in an embodiment of this application;

[0059] Figures 2-9 This is the synthetic dataset provided in the embodiments of this application. Figure 2 , Figure 4 , Figure 6 and Figure 8 These are composite images of foreign objects in the track bed. Figure 3 , Figure 5 , Figure 7 and Figure 9 The foreign object mask image corresponding to the synthesized image of foreign objects in the track bed;

[0060] Figure 10 This is a schematic diagram of the composition of a foreign object detection system for tracks provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0063] Example 1

[0064] In existing technologies, supervised methods train models directly on grayscale images. However, due to limitations in the types and quantities of training data, they are poorly adaptable to unknown foreign objects and complex track bed backgrounds, resulting in a high false detection rate. Their detection performance heavily relies on a large amount of manually labeled real foreign object data. Moreover, in real-world scenarios, the following problems exist: real foreign objects have diverse forms and low occurrence probabilities, leading to difficulties in sample collection and scarcity of samples, making it difficult to cover all types of foreign objects; sample labeling is costly and prone to imbalanced sample problems, directly limiting the model training effect and resulting in extremely poor generalization ability for unknown foreign objects.

[0065] Therefore, this application provides a method for detecting foreign objects in orbit, such as... Figure 1 As shown, it includes:

[0066] Given the maximum and minimum thresholds for the three dimensions of the foreign object, the Gaussian probability distribution function is used to randomly generate the dimensions of the foreign object. Within the given threshold range of the dimensions of the foreign object, random three-dimensional points of a specified length are generated. An irregular polyhedron network is constructed using the convex hull algorithm and noise is added to the surface to complete the modeling of the unknown foreign object.

[0067] Based on prior data, a 3D model of a known irregular object is established and noise is added to the surface to complete the modeling of the known irregular object; the modeling of the unknown irregular object and the modeling of the known irregular object together constitute the model of the irregular object.

[0068] The track bed depth map is subjected to gradient segmentation and curvature filtering, and the flat area of ​​the track bed is obtained by binarization and morphological calculation.

[0069] In the track bed depth map, a suitable location for the superposition of foreign objects is randomly selected in the flat area of ​​the track bed. Based on the 3D camera imaging model and the shooting position and angle of the 3D camera, the foreign object model is projected into a foreign object depth map.

[0070] Based on the foreign object depth map, the position of the foreign object in the track bed depth map is determined. Combined with the installation position of the 3D camera and the 3D camera imaging model, the shadow area formed on the track bed surface after the foreign object blocks the optical path of the 3D camera is calculated. The shadow area corresponds to the position of the foreign object in the foreign object depth map.

[0071] The foreign object depth map and the shadow area are superimposed on the track bed image without foreign objects, and the foreign object depth map is filtered to obtain a track bed image containing foreign objects, which is denoted as the track bed foreign object composite image; based on the foreign object superposition position and the shadow area, the track bed foreign object composite image is automatically labeled to obtain the corresponding mask image;

[0072] The labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed are used as datasets to train the foreign object segmentation network.

[0073] The trained foreign object segmentation network is used to perform forward reasoning on the ballastless track depth map to predict and segment potential foreign object regions, generating a probability distribution heatmap. The probability distribution heatmap is then binarized based on a set threshold to obtain a binarized image. Connected component analysis is performed on the binarized image, and regions with actual sizes smaller than the detection threshold are removed in conjunction with the ballastless track depth map to obtain the mask region of potential foreign objects.

[0074] Extract grayscale images of the inherent facilities of the track as a secondary classification dataset to train a secondary classification network;

[0075] The trained secondary classification network is used to perform secondary reasoning on the masked region of potential foreign objects, eliminating the region corresponding to the inherent facilities of the track, to obtain the masked region of the real foreign objects, and finally output the detection result of the real foreign objects, and output alarm information based on the detection threshold.

[0076] This embodiment provides a method for detecting foreign objects on a track, which is implemented through the following steps:

[0077] 1. Modeling of unknown foreign objects

[0078] Given a threshold range for the three dimensions of the foreign object (length, width, and height), such as length ∈ [2cm, 30cm], width ∈ [2cm, 30cm], and height ∈ [1cm, 15cm], then based on the Gaussian distribution function (… , The size of the foreign object is randomly generated, and 35 random points are generated in the corresponding space. These point clouds will serve as the core skeleton for constructing the polyhedron using the convex hull algorithm. The Quickhull convex hull algorithm is used to perform minimum closure wrapping on the generated discrete 3D point clouds to form an irregular polyhedron network, and Gaussian noise with a mean of 0 and a variance of 0.5 mm is added to the surface.

[0079] By randomizing (object size + point cloud) to solve the problem of scarce samples, by geometric reconstruction (convex hull algorithm) to solve the problem of irregular shape, and by realism enhancement to solve the problem of distortion of synthetic data, the resulting unknown object model can be directly used for subsequent road bed scene synthesis, providing diverse training samples for the object segmentation network.

[0080] 2. Modeling a known foreign object

[0081] Based on accurate 3D reconstruction of prior data and realistic texture restoration of surface noise, for foreign objects with fixed shapes and known types (such as fasteners, water bottles, wrenches, headlamps, gloves, screwdrivers, etc.), a standard 3D model consistent with the real shape is first constructed. Then, by adding noise to simulate the actual texture, a regular foreign object model with accurate shape and realistic details is generated, which complements the known foreign object model and enriches the diversity of synthetic data.

[0082] 3. Obtain the flat area of ​​the track bed

[0083] The core principle is to initially screen smooth areas through gradient segmentation, accurately purify the area through curvature filtering, perform binarization, optimize the contour through morphology, and match the size of foreign objects to the appropriate area. By selecting flat areas with smooth textures, no protrusion interference, and sufficient size from the complex track bed depth map, it provides an appropriate background for subsequent foreign object superposition, avoids feature confusion caused by complex background during foreign object synthesis, and ensures the authenticity of the synthesized data.

[0084] Gradient segmentation uses gradient as a metric to measure the rate of change of pixel depth values. The depth values ​​in flat areas of the track bed remain almost constant, while the depth values ​​in textured areas, fastener edges, and other regions change abruptly. The specific steps are as follows: For each pixel in the track bed depth map, calculate the depth difference between it and its neighboring pixels to generate a gradient map; set a gradient threshold, retaining pixels with gradients less than or equal to the threshold and discarding pixels with gradients greater than the threshold; quickly remove obvious protrusions, edges, and other interfering regions from the track bed to obtain a preliminary candidate set of smooth areas.

[0085] Curvature filtering focuses on the curvature of pixel regions, complementing gradient segmentation. Gradients focus on the rate of change of depth, while curvature focuses on the curvature of the surface. This dual filtering further removes the tiny textures, impurities, and protrusions left by gradient segmentation.

[0086] Binarization works by setting a grayscale threshold and converting the image after gradient-curvature dual filtering into a black-and-white binary image. Pixels in flat areas are set to white (foreground) and pixels in interfering areas are set to black (background). Its purpose is to clearly separate flat areas from the grayscale image, laying the foundation for subsequent morphological processing and size calculation.

[0087] Morphological calculations employ morphological opening operations of erosion and dilation. Erosion eliminates minute noise in the white area, while dilation fills in small voids in the white area. The purpose is to make the flat area continuous and complete, preventing subsequent foreign objects from being unable to be completely superimposed due to regional fragmentation.

[0088] 4. Foreign object depth map projection

[0089] The core principle is flat area adaptive positioning + 3D camera imaging simulation + controllable occlusion ratio. A reasonable location is locked in a flat area of ​​the track bed, and the foreign object model is projected into a depth map according to the logic of a real camera. Then, diverse samples are generated through adjustable occlusion ratios. This ensures the authenticity of the synthetic data while solving the problem of a single training sample scenario. The specific steps are as follows:

[0090] Based on the selected flat areas of the track bed, the locations are selected according to the principles of spatial adaptation and random distribution to ensure that the superposition of foreign objects conforms to the real physical laws.

[0091] By utilizing a 3D camera imaging model and the shooting position and angle of the 3D camera, the actual shooting process of the 3D camera is simulated to transform the foreign object model into an overlayable foreign object depth map. The principle is to rely on the camera's intrinsic and extrinsic parameters (which are completely consistent with the parameters of the actual inspection camera) to map the three-dimensional coordinates of the foreign object model into pixel coordinates + physical depth through the 3D imaging model, forming a foreign object depth map. Each pixel contains both position information and distance (depth value) from the camera.

[0092] By setting an occlusion ratio to simulate real-world scenarios where foreign objects are partially occluded, the model enriches its sample diversity. The principle is to mask the depth information of a portion of the foreign object's depth map according to a set ratio, simulating situations where foreign objects are not fully exposed in real-world scenarios. This addresses the problem of weak model generalization caused by relying on single, unoccluded samples, allowing the model to learn the features of partially visible foreign objects during training and improving detection accuracy in real-world scenarios.

[0093] 5. Shadow area generation

[0094] The core principle is 3D camera optical path simulation, foreign object occlusion judgment and shadow projection calculation. Based on the actual installation position of the 3D camera and the 3D imaging model, the physical process of foreign object occlusion of the camera optical path is simulated. A shadow area matching the position and shape of the foreign object is generated in the track bed depth map, so that the light and shadow effect of the synthetic data is consistent with the real inspection scene, and the realism of subsequent model training is improved.

[0095] Essentially, it is a mathematical simulation of the physical light and shadow process. By replicating the light path propagation of a 3D camera and the occlusion effect of foreign objects, a realistic shadow area is generated in the track bed depth map. This allows the synthetic data to not only contain the depth and shape information of foreign objects, but also the light and shadow details of physical laws, providing more realistic materials for subsequent model training and improving the robustness of the detection model.

[0096] 6. Perform foreign object synthesis to generate a foreign object mask image (see reference). Figures 2-9 (This refers to a composite image of foreign objects in the track bed and the corresponding foreign object mask image.)

[0097] The core principle is accurate fusion of depth maps, smoothing optimization by Gaussian filtering, and accurate mask annotation. The mask image is generated through three collaborative steps, which not only ensures the realism of the synthesized image, but also provides clear foreign object feature annotations for model training, supporting the efficient learning of the subsequent foreign object segmentation network.

[0098] Depth map overlay: The depth map of foreign objects is fused pixel-by-pixel with the depth map of clean track bed (without foreign objects) according to the selected flat areas of the track bed to form a complete track bed image containing foreign objects;

[0099] The filtering process involves applying Gaussian filtering to the superimposed depth map. The core of this process is to simulate the edge blurring effect of real imaging and avoid the distortion caused by overly sharp edges of foreign objects.

[0100] Mask annotation: The mask image is the label for model training. The core is to clearly label the pixel range of foreign objects and shadows in the depth map containing foreign objects, providing a learning target for the foreign object segmentation network. This allows the foreign object segmentation network to accurately identify the location range of foreign objects and shadows in the image through learning. That is, the pixels of the foreign object area and the shadow area are set to 1 (foreground, the target that needs to be identified by the model), and the pixels of the other track bed background area are set to 0 (background, no need to be identified).

[0101] Foreign objects are embedded by overlaying depth maps, Gaussian filtering is used to enhance realism, and mask images provide training labels. Finally, high-quality image-label training pairs are generated to support the efficient training of the foreign object segmentation network.

[0102] 7. Training the foreign object segmentation network

[0103] The principle is to train synthetic and real data together, extract precise features from encoder and decoder, and optimize multiple loss functions in a targeted manner. Through the collaborative design of data, architecture and loss, the segmentation network can accurately learn the pixel-level features of foreign objects in the track, and finally output a probability map that can distinguish foreign objects from the background, providing core model support for subsequent foreign object detection.

[0104] Synthetic and real data are used for collaborative training. The dataset is formed by a ratio of more than 90% synthetic data and a small amount of real data. For example, the synthetic data consists of 120,000 images and the real data consists of 800 images (accounting for 0.67%). The synthetic data ensures the quantity and diversity of the samples, while the real data ensures the adaptability to the scenario.

[0105] The encoder-decoder architecture is designed for precise feature extraction, specifically tailored to the core requirements of trackside object detection (multi-scale, small targets, and complex backgrounds). The encoder employs a ResNet50 deep convolutional neural network and an attention mechanism for high-dimensional feature extraction. Its core function is to transform the input track bed image containing objects from pixel-level raw information into high-dimensional semantic features, while suppressing background interference. ResNet50's advantage lies in its ability to extract more abstract high-dimensional features and its residual connections address the gradient vanishing problem in deep networks, making it suitable for feature learning in complex track bed scenarios. The attention mechanism, through interlayer attention modules, allows the network to automatically focus on the object region while minimizing irrelevant background elements such as track bed texture, thus improving the recognizability of object features. The decoder uses a Feature Pyramid Network (FPN) structure to restore the high-dimensional features extracted by the encoder to pixel-level outputs consistent with the input image size, accommodating both large and small objects. The final output is a probability map corresponding to the pixels of the input image, where the value of each pixel represents the confidence level that the location contains an object. This provides the core basis for subsequent binarization processing, achieving pixel-level precise segmentation.

[0106] Targeted optimization using multiple loss functions, achieved through the design of loss functions:

[0107]

[0108] Assigning weights , The function principle measures the error by calculating the overlap between the predicted region and the real region, and is more sensitive to sample imbalance. The function principle is to quantify the difference between the predicted edge and the actual edge, and constrain the network to learn the edge features of the foreign object; The function works by calculating the mean square error between the predicted depth map and the true depth map, thus constraining the accuracy of depth feature restoration.

[0109] That is, the sample problem is solved through data collaboration, the feature extraction problem is solved through architecture adaptation, and the detection pain point is solved through multi-loss optimization. Finally, a network that can accurately segment foreign objects in the track is trained, laying the core foundation for subsequent inference detection.

[0110] 8. Foreign Object Segmentation Network Reasoning

[0111] The core principle is model probability detection, threshold binarization, connected component clustering, and size verification filtering. Through four collaborative steps, potential foreign object regions with sufficient confidence, intact shape, and size meeting safety thresholds are accurately screened from the ballastless track depth map, providing an accurate candidate set for subsequent false alarm removal. The specific steps are as follows:

[0112] Step 1: Generate a probability distribution heatmap using forward reasoning

[0113] The trained foreign object segmentation network performs forward inference on the input ballastless track depth map. Essentially, the model probabilistically determines the foreign object attribute of each pixel in the image. The value of each pixel (0-1) represents the confidence level of the foreign object; the closer the value is to 1, the higher the confidence level of identifying it as a foreign object, and the closer it is to 0, the more likely it is the track bed background. This pixel-level foreign object identification is transformed into probability quantification, providing an objective basis for subsequent screening and avoiding missed / false detections caused by direct judgment.

[0114] Step 2: Threshold binarization to initially distinguish foreign objects from the background.

[0115] A fixed confidence threshold is set, and the heatmap is binarized to transform probabilistic judgments into explicit pixel classifications. Pixels with a confidence level greater than or equal to the threshold are identified as suspected foreign objects and marked as white; pixels with a confidence level less than the threshold are identified as background and marked as black. This quickly removes most background areas, focusing on white pixels suspected of being foreign objects, thus reducing the amount of data required for subsequent processing.

[0116] Step 3: 8-neighborhood connectivity analysis to cluster suspected foreign object regions.

[0117] An 8-neighborhood connectivity analysis is performed on the binarized black-and-white image to cluster scattered white pixels into complete connected regions, avoiding interference from single noise points or fragmented pixels. An 8-neighborhood determines whether two white pixels are adjacent in eight directions (horizontal, vertical, and diagonal); if adjacent, they are grouped into the same connected region. The purpose is to integrate scattered suspected pixels into morphologically complete suspected foreign object regions, facilitating subsequent size calculations and filtering. Pixels of genuine foreign objects are necessarily continuously connected, while isolated noise points are mostly single pixels, which can be initially filtered out through clustering.

[0118] Step 4: Size verification and filtering to eliminate small, non-threatening areas.

[0119] Combining the 3D information from the ballastless track depth map and camera calibration parameters, the actual physical size of each connected region is calculated and compared with the detection thresholds (length ≥ 2cm, width ≥ 2cm, height ≥ 1cm). Only connected regions with all three dimensions ≥ the detection threshold are retained, while small-sized regions posing no safety threat are eliminated. The actual size calculation logic converts the pixel range of the connected region into the actual physical length / width using the camera calibration parameters. Combined with the depth map, the difference between the maximum depth of the connected region and the track bed reference depth is calculated as the actual height. This process accurately filters out interference regions that meet the confidence criteria but are not threatening in size, ensuring that the final potential foreign object mask areas are all candidate objects that may threaten driving safety.

[0120] 9. Training a secondary classification network

[0121] The core principle is the construction of targeted datasets, the multi-head classification architecture adapted to YOLOv5, and the optimization of hybrid loss functions. It focuses on the accurate identification and elimination of known inherent facilities without relying on a large number of samples. Through hierarchical classification logic and loss function constraints, it efficiently eliminates inherent facilities on the track that resemble foreign objects, thus solving the problem of false alarms of inherent facilities in foreign object detection.

[0122] By extracting grayscale images of potential foreign object regions to create a secondary classification dataset, only grayscale images of inherent facilities (observation mirrors, shock absorbers, cables, etc.) that resemble foreign objects need to be collected as positive samples, without the need to collect a large number of unknown negative samples such as non-inherent facilities / real foreign objects.

[0123] Using YOLOv5 as the network backbone, the core requirements of the track scenario are adapted, and the last layer of YOLOv5 is modified to a dual-branch classification system to achieve hierarchical judgment and avoid confusion caused by direct multi-classification. The advantage of hierarchical classification is that it first makes a coarse judgment on "whether it is an inherent facility" and then makes a fine judgment on "specific type". This avoids the confusion between the characteristics of real foreign objects and similar inherent facilities (such as misjudging shock absorbers and cylindrical foreign objects) when directly multi-classifying, thus improving the accuracy of rejection.

[0124] The prediction accuracy of the dual-classification head is constrained by designing a loss function weighting:

[0125]

[0126] like , The function is the mean squared error loss of the first classification head (whether it is an inherent facility), which quantifies the error between predicting whether it is an inherent facility and the true label; The function is the mean squared error loss of the second classification head (which inherent facility), which quantifies the error between the predicted facility type and the actual type.

[0127] 10. Secondary Classification Network Reasoning

[0128] The principle is to expand the sampling of the outer rectangle, accurately remove objects through secondary classification, calculate the physical size and trigger an alarm based on the threshold. Finally, it removes false alarms from the potential foreign objects, outputs the physical size of the actual foreign objects and triggers an alarm, ensuring the accuracy of the detection results and the practicality of the project.

[0129] Step 1: Expand and cut the minimum bounding rectangle

[0130] Calculate the minimum bounding rectangle for the connected components of the potential foreign object mask, and then cut the grayscale image after doubling the length and width.

[0131] Step 2: Secondary classification network identification and elimination

[0132] The pre-trained secondary classification network performs inference on the segmented grayscale image, eliminating inherent facilities through dual verification of confidence level and size prior. The first step uses the first classification head to determine if a facility is inherent and outputs a confidence level. The second step filters the confidence levels, proceeding to the next verification step only when the confidence level exceeds a set threshold. The third step uses the second classification head to determine the specific type and compares it with the corresponding inherent facility prior size. If the actual size of the segmented area matches the prior size, it is determined to be an inherent facility, and its corresponding Mask connected component is removed. True foreign object Masks are retained, and the effective region is output after filtering.

[0133] Step 3: Physical Dimension Calculation

[0134] Based on the actual foreign object mask area and the ballastless track depth map, combined with camera calibration parameters, the pixel dimensions are converted into actual physical dimensions. For length / width calculation, the minimum bounding rectangle of the connected region of the foreign object mask is extracted, and the pixel length / width of the rectangle is converted into the actual physical length and width using camera calibration parameters. For height calculation, with the track bed area within 5cm of the foreign object mask area as the background, the average depth of this area is calculated, and the maximum depth within the connected region of the foreign object mask is extracted; the difference between the two is the actual physical height of the foreign object.

[0135] Step 4: Threshold-triggered alarm

[0136] The calculated length, width, and height of the foreign object are compared with the detection thresholds (length ≥ 2cm, width ≥ 2cm, height ≥ 1cm). When the length, width, and height of the foreign object are all ≥ the detection thresholds, it is determined to be a foreign object threatening train operation safety, and an alarm message is output. When the size does not reach the detection threshold, it is determined to be a small foreign object without safety threat, and only the foreign object information is recorded to avoid invalid alarm interference. The detection thresholds are compatible with railway safety regulations to ensure that alarm messages are only for foreign objects that truly pose a safety risk, balancing safety and practicality.

[0137] In some embodiments, the foreign object segmentation network adopts an encoder-decoder architecture, specifically:

[0138] The encoder uses a ResNet50 deep convolutional neural network for high-dimensional feature extraction, and an attention mechanism network is used to connect the layers.

[0139] The decoder uses a feature pyramid (FPN) structure for multi-scale feature extraction and outputs a pixel-level foreign object probability map.

[0140] The loss function of the heterogeneous object segmentation network for:

[0141]

[0142] , and These are the weighting coefficients. To quantify the difference between small target predictions and reality, To quantify the difference between the predicted edge and the actual edge, To quantify the difference between deep feature predictions and reality.

[0143] In some embodiments, the backbone of the secondary classification network is based on the YOLOv5 model, and the last layer of the YOLOv5 model is split into two parallel fully connected classification heads, using a multi-class mixture loss function, specifically:

[0144]

[0145] and These are the weighting coefficients. To quantify whether the prediction is due to the inherent facilities and the true label, To quantify the error between the predicted facility type and the actual type.

[0146] In some embodiments, projecting the foreign object model into a foreign object depth map includes:

[0147] By setting the occlusion ratio, the foreign object model is projected into a foreign object depth map with different occlusion states, thus obtaining diverse foreign object samples.

[0148] In some embodiments, performing connected component analysis on the binarized image includes:

[0149] Eight-neighborhood connectivity analysis is used to cluster white pixels in the binarized image, specifically as follows:

[0150] Determine whether two white pixels are adjacent in the horizontal, vertical, and diagonal directions. If they are, they are considered a connected region; otherwise, they are considered a disconnected region and are removed.

[0151] In some embodiments, the mask region for obtaining potential foreign objects includes:

[0152] Based on the three-dimensional information of the ballastless track depth map and combined with the calibration parameters of the 3D camera, the pixel range of each connected region is converted into the actual physical size.

[0153] The actual physical dimensions are compared with the preset detection thresholds. If the length, width, and height of the connected region are all smaller than the corresponding detection thresholds, it is determined to be a type of connected region and is removed. At the same time, the remaining connected regions are used as mask regions for potential foreign objects.

[0154] In some embodiments, excluding the area corresponding to the inherent facilities of the track includes:

[0155] Calculate the minimum bounding rectangle of any connected region in the mask region, and then cut out the corresponding grayscale image after multiplying the length and width of the minimum bounding rectangle by 1.

[0156] The trained secondary classification network is used to identify the grayscale image and determine whether it is an inherent facility of the track. If it is, and the confidence level is greater than a set threshold and the corresponding size matches the prior data, the inherent facility of the track corresponding to the grayscale image is removed. If not, no action is taken.

[0157] In some embodiments, outputting alarm information based on the detection threshold includes:

[0158] Based on the mask area of ​​the real foreign object, locate the connected region corresponding to the foreign object, and simultaneously obtain the minimum bounding rectangle of the corresponding connected region;

[0159] Based on the minimum bounding rectangle of the corresponding connected region of the foreign object, determine the pixel length of the foreign object and convert it into the length and width corresponding to the actual physical size;

[0160] The track bed area without foreign objects around the foreign object mask area in the ballastless track depth map is taken as the background area, and the average depth of the corresponding pixels in the background area is calculated as the reference depth of the track bed.

[0161] Extract the maximum depth value in the connected region corresponding to the foreign object, and calculate the difference between the maximum depth value and the reference depth of the track bed as the height of the foreign object;

[0162] An alarm message is output only when the length, width, and height of the foreign object are all greater than or equal to the detection threshold; otherwise, the foreign object information is recorded.

[0163] In some embodiments, it also includes:

[0164] In the dataset consisting of labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed, the synthetic images of foreign objects account for more than 90% of the dataset.

[0165] Example 2

[0166] Based on the orbital foreign object detection method provided in Embodiment 1 of this application, correspondingly, Embodiment 2 of this application also provides an orbital foreign object detection system, such as... Figure 10 As shown, the system includes:

[0167] The unknown foreign object modeling unit generates the length, width, and height values ​​of the foreign object randomly using a Gaussian probability distribution function, given the maximum and minimum thresholds for the length, width, and height of the foreign object. Within the given threshold range, it generates random 3D points of a specified length, constructs an irregular polyhedron network using a convex hull algorithm, and adds noise to the surface to complete the modeling of the unknown foreign object.

[0168] The known foreign object modeling unit, based on prior data, establishes a 3D model of a known regular foreign object and adds noise to the surface to complete the known foreign object modeling; the unknown foreign object modeling and the known foreign object modeling together constitute the foreign object model;

[0169] The track bed flat area acquisition unit performs gradient segmentation and curvature filtering on the track bed depth map, and obtains the track bed flat area through binarization and morphological calculation.

[0170] The foreign object depth map unit randomly selects an appropriate foreign object superposition position in the flat area of ​​the track bed in the track bed depth map, and projects the foreign object model into a foreign object depth map based on the 3D camera imaging model and the shooting position and angle of the 3D camera.

[0171] The shadow generation unit determines the position of the foreign object in the track bed depth map based on the foreign object depth map, and calculates the shadow area formed on the track bed surface after the foreign object blocks the optical path of the 3D camera, in combination with the installation position of the 3D camera and the 3D camera imaging model. The shadow area corresponds to the position of the foreign object in the foreign object depth map.

[0172] The foreign object synthesis unit overlays the foreign object depth map and the shadow area onto the track bed image without foreign objects, and filters the foreign object depth map to obtain a track bed image containing foreign objects, denoted as the track bed foreign object synthesis image; based on the foreign object overlay position and the shadow area, the track bed foreign object synthesis image is automatically labeled to obtain a corresponding mask image;

[0173] The foreign object training unit uses labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed as datasets to train the foreign object segmentation network.

[0174] The foreign object inference unit uses the trained foreign object segmentation network to perform forward inference on the ballastless track depth map, predicts and segments potential foreign object regions, and generates a probability distribution heatmap; it performs binarization processing on the probability distribution heatmap based on a set threshold to obtain a binarized image; it performs connected component analysis on the binarized image, and combines it with the ballastless track depth map to remove regions whose actual size is smaller than the detection threshold, thereby obtaining the mask region of potential foreign objects;

[0175] The inherent facility training unit extracts grayscale images of the inherent facilities of the track as a secondary classification dataset to train the secondary classification network.

[0176] The inherent facility reasoning unit uses the trained secondary classification network to perform secondary reasoning on the mask region of potential foreign objects, eliminates the region corresponding to the inherent facilities of the track, obtains the mask region of the real foreign objects, and finally outputs the detection result of the real foreign objects, and outputs alarm information based on the detection threshold.

[0177] Furthermore, this embodiment provides a foreign object detection system based on ballastless track, mounted on a track inspection vehicle, which employs two or more line structured light 3D cameras. This type of camera can simultaneously acquire pixel-aligned two-dimensional images and corresponding three-dimensional information. This system provides a solution for foreign object detection on ballastless track, aiming to improve the detection efficiency and safety assurance capabilities of railway equipment.

[0178] The system's hardware components include:

[0179] The system is mounted on a track inspection vehicle, which provides power to the entire system.

[0180] The vision module uses two line structured light 3D cameras, which are installed 0.55m directly above the left and right rails respectively. The combined field of view of the two cameras is 3.4m, which can completely cover the track bed area.

[0181] The imaging control module reads mileage data and triggers the vision module to scan and image at 1mm intervals.

[0182] The software has the following functions:

[0183] Camera calibration is achieved by acquiring images of the calibration board, extracting control points, and calculating the rotation and translation matrices of the left and right cameras.

[0184] Image acquisition: The vision module scans the 3D model of the track bed based on a 1mm trigger and outputs a track bed depth image. The left and right images are then stitched and fused together using calibrated camera extrinsic parameters.

[0185] Image preprocessing involves using Gaussian filtering to remove image noise and using log nonlinear image enhancement to improve the image signal-to-noise ratio and contrast.

[0186] Model training involves using a large dataset (synthesized images of foreign objects in the track bed) synthesized with positive samples and a small number of real images of foreign objects in the track bed to train the foreign object segmentation network, and using grayscale images of inherent track facilities to train the secondary classification network.

[0187] Foreign object segmentation network inference: The trained foreign object segmentation network performs forward inference on the preprocessed image to obtain a heatmap of the probability distribution of potential foreign objects. The heatmap is then processed according to a set threshold. Figure 2 Binarization; perform 8-neighborhood connectivity analysis on the binarized image, and retain only the connected regions whose length, width and height are greater than or equal to the detection threshold as potential foreign object mask regions;

[0188] For the removal of inherent facilities, the minimum bounding rectangle of the connected components in the mask region is calculated. After doubling the length and width, the corresponding grayscale image is cut out. A quadratic classification network is used to identify the grayscale image to determine whether it is an inherent facility of the track. If the confidence level is greater than the threshold and the size matches the prior, it is removed. The mask region that is finally retained is the foreign object region.

[0189] The results are reported, outputting the length and width of the foreign object region in the Mask. The maximum depth value in the depth map is calculated based on the Mask and used as the height output. At the same time, alarm information is output based on the detection threshold.

[0190] The track foreign object detection system in this embodiment solves the problems of low efficiency, high false alarm rate and difficulty in covering complex scenarios caused by traditional detection methods that rely on manual inspection or fixed sensors. At the same time, it solves the problem of scarcity of real foreign object samples in railway track bed foreign object detection, improves the generalization ability of unknown foreign objects, and reduces the overall false alarm rate by more than 60%.

[0191] The specific principles and execution processes of each unit in the track foreign object detection system disclosed in Embodiment 2 of this application are the same as those of the track foreign object detection method disclosed in Embodiment 1 of this application. Please refer to the corresponding part of the track foreign object detection method disclosed in Embodiment 1 of this application. They will not be repeated here.

[0192] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computing software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0194] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting foreign objects on a track, characterized in that, include: Given the maximum and minimum thresholds for the three dimensions of the foreign object, the Gaussian probability distribution function is used to randomly generate the dimensions of the foreign object. Within the given threshold range of the dimensions of the foreign object, random three-dimensional points of a specified length are generated. An irregular polyhedron network is constructed using the convex hull algorithm and noise is added to the surface to complete the modeling of the unknown foreign object. Based on prior data, a 3D model of a known irregular object is established and noise is added to the surface to complete the modeling of the known irregular object; the modeling of the unknown irregular object and the modeling of the known irregular object together constitute the model of the irregular object. The track bed depth map is subjected to gradient segmentation and curvature filtering, and the flat area of ​​the track bed is obtained by binarization and morphological calculation. In the track bed depth map, a suitable location for the superposition of foreign objects is randomly selected in the flat area of ​​the track bed. Based on the 3D camera imaging model and the shooting position and angle of the 3D camera, the foreign object model is projected into a foreign object depth map. Based on the foreign object depth map, the position of the foreign object in the track bed depth map is determined. Combined with the installation position of the 3D camera and the 3D camera imaging model, the shadow area formed on the track bed surface after the foreign object blocks the optical path of the 3D camera is calculated. The shadow area corresponds to the position of the foreign object in the foreign object depth map. The foreign object depth map and the shadow area are superimposed on the track bed image without foreign objects, and the foreign object depth map is filtered to obtain a track bed image containing foreign objects, which is denoted as the track bed foreign object composite image; based on the foreign object superposition position and the shadow area, the track bed foreign object composite image is automatically labeled to obtain the corresponding mask image; The labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed are used as datasets to train the foreign object segmentation network. The trained foreign object segmentation network is used to perform forward reasoning on the ballastless track depth map to predict and segment potential foreign object regions, and generate a probability distribution heatmap. The probability distribution heatmap is binarized based on a set threshold to obtain a binarized image; connected component analysis is performed on the binarized image, and the area corresponding to the actual size being smaller than the detection threshold is removed by combining the ballastless track depth map to obtain the mask area of ​​potential foreign objects. Extract grayscale images of the inherent facilities of the track as a secondary classification dataset to train a secondary classification network; The trained secondary classification network is used to perform secondary reasoning on the masked region of potential foreign objects, eliminating the region corresponding to the inherent facilities of the track, to obtain the masked region of the real foreign objects, and finally output the detection result of the real foreign objects, and output alarm information based on the detection threshold.

2. The method for detecting foreign objects on a track according to claim 1, characterized in that, The foreign object segmentation network adopts an encoder-decoder architecture, specifically: The encoder uses a ResNet50 deep convolutional neural network for high-dimensional feature extraction, and an attention mechanism network is used to connect the layers. The decoder uses a feature pyramid (FPN) structure for multi-scale feature extraction and outputs a pixel-level foreign object probability map. The loss function of the heterogeneous object segmentation network for: , and These are the weighting coefficients. To quantify the difference between small target predictions and reality, To quantify the difference between the predicted edge and the actual edge, To quantify the difference between deep feature predictions and reality.

3. The method for detecting foreign objects on a track according to claim 1, characterized in that, The backbone of the secondary classification network is based on the YOLOv5 model, and the last layer of the YOLOv5 model is split into two parallel fully connected classification heads, using a multi-class mixture loss function, specifically: and These are the weighting coefficients. To quantify whether the prediction is due to the inherent facilities and the true label, To quantify the error between the predicted facility type and the actual type.

4. The method for detecting foreign objects on a track according to claim 1, characterized in that, Projecting the foreign object model into a foreign object depth map includes: By setting the occlusion ratio, the foreign object model is projected into a foreign object depth map with different occlusion states, thus obtaining diverse foreign object samples.

5. The method for detecting foreign objects on a track according to claim 1, characterized in that, Connectivity analysis of the binarized image includes: Eight-neighborhood connectivity analysis is used to cluster white pixels in the binarized image, specifically as follows: Determine whether two white pixels are adjacent in the horizontal, vertical, and diagonal directions. If they are, they are considered a connected region; otherwise, they are considered a disconnected region and are removed.

6. The method for detecting foreign objects on a track according to claim 5, characterized in that, The masked area for obtaining potential foreign objects includes: Based on the three-dimensional information of the ballastless track depth map and combined with the calibration parameters of the 3D camera, the pixel range of each connected region is converted into the actual physical size. The actual physical dimensions are compared with the preset detection thresholds. If the length, width, and height of the connected region are all smaller than the corresponding detection thresholds, it is determined to be a type of connected region and is removed. At the same time, the remaining connected regions are used as mask regions for potential foreign objects.

7. The method for detecting foreign objects on a track according to claim 5, characterized in that, The areas corresponding to existing track facilities are excluded, including: Calculate the minimum bounding rectangle of any connected region in the mask region, and then cut out the corresponding grayscale image after multiplying the length and width of the minimum bounding rectangle by 1. The trained secondary classification network is used to identify the grayscale image and determine whether it is an inherent facility of the track. If it is, and the confidence level is greater than a set threshold and the corresponding size matches the prior data, the inherent facility of the track corresponding to the grayscale image is removed. If not, no action is taken.

8. The method for detecting foreign objects on a track according to claim 7, characterized in that, And based on the detection threshold, the alarm information output includes: Based on the mask area of ​​the real foreign object, locate the connected region corresponding to the foreign object, and simultaneously obtain the minimum bounding rectangle of the corresponding connected region; Based on the minimum bounding rectangle of the corresponding connected region of the foreign object, determine the pixel length of the foreign object and convert it into the length and width corresponding to the actual physical size; The track bed area without foreign objects around the foreign object mask area in the ballastless track depth map is taken as the background area, and the average depth of the corresponding pixels in the background area is calculated as the reference depth of the track bed. Extract the maximum depth value in the connected region corresponding to the foreign object, and calculate the difference between the maximum depth value and the reference depth of the track bed as the height of the foreign object; An alarm message is output only when the length, width, and height of the foreign object are all greater than or equal to the detection threshold; otherwise, the foreign object information is recorded.

9. The method for detecting foreign objects on a track according to claim 1, characterized in that, Also includes: In the dataset consisting of labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed, the synthetic images of foreign objects account for more than 90% of the dataset.

10. A foreign object detection system for railway tracks, characterized in that, include: The unknown foreign object modeling unit generates the length, width, and height values ​​of the foreign object randomly using a Gaussian probability distribution function, given the maximum and minimum thresholds for the length, width, and height of the foreign object. Within the given threshold range, it generates random 3D points of a specified length, constructs an irregular polyhedron network using a convex hull algorithm, and adds noise to the surface to complete the modeling of the unknown foreign object. The known foreign object modeling unit, based on prior data, establishes a 3D model of a known regular foreign object and adds noise to the surface to complete the known foreign object modeling; the unknown foreign object modeling and the known foreign object modeling together constitute the foreign object model; The track bed flat area acquisition unit performs gradient segmentation and curvature filtering on the track bed depth map, and obtains the track bed flat area through binarization and morphological calculation. The foreign object depth map unit randomly selects an appropriate foreign object superposition position in the flat area of ​​the track bed in the track bed depth map, and projects the foreign object model into a foreign object depth map based on the 3D camera imaging model and the shooting position and angle of the 3D camera. The shadow generation unit determines the position of the foreign object in the track bed depth map based on the foreign object depth map, and calculates the shadow area formed on the track bed surface after the foreign object blocks the optical path of the 3D camera, in combination with the installation position of the 3D camera and the 3D camera imaging model. The shadow area corresponds to the position of the foreign object in the foreign object depth map. The foreign object synthesis unit overlays the foreign object depth map and the shadow area onto the track bed image without foreign objects, and filters the foreign object depth map to obtain a track bed image containing foreign objects, denoted as the track bed foreign object synthesis image; based on the foreign object overlay position and the shadow area, the track bed foreign object synthesis image is automatically labeled to obtain a corresponding mask image; The foreign object training unit uses labeled synthetic images of foreign objects in the track bed and real images of foreign objects in the track bed as datasets to train the foreign object segmentation network. The foreign object inference unit uses the trained foreign object segmentation network to perform forward inference on the ballastless track depth map, predicts and segments potential foreign object regions, and generates a probability distribution heatmap; it performs binarization processing on the probability distribution heatmap based on a set threshold to obtain a binarized image; it performs connected component analysis on the binarized image, and combines it with the ballastless track depth map to remove regions whose actual size is smaller than the detection threshold, thereby obtaining the mask region of potential foreign objects; The inherent facility training unit extracts grayscale images of the inherent facilities of the track as a secondary classification dataset to train the secondary classification network. The inherent facility reasoning unit uses the trained secondary classification network to perform secondary reasoning on the mask region of potential foreign objects, eliminates the region corresponding to the inherent facilities of the track, obtains the mask region of the real foreign objects, and finally outputs the detection result of the real foreign objects, and outputs alarm information based on the detection threshold.

Images