Track obstacle invasion detection method without category limitation

Abstract

The invention discloses a category-unlimited track obstacle boundary invasion detection method. The method comprises the following steps: S1) constructing an early warning boundary generation method and a category-unlimited obstacle detection method; s2) acquiring an image of a track area to be detected, and generating an early warning boundary on the image through an early warning boundary generation method; s3) intercepting a small image on the image of the track area to be detected based on the generated early warning boundary; s4) obtaining a normal graph from the small screenshot through a category-unlimited obstacle detection method; and S5) carrying out image processing and segmentation based on the normal graph, extracting the contour of an obstacle higher than the ground, comparing the extracted contour of the obstacle with the early warning boundary, and if there is an intersection, determining that foreign matter invasion exists and giving an early warning, thereby solving the problem of missing detection of foreign matters of unknown categories in a traditional method, and realizing unlimited category detection.

Description

Technical Field

[0001] This invention belongs to the field of track obstacle intrusion detection technology, specifically relating to a method for detecting track obstacles of any type. Background Technology

[0002] With the rapid development of rail transit, train operation safety has received increasing attention. Foreign object intrusion into the track clearance is a significant factor threatening train safety; therefore, accurate and real-time detection of foreign objects on the track has become a key technology for ensuring safe train operation. Currently, mainstream track foreign object detection methods mainly include deep learning-based target detection methods, lidar-based target detection methods, and multimodal target detection methods.

[0003] I. Deep Learning-Based Object Detection Methods The method is implemented through the following steps: First, collect images containing obstacles to construct a dataset; second, label the images with the algorithm to construct a dataset for training the algorithm; then, use a one-stage or two-stage object detection algorithm to train the data and obtain a detection model; finally, deploy the model to obtain image data in front of the locomotive for prediction.

[0004] However, this method has the following technical drawbacks: First, it can only identify the types of foreign objects that the detection model has trained and learned. It cannot identify abnormal data that has not been learned, which affects the detection performance. Second, it is greatly affected by environmental factors such as lighting. Third, the first-stage algorithm has limited accuracy but is fast, while the second-stage algorithm has higher accuracy but is slow, resulting in a contradiction between speed and accuracy.

[0005] II. Target Detection Methods Based on LiDAR The method is implemented through the following steps: First, acquire point cloud data and define the size of the foreign object; second, process the point cloud data using a point cloud processing algorithm, and filter point cloud clusters that meet the conditions through a clustering algorithm or a point cloud object detection algorithm; finally, deploy the algorithm for detection.

[0006] The main problems with this method are: first, the point cloud processing algorithm is inefficient, with a maximum of 10fps, which is difficult to meet real-time requirements; second, hardware devices such as LiDAR are expensive, with mainstream 128-line LiDAR costing tens of thousands of yuan, making it unsuitable for large-scale deployment.

[0007] III. Multimodal Target Detection Methods This method is implemented through the following steps: First, acquire data from multiple sensors, such as video streams and point cloud data; second, learn data features based on the types of data acquired and train a model; then, use the learned model, image processing algorithms, and point cloud processing algorithms to process the raw data and obtain the objects to be detected; finally, use a multimodal fusion algorithm to process the results and obtain the final target detection result.

[0008] This method has the following technical drawbacks: First, the processing logic of multimodal algorithms is complex and not suitable for real-time calculation by vehicle-mounted devices; second, multimodal data acquisition equipment is expensive, such as the mainstream 128-line LiDAR, which costs tens of thousands of yuan and is difficult to promote and apply on a large scale.

[0009] In summary, existing technologies suffer from the following core problems: First, general machine learning (deep learning, point cloud) has a limited range of detection categories, making it unable to detect obstacles not included in the model. In real-world scenarios, there are many types of obstacles, including unpredictable ones on tracks, resulting in limited data collection and a limited range of detection categories, making it prone to missed detections. Second, processing algorithms based on multimodal devices are complex and have high hardware costs, making them unsuitable for large-scale deployment. Third, the detection range is the entire image, which cannot accurately determine the train's running track and generate warning boundaries, easily leading to false warnings, such as issuing a warning after detecting an obstacle on a non-running track. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides a method for detecting track obstacle intrusions that is not limited to any particular type.

[0011] The specific plan is as follows: A method for detecting encroachment of track obstacles regardless of category, comprising the following steps: S1): Construct a method for generating early warning boundaries and a method for detecting obstacles of any category; S2): Acquire an image of the track area to be detected, and generate a warning boundary on the image using a warning boundary generation method. The warning boundary is used to define the safe operating area of ​​the track. S3): Based on the generated warning boundary, a small image is cropped from the image of the track area to be detected, and the cropped small image is the image within the warning boundary area; S4): Obtain the normal map of the small screenshot using an obstacle detection method that does not limit the types of obstacles. The normal map is used to characterize the normal direction information of the surface of the object in the image. S5): Based on the normal map, perform image processing, segmentation, and adaptive filtering to extract the outline of obstacles higher than the ground; compare the extracted obstacle outline with the warning boundary, and if there is an intersection, determine that it is an intrusion of foreign object and issue a warning.

[0012] The method for generating early warning boundaries includes the following steps: P1): Track data collection and processing: Install equipment on the roof of the vehicle, ride along the vehicle to collect track data, organize and filter the track dataset, and create a track instance segmentation dataset. P2): Training and generation of the orbital contour extraction model: A lightweight algorithm is selected to train the orbital contour extraction model and generate the orbital contour extraction model. The lightweight algorithm is YOLOv8. P3): Model Deployment: Deploy the track contour extraction model to an embedded device application; P4): Warning boundary generation: Based on the track contour extraction model, the image of the track area to be detected is used to extract the contour, and the warning boundary is generated by expanding the extracted contour.

[0013] In P4), the warning boundary is generated by expanding the extracted contour, including the following steps: P41): Calculate the left line segment point set, the right line segment point set, and the track centerline point set of the profile; P42): Calculate the normal corresponding to the right line segment point set. The normal direction corresponding to each point on the right line segment point set is perpendicular to the track edge and inward at each point. Calculate the perpendicular distance from each point on the centerline point set to the left and right edges respectively. The sum of the two perpendicular distances is the track width at that point. The normal corresponding to the right line segment point set helps determine the measurement direction of the width. P43): Calculate the area range for the width at the end of the track; P44): Set the magnification factor, and based on the magnification factor, multiply by the track width described in step P42) to obtain the magnified track width; P45): Based on the expansion of the central axis to both sides, each point on the central axis moves to both sides by half the distance of the "expanded track width"; P46): Based on the magnification factor and the width at the end of the track, the track area is extended to obtain the expanded boundary, i.e., the warning boundary.

[0014] The obstacle detection method, which is not limited to any particular category, uses the lightweight Metric3Dv2 algorithm to obtain the normal map based on the visible light image. The construction of the obstacle detection method includes the following steps: T1): Preprocessing the input image, the preprocessing including image normalization and boundary expansion; T2): An encoder-decoder architecture is adopted. The encoder uses ViT (Vision Transformer) to extract multi-level features from the input image to obtain semantic information and spatial detail features at different levels. The decoder receives the multi-scale features extracted by the encoder, restores the spatial resolution through feature fusion and upsampling operations, and outputs dense features that match the resolution of the input image. T3): Post-processing and 2D feature format conversion of ViT features: unpacking feature dimensionality information and separating spatial features, reshaping the spatial image block tokens in each feature batch into a standard 4D feature map format [B, H, W, C] for subsequent dense prediction, while extracting classification tokens and registration tokens and flattening them into global context features [B, 1, 1, C × (1 + num_register_tokens)]. In this process, local spatial information and global semantic representation are preserved, providing structured feature input for the depth estimation task; where B represents the number of inputs, H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map; T4): The feature optimizer is used to process the input features to generate a shared feature representation for joint prediction of depth and normal. At this time, the feature map resolution is 1 / 4 of the input. T5): The optimized feature channels are split into three parts: basic features, depth confidence, and normal confidence; the basic features include the basic features shared by depth and normal predictions; the basic features, depth confidence, and normal confidence together constitute the feature foundation for multi-task learning; T6): Processing the shared feature map: Use the depth regressor to generate a probability distribution of 256 / 512 depth intervals. After softmax normalization, use the depth expectation calculation module to convert the discrete probability distribution into continuous depth values, and perform numerical clipping and scale normalization to obtain the measurement depth prediction result. T7): Processing the shared feature map and normal confidence: The normal predictor generates an initial normal vector from the feature map, then concatenates it with the confidence map, and finally performs vector normalization to ensure that the output normal vector is of unit length; T8): The acquired depth map and normal map features are optimized in 8 iterations: After each iteration, the low-resolution flow field is upsampled to the original resolution using a learnable upsampling module, the depth component is scaled back and numerically clipped, the normal vector is normalized, and the depth prediction, confidence and normal prediction results are stored respectively. T9): Map the obtained normal map colors to 0 and 255 to obtain the normal map.

[0015] S5) The image processing, segmentation, and adaptive filtering based on the normal map includes the following steps: M1): Separate the RGB channels of the generated normal map into R channel map, G channel map and B channel map. The channel map is a grayscale image. Select the R channel image for the next step of processing. M2): Image binarization. By setting one or more thresholds, the intensity value of each pixel in the grayscale image is compared with the threshold, thereby dividing it into two categories: foreground or background. It is then converted into a binary image containing only pure black and pure white pixel values, thus achieving the separation of the target from the background. M3): Contour adaptive filtering divides the image into several continuous regions along the vertical or height direction, dynamically assigns different filtering thresholds to each region, and for an extracted contour, determines the image region where the contour center point or pixel distribution is located, and then uses the preset threshold of that region for filtering. Only contours whose area is greater than the threshold of their region will be retained.

[0016] The extracted obstacle contours are compared with the warning boundaries, including the following steps: N1): Perform a Boolean operation to find the intersection of the obstacle outline and the warning boundary outline; N2): Determine if the number of intersections is positive. If it is positive, that is, the obstacle is within the boundary, then issue a voice warning.

[0017] The detection method is deployed on edge devices.

[0018] This invention discloses a method for detecting track obstacles without class restrictions. It constructs a warning boundary generation method and a method for detecting obstacles without class restrictions. The warning boundary generation method obtains the warning boundary of the track, extracts a small image based on the warning boundary, and processes only the image region within the warning boundary, reducing computational load. The method for detecting obstacles without class restrictions obtains a normal map, which renders surfaces at different angles with different colors. For example, objects perpendicular to the ground are given a color significantly different from the ground, thus being "highlighted." This overcomes camera parameter differences and lighting interference, providing stable output results. Based on the normal map, image processing and segmentation are performed to extract the contours of obstacles above the ground, achieving detection of all obstacle types above the ground without distinguishing their specific categories. This solves the problem of traditional methods missing unknown types of foreign objects, realizing detection without class restrictions. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0020] Figure 2 This is a schematic diagram of the early warning boundary generation.

[0021] Figure 3 This is a schematic diagram of the early warning boundary generation in a turnout scenario.

[0022] Figure 4 This is a schematic diagram of early warning boundary generation in a complex turnout scenario.

[0023] Figure 5 This is a schematic diagram of the early warning boundary generation in a straight road scenario.

[0024] Figure 6 This is a schematic diagram illustrating the effect of generating a normal map from a visible light image in a straight-line scene.

[0025] Figure 7 This is a schematic diagram illustrating the effect of generating a normal map from a visible light image in a complex turnout scenario.

[0026] Figure 8 This is a schematic diagram illustrating the effect of generating a normal map from a visible light image in a turnout scenario.

[0027] Figure 9 This is a schematic diagram of the overall testing process. Detailed Implementation

[0028] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the implementation of the present invention, and not all of it. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0029] like Figure 1 As shown, a method for detecting intrusions into orbital obstacles of any category includes the following steps: S1): Construct a method for generating early warning boundaries and a method for detecting obstacles of any category; S2): Acquire an image of the track area to be detected, and generate a warning boundary on the image using a warning boundary generation method. The warning boundary is used to define the safe operating area of ​​the track. The warning boundary is not a fixed detection area, but is dynamically generated based on the real-time extracted track alignment, such as straight sections, curves, or turnouts, ensuring a high degree of consistency between the detection area and the train's running trajectory, and reducing false alarms and missed alarms. S3): Based on the generated warning boundary, a small image is cropped from the image of the track area to be detected. The cropped small image is the image within the warning boundary area. The warning boundary module ensures the regional relevance of the detection. By cropping a small image based on the warning boundary, only the image area within the warning boundary is processed, reducing the amount of computation. S4): Obtain the normal map of the small screenshot using an obstacle detection method that does not limit the types of obstacles. The normal map is used to characterize the normal direction information of the surface of the object in the image. S5): Based on the normal map, perform image processing, segmentation, and adaptive filtering to extract the outline of obstacles higher than the ground; compare the extracted obstacle outline with the warning boundary, and if there is an intersection, determine that it is an intrusion of foreign object and issue a warning.

[0030] This method for detecting unrestricted obstacle intrusions into a track first generates a dynamic warning boundary that closely matches the track geometry. Then, within this boundary, computer vision technology is used to detect any object that intrudes, without prior knowledge of the object's category. It does not rely on pre-trained object detection models, thus eliminating the need to collect and label thousands of obstacle images such as rocks, pedestrians, animals, and falling rocks. This embodiment identifies all "protrusions" based on their physical geometric characteristics, fundamentally solving the problem of detecting rare or unknown obstacle categories.

[0031] The method for generating early warning boundaries includes the following steps: P1): Track data collection and processing: Install equipment on the roof of the vehicle, ride along the vehicle to collect track data, organize and filter the track dataset, and create a track instance segmentation dataset. P2): Training and generation of the orbital contour extraction model: A lightweight algorithm is selected to train the orbital contour extraction model and generate the orbital contour extraction model. The lightweight algorithm is YOLOv8. P3): Model Deployment: The track contour extraction model is deployed to an embedded device application. In this embodiment, hardware acceleration is used for model deployment. Hardware acceleration refers to using dedicated hardware such as GPUs and DSPs to handle computationally intensive tasks originally executed by the central processing unit (CPU), thereby improving performance and efficiency. Specifically, some software running on the CPU is moved to idle hardware resources to reduce the CPU's burden. Through hardware acceleration, the system can complete complex computational tasks more efficiently, thus improving overall performance. The selection of lightweight algorithms and hardware acceleration optimization meet the stringent real-time processing requirements of in-vehicle equipment. P4): Warning boundary generation: Based on the track contour extraction model, the image of the track area to be detected is used to extract the contour, and the warning boundary is generated by expanding the extracted contour.

[0032] like Figure 2 As shown in Figure P4), the early warning boundary is generated by expanding the extracted contour, including the following steps: P41): Calculate the left line segment point set, the right line segment point set, and the track centerline point set of the profile; these point sets are the basis for all subsequent calculations and define the basic shape and orientation of the track; P42): Calculate the normal corresponding to the right line segment point set. The normal direction corresponding to each point on the right line segment point set is perpendicular to the track edge and inward at each point. Calculate the perpendicular distance from each point on the centerline point set to the left and right edges. The sum of the two perpendicular distances is the track width at that point. The normal corresponding to the right line segment point set helps determine the measurement direction of the width. The calculated track width provides an accurate dimensional basis for the subsequent "proportional enlargement". The track width is not equal everywhere, especially at curves, so it needs to be calculated point by point. P43): Calculate the area range of the width at the end of the track; ensure that the generated warning boundary can smoothly cover the distance, and avoid sudden or unreasonable boundary cutoffs at the end of the field of view; P44): Set the magnification factor, and based on the magnification factor, multiply by the track width described in step P42) to obtain the magnified track width; the width expansion ratio is controllable. For example, if the ratio coefficient is set to 3, and the track width at a certain point is 1.435 meters (standard gauge), and the magnification factor is 3, then the warning boundary width at this point is 1.435 * 3 = 4.305 meters; the width expansion ratio is controllable, so that the expansion range of the safety zone can be flexibly adjusted to adapt to different safety standards and scenario requirements; P45): Based on the expansion to both sides of the central axis, each point on the central axis moves to both sides by half the distance of the "expanded track width"; thus achieving uniform and symmetrical expansion with the track center as the reference. P46): Based on the magnification factor and the width at the end of the track, the track area is extended to obtain the expanded boundary, i.e. the warning boundary, to ensure that the warning area is a complete "electronic fence" that covers a sufficient distance in front of the train, and not just the two sides of the currently visible track.

[0033] like Figures 3 to 5 The image shows the warning boundaries generated for the track under different scenarios. Figure 3 This refers to the generation of early warning boundary maps in simple turnout scenarios. Figure 3 The three images in the image, from top to bottom, are the original image, the extracted orbital contour image, and the expanded warning boundary image; Figure 4 It is for generating early warning boundary maps in complex turnout scenarios. Figure 4 The three images in the image, from top to bottom, are the original image, the extracted orbital contour image, and the expanded warning boundary image; Figure 5 This refers to the generation of warning boundary maps in straight-road scenarios. Figure 5 The three images in the image, from top to bottom, are the original image, the extracted orbital contour image, and the warning boundary image.

[0034] In the method for generating early warning boundaries, the running track is first extracted, and then the track contour is expanded to both sides with the central axis as the reference. The expansion direction is perpendicular to the central axis. The geometric similarity, morphological features and spatial orientation of the expanded boundary are kept the same as those of the original contour. The expansion factor can be configured by parameters. Accurate early warning boundaries have been obtained in scenarios such as curves, straight roads and turnouts, which solves the problem of inaccurate early warning boundaries.

[0035] The warning boundary is based on the extracted real-time train track and extends to the left, right and front of the track to form an electronic fence. It has the function of quickly generating warning boundaries. The warning boundary and the track maintain the same geometric similarity, morphological characteristics and spatial orientation. It is applicable to straight tracks, curves and turnouts and other scenarios, providing a safety protection boundary for train operation and can be used for the detection and warning of a full range of obstacles.

[0036] The obstacle detection method, which is not limited to any particular category, uses the lightweight Metric3Dv2 algorithm to obtain the normal map based on the visible light image. The construction of the obstacle detection method includes the following steps: T1): Preprocessing the input image, the preprocessing including image normalization and boundary expansion; T2): An encoder-decoder architecture is adopted. The encoder uses ViT (Vision Transformer) to extract multi-level features from the input image to obtain semantic information and spatial detail features at different levels. The decoder receives the multi-scale features extracted by the encoder, restores the spatial resolution through feature fusion and upsampling operations, and outputs dense features that match the resolution of the input image. T3): Post-processing and 2D feature format conversion of ViT features: unpacking feature dimensionality information and separating spatial features, reshaping the spatial image block tokens in each feature batch into a standard 4D feature map format [B, H, W, C] for subsequent dense prediction, while extracting classification tokens and registration tokens and flattening them into global context features [B, 1, 1, C × (1 + num_register_tokens)]. In this process, local spatial information and global semantic representation are preserved, providing structured feature input for the depth estimation task; where B represents the number of inputs, H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map; T4): The feature optimizer is used to process the input features to generate a shared feature representation for joint prediction of depth and normal. At this time, the feature map resolution is 1 / 4 of the input. T5): The optimized feature channels are split into three parts: basic features (feature_map), depth prediction confidence (depth_confidence_map), and normal prediction confidence (normal_confidence_map). The basic features include the basic features shared by depth and normal prediction. The basic features, depth prediction confidence, and normal prediction confidence together constitute the feature foundation for multi-task learning. This design allows depth estimation and normal prediction to share low-level features, while performing task-specific optimizations through their respective confidence maps. T6): Processing the shared feature map: Use the depth regressor to generate a probability distribution of 256 / 512 depth intervals. After softmax normalization, use the depth expectation calculation module to convert the discrete probability distribution into continuous depth values, and perform numerical clipping and scale normalization to obtain the measurement depth prediction result. T7): Processing the shared feature map and normal confidence: The normal predictor generates an initial normal vector from the feature map, then concatenates it with the confidence map, and finally performs vector normalization to ensure that the output normal vector is of unit length; T8): The acquired depth map and normal map features are optimized through 8 iterations: After each iteration, the low-resolution flow field (depth change) is upsampled to the original resolution using a learnable upsampling module, scale recovery and numerical clipping are performed on the depth components, and the normal vector is normalized. The depth prediction, confidence and normal prediction results are stored respectively. In the above process, the geometric estimation is gradually optimized from the initial prediction. By combining multi-scale contextual information and the memory mechanism of GRU, it can handle the deep ambiguity of complex scenes and produce high-precision, well-defined metric geometric output.

[0037] T9): Map the obtained normal map colors to 0 and 255 to obtain the normal map.

[0038] Normal maps render surfaces at different angles in different colors. In particular, objects perpendicular to the ground are given a color significantly different from the ground (such as a horizontal plane), thus being "highlighted." For example... Figures 6 to 8 The image shown is a schematic diagram illustrating the effects of converting visible light images to normal maps in different scenarios. Figure 6 This is a schematic diagram illustrating the effect of converting a visible light image to a normal map in a straight-line scene. Figure 6 The two images in the image, from top to bottom, are the original visible light image and the transformed normal image; Figure 7 This is a schematic diagram illustrating the effect of converting a visible light image to a normal image in a complex turnout scenario. Figure 7 The two images in the image, from top to bottom, are the original visible light image and the transformed normal image; Figure 8This is a schematic diagram illustrating the effect of converting a visible light image to a normal image in a simple turnout scenario. Figure 8 The two images in the image, from top to bottom, are the original visible light image and the transformed normal image.

[0039] The Metric3Dv2 algorithm boasts exceptional zero-shot generalization capabilities. It is trained and tested using 16 authoritative benchmarks, including NYUv2 (indoor) and KITTI (autonomous driving), and addresses scale ambiguity caused by camera intrinsic parameter variations by employing a canonical camera space transformation module. Its core idea is to define a virtual "canonical camera" with a fixed focal length, and to unify all training data (regardless of the camera used) into this canonical space through geometric transformations for learning. This allows the network to learn perspective rules under only one camera model, completely eliminating the confusion caused by camera differences and effectively solving the scale uncertainty problem in traditional monocular vision caused by unknown camera intrinsic parameters. Simultaneously, it can recover depth information and surface normals with true physical scale from a single image with zero samples. Furthermore, it unifies the model to solve dual tasks, using the same set of model parameters to simultaneously output scaled depth maps and surface normal maps with high accuracy, simplifying the system structure.

[0040] The lightweight Metric3Dv2 algorithm is used to recover surface normal information with real physical scale from a single visible light image. The Metric3Dv2 algorithm processes small images based on warning areas, effectively overcomes lighting interference, highlights all objects perpendicular to the ground, and effectively separates objects from the ground, providing a basis for obstacle detection of all categories. Furthermore, it integrates visible light image, depth map, and normal map information for multi-dimensional fusion.

[0041] S5) The image processing, segmentation, and adaptive filtering based on the normal map includes the following steps: M1): The generated normal map is separated into RGB channels into R channel, G channel and B channel images. The channel images are grayscale images. The R channel image is selected for the next step of processing. The purpose of R channel extraction is to highlight the color features of objects perpendicular to the ground in the normal map and to provide high-quality input for binarization and contour extraction. A normal map is a three-channel RGB image where the color value of each pixel represents the direction of the normal to the surface at that point. By separating the three RGB channels into independent grayscale images, features in different directions can be analyzed separately. The R channel was chosen because objects perpendicular to the ground (such as obstacles on a track) exhibit the most significant feature responses in the R channel of the normal map, providing the most ideal input for subsequent binarization processing.

[0042] M2: Image binarization. By setting one or more thresholds, the intensity value of each pixel in the grayscale image is compared with the threshold, thereby dividing it into two categories: foreground (target object) or background. It is converted into a binary image containing only two pixel values: pure black (0) and pure white (255), thus achieving the separation of target and background. M3): Contour adaptive filtering divides the image into several continuous regions along the vertical or height direction, such as three regions: far, middle, and near, or more fine-grained partitions; and dynamically assigns different filtering thresholds to each region. In this embodiment, the threshold is an area threshold. The threshold for the upper (far) region of the image is set to a smaller value, and the threshold for the lower (near) region of the image is set to a larger value. For an extracted contour, determine the image region where the contour's center point or pixel distribution is located, and then use a preset threshold for that region to filter it. Only contours whose area is greater than the threshold of their region will be retained.

[0043] The contour adaptive filtering process processes the binarized image, filtering out interference and retaining obstacles. It avoids setting a single threshold too high, filtering out distant interference targets with few pixels, while also avoiding setting a single threshold too low, which would fail to filter out nearby interference targets with many pixels. Moreover, it does not focus on the target category, and all protruding objects can be detected, achieving obstacle detection without limiting the category, that is, obstacle detection of all categories.

[0044] Contour adaptive filtering addresses the "near objects appear larger, far objects appear smaller" visual problem. Due to perspective effects, distant obstacles occupy a small pixel area in an image, while nearby obstacles occupy a large pixel area. Using a single global threshold can lead to missed detections of small distant objects (threshold too large) or false detections of nearby noise (threshold too small). A dynamic thresholding strategy is employed, dividing the image into multiple regions based on distance (far, medium, near), and setting different contour area thresholds for each region. A smaller threshold for distant regions avoids missing small distant objects with small pixel areas, while a larger threshold for near regions avoids false detections of nearby noise with large pixel areas. Simultaneously, multi-feature fusion technology further improves discrimination accuracy. Ensuring accurate and reliable identification of real orbital obstacles at different distances is the core algorithmic guarantee for achieving high-precision detection.

[0045] Contour adaptive screening addresses the scale issue through dynamic thresholding and improves discrimination accuracy through multi-feature fusion. Together, they ensure that the system can accurately and reliably identify real foreign objects on the track at different distances, which is the core algorithm guarantee for achieving high-precision detection.

[0046] The extracted obstacle contours are compared with the warning boundaries, including the following steps: N1): Perform a Boolean operation to find the intersection of the obstacle outline and the warning boundary outline; N2): Determine if the number of intersections is positive. If it is positive, that is, the obstacle is within the boundary, then issue a voice warning.

[0047] like Figure 9 The image shown is the overall image processing flow of the obstacle intrusion detection method for tracks without limitation of categories. Figure 9 In section (a), a visible light image of the track to be detected is acquired using a camera. Then, a boundary warning generation method is used to extract the track contour from the visible light image of the track to be detected. Figure 9 Figure (b) shows the track profile. The track profile is then expanded to obtain the early warning boundary map, as shown below. Figure 9 As shown in Figure (c); exist Figure 9 Based on the warning boundary in (c), the region of the warning boundary is extracted from the visible light image of the track to be detected, i.e., a small extracted image. The visible light image of the extracted warning boundary is then converted into a normal image using an obstacle detection method that does not limit the types of obstacles, such as... Figure 9 As shown in (d); Based on the normal map, RGB channels are separated and the R channel is extracted. Figure 9 (e) in the image represents the extracted R-channel image. Then, based on the extracted R-channel image, binarized contour filtering is performed, such as... Figure 9 As shown in (f); finally, the extracted obstacle contours are compared with the warning boundaries to make a warning judgment, and the results are mapped onto the visible light map of the track to be detected, as shown in (f). Figure 9 As shown in (g) in the diagram.

[0048] The detection method is deployed on edge devices for engineering applications. This embodiment comprehensively considers cost, speed, and accuracy, selecting low-cost edge computing devices and optimizing high-time consumption. It improves speed while maintaining prediction accuracy. Moreover, this embodiment only relies on low-cost monocular cameras and edge computing devices, without the need for expensive radar, thus providing a solid foundation for large-scale deployment and solving the problems of slow inference speed and high cost that hinder promotion.

[0049] A non-class-restricted obstacle intrusion detection method combines multimodal information such as depth information of visible light images, relative position of objects to the camera, and surface normals. It assigns different colors to objects at different angles in the image, highlighting objects perpendicular to the ground (e.g., one color for objects perpendicular to the ground and another color for the ground), thus achieving limited distinction between the ground and objects above it. Simultaneously, by incorporating image morphology operations such as channel extraction, binarization, and contour filtering, it achieves the detection and identification of all obstacle types above the ground without distinguishing specific categories, solving the problem of limited detection categories.

[0050] The aforementioned method for detecting unrestricted track obstacle intrusions has the following beneficial effects: First, this invention comprehensively addresses the problems faced in detecting foreign object intrusions into orbit, expanding the detection categories to all categories while limiting the warning area to within the orbital clearance zone. This helps improve the effectiveness of warnings and the accuracy of detection.

[0051] Secondly, the early warning boundary generation method proposed in this invention can adjust the range of expanded parameters to match the geometric orientation of the original contour, resulting in accurate generated boundaries. Furthermore, it is universally applicable and can be transferred to other application scenarios.

[0052] Third, the class-inclusive obstacle detection method in this invention comprehensively considers visible light image information, image depth information, and image normal information, and combines morphological operations to effectively separate the ground and objects on the ground, highlighting objects above the ground. Combined with filtering rules, it achieves class-inclusive obstacle detection in the track scene domain. Furthermore, this method is universal, not dependent on data training for specific scenarios, and can work stably in different track environments such as straight tracks, curves, and switches, and has the potential to be transferred to other scenarios such as highways.

[0053] Fourth, the method of the present invention uses only low-cost vision hardware to obtain the depth and normal information of the image, saving the need for radar, high-cost depth cameras and other equipment, thus saving a lot of costs and having a basis for widespread application.

[0054] Fifth, use multi-level threshold contour filtering to avoid inconsistencies such as distant targets being small and nearby targets being large and occupying more pixels, which could lead to false deletions or missed deletions due to a single threshold, thus ensuring the accuracy of the detection results and eliminating environmental interference.

[0055] Sixth, the method is simple to deploy, time-saving, and easy to apply in engineering.

[0056] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A method for detecting encroachment of track obstacles regardless of category, characterized in that: Includes the following steps: S1): Construct a method for generating early warning boundaries and a method for detecting obstacles of any category; S2): Acquire an image of the track area to be detected, and generate a warning boundary on the image using a warning boundary generation method. The warning boundary is used to define the safe operating area of ​​the track. S3): Based on the generated warning boundary, a small image is cropped from the image of the track area to be detected, and the cropped small image is the image within the warning boundary area; S4): Obtain the normal map of the small screenshot using an obstacle detection method that does not limit the types of obstacles. The normal map is used to characterize the normal direction information of the surface of the object in the image. S5): Based on the normal map, perform image processing, segmentation, and adaptive filtering to extract the outline of obstacles higher than the ground; compare the extracted obstacle outline with the warning boundary, and if there is an intersection, determine that it is an intrusion of foreign object and issue a warning.

2. The track obstacle intrusion detection method according to claim 1, which is not limited to any type, is characterized in that: The method for generating early warning boundaries includes the following steps: P1): Track data collection and processing: Install equipment on the roof of the vehicle, ride along the vehicle to collect track data, organize and filter the track dataset, and create a track instance segmentation dataset. P2): Training and generation of the orbital contour extraction model: A lightweight algorithm is selected to train the orbital contour extraction model and generate the orbital contour extraction model. The lightweight algorithm is YOLOv8. P3): Model Deployment: Deploy the track contour extraction model to an embedded device application; P4): Warning boundary generation: Based on the track contour extraction model, the image of the track area to be detected is used to extract the contour, and the warning boundary is generated by expanding the extracted contour.

3. The track obstacle intrusion detection method according to claim 2, which is not limited to any type, is characterized in that: In P4), the warning boundary is generated by expanding the extracted contour, including the following steps: P41): Calculate the left line segment point set, the right line segment point set, and the track centerline point set of the profile; P42): Calculate the normal corresponding to the right line segment point set. The normal direction corresponding to each point on the right line segment point set is perpendicular to the track edge and inward at each point. Calculate the perpendicular distance from each point on the centerline point set to the left and right edges respectively. The sum of the two perpendicular distances is the track width at that point. The normal corresponding to the right line segment point set helps determine the measurement direction of the width. P43): Calculate the area range for the width at the end of the track; P44): Set the magnification factor, and based on the magnification factor, multiply by the track width described in step P42) to obtain the magnified track width; P45): Based on the expansion of the central axis to both sides, each point on the central axis moves to both sides by half the distance of the "expanded track width"; P46): Based on the magnification factor and the width at the end of the track, the track area is extended to obtain the expanded boundary, i.e., the warning boundary.

4. The track obstacle intrusion detection method according to claim 1, which is not limited to any type, is characterized in that: The obstacle detection method, which is not limited to any particular category, uses the lightweight Metric3Dv2 algorithm to obtain the normal map based on the visible light image. The construction of the obstacle detection method includes the following steps: T1): Preprocessing the input image, the preprocessing including image normalization and boundary expansion; T2): An encoder-decoder architecture is adopted. The encoder uses ViT (Vision Transformer) to extract multi-level features from the input image to obtain semantic information and spatial detail features at different levels. The decoder receives the multi-scale features extracted by the encoder, restores the spatial resolution through feature fusion and upsampling operations, and outputs dense features that match the resolution of the input image. T3): Post-processing and 2D feature format conversion of ViT features: unpacking feature dimensionality information and separating spatial features, reshaping the spatial image block tokens in each feature batch into a standard 4D feature map format [B, H, W, C] for subsequent dense prediction, while extracting classification tokens and registration tokens and flattening them into global context features [B, 1, 1, C × (1 + num_register_tokens)]. In this process, local spatial information and global semantic representation are preserved, providing structured feature input for the depth estimation task; where B represents the number of inputs, H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map; T4): The feature optimizer is used to process the input features to generate a shared feature representation for joint prediction of depth and normal. At this time, the feature map resolution is 1 / 4 of the input. T5): The optimized feature channels are split into three parts: basic features, depth confidence, and normal confidence; the basic features include the basic features shared by depth and normal predictions; the basic features, depth confidence, and normal confidence together constitute the feature foundation for multi-task learning; T6): Processing the shared feature map: Use the depth regressor to generate a probability distribution of 256 / 512 depth intervals. After softmax normalization, use the depth expectation calculation module to convert the discrete probability distribution into continuous depth values, and perform numerical clipping and scale normalization to obtain the measurement depth prediction result. T7): Processing the shared feature map and normal confidence: The normal predictor generates an initial normal vector from the feature map, then concatenates it with the confidence map, and finally performs vector normalization to ensure that the output normal vector is of unit length; T8): The acquired depth map and normal map features are optimized in 8 iterations: After each iteration, the low-resolution flow field is upsampled to the original resolution using a learnable upsampling module, the depth component is scaled back and numerically clipped, the normal vector is normalized, and the depth prediction, confidence and normal prediction results are stored respectively. T9): Map the obtained normal map colors to 0 and 255 to obtain the normal map.

5. The track obstacle intrusion detection method according to claim 1, which is not limited to any type, is characterized in that: S5) The image processing and segmentation based on the normal map includes the following steps: M1): Separate the RGB channels of the generated normal map into R channel map, G channel map and B channel map. The channel map is a grayscale image. Select the R channel image for the next step of processing. M2): Image binarization. By setting one or more thresholds, the intensity value of each pixel in the grayscale image is compared with the threshold, thereby dividing it into two categories: foreground or background. It is then converted into a binary image containing only pure black and pure white pixel values, thus achieving the separation of the target from the background. M3): Contour adaptive filtering divides the image into several continuous regions along the vertical or height direction, dynamically assigns different filtering thresholds to each region, and for an extracted contour, determines the image region where the contour center point or pixel distribution is located, and then uses the preset threshold of that region for filtering. Only contours whose area is greater than the threshold of their region will be retained.

6. The track obstacle intrusion detection method according to claim 1, which is not limited to any type, is characterized in that: The extracted obstacle contours are compared with the warning boundaries, including the following steps: N1): Perform a Boolean operation to find the intersection of the obstacle outline and the warning boundary outline; N2): Determine if the number of intersections is positive. If it is positive, that is, the obstacle is within the boundary, then issue a voice warning.

7. The track obstacle intrusion detection method according to claim 1, which is not limited to any type, is characterized in that: The detection method is deployed on edge devices.

Images