A track area foreign matter detection method and system based on distillation perception

Abstract

This invention belongs to the field of intelligent perception and target detection technology for railway environments, specifically disclosing a method and system for detecting foreign objects in track areas based on distillation perception. The method utilizes a semantic segmentation model as the teacher network to extract semantic structural information of the track area, and guides a semantically enhanced target detection model (actually a student model) to focus on key areas through distillation learning, thereby enhancing its area perception capability and achieving accurate identification of foreign objects. To this end, a semantic perception module based on spatial attention guidance is designed in the semantically enhanced target detection model to model the track spatial response area under the track semantic guidance provided by the teacher model. Simultaneously, this invention also constructs a semantic region enhancement fusion module to integrate features with semantic perception capabilities with basic target detection features, effectively mitigating interference between the foreign object target and the background environment, thus improving the accuracy and robustness of foreign object detection in track areas under complex railway environments.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing and target detection technology for railway environment, specifically involving a method and system for detecting foreign objects in track areas based on distillation sensing, which is particularly suitable for identifying foreign objects in track areas under complex railway scenarios. Background Technology

[0002] In railway transportation systems, foreign object intrusion refers to the minimum safe space required for train operation that non-track-running equipment or objects may enter. This can include tools or construction debris that have fallen onto the tracks, mudslides or fallen trees caused by natural disasters obstructing the tracks, or even unauthorized personnel or animals. If such intrusions are not detected and addressed promptly, they can easily lead to emergency braking, operational disruptions, and even serious safety accidents such as equipment damage and personal injury, posing significant safety hazards and operational risks to the transportation system. Therefore, real-time detection and intelligent sensing of foreign object intrusions in railway track areas has become a crucial technical requirement for ensuring railway operational safety. Currently, manual inspection methods suffer from low efficiency, high labor costs, and slow response times, while radar-based sensor solutions are limited by equipment costs and installation conditions. With the rapid development of computer vision and artificial intelligence technologies, in many application scenarios, cameras combined with deep learning networks are often used to replace radar and other sensors. Railway environmental sensing systems are no exception. Therefore, extracting motion-related information from raw camera images is crucial for trains to achieve efficient foreign object detection.

[0003] Railway scenarios are characterized by their unique structure and clearly defined safety zone boundaries. Foreign objects are mainly concentrated in the track area, while a large portion of the image area outside the track is of no practical value for detection. This not only results in computational redundancy but also introduces unnecessary interference, making traditional general-purpose detection models inefficient in such tasks. With the rise of deep learning network architectures, target detection technology in railway environments is constantly improving. Simultaneously, to efficiently detect foreign object intrusions, research has emerged focusing solely on target detection within specific track areas. Some methods propose first using semantic segmentation networks to extract the track area, then limiting the detection range to only within this area. This two-stage "segmentation + detection" approach improves the focus of detection to some extent, but it also introduces significant problems. Because segmentation and detection processes belong to different modules, information cannot be jointly modeled, leading to fragmented system training. Furthermore, segmentation errors are easily propagated to the detection stage, affecting overall detection performance.

[0004] In summary, existing methods for detecting foreign object intrusions in railway tracks generally employ image-based target detection techniques. While these techniques possess some recognition capabilities in static scenes, they still face numerous limitations in structural design and task coordination. On one hand, due to the mechanism of existing detection models performing target searches across the entire map, whole-map detection is susceptible to interference from non-target regions. On the other hand, to improve detection focus, some methods employ a multi-task structure that first segments the track area and then performs target detection. However, because the segmentation and detection networks are independent and training is decoupled, problems such as error accumulation and information transmission interruptions arise, severely restricting the overall system performance and practicality. Summary of the Invention

[0005] The purpose of this invention is to propose a foreign object detection method for track areas based on distillation perception. This method uses a semantic segmentation model as the teacher network to extract semantic structural information of the track area. Through distillation learning, the student model is guided to focus on key areas to enhance the area perception capability, which is beneficial to improving the accuracy and robustness of detecting foreign object intrusion targets in railway scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for detecting foreign objects in orbital regions based on distillation sensing includes the following steps: Step 1. Obtain the original image dataset of the railway scene, label the semantic ground truth of the track region and the ground truth of the target box for each image, obtain the range of the track and the location and category information of foreign objects, and use them as label data to construct the training dataset; Step 2. Build a semantically enhanced target detection model based on cross-task knowledge distillation, including a foreign object feature encoder, a semantic perception module, an interactive distillation module, a semantic region enhancement fusion module, and a fusion feature decoder; Foreign object feature encoder is used to extract foreign object features at different scales from the input image, which serve as the basic features for target detection; There are multiple semantic awareness modules, which are used to further extract the basic features of the corresponding scale, and output features with semantic awareness capabilities through knowledge distillation by learning the high-level semantic information of the teacher model. There are multiple interactive distillation modules, each responsible for transmitting the semantic features of the track region at different scales extracted by the teacher model to each semantic perception module, so that each semantic perception module can focus on foreign object detection in the track region. There are multiple semantic region enhancement and fusion modules, which are used to fuse semantically perceptive features and basic features at corresponding scales through cross attention, and output multi-scale enhanced target detection feature maps. There are multiple fusion feature decoders, each used to decode the enhanced target detection feature map of the corresponding scale into target detection results, including the location and category information of the foreign object and the confidence level; Step 3. Train a semantically enhanced object detection model based on cross-task knowledge distillation using the training dataset; input the real-time acquired railway forward-looking images into the trained object detection model to obtain the foreign object detection results.

[0007] Preferably, step 1 specifically comprises: The train uses onboard cameras to collect raw image datasets from the train's forward-looking perspective. The images cover railway environments under different weather and lighting conditions to ensure the diversity of the raw image dataset. The semantic ground truth of the track region is labeled for each image, the ground truth of the foreign object target box is labeled for the image containing foreign objects, and each target is assigned a corresponding label. A sample dataset of railway foreign object intrusion scenarios containing five types of foreign objects is established as a training dataset. Foreign objects include maintenance personnel, abandoned equipment, scattered goods, animals, and fallen rocks; The track region of each image is labeled using semantic segmentation labels. The labeled track region includes all pixels of the track, track bed and surrounding area. The labeling method is used, with the track region labeled as 1 and other regions labeled as 0. Each target bounding box needs to provide the following information: Target category information: Category label for each foreign object; Target location information: the bounding box coordinates of the foreign object, including the coordinates of the top left and bottom right corners of the object in the image; the size and proportion of the bounding box of the foreign object. To ensure the high quality of the labeled data, the target box needs to define the boundary of the foreign object.

[0008] Preferably, in step 2, the teacher model is a semantic segmentation model for track region supervision; wherein the constructed semantically enhanced target detection model serves as a student model, used to learn key track regions from the teacher model for foreign object detection.

[0009] Preferably, the semantic segmentation model adopts the Segformer model, which uses four Transformer coding blocks as the backbone network, and each coding block outputs feature maps with different resolutions and semantic depths. The output features of the second, third, and fourth Transformer coding blocks of the Segformer model are selected as semantic guiding features for distillation, which are used to characterize the boundaries, contours, and overall structure of the orbital region. The foreign object feature encoder in the student model selects the backbone network and neck network of Yolov8n. The neck network outputs three feature maps P3, P4 and P5 at different scales, which are used to detect small, medium and large scale targets, respectively. P3, P4, and P5 each obtain three semantic perception features through the semantic perception module, which are used as student features to be distilled. These features are then aligned and learned with the features output by the second, third, and fourth Transformer encoding blocks in the teacher model. During the training phase, the SegFormer model performs knowledge distillation on the semantic perception module of the student model, indirectly guiding the student model to focus on foreign objects within the orbital region, thereby improving the regional sensitivity and discrimination ability of the detection model.

[0010] Preferably, the semantic perception module aligns the feature maps of the student model with the semantic information from the teacher model, and uses deep convolution to enhance the semantic context information of the multi-scale feature maps; at the same time, the spatial attention mechanism is used to weight the spatial dimension of the feature maps, and by assigning higher weights to the track regions, it helps the object detection model automatically focus on the track regions in the image.

[0011] Preferably, the semantic awareness module includes two 3×3 depthwise convolutional layers, three 1×1 convolutional layers, a spatial attention mechanism layer, a channel splitting layer, and a feature concatenation layer; The input to the semantic awareness module is the feature map F. p F is adjusted by 1×1 convolution. p The number of channels is aligned with the orbital semantic features, and then a channel splitting layer is used to split the channel into two sub-feature maps, F1 and F2, along the channel dimension. The F1 branch first extracts local spatial features through a 3×3 depthwise convolution, then performs feature compression and dimensionality reduction through a 1×1 convolution, then expands features through a second 1×1 convolution, and finally enhances spatial structural features through a second 3×3 depthwise convolution, resulting in an F1 with enhanced spatial expressive power. F1, which enhances spatial representation, is combined with F2, which preserves original details, to generate F. concat This enables the complementary fusion of deep semantic features and shallow detailed features; Subsequently, further investigation into F concat The spatial attention calculation process is as follows: First F concatFeatures are compressed through 1×1 convolutions, followed by feature distribution stabilization using Batch Normalization (BN) layers, and finally spatial attention map A is generated using Sigmoid activation. sa ; Then spatial attention map A sa With F concat Element-wise multiplication is used to achieve attention weighting, resulting in attention-weighted features F. mul This enhances the response in the orbital-dependent region and suppresses background interference; finally, F mul With F concat The semantic-aware feature F is obtained by adding elements one by one. sp This approach preserves the original feature information to prevent gradient vanishing while ensuring that key semantic features are not lost.

[0012] Among them, the feature map is F p That is, the output feature maps P3, P4, and P5 of the foreign object feature encoder; Spatial attention map A sa The subscript 'sa' represents spatial attention. Attention-weighted features are F mul The subscript mul represents multiplication; The semantic perception feature is F sp The subscript sp represents semantic perception.

[0013] Preferably, the processing flow of the interactive distillation module is as follows: Define semantic perception features as , Aligned semantic-aware features and orbital semantic features , ; Semantic awareness features after alignment The orbital semantic feature is F seg ; Where C1 and C2 represent the number of channels, H represents the height, and W represents the width; Multi-task collaboration features are subscript Represents cooperation; First, respectively and Perform a flattening operation in the spatial dimension followed by self-multiplication to obtain their respective spatial correlation matrices. , ,in ; The process of obtaining the spatial correlation matrix described above is expressed as follows: , ; in express The spatial correlation matrix, express Spatial correlation matrix; This represents a flattening operation in the spatial dimension, where i=3,4,5 and j=2,3,4; Where i is the hierarchical index of the multi-scale features output by the foreign object feature encoder, and j is the index of the feature map corresponding to the multiple encoders of the teacher model Segformer; Represents semantic-aware features Features aligned with the teacher model at the same scale via channels This represents the features extracted by the j-th layer encoder in the teacher network. ; Subsequently and Multiplying and then performing a softmax operation yields the spatial correlation weights between the feature maps extracted by the student model and the teacher model, and then multiplying them by... The products are multiplied, and the final result is used to generate multi-task collaborative features through residual connections. , k =1,2,3; where Hierarchical numbering for the newly generated multi-task collaborative features; Will Perform a flattening operation in the spatial dimension followed by self-multiplication to obtain a multi-task collaborative space matrix. ; then execute and Cross-domain knowledge distillation between the two domains is constrained by distillation loss due to the distributional differences between them.

[0014] Preferably, the semantic region enhancement fusion module includes a Concat connection, a three-level cascaded C2f module, and a cross-attention module; The input to the semantic region enhancement fusion module is a semantically perceptive feature F at the corresponding scale. sp With the basic features F of object detection p The processing flow of this semantic region enhancement and fusion module is as follows: Define the base feature for feature fusion after feature fusion and before it enters the subsequent attention module as F. trans The subscript "trans" indicates a transition. The semantically enhanced fusion feature resulting from the deep fusion of spatial and semantic information is F. com The subscript .com represents combination; First, the semantically perceptive features F at the corresponding scale are... sp With the basic features F of object detection p Perform concat connections along the channel dimension to form the fused basic feature F. trans ;F afterwards trans The C2f module is cascaded through three levels; and the features output after the three-level cascade of the C2f module are projected as the Q vector of the cross-attention module. F sp The K and V vectors are projected as the cross-attention module. Next, attention weights are generated by matrix multiplication of the transposes of Q and K, stabilized through scaling factors, and then Softmax is applied along the K dimension to generate a probability distribution. Finally, the attention weights are used to weight and sum V, injecting semantic information into the spatial features to obtain the semantically enhanced fusion feature F. com .

[0015] Preferably, in step 3, the training process of the semantically enhanced target detection model is as follows: The semantically enhanced object detection model is trained using the training dataset until the accuracy of the semantically enhanced object detection model in detecting foreign objects reaches the preset accuracy requirement or the maximum number of training iterations, thus obtaining a well-trained object detection model. During training, the parameters of the semantically enhanced target detection model are modified by combining the loss of the location and category information of the foreign object in the foreign object prediction results with the loss in the knowledge distillation process. The total loss is defined as consisting of two parts: knowledge distillation loss and foreign object detection loss. Knowledge distillation loss enhances the attention of foreign object detection to regional features by performing layer-by-layer matching of spatial correlation matrices at three different scales. This allows the student model to inherit the structural modeling and spatial attention capabilities of the teacher model in the track region.

[0016] in L kd This represents the knowledge distillation loss. , , The weighting coefficients for the feature map distillation loss satisfy the following conditions: , Represents the L2 norm; since the feature maps involved in knowledge distillation have three scales... N 1 、N 2 、N 3 These represent the size of the feature map at this scale when calculating the loss; The foreign object detection loss consists of classification loss and bounding box regression loss; the classification loss is the binary cross-entropy loss, and the bounding box regression loss includes CIoU loss and distribution focus loss; the foreign object detection loss... L D We obtain it from the following formula: ; in, This is a binary cross-entropy loss, used to correctly determine the category of foreign objects. CIoU loss is used to optimize the position and shape of the target bounding box. To distribute the focus loss, the regression accuracy of the target box is further optimized; They are respectively , , Weighting coefficients for the three types of losses; - Distribution Focal Loss (DFL) is a loss function used for bounding box regression in object detection.

[0017] Therefore, the overall loss function is expressed as: ; in, L Indicates the total loss. Represents knowledge distillation loss The weight.

[0018] - Knowledge distillation.

[0019] Furthermore, based on the aforementioned foreign object detection method in the orbital region based on distillation sensing, this invention also proposes a foreign object detection system in the orbital region based on distillation sensing, which adopts the following technical solution: A foreign object detection system for track areas based on distillation sensing includes a camera and a computer, both mounted on a train; the camera is used to acquire image data of the railway environment in real time and upload it to the computer. The computer device includes a memory and a processor; executable code is stored in the memory, and when the processor executes the executable code, it is used to implement the steps of the above-described method for detecting foreign objects in the orbital region based on distillation sensing.

[0020] The present invention has the following advantages: As described above, this invention relates to a method and system for detecting foreign objects in track areas based on distillation perception. The method of this invention constructs a semantically enhanced target detection model based on cross-task knowledge distillation. This semantically enhanced target detection model includes a foreign object feature encoder, a semantic perception module, an interactive distillation module, a semantic region enhancement fusion module, and a fusion feature decoder. Furthermore, this invention introduces a semantic segmentation model as a teacher network to extract semantic structural information of the track area. Through distillation learning, the semantically enhanced target detection model, acting as a student model, focuses on key areas, enhancing its region perception capability and thus achieving accurate identification of foreign objects. To this end, this invention designs a semantic perception module based on spatial attention guidance in the student model to model the track spatial response area under the track semantic guidance provided by the teacher model. Simultaneously, a semantic region enhancement fusion module is constructed to fuse features with semantic perception capability with the basic features of target detection, effectively mitigating interference between the foreign object target and the background environment, thereby improving the detection accuracy and robustness in complex railway environments. This invention is beneficial for improving the detection accuracy and region perception capability of foreign object intrusion targets in railway scenarios. Attached Figure Description

[0021] Figure 1 This is a network structure diagram of the semantically enhanced target detection model based on cross-task knowledge distillation in an embodiment of the present invention; Figure 2 This is a network structure diagram of the semantic perception module in an embodiment of the present invention; Figure 3 This is a network structure diagram of the spatial attention module in an embodiment of the present invention; Figure 4 This is a network structure diagram of the interactive distillation module in an embodiment of the present invention; Figure 5 This is a network structure diagram of the semantic region enhancement fusion module in an embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0023] Example 1

[0024] In the overall train front view, foreign objects are often small in size, have an irregular appearance, and are uncertain in location. However, the foreign objects that actually affect train operation are concentrated on the track. Compared with the existing two-stage method of "segmentation + detection" for track foreign object detection, this invention does not use separate "segmentation + detection" task learning networks or parallel "segmentation + detection" task learning networks. Instead, it deeply coordinates the segmentation and detection tasks through cross-task knowledge distillation.

[0025] Since segmentation and detection tasks differ in target representation methods, output formats, and feature focus, directly sharing features can easily lead to feature conflicts and performance loss. Therefore, this invention introduces a semantic awareness module specifically designed for semantic alignment in knowledge distillation.

[0026] This semantic awareness module utilizes deep convolution and attention mechanisms to model and refine the orbital features of the student model, enabling it to better learn from the orbital features extracted by the teacher model. Leveraging its structural advantages, the semantic awareness module significantly enhances the student model's orbital space modeling capabilities, effectively breaking down the inherent barriers between different tasks and achieving deep fusion and complementarity between segmentation and detection.

[0027] By employing a cross-task knowledge distillation mechanism, the high-precision semantic features learned by the teacher model in track region segmentation are implicitly transferred to the semantic perception module of the student model. This enables the detection model to autonomously build semantic representations of track regions and focus on key areas. Running the detection model during the inference phase eliminates the need for explicit output of segmentation results, thus achieving implicit region focusing. This design effectively avoids the propagation of errors from explicit segmentation results in subsequent foreign object detection, ultimately improving the accuracy and robustness of foreign object detection in complex railway environments.

[0028] This invention utilizes a semantic perception module to acquire inherent focusing capabilities on the orbital region during cross-task knowledge distillation. Through a semantic region enhancement fusion module, the orbital spatial information is fused with the detection information, enabling the target detection model to stably identify foreign objects on the orbit even in complex backgrounds. Even with cluttered backgrounds and numerous irrelevant targets in non-orbital regions, it effectively highlights key features of the orbital region and suppresses interference from irrelevant backgrounds, thus maintaining the accuracy and robustness of the detection results.

[0029] The present invention enables efficient end-to-end target detection under orbital region enhancement, reducing unnecessary calculations and error propagation, thereby significantly improving the overall detection efficiency of the present invention.

[0030] Specifically, the foreign object detection method in the orbital region based on distillation sensing in this embodiment includes the following steps: Step 1. Obtain the original image dataset of the railway scene, label the semantic ground truth of the track region and the ground truth of the target box for each image, obtain the range of the track and the location and category information of foreign objects, and use them as label data to construct the training dataset.

[0031] Raw image datasets were collected using onboard cameras from the train's forward-looking perspective. The images cover railway environments under varying weather and lighting conditions to ensure dataset diversity. The LabelMe tool was used to precisely delineate the track area outline in each image, generating semantic ground truth values ​​for the track area. Additionally, each object was selected and a category ID was assigned to each bounding box, serving as the ground truth value between the bounding box and the object. This yielded the track extent and the location and category of the object, which were then used as label data.

[0032] Specifically, the training dataset constructed in this embodiment contains multiple sample images of foreign object intrusion taken by railway forward-looking cameras, and these images are labeled with information on the location, shape, category of the foreign object, and semantic information of the track area.

[0033] The onboard camera is installed inside the train's front windshield, positioned at the front center or slightly above the train, at a height of 2.5 to 3 meters above the ground, to ensure coverage of the entire track area in front of the train. The camera's field of view (FOV) needs to cover at least the areas on both sides of the track, with a horizontal field of view of no less than 60° and a vertical field of view of no less than 30°.

[0034] To ensure safety, a railway foreign object intrusion test site was established. Cameras captured and saved images at a frequency of 30Hz, while various working conditions were simulated in the track area in front of the field of view. Specific working conditions included: a. People of different heights wearing maintenance uniforms making lateral intrusions, as well as longitudinal intrusions from near to far and from far to near; b. Animals of different types and sizes making lateral intrusions; c. Intrusions by falling rocks, goods, or abandoned equipment at different distances.

[0035] By continuously simulating the above working conditions and collecting images, a sufficient number of foreign object intrusion image samples can be collected.

[0036] The above simulated working conditions revealed five types of foreign object intrusions that frequently occur in freight railway environments: maintenance personnel, abandoned equipment, scattered goods, animals, and falling rocks. These five types of foreign objects are defined as foreign object tags.

[0037] Data acquisition should include different lighting conditions. During clear daylight, railway track images should be collected at three different times: morning, noon, and dusk. The complexity of the data should be increased by varying light intensity, color temperature, and shadow length to improve the model's detection performance under different lighting conditions. After the foreign object images are collected, the semantic ground truth of the track region is manually labeled for each image, and the ground truth of the foreign object target bounding box is labeled for images containing foreign objects. Each target is then assigned a corresponding label, thus establishing a sample dataset of railway foreign object intrusion scenarios containing five types of foreign objects.

[0038] The track region in each image is labeled using semantic segmentation tags. The labeled track region includes all pixels of the track, track bed, and surrounding area. Binarization is used, with the track region labeled as 1 and other areas labeled as 0, ensuring the teacher model can accurately segment the track region. Each bounding box needs to provide the following information: a. Target category information: Category label for each foreign object; b. Target location information: the bounding box coordinates of the foreign object, including the coordinates of the top left and bottom right corners of the object in the image; the size and proportion of the bounding box of the foreign object. To ensure the high quality of the labeled data, the target box needs to define the boundary of the foreign object.

[0039] Step 2. Construct a semantically enhanced object detection model based on cross-task knowledge distillation, including a foreign object feature encoder, a semantic perception module, an interactive distillation module, a semantic region enhancement and fusion module, and a fusion feature decoder, such as... Figure 1 As shown.

[0040] Foreign object feature encoders are used to extract foreign object features at different scales from the input image, serving as the basic features for target detection.

[0041] There are multiple semantic awareness modules, which are used to further extract basic features at the corresponding scales and output features with semantic awareness capabilities through knowledge distillation by learning the high-level semantic information of the teacher model.

[0042] There are multiple interactive distillation modules, each responsible for transmitting the semantic features of the track region at different scales extracted by the teacher model to each semantic perception module, enabling each semantic perception module to focus on foreign object detection in the track region.

[0043] There are multiple semantic region enhancement fusion modules, which are used to fuse semantically perceptive features at corresponding scales with basic features through cross attention, and output multi-scale enhanced object detection feature maps.

[0044] There are multiple fusion feature decoders, each used to decode the enhanced target detection feature map of the corresponding scale into target detection results, including the location and category information of the foreign object and the confidence level.

[0045] In this embodiment, the teacher model is a semantic segmentation model used for track region supervision. The aforementioned semantically enhanced object detection model serves as the student model, learning from the teacher model to perform foreign object detection in key track regions.

[0046] This invention uses a semantic segmentation model as the teacher network and transmits track semantic information to the semantic perception module of the student model through knowledge distillation. Then, the output of the semantic perception module is fused with the target detection backbone features (basic features), thereby realizing the student model's implicit attention to key areas and improving the model's target detection performance in the track environment.

[0047] The teacher model is, for example, the SegFormer model. SegFormer is an efficient semantic segmentation model with the advantages of being lightweight, having no positional encoding, and strong semantic modeling capabilities. This model uses a multi-layer (e.g., four-layer) Transformer encoder as the backbone network, which can extract semantic information at different scales in the image layer by layer, making it suitable for scenarios that require precise region perception.

[0048] In this embodiment, the SegFormer model is used as the teacher model to perform semantic segmentation of the track region on the input image. The semantic features output by the intermediate layer (second, third, and fourth Transformer coding blocks) express the spatial layout and structural features of the track. During the training phase, knowledge distillation is performed on the semantic perception module of the student model to indirectly guide the student model to pay attention to foreign objects in the track region, thereby improving the regional sensitivity and discrimination ability of the detection model.

[0049] The pre-training process of the SegFormer model is as follows: The training input for SegFormer is the original image dataset of railway scenes. Each image is labeled with ground truth semantic values ​​for the track region, using a binary format: track region pixels are marked as 1, and non-track region pixels are marked as 0. The teacher model is pre-trained on the public railway semantic segmentation dataset Railsem19, retaining only background and track categories to obtain preliminary track semantic feature representation capabilities. It is then fine-tuned using the railway scene dataset with track semantic annotations collected in this invention to further adapt to the track region feature distribution under specific railway environments. The trained teacher model can accurately identify the overall structure of the track region, providing reliable semantic guidance features for subsequent knowledge distillation.

[0050] Foreign object feature encoders can employ the backbone and neck network of YOLOv8n to extract multi-scale foreign object feature maps from input images, covering targets of different scales. The output multi-scale feature map data of YOLOv8n has two flows: On the one hand, the semantic perception module further extracts and performs knowledge distillation with the semantic information of the teacher model to learn the orbital semantic information and output features with semantic perception capabilities. On the other hand, the multi-scale feature map output by Yolov8n is fused with the semantically perceptive features obtained through knowledge distillation to enhance the accuracy of target detection within the orbital region.

[0051] Yolov8n is a lightweight object detection model with the fewest parameters and fastest speed in the Yolov8 series, featuring an end-to-end architecture. In this embodiment, Yolov8n is used as a student model to learn to identify and locate foreign objects within the orbital region. Its structure consists of a backbone network, a neck network, and a detection head. The backbone network is responsible for extracting low- and mid-level features from the image, while the neck network fuses multi-scale information through a feature pyramid structure. Finally, the detection head outputs the predicted object category and location. By combining a semantic perception module and a semantic region enhancement fusion module, Yolov8n achieves fine-grained perception of key regions, improving the model's robustness in detecting small targets and occluded foreign objects in orbital scenes without altering the original detection structure.

[0052] The semantic perception module aims to provide stronger semantic understanding and target localization capabilities in railway freight scenarios, especially in complex track areas. This ensures the model can accurately detect foreign objects within the track area, improving railway safety monitoring and automated detection capabilities. (Yolov8n multi-scale feature map F) p While P3, P4, and P5 can capture targets at different scales, they lack an understanding of the semantic context of the track regions. To address this issue of their outputs not fully utilizing this contextual information, the semantic awareness module enhances the semantic context of the multi-scale feature maps by aligning the feature maps of the student model with the semantic information from the teacher model and using deep convolution (DWConv). Simultaneously, a spatial attention mechanism (SA) is used to weight the spatial dimension of the feature maps, assigning higher weights to the track regions and helping the model automatically focus on the track regions in the image.

[0053] like Figure 2 As shown, the semantic awareness module includes two 3×3 depth convolutional layers, three 1×1 convolutional layers, a spatial attention mechanism layer, a channel splitting layer, and a feature concatenation layer.

[0054] The F1 branch is processed by DWConv with a kernel size of 3×3 and a stride of 1. Then, batch normalization (BN) and ReLU activation function are used to normalize and nonlinearly transform the features. DWConv can improve the model's sensitivity to the shape and structure of the orbital region, extract spatial information from the features, and significantly reduce computational cost.

[0055] The specific processing flow of the semantic awareness module is as follows: The input is the feature map F p F is adjusted by 1×1 convolution. pThe number of channels is aligned with the orbital semantic features, and then a channel splitting layer is used to split the channel into two sub-feature maps, F1 and F2, along the channel dimension. This aims to preserve both shallow details and deep semantic representations, achieving complementary fusion of different feature levels, thereby improving the model's ability to distinguish in complex contexts.

[0056] The F1 branch first extracts local spatial features through a 3×3 depthwise convolution, then performs feature compression and dimensionality reduction through a 1×1 convolution, followed by feature expansion through a second 1×1 convolution, and finally enhances spatial structural features through a second 3×3 depthwise convolution. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function operation to ensure training stability and non-linear expressive power, resulting in an F1 with enhanced spatial expressive power.

[0057] F1, which enhances spatial representation, is combined with F2, which preserves original details, to generate F. concat This achieves complementary fusion of deep semantic features and shallow detail features; subsequently, further research was conducted on F... concat Perform spatial attention calculations, such as Figure 3 As shown.

[0058] First F concat Features are compressed through 1×1 convolutions, followed by feature distribution stabilization using Batch Normalization (BN) layers, and finally spatial attention map A is generated using Sigmoid activation. sa Then the spatial attention map A sa With F concat Element-wise multiplication is used to achieve attention weighting, resulting in attention-weighted features F. mul This enhances the response in the orbital-related region and suppresses background interference.

[0059] Finally F mul With F concat The semantic-aware feature F is obtained by adding elements one by one. sp This approach preserves original feature information to prevent gradient vanishing while ensuring that key semantic features are not lost. The computational flow of the spatial attention mechanism is as follows: ; in, This represents a 1×1 convolution operation. This indicates a batch normalization operation. This represents the Sigmoid activation function, which restricts the weights to between 0 and 1.

[0060] The semantic awareness module is an intermediate submodule built in the student model that can perceive the semantics of the orbital regions, so as to more effectively align with the orbital semantic features output by the teacher model during the knowledge distillation process.

[0061] Since its input comes from the multi-scale foreign object detection features of the student model, while the teacher model provides semantic structural features focused on the track region, the semantic perception module uses channel segmentation to preserve the original track details, improves the model's ability to express the long track structure through DWConv and multi-layer convolution design, and focuses on the track region through spatial attention mechanism, actively adjusting and enhancing its own semantic modeling ability, so that its output features are closer to the expression of the track region in the semantic space. Thus, it serves as a bridge module for distillation learning, ensuring that the gradient is more stable and the training is easier to converge.

[0062] The semantic perception module proposed in this invention, based on spatial attention guidance, independently undertakes the tasks of receiving and modeling semantic information. This semantic perception module uses spatial attention mechanisms and lightweight convolutional structures to model the expressive power of elongated track structures and focus on track regions. During the training phase, it enables the student model to more effectively align with the track semantic features output by the intermediate layer of the teacher model during knowledge distillation, breaking down the inherent barriers between different tasks.

[0063] In this embodiment, to enhance the object detection model's ability to perceive foreign objects in the track area, a cross-task knowledge distillation mechanism is proposed. This mechanism guides the detection model by introducing track semantic knowledge, thereby improving the model's recognition accuracy in the track area. This module consists of a teacher model (performing the track semantic segmentation task), a student model (performing the object detection task), and an interactive distillation module. The teacher model uses SegFormer as its backbone structure, which is lightweight, high-precision, and has good context modeling capabilities, making it suitable for track area semantic segmentation tasks in railway scenarios. The input image is a three-channel RGB image from the train's forward-looking perspective, and the output is a semantic classification map for each pixel, identifying the semantic mask of the track area in the image.

[0064] The SegFormer model contains multiple Transformer coding blocks, where each block outputs feature maps with different resolutions and semantic depths. This embodiment selects the outputs of Block 2, Block 3, and Block 4 as the semantic guidance features for distillation, which can characterize the boundaries, contours, and overall structure of the orbital region, such as... Figure 1 As shown.

[0065] Figure 1 middle, - The total number of categories of foreign objects in the dataset represents the number of output channels of the classification head, with each channel corresponding to the predicted probability of a category.

[0066] like Figure 4As shown, the student model uses YOLOv8n as the backbone network to perform object detection tasks, locating and classifying five types of railway foreign objects in images. The neck of YOLOv8n outputs three feature maps P3, P4, and P5 at different scales, used to detect small, medium, and large-scale targets, respectively. After passing through the semantic perception module, three semantic perception features are obtained, which are used as student features to be distilled and aligned with the output features of Block2, Block3, and Block4 in the teacher model.

[0067] In the interactive distillation module, although the student model has already aligned its detected features with the teacher's track semantic features through the semantic awareness module, differences still exist between the two in the task representation space. To further reduce this difference and make gradient updates more targeted in enhancing the response of the track region, this method first fuses and then aligns the features. That is, after fusing the student features with the teacher's spatial structure attention weights, residual preservation is performed to generate a collaborative feature F. co This approach avoids directly mapping the students' original noisy spatial relationships to the teacher's space, which could lead to convergence instability, while preserving the discriminative information required for detection. Based on this, the spatial correlation matrix of the collaborative features is used as the distillation object, and the L2 norm is used to constrain its similarity to the spatial correlation matrix of the teacher's orbital semantic features.

[0068] Because the teacher and student models have different network structures, their output feature maps are the same in size but different in the number of channels. Therefore, a 1×1 convolutional layer is added to the interactive distillation module to distill the feature maps F output by the student model. sp The output channel is actively adjusted to match the teacher model, resulting in a model with the same size. .

[0069] like Figure 5 As shown, the processing flow of the interactive distillation module in this embodiment is as follows: Define semantic perception features as , Aligned semantic-aware features and orbital semantic features , ; Where C1 and C2 represent the number of channels, H represents the height, and W represents the width.

[0070] First, respectively and Perform a flattening operation in the spatial dimension followed by self-multiplication to obtain their respective spatial correlation matrices. , ,in .

[0071] Spatial correlation matrix enables structure-aware knowledge distillation, revealing the structural relationships and semantic consistency between different spatial locations in an image. This allows student models to learn, like teachers, to consider pixels in the orbital region as highly similar and those between the orbit and the background as lowly similar, thereby learning to extract the orbital spatial location. The process of obtaining the spatial correlation matrix described above is represented as follows: , .

[0072] in express The spatial correlation matrix, express Spatial correlation matrix; This represents a flattening operation in the spatial dimension, where i=3,4,5 and j=2,3,4; Represents semantic-aware features Features aligned with the teacher model at the same scale via channels This represents the features extracted by the j-th layer encoder in the teacher network. .

[0073] Subsequently and Multiplying and then performing a softmax operation yields the spatial correlation weights between the feature maps extracted by the student model and the teacher model, and then multiplying them by... The products are multiplied, and the final result is used to generate multi-task collaborative features through residual connections. , k =1,2,3.

[0074] The above operations obtain the weight distribution of students and teachers in the spatial structure, apply this weight to student features to enhance the response of the track region, and at the same time use residual connections to preserve the original feature details, so as to achieve complementary fusion of key semantics and global information.

[0075] Finally, to prepare for subsequent knowledge distillation, multi-task collaborative features will be incorporated. Perform a flattening operation in the spatial dimension followed by self-multiplication to obtain a multi-task collaborative space matrix. The formula is expressed as follows: ; .

[0076] Subsequently, execution and Cross-domain knowledge distillation between the two domains, where the distributional differences are constrained by distillation loss. This makes... Sensitivity to semantic information and Similar to the sensitivity, students are continuously constrained to approximate the teacher in spatial relationship modeling, thereby forming an inherent focusing ability of the orbital region without relying on explicit segmentation results.

[0077] Through the interactive distillation mechanism, the target detection model of the student network in this embodiment can learn the feature representation that integrates the semantics of teachers and students, thereby effectively eliminating the interference effect that may occur when cross-task feature interaction.

[0078] During model training, the student model's orbital region focusing ability is implicitly learned through the interactive distillation module.

[0079] Specifically, the interactive distillation module first extracts the orbital semantic features of the teacher model and the feature representations of the student model's semantic perception module, calculates their spatial correlation matrix and matches them, thereby obtaining the distillation loss. During backpropagation, the teacher model does not participate in parameter updates. The gradient generated by this distillation loss is backpropagated layer by layer according to the chain rule, driving the parameters of the convolutional kernel and attention mechanism in the student model's semantic perception module to approximate the teacher's orbital semantic representation, thereby enhancing the response in the orbital region and suppressing the response in the non-orbital region, gradually enhancing the feature response in the orbital region during parameter updates.

[0080] The key to high-precision foreign object detection in complex railway scenarios is maintaining the discriminative power of detection features while focusing on the track area. This invention designs a semantic region enhancement fusion module to fuse distilled and optimized semantic-aware features with the detection features output by the student model's neck network. This module primarily performs three operations: channel splicing, feature fusion and channel compression, and important feature enhancement. This part consists of three semantic region enhancement fusion modules, which respectively achieve feature fusion at three scales: large, medium, and small detection. Figure 1 As shown.

[0081] The detection features contain rich target discrimination information, but their focus on the orbital region is not strong enough; while the semantic perception features have a strong saliency response in the orbital region, but lack the ability to identify foreign objects.

[0082] To combine the advantages of both, this invention integrates three detection features (P3, P4, P5) of the YOLOv8n neck network with three region enhancement features of the semantic perception module. Concat (i=3,4,5) along the channel dimension to form the fused basic feature F. trans This allows the detected features to gain spatial guidance within the orbital region while preserving their class discriminative power.

[0083] Subsequently, the C2f structure of Yolov8n is used as the core unit for feature fusion and transformation, and a three-level cascade is performed to achieve cross-channel feature interaction while maintaining lightweight design, thus realizing feature fusion and channel compression. Finally, its output is used as the query vector (Query, Q) for cross-attention, which combines the original semantic features F from the semantic enhancement module. sp Simultaneously serving as both key (K) and value (V), the Cross-Attention (CA) mechanism is applied to enhance important features.

[0084] In this embodiment, the semantic region enhancement fusion module includes a Concat connection, a three-level cascaded C2f module, and a cross-attention module; the input of the semantic region enhancement fusion module is a semantically aware feature F at the corresponding scale. sp With the basic features F of object detection p The processing flow of this semantic region enhancement and fusion module is as follows: First, the semantically perceptive features F at the corresponding scale are... sp With the basic features F of object detection p Perform concat connections along the channel dimension to form the fused basic feature F. trans ;F afterwards trans The C2f module is cascaded through three levels.

[0085] In the semantic region enhancement fusion module, each C2f module adopts a multi-branch architecture and is configured with shortcut=False and bottleneck=1: The input features are first adjusted for channels through 1×1 convolutions, and then evenly divided into a main branch and a transform branch, with the number of channels in each branch halved; the transform branch contains a single bottleneck unit, which consists of two standard 3×3 convolutions. The first convolution compresses the channels to 1 / 4 of the input channels, and the second convolution restores them to 1 / 2 of the input channels. Each layer is followed by a BN layer and a SiLU activation function, and residual connections are disabled; the processed transform branch features are concatenated with the main branch features by channel dimension, and finally fused and output through 1×1 convolutions. Feature fusion and channel compression are achieved through three cascaded C2f modules to obtain F. trans The input is then fed into the cross-attention mechanism.

[0086] In the semantic region enhancement fusion module, the cross-attention mechanism projects the outputs of the three cascaded C2f modules as Q. First, a 1×1 convolution is used to project the input features onto the attention space. The projection process is as follows: , , .

[0087] Next, attention weights are generated by matrix multiplication of the transposes of Q and K, and training is stabilized using a scaling factor. Softmax is then applied along the Key dimension to generate a probability distribution, as shown in the following formula: .

[0088] Subsequently, attention weights are used to weight and sum the Values, injecting semantic information into the spatial features to obtain the semantically enhanced fusion feature F. com The formula is as follows: .

[0089] The three feature outputs P3, P4, and P5 of the Yolov8n neck network are mapped to the three region enhancement features of the semantic awareness module. The feature maps F3, F4, and F5 are fused separately for i=3, 4, and 5, respectively, to obtain three semantically enhanced multi-scale feature maps F3, F4, and F5. Then, the feature maps F3, F4, and F5 are input into the corresponding fused feature decoders to obtain the target detection results.

[0090] The fusion feature decoder is responsible for decoding the enhanced object detection feature map into object detection results, including the location information, category, and confidence level of the foreign object. The fusion feature decoder uses the native object detection head of YOLOv8n. The YOLOv8n detection head consists of multi-scale branches, each branch corresponding to a feature map (F3, F4, F5) output by the semantic region enhancement fusion module.

[0091] Each branch contains two sub-modules: a regression branch and a classification branch, both employing a lightweight 1×1 convolutional structure. The regression branch outputs 4×reg_max channels for distributed regression to represent the probability distribution in the four boundary directions, which is then decoded into continuous bounding box coordinates by the Distributed Focus Loss (DFL) module. The classification branch outputs num_classes channels and is activated by a sigmoid function for class probability estimation. These three branches operate on feature maps at different scales, enabling multi-scale object detection.

[0092] During inference, the outputs of all detection layers (including bounding boxes and category information) are merged, with the bounding box portion and the category probability portion being processed separately. The bounding boxes are decoded using DFL to obtain regression coordinates, the output of the category branch is normalized using Sigmoid, and finally, the final detection result is filtered out using Non-Maximum Suppression (NMS).

[0093] During training, the object detection model optimizes its detection performance by initializing and adjusting biases. Classification biases are typically set to negative values ​​to make the network's initial state more biased towards the background class; regression biases are initialized based on preset bounding box sizes. These convolutional layers and modules work together to enable Yolov8n to perform object detection efficiently and accurately.

[0094] The semantic region enhancement fusion module enables the interactive integration of detection features and distilled semantic features. This module introduces a cross-attention mechanism, using detection features as query vectors and distilled semantic features as keys, to implicitly enhance the attention representation of the orbital region without altering the core structure, thereby improving the sensitivity and robustness of foreign object detection.

[0095] Step 3. Train a semantically enhanced object detection model based on cross-task knowledge distillation using the training dataset; input the real-time acquired railway forward-looking images into the trained object detection model to obtain the foreign object detection results.

[0096] The deep learning network is trained by iteratively optimizing the network parameters using stochastic gradient descent.

[0097] The training process of the semantically enhanced object detection model is as follows: The semantically enhanced object detection model is trained using the training dataset until the accuracy of the semantically enhanced object detection model in detecting foreign objects reaches the preset accuracy requirement or the maximum number of training iterations, thus obtaining a well-trained object detection model.

[0098] During training, the parameters of the semantically enhanced target detection model are modified by combining the loss of the location and category information of the foreign object in the foreign object prediction results with the loss of the ground truth, as well as the loss in the knowledge distillation process.

[0099] Specifically, the total loss is defined as consisting of two parts: knowledge distillation loss and foreign object detection loss.

[0100] Knowledge distillation loss enhances the attention of foreign object detection to regional features by performing layer-by-layer matching of spatial correlation matrices at three different scales, enabling the student model to inherit the teacher model's structural modeling and spatial attention capabilities in the orbital region.

[0101]

[0102] in L kd This represents the knowledge distillation loss. , , The weighting coefficients for the feature map distillation loss satisfy the following conditions: , Represents the L2 norm; since the feature maps involved in knowledge distillation have three scales... N 1 、N 2 、N 3 These represent the size of the feature map at this scale when calculating the loss; The foreign object detection loss consists of classification loss and bounding box regression loss; the classification loss is the binary cross-entropy loss, and the bounding box regression loss includes CIoU loss and distribution focus loss; the foreign object detection loss... L D We obtain it from the following formula: .

[0103] in, This is a binary cross-entropy loss, used to correctly determine the category of foreign objects. CIoU loss is used to optimize the position and shape of the target bounding box. To distribute the focus loss, the regression accuracy of the target bounding box is further optimized.

[0104] They are respectively , , Weighting coefficients for the three types of losses.

[0105] Therefore, the overall loss function is expressed as: ;in, L Indicates the total loss. Represents knowledge distillation loss The weight.

[0106] In the training process, this invention compares the predicted location, shape, and category of foreign objects by the preset network model with the corresponding labeled real foreign object information and adjusts the network model parameters accordingly. Then, iterative training is performed using images containing foreign object intrusions in the training sample set until the output foreign object detection results meet the set accuracy requirements, thus obtaining a trained foreign object recognition model. During the model training phase, the teacher model's output of track semantic features guides the student model to perform implicit semantic enhancement on the track region, enabling the student model to efficiently detect foreign objects within the track region.

[0107] After training, the trained model can be deployed on computer equipment in a railway scenario to perform actual foreign object intrusion detection tasks. During the inference phase, the model receives image input from onboard cameras and can independently complete foreign object detection inference without requiring additional explicit semantic information about the track area or auxiliary segmentation results.

[0108] Because the student model receives effective implicit semantic guidance from the teacher model's output of track semantic features during training, it develops spatial focus on the track region. Therefore, during inference, the model automatically tends to focus on the feature representation of the target of interest within the track region, increasing detection confidence within the track region and suppressing redundant responses in non-target areas. In practical deployment, to further improve the reliability and accuracy of detection results, a confidence threshold parameter is set at the model output stage, retaining only target detection boxes with confidence levels higher than this threshold. Since the model has already learned to focus on track region features through the track semantic guidance mechanism, under this confidence filtering mechanism, most of the retained detection targets are concentrated within the track region. In other words, the model outputs high-confidence foreign object detection results almost exclusively within the track region, while its ability to suppress non-target areas outside the track is enhanced, reducing false alarms. Based on this characteristic, the model can be further used for the automatic determination of foreign object intrusion events, outputting event-level determination results. Specifically, if the model outputs at least one detection box that meets the confidence requirement in the image, it can be determined that there is a foreign object target within the track area in the current image, i.e., a "foreign object intrusion event" has occurred; otherwise, it is determined that there is no intrusion risk. This event-level output format is simple and efficient, and is suitable for the triggering mechanism of safety monitoring, intelligent alarms, and subsequent control strategies in railway systems.

[0109] The confidence score predicts the probability that the bounding box contains a target, ranging from [0,1]. A higher value indicates that the model is more certain that the box contains a target. The confidence score is determined by both the classification probability and the regression matching quality. The classification probability is calculated by using BCE (Binary Cross Entropy) to independently model each category in the classification branch of the detector head, outputting the probability p(c) for each category. For each predicted bounding box, five probabilities are output, representing the likelihood that it is a repairman, lost equipment, scattered goods, an animal, or a falling rock. The regression matching quality represents the overlap quality between the predicted and ground truth bounding boxes. For each predicted bounding box, a regression matching quality score is obtained to evaluate the reasonableness of the regression results. The classification probability multiplied by the regression matching quality yields five final scores for the predicted bounding box. The category corresponding to the highest score is taken as the final category, and the final score of that category is retained as the confidence score for the bounding box.

[0110] During the training phase, the student model learns to focus on the track area through track semantic feature distillation from the teacher model. Therefore, the feature weights within the track area are higher, while the feature weights outside the track area are lower. This results in a generally higher confidence level for foreign object detection within the track area during inference. A confidence threshold τ is set, and the model determines whether to retain a predicted bounding box by comparing its confidence level with that of the predicted bounding box. If the confidence level is less than τ, the predicted bounding box is discarded, thus filtering out foreign objects in non-track areas that do not affect train operation. Based on this step, the model can directly output event-level judgment results. Specifically, if the model outputs at least one detection box that meets the confidence requirement in the image, it can determine that a foreign object target exists within the track area in the current image, i.e., a "foreign object intrusion event" has occurred, and outputs this information to the driver's cab human-machine interface and alarm system; otherwise, it is determined that there is no intrusion risk. When a "foreign object intrusion event" occurs in the event-level output, NMS (non-maximum suppression) is applied to the remaining boxes to remove duplicate boxes near the same target. Only the box with the highest score is retained as the final predicted box for that target, resulting in a clear target box location and foreign object category, which can be used for human-computer interface display and post-event analysis.

[0111] Example 2

[0112] This embodiment 2 describes a foreign object detection system for track areas based on distillation sensing. The foreign object detection system for track areas includes a camera and a computer device, both of which are mounted on a train. The camera is used to acquire image data of the railway environment in real time and upload it to the computer device. The computer device includes a memory and a processor.

[0113] The executable code is stored in the memory, and when the processor executes the executable code, it is used to implement the foreign object detection method in the orbital region based on distillation sensing as described in Embodiment 1 above.

[0114] Specifically, the target (foreign object) detection model is trained by inputting real-time railway forward-looking images acquired by the camera. The model then determines whether an obstacle has been detected ahead and outputs an event-level foreign object intrusion result.

[0115] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A method for detecting foreign objects in an orbital region based on distillation sensing, characterized in that, Includes the following steps: Step 1. Obtain the original image dataset of the railway scene, label the semantic ground truth of the track region and the ground truth of the target box for each image, obtain the range of the track and the location and category information of foreign objects, and use them as label data to construct the training dataset; Step 2. Build a semantically enhanced target detection model based on cross-task knowledge distillation, including a foreign object feature encoder, a semantic perception module, an interactive distillation module, a semantic region enhancement fusion module, and a fusion feature decoder; Foreign object feature encoder is used to extract foreign object features at different scales from the input image, which serve as the basic features for target detection; There are multiple semantic awareness modules, which are used to further extract the basic features of the corresponding scale, and output features with semantic awareness capabilities through knowledge distillation by learning the high-level semantic information of the teacher model. There are multiple interactive distillation modules, each responsible for transmitting the semantic features of the track region at different scales extracted by the teacher model to each semantic perception module, so that each semantic perception module can focus on foreign object detection in the track region. There are multiple semantic region enhancement and fusion modules, which are used to fuse semantically perceptive features and basic features at corresponding scales through cross attention, and output multi-scale enhanced target detection feature maps. There are multiple fusion feature decoders, each used to decode the enhanced target detection feature map of the corresponding scale into target detection results, including the location and category information of the foreign object and the confidence level; Step 3. Train a semantically enhanced object detection model based on cross-task knowledge distillation using the training dataset; input the real-time acquired railway forward-looking images into the trained object detection model to obtain the foreign object detection results.

2. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 1, characterized in that, Step 1 specifically involves: The train uses onboard cameras to collect raw image datasets from the train's forward-looking perspective. The images cover railway environments under different weather and lighting conditions to ensure the diversity of the raw image dataset. The semantic ground truth of the track region is labeled for each image, the ground truth of the foreign object target box is labeled for the image containing foreign objects, and each target is assigned a corresponding label. A sample dataset of railway foreign object intrusion scenarios containing five types of foreign objects is established as a training dataset. Foreign objects include maintenance personnel, abandoned equipment, scattered goods, animals, and fallen rocks; The track region of each image is labeled using semantic segmentation labels. The labeled track region includes all pixels of the track, track bed and surrounding area. The labeling method is used, with the track region labeled as 1 and other regions labeled as 0. Each target bounding box needs to provide the following information: Target category information: Category label for each foreign object; Target location information: the bounding box coordinates of the foreign object, including the coordinates of the top left and bottom right corners of the object in the image; the size and proportion of the bounding box of the foreign object. To ensure the high quality of the labeled data, the target box needs to define the boundary of the foreign object.

3. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 1, characterized in that, In step 2, the teacher model is a semantic segmentation model used for track region supervision; the semantically enhanced target detection model is built as a student model, which is used to learn key track regions from the teacher model to detect foreign objects.

4. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 3, characterized in that, The semantic segmentation model adopts the Segformer model, which uses four Transformer coding blocks as the backbone network. Each coding block outputs feature maps with different resolutions and semantic depths. The output features of the second, third, and fourth Transformer coding blocks of the Segformer model are selected as semantic guiding features for distillation, which are used to characterize the boundaries, contours, and overall structure of the orbital region. The foreign object feature encoder in the student model selects the backbone network and neck network of Yolov8n. The neck network outputs three feature maps P3, P4 and P5 at different scales, which are used to detect small, medium and large scale targets, respectively. P3, P4, and P5 each obtain three semantic perception features through the semantic perception module, which are used as student features to be distilled. These features are then aligned and learned with the features output by the second, third, and fourth Transformer encoding blocks in the teacher model. During the training phase, the SegFormer model performs knowledge distillation on the semantic perception module of the student model, indirectly guiding the student model to focus on foreign objects within the orbital region, thereby improving the regional sensitivity and discrimination ability of the detection model.

5. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 3, characterized in that, The semantic awareness module aligns the feature maps of the student model with the semantic information from the teacher model, and uses deep convolution to enhance the semantic context information of the multi-scale feature maps. At the same time, the spatial attention mechanism is used to weight the spatial dimension of the feature maps, and by giving higher weights to the track regions, it helps the object detection model to automatically focus on the track regions in the image.

6. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 5, characterized in that, The semantic awareness module includes two 3×3 depth convolutional layers, three 1×1 convolutional layers, a spatial attention mechanism layer, a channel splitting layer, and a feature splicing layer. The input to the semantic awareness module is the feature map F. p F is adjusted by 1×1 convolution. p The number of channels is aligned with the orbital semantic features, and then a channel splitting layer is used to split the channel into two sub-feature maps, F1 and F2, along the channel dimension. The F1 branch first extracts local spatial features through a 3×3 depthwise convolution, then performs feature compression and dimensionality reduction through a 1×1 convolution, then expands features through a second 1×1 convolution, and finally enhances spatial structural features through a second 3×3 depthwise convolution, resulting in an F1 with enhanced spatial expressive power. F1, which enhances spatial representation, is combined with F2, which preserves original details, to generate F. concat This enables the complementary fusion of deep semantic features and shallow detailed features; Subsequently, further investigation into F concat The spatial attention calculation process is as follows: First F concat Features are compressed through 1×1 convolutions, followed by feature distribution stabilization using Batch Normalization (BN) layers, and finally spatial attention map A is generated using Sigmoid activation. sa ; Then spatial attention map A sa With F concat Element-wise multiplication is used to achieve attention weighting, resulting in attention-weighted features F. mul This enhances the response in the orbital-dependent region and suppresses background interference; finally, F mul With F concat The semantic-aware feature F is obtained by adding elements one by one. sp This approach preserves the original feature information to prevent gradient vanishing while ensuring that key semantic features are not lost.

7. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 4, characterized in that, The processing flow of the interactive distillation module is as follows: Define semantic perception features as , Aligned semantic-aware features and orbital semantic features , ; Semantic awareness features after alignment The orbital semantic feature is F seg ; Where C1 and C2 represent the number of channels, H represents the height, and W represents the width; First, respectively and Perform a flattening operation in the spatial dimension followed by self-multiplication to obtain their respective spatial correlation matrices. , ,in ; The process of obtaining the spatial correlation matrix described above is expressed as follows: ， ； in express The spatial correlation matrix, express Spatial correlation matrix; This represents a flattening operation in the spatial dimension, where i=3,4,5 and j=2,3,4; Represents semantic-aware features Features aligned with the teacher model at the same scale via channels This represents the features extracted by the j-th layer encoder in the teacher network. ; Subsequently and Multiplying and then performing a softmax operation yields the spatial correlation weights between the feature maps extracted by the student model and the teacher model, and then multiplying them by... The products are multiplied, and the final result is used to generate multi-task collaborative features through residual connections. , k =1,2,3; Will Perform a flattening operation in the spatial dimension followed by self-multiplication to obtain a multi-task collaborative space matrix. ; then execute and Cross-domain knowledge distillation between the two domains is constrained by distillation loss due to the distributional differences between them.

8. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 1, characterized in that, The semantic region enhancement fusion module includes Concat connection, a three-level cascaded C2f module, and a cross-attention module; The input to the semantic region enhancement fusion module is a semantically perceptive feature F at the corresponding scale. sp With the basic features F of object detection p The processing flow of this semantic region enhancement and fusion module is as follows: First, the semantically perceptive features F at the corresponding scale are... sp With the basic features F of object detection p Perform concat connections along the channel dimension to form the fused basic feature F. trans ;F afterwards trans The C2f module is cascaded through three levels; and the features output after the three-level cascade of the C2f module are projected as the Q vector of the cross-attention module. F sp The K and V vectors are projected to form the cross-attention module. Next, attention weights are generated by matrix multiplication of the transposes of Q and K, stabilized through scaling factors, and then Softmax is applied along the K dimension to generate a probability distribution. Finally, the attention weights are used to weight and sum V, injecting semantic information into the spatial features to obtain the semantically enhanced fusion feature F. com .

9. The method for detecting foreign objects in the orbital region based on distillation sensing according to claim 7, characterized in that, In step 3, the training process of the semantically enhanced target detection model is as follows: The semantically enhanced object detection model is trained using the training dataset until the accuracy of the semantically enhanced object detection model in detecting foreign objects reaches the preset accuracy requirement or the maximum number of training iterations, thus obtaining a well-trained object detection model. During training, the parameters of the semantically enhanced target detection model are modified by combining the loss of the location and category information of the foreign object in the foreign object prediction results with the loss in the knowledge distillation process. The total loss is defined as consisting of two parts: knowledge distillation loss and foreign object detection loss. Knowledge distillation loss enhances the attention of foreign object detection to regional features by performing layer-by-layer matching of spatial correlation matrices at three different scales. This allows the student model to inherit the structural modeling and spatial attention capabilities of the teacher model in the track region. ； in L kd This represents the knowledge distillation loss. , , The weighting coefficients for the feature map distillation loss satisfy the following conditions: , Represents the L2 norm; since the feature maps involved in knowledge distillation have three scales... N 1 、N 2 、N 3 These represent the size of the feature map at this scale when calculating the loss; The foreign object detection loss consists of classification loss and bounding box regression loss; the classification loss is the binary cross-entropy loss, and the bounding box regression loss includes CIoU loss and distribution focus loss; the foreign object detection loss... L D We obtain it from the following formula: ； in, This is a binary cross-entropy loss, used to correctly determine the category of foreign objects. CIoU loss is used to optimize the position and shape of the target bounding box. To distribute the focus loss, the regression accuracy of the target box is further optimized; They are respectively , , Weighting coefficients for the three types of losses; Therefore, the overall loss function is expressed as: ； in, L Indicates the total loss. Represents knowledge distillation loss The weight.

10. A foreign object detection system for track areas based on distillation sensing, comprising a camera and a computer device, both mounted on a train; the camera is used to acquire image data of the railway environment in real time and upload it to the computer device; Computer devices include memory and processors; characterized in that, The executable code is stored in the memory, and when the processor executes the executable code, it is used to implement the orbital region foreign object detection method based on distillation sensing as described in any one of claims 1 to 9.

Images