Railway perimeter intrusion monitoring method based on multi-sensor fusion

Abstract

The invention discloses a railway perimeter intrusion monitoring method based on multi-sensor fusion, and belongs to the technical field of railway safety monitoring, a multi-sensor sample library containing different scenes, climates, distances, intrusion behaviors and intrusion actions is constructed, and the intrusion behaviors and the intrusion actions are determined according to topographic features, interference factors and monitoring requirements of different typical scenes of a high-speed rail perimeter. A coordinate attention mechanism is introduced into an image detection module; foreground target feature representation under a complex background is enhanced; intrusion target detection is performed on fused feature data based on an improved ByteTrack algorithm; a unique ID is allocated to each target; differentiated configuration is achieved through scenarized multi-sensor deployment, multi-source data collaborative preprocessing and complex environment adaptive detection optimization, the monitoring problem under foggy days, complex backgrounds, small targets and shielding scenes is solved, and stable monitoring of the whole railway perimeter scene is achieved.

Description

Technical Field

[0001] This invention belongs to the field of railway safety monitoring technology, specifically relating to a railway perimeter intrusion monitoring method based on multi-sensor fusion. Background Technology

[0002] During the 13th Five-Year Plan period, China has built the world's most developed high-speed rail network. High-speed rail, with its advanced technology, safety, punctuality, convenience, and efficiency, has become a shining symbol of China. With the formation of the "eight vertical and eight horizontal" high-speed rail network, high-speed rail has become an important means of transportation for the people. According to the "National Comprehensive Three-Dimensional Transportation Network Planning Outline," by 2035, the scale of high-speed rail (including intercity rail) will reach approximately 70,000 kilometers. However, with the continuous expansion of the high-speed rail network and its mileage, as well as the increase in high-speed rail speed, the safety of the high-speed rail operating environment has become particularly important.

[0003] High-speed rail perimeter intrusions pose a potential threat to the safety of high-speed rail operations, potentially leading to train delays, casualties, and economic losses. Currently, my country has initially established a three-pronged security system for its high-speed rail perimeter, integrating human, physical, and technological defenses. Human defenses are ensured through regular patrols by personnel; physical defenses primarily utilize barriers such as fencing and barbed wire; and technological defenses mainly employ sensor technology and electronic information technology for perimeter protection.

[0004] Existing security technologies include video surveillance, vibration fiber optic cables, pulsed electronic fences, and radar. However, single monitoring technologies are insufficient to address the complex and ever-changing environment around high-speed railways. Video surveillance systems are susceptible to interference under varying lighting conditions and inclement weather. Vibration fiber optic cables and pulsed electronic fences suffer from blind spots and false alarms when facing complex terrain around high-speed railways. While radar technology has advantages in target detection, its application around high-speed railways is still in the exploratory stage. The accuracy, reliability, and robustness of existing monitoring technologies need further improvement to achieve effective monitoring of key areas around high-speed railways.

[0005] Therefore, there is an urgent need to design a railway perimeter intrusion detection method that can integrate the advantages of multiple sensors, adapt to complex environments, and achieve lightweight deployment, in order to overcome the shortcomings of existing technologies. Summary of the Invention

[0006] The purpose of this invention is to provide a railway perimeter intrusion detection method based on multi-sensor fusion to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a railway perimeter intrusion detection method based on multi-sensor fusion, wherein the intrusion detection method is as follows:

[0008] S1. Construction of Railway Perimeter Intrusion Sample Library: Construct a multi-sensor sample library containing different scenarios, climates, distances, intrusion behaviors, and intrusion actions;

[0009] S2. Scenario-based multi-sensor deployment: Based on the terrain features, interference factors and monitoring requirements of different typical scenarios around the high-speed railway, sensor combinations are configured differently. Millimeter-wave radar and high-definition camera combinations are deployed in the roadbed sections and bridgeheads in open areas, while lidar and high-definition camera combinations are deployed in the roadbed sections at tunnel entrances, road cuts and adjacent mountains to collect multi-source monitoring data simultaneously.

[0010] S3. Multi-sensor joint calibration: It adopts a combination of automatic and manual calibration to achieve spatiotemporal synchronization of millimeter-wave radar, lidar and high-definition camera, and ensure the consistency of multi-source data in timestamps and spatial coordinates.

[0011] S4. Multi-source data collaborative preprocessing: Image data acquired by high-definition cameras is preprocessed using an image enhancement and machine learning fusion dehazing algorithm. Point cloud data acquired by lidar is processed by voxelization and key point sampling to remove noise interference and retain target detail features.

[0012] S5, Multi-level Intelligent Fusion: Constructing a fusion tensor at the feature layer;

[0013] S6. Adaptive Detection Optimization in Complex Environments: A coordinate attention mechanism is introduced into the image detection module to enhance the feature representation of foreground targets in complex backgrounds;

[0014] S7. Real-time target detection and trajectory tracking: Based on the improved ByteTrack algorithm, intrusion target detection is performed on the fused feature data. A unique ID is assigned to each target, the real-time position of the target is recorded and the motion trajectory is drawn. The target behavior is analyzed to determine whether there is an intrusion risk. If a risk is determined, a real-time alarm is triggered.

[0015] S8. Lightweight optimization of the monitoring model: Sparse training is performed on the fusion network in S5. The pruning parameters are determined through sparse training. Combined with knowledge distillation technology, knowledge from the large model is transferred to the small model, thereby improving the model's running speed while ensuring detection accuracy.

[0016] Preferably, in S1, the sample library includes an image sample library, a 3D point cloud sample library, and a fused sample library, which are used for subsequent model training and validation.

[0017] Preferably, in S4, the implementation process of the image enhancement and machine learning fusion dehazing algorithm is as follows: first, the image contrast is enhanced by histogram equalization, and then the pre-trained U-Net dehazing model is used to automatically remove the fog interference in the image. The training samples of the U-Net dehazing model come from image datasets of different foggy scenes around the high-speed rail perimeter.

[0018] The specific parameters for voxelization and key point sampling are as follows: the voxel mesh size is set to 10cm×10cm×10cm, the farthest point sampling algorithm is used to select key points, and the sampling density is controlled at 50 to 80 key points per cubic meter to ensure that the target contour features are preserved while reducing data redundancy.

[0019] Preferably, the training process of the U-Net dehazing model includes: forming training pairs with foggy images and corresponding clear images from the image sample library, using mean squared error as the loss function, iteratively training with the Adam optimizer for 100 to 150 rounds, with the initial learning rate set to 1e-4, and decaying to 1 / 10 of the original rate every 50 rounds.

[0020] Preferably, in S5, the construction process of the fusion tensor is as follows: the preprocessed 2D target candidate features of the image and the 3D target candidate features of the point cloud are integrated into the fusion tensor with the overlap, confidence of 2D detection candidate objects, confidence of 3D detection candidate objects, and normalized distance as feature dimensions. The fusion tensor is jointly learned by the improved fusion network in the decision layer, wherein the fusion network uses Focalloss as the loss function.

[0021] Preferably, S6 also includes: a design space and depth transformation module to improve the problem of small target feature loss during downsampling, and introduces adaptive deformable convolution during point cloud detection to automatically adjust the shape of the convolution kernel according to the target shape and position, and generate an anchor box adapted to the target size by combining the K-Means clustering algorithm.

[0022] Preferably, in S6, the coordinate attention mechanism is implemented as follows: a coordinate information encoding layer is added to the backbone network of the target detection model to encode the spatial position information of the target's x-axis and y-axis, generate a coordinate attention weight map, and enhance the features of the target region and suppress background interference through weighted summation.

[0023] Preferably, the specific process of K-Means clustering is as follows: extract 3D bounding box size data of 1000 to 2000 intrusion targets from the 3D point cloud sample library in S1, set the number of clusters to 3 to 5, calculate the sample similarity through Euclidean distance, iteratively update the cluster centers until convergence, and generate anchor box sizes that are adapted to different target types.

[0024] Preferably, in S8, the specific process of model pruning and knowledge distillation is as follows: set a sparsity threshold of 0.6 to 0.8, use L1 regularization to make the weights of unimportant channels approach 0, then remove channel connections with weights below the threshold, compress the number of model parameters, and then use the pruned model as the teacher model to construct a small student model, and realize knowledge transfer through the distillation loss function.

[0025] Preferably, the distillation loss function is calculated as follows: Distillation loss = α × classification loss + β × feature loss, where α = 0.7, β = 0.3, the classification loss is the cross-entropy between the outputs of the teacher model and the student model, and the feature loss is the L2 distance between the intermediate layer features of the teacher model and the student model.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] 1. This invention achieves differentiated configuration through scenario-based multi-sensor deployment, multi-source data collaborative preprocessing, and adaptive detection optimization in complex environments. It solves the monitoring challenges in foggy weather, complex backgrounds, small targets, and occluded scenarios, enabling accurate identification, dynamic tracking, and real-time early warning of intrusion targets in different scenarios such as railway perimeter roadbed sections, bridgeheads, tunnel entrances, and road cuts. The fusion detection module uses feature layer fusion tensors and decision layer improved fusion networks for multi-level intelligent fusion, effectively solving the problems of single sensors being susceptible to interference from complex environments and insufficient detection accuracy, and significantly improving the target recognition accuracy, especially for small targets.

[0028] 2. This invention enhances the features of the target area through a coordinate attention mechanism, adapts the target shape to adaptive deformable convolution, and combines image dehazing and point cloud noise reduction in the data preprocessing module, effectively overcoming the interference of severe weather and complex terrain on monitoring, and significantly reducing the false alarm rate and missed alarm rate.

[0029] 3. This invention, through multi-sensor redundancy design and multi-level fusion strategy, enables the system to adapt to the diverse and complex environment of the high-speed rail perimeter, thereby improving the overall reliability and robustness of the system.

[0030] 4. This invention, by employing an improved ByteTrack algorithm combined with Kalman filtering, can achieve real-time detection and accurate trajectory tracking of intrusion targets, providing support for timely early warning and response. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention;

[0032] Figure 2 A diagram showing the PV-RCNN network structure with adaptive convolution added in this invention;

[0033] Figure 3 This invention relates to a tensor-based fusion method. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figures 1-3 This invention provides a railway perimeter intrusion detection method based on multi-sensor fusion. The intrusion detection method is as follows:

[0036] S1. Construction of Railway Perimeter Intrusion Sample Library: Construct a multi-sensor sample library containing different scenarios, climates, distances, intrusion behaviors, and intrusion actions. The sample library includes an image sample library, a 3D point cloud sample library, and a fusion sample library, which will be used for subsequent model training and validation.

[0037] It should be noted that three sub-libraries are constructed: an image sample library, a 3D point cloud sample library, and a fusion sample library. The image sample library collects images of intrusion targets under different lighting conditions, such as strong daylight, low light at night, and strong nightlight, as well as weather conditions such as sunny, rainy, snowy, and foggy conditions, and at different distances, labeling the target categories and bounding boxes. The 3D point cloud sample library collects point cloud data of intrusion targets under different scenarios, labeling the target's 3D bounding boxes and semantic information. The fusion sample library associates the synchronously collected images with the point cloud data, labeling the correspondence between fused features.

[0038] Furthermore, the sample augmentation strategy employs data augmentation techniques such as flipping, rotating, and adding noise for image samples, and operations such as translation, rotation, and local point cloud deletion for 3D point cloud samples to ensure that the total number of samples in the sample library exceeds 100,000, with intrusion scene samples accounting for 30%, thus meeting the diversity requirements of model training.

[0039] S2. Scenario-based multi-sensor deployment: Based on the terrain features, interference factors and monitoring requirements of different typical scenarios around the high-speed railway, sensor combinations are configured differently. Millimeter-wave radar and high-definition camera combinations are deployed on the roadbed sections and bridgeheads in open areas, while lidar and high-definition camera combinations are deployed on the roadbed sections at tunnel entrances, road cuts and adjacent mountains, to simultaneously collect multi-source monitoring data.

[0040] It should be noted that in open scenarios such as roadbed sections or bridgeheads: these scenarios offer a wide field of view with no significant obstructions, and a combination of millimeter-wave radar and high-definition cameras is deployed. The millimeter-wave radar uses the 24GHz or 77GHz frequency band, with a detection range of 100–200m and a detection angle of 120°–150°, enabling rapid acquisition of the position and speed of moving targets for long-range early warning. The high-definition cameras employ 4K resolution and a frame rate of 30 frames per second, featuring a wide dynamic range, clearly capturing target details such as clothing and the shape of foreign objects to aid in target classification. Sensors are deployed in groups every 50–80m along the route, with adjacent sensors overlapping their monitoring ranges by 20%–30% to ensure no blind spots.

[0041] For complex scenarios such as tunnel entrances, road cuts, or roadbed sections near mountains: These scenarios present challenges due to terrain obstruction and environmental interference. A combination of LiDAR and high-definition cameras is recommended. The LiDAR should be a 16-line or 32-line device with a point cloud density of 150–200 points / m² and a detection range of 150–200m. This generates high-precision 3D point cloud data, enabling the construction of complex terrain models and effectively avoiding target loss due to obstruction. The high-definition cameras have night vision capabilities, adapting to the alternating light and dark environments at tunnel entrances and nighttime monitoring. Sensors are installed on both sides of the tunnel entrance, avoiding direct headlights and the top of the road cut slope. The installation height should be 1–2m above vegetation height, with a coverage area of ​​50–80m, ensuring that the tunnel entrance area and the bottom of the road cut slope are within the monitoring range.

[0042] S3. Multi-sensor joint calibration: It adopts a combination of automatic and manual calibration to achieve spatiotemporal synchronization of millimeter-wave radar, lidar and high-definition camera, ensuring the consistency of multi-source data in timestamps and spatial coordinates.

[0043] It should be noted that the intrinsic parameter calibration is performed as follows: For high-definition cameras, a black and white checkerboard calibration board measuring 1m×1m with a grid size of 5cm×5cm is used. This board is placed at different locations within the overlapping area monitored by multiple sensors, and images of the calibration board are acquired. Then, the camera's intrinsic parameter matrix and distortion coefficients are calculated using Zhang Zhengyou's calibration algorithm to eliminate the influence of lens distortion on the image.

[0044] Extrinsic parameter alignment: For spatial coordinate matching between millimeter-wave radar, lidar, and camera, the calibration board is placed at a point with known 3D coordinates. Data from each sensor is collected synchronously. The 3D coordinates of the calibration board in the lidar point cloud and the 2D coordinates of the calibration board in the camera image are used. Then, the RANSAC matching algorithm is used to solve the extrinsic parameter matrix between the sensors to achieve spatial coordinate alignment. For time synchronization, the timestamps of each sensor are unified to the same clock source through the sensor's PTP interface, initially controlling the time error to ≤20ms.

[0045] Error correction: In actual railway perimeter scenarios, select 3 to 5 reference objects with known precise coordinates, such as track mileage markers and signal machine bases, collect detection data of each sensor on the reference objects, manually fine-tune the extrinsic parameter matrix and time offset, and finally control the spatial coordinate error ≤5cm and the time synchronization error ≤10ms to ensure that the multi-source data are completely matched in the spatiotemporal dimension.

[0046] S4. Multi-source data collaborative preprocessing: Image data acquired by high-definition cameras is preprocessed using an image enhancement and machine learning fusion dehazing algorithm. Point cloud data acquired by LiDAR is processed using voxelization and keypoint sampling to remove noise interference and retain target detail features. The implementation process of the image enhancement and machine learning fusion dehazing algorithm is as follows: First, image contrast is enhanced through histogram equalization, and then the pre-trained U-Net dehazing model is used to automatically remove fog interference from the image. The training samples of the U-Net dehazing model come from image datasets of different foggy scenes around the high-speed rail perimeter. The training process of the U-Net dehazing model includes: forming training pairs with foggy images and corresponding clear images in the image sample library, using mean squared error as the loss function, and iteratively training with the Adam optimizer for 100-150 rounds. The initial learning rate is set to 1e-4, and it decays to 1 / 10 of the original rate every 50 rounds. The specific parameters for voxelization and key point sampling are as follows: the voxel mesh size is set to 10cm×10cm×10cm, the farthest point sampling algorithm is used to select key points, and the sampling density is controlled at 50 to 80 key points per cubic meter to ensure that the target contour features are preserved while reducing data redundancy.

[0047] It should be noted that image data preprocessing focuses on addressing image quality degradation issues in environments such as fog, low light, and backlighting. A three-stage processing flow is employed: distortion correction, contrast enhancement, and dehazing optimization. Distortion correction inputs the acquired raw image into a correction model generated from camera intrinsic parameter calibration, eliminating radial and tangential distortion of the lens to ensure that the geometry of targets in the image matches reality and avoids target position shifts caused by distortion. Contrast enhancement targets low-contrast scenes such as cloudy days and nighttime, employing an adaptive histogram equalization algorithm. The image is divided into 8×8 sub-blocks, and histogram equalization is performed on each sub-block, limiting the contrast threshold to avoid local over-enhancement caused by traditional histogram equalization and improving the clarity of local image details. Dehazing optimization addresses image degradation in foggy conditions, using a fusion of image enhancement and machine learning for dehazing. First, a dark channel prior algorithm is used to initially estimate the fog concentration, generating a coarsely dehazed image. Secondly, the roughly dehazed image is input into the pre-trained U-Net dehazing model. The training samples are from the foggy image sample library constructed in S1. The model extracts fog features through the Encoder layer and performs pixel-level dehazing through the Decoder layer, outputting a clear image. Experiments show that this method can improve the peak signal-to-noise ratio of foggy images from 20-25 dB to 35-40 dB, and the structural similarity from 0.5-0.6 to 0.8-0.9, effectively restoring target details such as facial features and the contours of foreign objects.

[0048] Point cloud data preprocessing focuses on addressing the sparsity of LiDAR point clouds, noise points such as dust or raindrop reflections, and redundant data. A three-stage processing flow is employed: noise removal, voxelization dimensionality reduction, and keypoint sampling. Noise removal uses a statistical filtering algorithm to calculate the average distance to the 50 nearest neighbors of each point cloud, eliminating noise points with a mean distance exceeding twice the standard deviation, such as dust reflections or raindrops, while retaining valid point clouds of the target and environmental background, achieving a noise removal rate of ≥90%. Voxelization dimensionality reduction divides the denoised point cloud data into a 10cm×10cm×10cm voxel grid. The average coordinates, normal vector, and density features of points within each voxel are calculated, and a single voxel feature replaces all points within that voxel, achieving dimensionality reduction and significantly reducing the computational cost of subsequent algorithms while preserving the overall contour features of the target. Keypoint sampling uses a farthest-point sampling algorithm to select key points from the voxelized point cloud, with a sampling density controlled at 50–80 key points per cubic meter. This algorithm prioritizes selecting the points furthest from the selected points, ensuring that key points uniformly cover the target surface, preserving the target's detailed features, and avoiding feature loss due to uneven sampling, thus providing high-quality data for subsequent point cloud detection.

[0049] Millimeter-wave radar data preprocessing: The focus is on extracting effective target information and filtering environmental interference signals. Clutter filtering is applied to the raw radar echo signal to remove static background interference such as ground and trees, as well as to detect moving targets and perform constant false alarm rate (CFAR) detection. The output includes target distance, velocity, angle information, and detection confidence. Interfering targets with a confidence level <0.3, such as birds or wind-blown branches, are removed, while potential intrusion target data is retained to provide reliable target location references for subsequent fusion.

[0050] S5. Multi-level intelligent fusion: A fusion tensor is constructed at the feature layer. The construction process of the fusion tensor is as follows: the preprocessed 2D target candidate features of the image and the 3D target candidate features of the point cloud are integrated into the fusion tensor with the overlap, confidence of 2D detection candidate objects, confidence of 3D detection candidate objects, and normalized distance as feature dimensions. The fusion tensor is jointly learned at the decision layer using an improved fusion network, where the fusion network uses Focalloss as the loss function.

[0051] It should be noted that the preprocessed 2D features of the image are associated and integrated with the 3D features of the point cloud to construct a fusion tensor of a unified dimension, thereby achieving deep coupling of multi-source features.

[0052] The feature extraction process is as follows: From the preprocessed image, 2D target candidate features are extracted using an improved YOLO algorithm, including target bounding box coordinates, detection confidence, target size, and semantic category. Then, from the preprocessed point cloud, 3D target candidate features are extracted using an improved PV-RCNN algorithm, including target 3D bounding box coordinates, volume, surface texture features, and detection confidence.

[0053] Feature association: Calculate the association parameters of each group of 2D target candidates and 3D target candidates, including overlap. Use the IOU algorithm to calculate the spatial overlap rate of the 2D bounding box and the 3D bounding box on the image projection plane, with a value range of 0-1. Calculate the Euclidean distance between the center of the 2D target and the center of the 3D target on the image projection plane using normalized distance, normalized to the range of 0-1, with smaller values ​​for closer distances.

[0054] Tensor Construction: Using overlap, 2D detection confidence, 3D detection confidence, and normalized distance as four feature dimensions, the associated 2D-3D features are combined into a 4-dimensional fusion tensor. Each tensor element corresponds to a set of potential intrusion target features, maximizing the retention of all possible target information and reducing the risk of missed detections.

[0055] Fusion Network Learning: A Transformer-based fusion network is designed to extract features and optimize decisions on the fusion tensor, outputting the final object detection result and addressing the problems of imbalanced positive and negative samples and difficulty in distinguishing between easy and difficult samples. The fusion network consists of an input layer, four Multi-HeadAttention layers, two Feed-Forward layers, and an output layer. The input layer receives a 4-dimensional fusion tensor and performs dimensionality mapping, transforming the 4-dimensional features into a 256-dimensional feature vector. The Multi-HeadAttention layers calculate attention weights between features to capture the complementary relationships between 2D and 3D features, enhancing the representational power of effective features. The Feed-Forward layers perform non-linear transformations of features, improving the network's ability to fit complex features. The output layer outputs the final category, location, and confidence score of the target through a fully connected layer.

[0056] Loss function optimization: Focalloss is adopted as the loss function. By setting the focusing parameter γ=2, the weights of positive and negative samples are dynamically adjusted. Easy-to-classify samples are assigned low weights, while difficult-to-classify samples are assigned high weights, addressing the imbalance problem of numerous normal scene samples and few intrusion samples in railway perimeter scenarios. Simultaneously, class balancing weights are introduced, assigning higher weights to target categories with smaller sample sizes to ensure balanced model recognition accuracy across different target types.

[0057] Model training: Using the fusion sample library built in S1 as training data, the Adam optimizer is used, with the initial learning rate set to 1e-4, which is decayed to 1 / 10 of the original rate every 50 rounds. The batch size is set to 32, and the training is iterated for 80-100 rounds until the average accuracy of the model on the validation set tends to stabilize.

[0058] S6. Adaptive Detection Optimization in Complex Environments: A coordinate attention mechanism is introduced into the image detection module to enhance the feature representation of foreground targets in complex backgrounds. A spatial and depth transformation module is designed to improve the problem of feature loss of small targets during downsampling. Adaptive deformable convolution is introduced in point cloud detection to automatically adjust the shape of the convolution kernel according to the target shape and position. Combined with the K-Means clustering algorithm, anchor boxes adapted to the target size are generated. The coordinate attention mechanism is implemented by adding a coordinate information encoding layer to the backbone network of the target detection model to encode the spatial position information of the target on the x-axis and y-axis, generating a coordinate attention weight map. The target region features are enhanced by weighted summation to suppress background interference. The specific process of K-Means clustering is as follows: 3D bounding box size data of 1000-2000 intrusion targets are extracted from the 3D point cloud sample library in S1. The number of clusters is set to 3-5. The sample similarity is calculated by Euclidean distance. The cluster centers are iteratively updated until convergence, generating anchor box sizes adapted to different target types.

[0059] It's important to note the introduction of a coordinate attention mechanism: a coordinate attention module is added to the backbone network of the improved YOLO model. This module encodes the spatial location information of the target along the x and y axes to generate a two-dimensional attention weight map. Specifically, the feature map output by the backbone network is first subjected to global average pooling to obtain a channel-dimensional feature vector. Then, two 1×1 convolutional layers are used to generate attention weights for the x and y axes, respectively. Finally, the weight map is multiplied element-wise with the original feature map to enhance the feature response of the target region and suppress interference from the background region.

[0060] The spatial and depth transformation design process is as follows: A spatial and depth transformation module is added to the model downsampling layer to solve the core problem of small target feature loss during downsampling. This module achieves small target feature preservation through three steps: spatial feature decomposition, depth dimension mapping, and residual fusion. First, the spatial feature map before downsampling is decomposed into four 1 / 2-size sub-feature maps, corresponding to the local details of the target. Second, the spatial dimension features of the sub-feature maps are mapped to the depth dimension through 1×1 convolution, enhancing the detailed feature representation of the small target. Finally, the mapped depth features are fused with the downsampled feature map through residual connections to ensure that the small target features are not diluted by the downsampling process.

[0061] Point cloud detection optimization and focusing on irregular targets and occluded scenes: Adaptive deformable convolution is introduced into the detection head of the improved PV-RCNN model to replace the traditional fixed-shape convolution kernel, adapting to the feature extraction needs of irregular targets such as twisted tree branches, irregular rocks, or huddled people. This convolution dynamically adjusts the sampling position of the convolution kernel by learning the target's offset field. For the edge regions of irregular targets, the convolution kernel sampling points are shifted towards the target contour. For the core regions of the target, the sampling points are evenly distributed to ensure that the convolution operation can accurately capture the target's morphological features.

[0062] K-Means Clustering Optimizes Anchor Boxes: Traditional PV-RCNN models use fixed-size anchor boxes, which are difficult to adapt to the size differences of people, foreign objects, and animals in the railway perimeter, easily leading to low target matching accuracy. This application optimizes anchor box generation through the K-Means clustering algorithm: 3D bounding box size data of 1000-2000 intrusion targets are extracted from the 3D point cloud sample library constructed in S1. Using Euclidean distance as the similarity metric, the number of clusters is set to 3-5, and the cluster centers are iteratively updated until convergence, generating 3-5 sets of anchor boxes that adapt to the target size. For example, the anchor box size for people is set to 1.7m×0.6m×0.5m, and the anchor box size for small foreign objects is set to 0.3m×0.3m×0.3m, which improves the target matching IOU value by 15%-20% and the positioning accuracy by 3.02%.

[0063] Point Cloud Completion for Occluded Targets: To address the issue of missing point clouds caused by tree and slope occlusion, a Transformer-based point cloud completion module is designed. By learning the point cloud feature distribution of the complete target, the module predicts and generates point clouds for the missing regions. First, the occluded point clouds are masked to identify the missing regions. Second, a Transformer encoder extracts the global features of the occluded point clouds, and the decoder generates point cloud data for the missing regions based on these global features and prior knowledge of the complete target. Finally, the completed point cloud is fused with the original point cloud for subsequent detection.

[0064] S7. Real-time target detection and trajectory tracking: Based on the improved ByteTrack algorithm, intrusion target detection is performed on the fused feature data. Each target is assigned a unique ID, the real-time position of the target is recorded and the motion trajectory is drawn. The target behavior is analyzed to determine whether there is an intrusion risk. If a risk is determined, a real-time alarm is triggered.

[0065] It should be noted that after target detection is completed, real-time tracking and behavior analysis are needed to determine intrusion risks and trigger early warnings, thus buying time for railway maintenance personnel to respond. Target ID allocation and trajectory initialization: Valid target screening: Based on the detection results output from S6, a confidence threshold of 0.5 is preset to screen valid target candidate boxes. A ResNet18 classifier is used to perform secondary classification of valid targets, clarifying the target category and hazard level.

[0066] Unique ID Assignment: The Hungarian algorithm is used to match valid targets in two consecutive frames. The matching criteria include target position IOU, size similarity, and category consistency. A unique ID is assigned to the first detected target, such as "PER-001" for the first person target and "OBJ-002" for the second foreign object target, to ensure the continuity of IDs during target movement. The ID switching rate is controlled to ≤0.5%.

[0067] Trajectory initialization: Initialize trajectory cache for each target with assigned ID, record the target's real-time position, 2D bounding box center coordinates, 3D bounding box center coordinates, motion speed, and motion direction to construct the target motion model.

[0068] Dynamic trajectory tracking and occlusion handling: Based on the target motion model, a Kalman filter algorithm is used to predict the target's position and velocity in the next frame. The prior state of the target is predicted through a state equation, and the prior state is corrected through an observation equation to obtain the posterior state of the target, achieving smooth prediction of the target's motion trajectory and reducing the impact of inter-frame jitter on tracking. For occlusion scenarios, the association logic of the ByteTrack algorithm is improved. In addition to high-confidence detection boxes, low-scoring detection boxes are included in the association range. By calculating the positional and size similarity between low-scoring boxes and the predicted trajectory, occluded targets are identified, enabling trajectory connection for targets with occlusion duration ≤ 3 seconds, avoiding trajectory breaks caused by occlusion.

[0069] Trajectory Anomaly Correction: When an abnormal jump occurs in the target trajectory, an anomaly correction mechanism is triggered. By reviewing the trajectory data of the previous 5 frames, the cause of the anomaly is determined. If it is a false detection, the abnormal frame data is removed and rematched. If it is occlusion, the missing data is supplemented by interpolation of the trajectory of adjacent frames to ensure the continuity and accuracy of the trajectory.

[0070] Intrusion behavior analysis and real-time early warning:

[0071] Risk area classification: In accordance with railway safety regulations, the monitoring area is divided into warning zones and danger zones, with different warning thresholds set for different zones.

[0072] Behavioral analysis model: This model determines whether a target poses an intrusion risk by analyzing its movement trajectory, dwell time, and speed. For example, if a person moves from the warning zone to the danger zone at a speed greater than 2 m / s, it is considered a high-risk intrusion. If a foreign object remains in the danger zone for more than 1 minute, it is also considered a high-risk intrusion.

[0073] Multi-level early warning response: Different early warning mechanisms are triggered based on the risk level. Low risk only triggers a pop-up notification on the local maintenance terminal without triggering an audible or visual alarm. Medium risk triggers a local audible and visual alarm and simultaneously pushes target images and location information to the maintenance terminal. High risk, in addition to a local alarm, pushes early warning information to the railway dispatch center via LoRa / NB-IoT wireless communication, including target category, real-time trajectory, location coordinates, and on-site images / point cloud data. It also coordinates with signals along the line to prompt trains to slow down or make emergency stops, minimizing the risk of accidents.

[0074] S8. Lightweight Optimization of the Monitoring Model: The fusion network in S5 is sparsely trained to determine pruning parameters. Combined with knowledge distillation, knowledge from the large model is transferred to the small model, improving model speed while maintaining detection accuracy. The specific process of model pruning and knowledge distillation is as follows: Set a sparsity threshold of 0.6–0.8, use L1 regularization to make the weights of unimportant channels approach 0, then remove channel connections with weights below the threshold to compress the model parameters. Then, use the pruned model as the teacher model to construct a small student model, and achieve knowledge transfer through the distillation loss function. The distillation loss function is calculated as: Distillation Loss = α × Classification Loss + β × Feature Loss, where α = 0.7, β = 0.3, the classification loss is the cross-entropy between the teacher and student models, and the feature loss is the L2 distance between the intermediate layer features of the teacher and student models.

[0075] It should be noted that the sparsification objective is to perform sparse training on the fusion network in S5, with the goal of making the weights of unimportant channels in the model approach 0, in preparation for subsequent pruning.

[0076] Training Strategy: During model training, an L1 regularization term is introduced to penalize the channel weights of each layer of the network, causing the weights of redundant channels to gradually decrease. A sparsity threshold of 0.6–0.8 is set, and iterative training is used to achieve the preset sparsity while maintaining an accuracy loss of ≤1%. For example, in the convolutional layer channel weights of the fusion network, 80% of the weights with an absolute value ≤0.01 can be identified as redundant parameters.

[0077] Pruning criteria: Based on the sparsely trained model, calculate the sum of the absolute values ​​of the weights of each channel, and remove channels whose sum of weights is lower than a threshold as redundant channels.

[0078] The pruning process is as follows: First, each convolutional layer and feed-forward layer of the fusion network is evaluated for channels, and redundant channels are marked. Second, redundant channels and their corresponding input or output connections are removed, and the dimensionality matching between network layers is adjusted. Finally, the pruned model is fine-tuned to recover the accuracy loss caused by pruning.

[0079] Teacher-student model construction: The pruned model is used as the teacher model to construct the student model. The number of network layers in the student model is reduced from 8 layers to 5 layers in the teacher model, the number of Multi-HeadAttention heads is reduced from 4 to 2, the hidden dimension of the Feed-Forward layer is reduced from 2048 to 1024, and the number of parameters is further compressed to below 10M.

[0080] Distillation Training: A dual-loss function, combining classification loss and feature loss, is used for distillation training. The classification loss calculates the cross-entropy of the output class probabilities of the teacher and student models, ensuring the student model inherits the teacher model's classification ability. The feature loss calculates the L2 distance between the intermediate layer features of the teacher and student models, ensuring the student model inherits the teacher model's feature extraction ability. During training, the teacher model's output is used as a soft label to guide the student model's learning. Iterative training is conducted for 50-80 epochs until the student model's accuracy reaches over 98% of the teacher model's.

[0081] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A railway perimeter intrusion detection method based on multi-sensor fusion, characterized in that, The intrusion detection methods are as follows: S1. Construction of Railway Perimeter Intrusion Sample Library: Construct a multi-sensor sample library containing different scenarios, climates, distances, intrusion behaviors, and intrusion actions; S2, Scenario-based multi-sensor deployment: Based on the terrain features, interference factors and monitoring requirements of different typical scenarios around the high-speed railway, sensor combinations are configured differently. Millimeter-wave radar and high-definition camera combinations are deployed in the roadbed sections and bridgeheads in open areas, while lidar and high-definition camera combinations are deployed in the roadbed sections at tunnel entrances, road cuts and adjacent mountains, to collect multi-source monitoring data simultaneously. S3. Multi-sensor joint calibration: It adopts a combination of automatic and manual calibration to achieve spatiotemporal synchronization of millimeter-wave radar, lidar and high-definition camera, and ensure the consistency of multi-source data in timestamps and spatial coordinates. S4. Multi-source data collaborative preprocessing: Image data acquired by high-definition cameras is preprocessed using an image enhancement and machine learning fusion dehazing algorithm. Point cloud data acquired by lidar is processed by voxelization and key point sampling to remove noise interference and retain target detail features. S5, Multi-level Intelligent Fusion: Constructing a fusion tensor at the feature layer; S6. Adaptive Detection Optimization in Complex Environments: A coordinate attention mechanism is introduced into the image detection module to enhance the feature representation of foreground targets in complex backgrounds; S7. Real-time target detection and trajectory tracking: Based on the improved ByteTrack algorithm, intrusion target detection is performed on the fused feature data. A unique ID is assigned to each target, the real-time position of the target is recorded and the motion trajectory is drawn. The target behavior is analyzed to determine whether there is an intrusion risk. If a risk is determined, a real-time alarm is triggered. S8. Lightweight optimization of the monitoring model: Sparse training is performed on the fusion network in S5. The pruning parameters are determined through sparse training. Combined with knowledge distillation technology, knowledge from the large model is transferred to the small model, thereby improving the model's running speed while ensuring detection accuracy.

2. The railway perimeter intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that, In S1, the sample library includes an image sample library, a 3D point cloud sample library, and a fused sample library, which are used for subsequent model training and validation.

3. The railway perimeter intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that, In S4, the implementation process of the image enhancement and machine learning fusion dehazing algorithm is as follows: first, the image contrast is enhanced by histogram equalization, and then the pre-trained U-Net dehazing model is used to automatically remove the fog interference in the image. The training samples of the U-Net dehazing model come from image datasets of different foggy scenes around the high-speed rail perimeter. The specific parameters for voxelization and key point sampling are as follows: the voxel mesh size is set to 10cm×10cm×10cm, the farthest point sampling algorithm is used to select key points, and the sampling density is controlled at 50 to 80 key points per cubic meter to ensure that the target contour features are preserved while reducing data redundancy.

4. The railway perimeter intrusion detection method based on multi-sensor fusion according to claim 3, characterized in that, The training process of the U-Net dehazing model includes: forming training pairs with foggy images and corresponding clear images from the image sample library, using mean squared error as the loss function, and iteratively training for 100 to 150 rounds with the Adam optimizer. The learning rate is initially set to 1e-4 and decays to 1 / 10 of the original rate every 50 rounds.

5. The railway perimeter intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that, In S5, the construction process of the fusion tensor is as follows: the preprocessed 2D target candidate features of the image and the 3D target candidate features of the point cloud are integrated into the fusion tensor with the overlap, confidence of 2D detection candidate objects, confidence of 3D detection candidate objects, and normalized distance as feature dimensions. The fusion tensor is jointly learned by the improved fusion network in the decision layer, where the fusion network uses Focalloss as the loss function.

6. The railway perimeter intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that, S6 also includes: a design space and depth transformation module to improve the problem of small target feature loss during downsampling, and introduces adaptive deformable convolution in point cloud detection to automatically adjust the shape of the convolution kernel according to the target shape and position, and combine K-Means clustering algorithm to generate anchor boxes that fit the target size.

7. The railway perimeter intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that, In S6, the coordinate attention mechanism is implemented by adding a coordinate information encoding layer to the backbone network of the target detection model to encode the spatial position information of the target's x-axis and y-axis, generating a coordinate attention weight map, and enhancing the features of the target region and suppressing background interference through weighted summaries.

8. A railway perimeter intrusion detection method based on multi-sensor fusion according to claim 6, characterized in that, The specific process of K-Means clustering is as follows: extract the 3D bounding box size data of 1000 to 2000 intrusion targets from the 3D point cloud sample library in S1, set the number of clusters to 3 to 5, calculate the sample similarity through Euclidean distance, iteratively update the cluster center until convergence, and generate the Anchor box size that is adapted to different target types.

9. A railway perimeter intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that, In S8, the specific process of model pruning and knowledge distillation is as follows: set the sparsity threshold to 0.6 to 0.8, use L1 regularization to make the weights of unimportant channels approach 0, then remove channel connections with weights below the threshold, compress the number of model parameters, and then use the pruned model as the teacher model to construct a small student model. Knowledge transfer is achieved through the distillation loss function.

10. A railway perimeter intrusion detection method based on multi-sensor fusion according to claim 9, characterized in that, The distillation loss function is calculated as follows: Distillation loss = α × classification loss + β × feature loss, where α = 0.7, β = 0.3, the classification loss is the cross-entropy between the outputs of the teacher model and the student model, and the feature loss is the L2 distance between the intermediate layer features of the teacher model and the student model.

Images