Railway station-oriented crowd safety monitoring method based on compressed optimization edge vision
By employing gradient and information entropy preprocessing, multi-stage pruning head detection, and fisheye-calibrated ground projection, the problems of detection accuracy, stability, and privacy compliance in railway station crowd monitoring systems were solved, achieving efficient and accurate edge-vision crowd safety monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHAANXI RAILWAY INST
- Filing Date
- 2026-03-13
- Publication Date
- 2026-05-29
Smart Images
Figure CN122116288A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of image or video recognition and understanding methods, specifically relating to a compressed and optimized edge visual crowd safety monitoring method for railway stations. Background Technology
[0002] As a high-density, safety-critical public infrastructure in urban transportation networks, railway stations often face challenges such as dense crowds, complex flow patterns, and congestion in certain areas during peak passenger periods. Timely and accurate detection and early warning of crowd safety incidents such as overcrowding, unsafe movement, and intrusion into restricted areas are crucial technological foundations for ensuring railway operational safety and public order. Traditional monitoring methods relying on manual monitoring are ill-suited to the real-time monitoring needs of multi-camera, large-scale, and high-density scenarios. Therefore, image or video recognition and understanding technologies based on closed-circuit television (CCTV) are gradually becoming the primary technical means for crowd safety monitoring at railway stations. However, the actual visual monitoring environment of railway stations presents several challenges to existing image or video recognition and understanding methods, specifically in the following aspects:
[0003] First, regarding scene complexity, railway station surveillance footage commonly suffers from visual degradation phenomena such as high-density crowd obstruction, severe geometric distortion introduced by fisheye or ultra-wide-angle lenses, low lighting and localized strong reflections, motion blur, and rapid changes in lighting. These factors significantly reduce the accuracy and stability of traditional detection models that target the entire body of pedestrians, while existing head detection methods still struggle to meet the practical needs of high-density railway station scenes in terms of feature extraction, scale adaptation, and occlusion robustness.
[0004] Secondly, regarding deployment constraints, railway visual analysis systems are typically deployed on edge computing devices for low-latency response and privacy protection. However, edge devices are severely limited in terms of computing power, storage, and power consumption, making it difficult to directly run uncompressed deep learning models with large parameter scales and high computational demands. Existing structured pruning and model compression techniques are mostly designed for natural images or general scenarios, failing to fully consider the characteristics of visual degradation and high-density crowds in railway stations. This can easily lead to problems such as decreased detection accuracy and deteriorated temporal stability after compression.
[0005] Furthermore, in terms of geometric calibration and physical quantity calculation, railway stations widely use fisheye or ultra-wide-angle cameras to cover a larger monitoring area. Their nonlinear imaging distortion makes it difficult for coordinate transformation methods based on simple planar homography mapping to accurately establish the correspondence between pixel coordinates and ground physical coordinates. This results in large calculation errors for key safety indicators such as crowd density, walking speed, and traffic flow, affecting the reliability of subsequent safety situation analysis.
[0006] Furthermore, regarding privacy and compliance, traditional cloud-based centralized visual analysis solutions require the transmission of raw video streams, which not only consumes significant network bandwidth but also poses a potential risk of personnel identity information leakage. This makes it difficult to meet the management requirements of railway stations for minimized data transmission and privacy compliance. Among existing image or video recognition and understanding technologies, end-to-end, privacy-friendly monitoring methods for railway station scenarios still require improvement. Summary of the Invention
[0007] The purpose of this invention is to provide a compressed and optimized edge visual crowd safety monitoring method for railway stations, which solves the problem of insufficient accuracy in existing railway station crowd monitoring.
[0008] The technical solution adopted in this invention is a compressed and optimized edge visual crowd safety monitoring method for railway stations, which is implemented according to the following steps: Step 1, Video input and hardware-accelerated decoding; Step 2, Degradation perception preprocessing; Step 3, head detection of multi-stage structured pruning; Step 4, multi-target tracking; Step 5: Fisheye calibration of ground projection; Step 6: Calculation and privacy-preserving output of crowd status indicators.
[0009] The invention is further characterized by: Step 1 specifically involves: accessing the wide-angle or fisheye CCTV video stream from the railway station, decoding it using an NVDEC or V4L2 hardware acceleration decoding module, and outputting original video frames adapted to the processing capabilities of edge devices.
[0010] In step 2, to quantify the visual degradation degree of video frames in the complex environment of railway stations, a frame-level quality measurement model based on gradient and information entropy is introduced. First, the Laplacian operator is used to calculate the image blur degree, and its gradient variance is defined as: (1) in, This represents the input video frame image. Represents the Laplace operator. This represents the statistical variance of the Laplacian response value over the entire image space. A smaller value indicates a higher degree of image blur. Simultaneously, the global information entropy of the image is calculated to measure the richness of detail, and it is defined as: (2) in, The probability of a pixel's grayscale value being k is represented by this value. Combining the aforementioned blurriness and entropy values, a frame-level visibility score is defined. for: (3) in, These are weighting coefficients used to balance the contributions of ambiguity and detailed information. This represents the maximum reference value of the Laplace gradient variance under given system settings and data distribution conditions, used to normalize ambiguity indices. This represents the maximum possible value of image information entropy under the same conditions, and is used to normalize the information entropy index.
[0011] In step 3, a head detection model is built based on a lightweight object detection network. Domain fine-tuning is performed for high-density occlusion and fisheye distortion scenarios in railway stations. Model compression optimization is achieved through multi-stage structured pruning.
[0012] Step 3 involves multi-stage structured pruning to achieve model compression optimization, specifically as follows: Step 3.1, Hybrid Regularization Bootstrapping During model training, a hybrid nonconvex sparse constraint consisting of transformation L1 regularization and differentiable Lq regularization is introduced, and its regularization loss term is defined as: (4) in, The regularization intensity coefficient is . The transformed L1 or differentiable Lq regularization function. The scaling parameter corresponding to the channel in the Batch Normalization layer is used to apply sparse constraints to the low importance channel through formula (4) to guide the network to generate structured redundancy; Step 3.2, Channel Importance Measurement For the network Each channel has a structural importance score, defined as the absolute value of the corresponding BatchNormalization scaling parameter: (5) in: Indicates the first The structural importance score of each channel.
[0013] This reflects the degree to which the channel contributes to the network output; the smaller the value, the weaker the influence of the channel on feature representation. Step 3.3, Determining the Multi-Stage Pruning Threshold In the Each pruning phase involves setting a set of structural importance scores for all pruningable channels in the network. Perform a global sort and determine the pruning threshold based on the preset stage pruning ratio. Its definition is: (6) in, Describes the quantile operator. Indicates the first The target structured pruning ratio for each stage, with a value range of [value range missing]. ; Through gradual increments or adaptive adjustments This enables a multi-stage, gradual model compression process, thereby avoiding performance degradation caused by a one-time large-scale pruning. Step 3.4, Channel Pruning Criteria and Structured Pruning Operators Based on pruning threshold , define the first The channel pruning criterion function for each stage is as follows: (7) in, Indicates the first The first channel in the The stage was identified as a low-importance channel and pruning was performed; This indicates that the channel is reserved for the current stage; when At the same time, the convolution kernels, feature maps, and their connections in subsequent network layers of the corresponding channels are all removed in a structured manner, thereby achieving the physical removal of redundant channels while maintaining the consistency of network topology. Step 3.5, Explanation of Multi-Stage Pruning and Model Stability By repeatedly performing the above channel importance assessment and pruning criterion calculation process in multiple training, pruning, and fine-tuning stages, the model structure is gradually compressed, so that the network can significantly reduce the parameter size and computational complexity while ensuring head detection accuracy and geometric consistency. Step 3.6, Domain Adaptation Fine-tuning The pruned model was fine-tuned using a dataset containing visual features of railway stations to recover the accuracy loss caused by pruning. Step 3.7, Memory Layout Reorganization and Inference Optimization The pruned model is restructured using tensors and adapted to TensorRT or ONNX Runtime inference acceleration frameworks to reduce inference latency and improve execution efficiency on edge devices.
[0014] In step 4, during the multi-target tracking stage, a state prediction model based on Kalman filtering is adopted, and its state transition equation is: (8) in, Indicates the target at time. The state vector, Here is the state transition matrix. For process noise, Indicates the target at time. The state vector of the previous moment; The data association cost between the detection results and the predicted trajectory is defined as: (9) Where IoU represents the intersection-union ratio of bounding boxes. To test the confidence level, These are the weighting coefficients.
[0015] Step 5 employs an isometric or polynomial fisheye imaging model. Camera intrinsic and extrinsic parameters and distortion coefficients are obtained through a calibration plate. A precise mapping relationship between pixel coordinates and ground physical coordinates is established, achieving high-precision positioning of the head target on the ground plane. This ensures that calculation errors for physical quantities such as velocity and density are within a controllable range. The forward projection model of the fisheye camera is defined as follows: (10) in, For pixel coordinates, As a world coordinate system, For the camera intrinsic parameter matrix, and These are the rotation matrix and the translation vector, respectively. For fisheye distortion parameters, For isometric or polynomial fisheye distortion mapping functions; The corresponding inverse projection model is: (11).
[0016] Step 6 calculates core safety indicators such as crowd density, walking speed, directional flow, and area occupancy rate based on ground physical coordinates and target trajectory. Only non-identifiable numerical metadata is output; original video frames or identifiable target information are not transmitted to meet the privacy protection and data governance requirements of railway stations. Population density was calculated using kernel density estimation. (12) in, For the first Ground coordinates of the head target For core bandwidth, is a Gaussian kernel function, and N represents the total number of head targets successfully detected and projected onto the ground plane within the current time frame or time window; Walking speed is defined as: (13) in, Indicates the target at time. Ground plane coordinates, Indicates the same target at the next moment. Ground plane coordinates, This represents the time interval between two adjacent frames, determined by the video frame rate.
[0017] In step 6, the following content will be used as the rule for determining the safe time: Crowding incidents: Density ≥ 5 people / m² 2 And the average speed is ≤0.3m / s, lasting for 5 frames; Stasis event: Single target speed ≤ 0.1m / s, lasting 8 frames; Restricted Area Invasion Event: The target enters the restricted area and remains there for 2 frames; Traffic anomaly event: The fluctuation of targeted traffic within a unit of time exceeds 50% of the historical average; Alarm suppression: All alarms must undergo 10 frames of continuous verification.
[0018] The beneficial effects of this invention are: 1) Significantly improved detection accuracy and temporal stability: This invention addresses complex scenarios such as high-density occlusion, fisheye distortion, and low light conditions in railway stations by employing a joint modeling method based on head target detection and multi-target tracking. The average head detection accuracy (mAP@0.5) reaches 83.5%, maintaining stable detection performance even under high-density occlusion, low light, and motion blur scenarios. In the multi-target tracking stage, a joint correlation strategy based on detection results and motion prediction enables the comprehensive tracking index (MOTA) to reach 79.1%, significantly reducing the number of identity switching, thereby effectively improving the continuity and temporal consistency of the target trajectory. 2) Significant model compression effect and high deployment efficiency: By introducing a multi-stage structured pruning strategy that uses the Batch Normalization layer scaling parameter as a measure of channel importance, the model parameter size and computational complexity are compressed simultaneously without introducing additional model structure. While maintaining the detection accuracy basically unchanged, the model's dependence on computing power and storage resources is greatly reduced, enabling the method of this invention to run stably on edge computing devices with limited computing power, meeting the engineering requirements of real-time processing of multiple video streams in railway stations. 3) High accuracy in calculating geometric quantification indicators and reliable safety assessment: This invention combines precise calibration of fisheye cameras with a ground inverse projection model to achieve high-precision mapping of head targets from pixel coordinates to ground physical coordinates; based on ground physical coordinates, it calculates crowd density and walking speed, effectively suppressing the systematic errors caused by fisheye distortion to geometric quantification indicators; 4) Robust privacy protection mechanism, in line with practical application standards: This invention realizes end-to-end edge processing of video decoding, detection, tracking and crowd status analysis, outputting only non-identifying statistical results such as crowd density, speed, and flow, without transmitting original video frames, target bounding boxes or any image information that can be used for identification; this processing method effectively avoids leakage of personnel privacy information while ensuring the safety monitoring function of crowds in railway stations, and meets the practical application requirements of the railway industry for data minimization and privacy compliance. 5) Strong generalization ability and good engineering application prospects: The present invention exhibits stable performance under different density levels, different viewing angle configurations and complex lighting conditions. It can adapt to railway station environments with different structural layouts and camera deployment methods. It has strong cross-scene adaptability and engineering scalability, and is suitable for deployment and application of actual railway station crowd safety monitoring systems. Attached Figure Description
[0019] Figure 1 This is an overall architecture diagram of the present invention; Figure 2 This is a schematic diagram of the multi-stage structured pruning process in this invention; Figure 3 This is a schematic diagram of the coordinate transformation between fisheye camera calibration and ground projection in this invention; Figure 4 This is a comparison chart of the detection effects of the present invention in different scenarios at railway stations. Detailed Implementation
[0020] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0021] This invention presents a compressed and optimized edge visual crowd safety monitoring method for railway stations, with the overall architecture as follows: Figure 1 As shown, please follow these steps: Step 1, Video Input and Hardware Accelerated Decoding Video streams are acquired by deploying wide-angle or fisheye surveillance cameras in railway stations, and hardware-accelerated decoding is used to decode the video streams in real time to reduce system latency and reduce the computing load on edge devices. Step 2, Degradation Detection Preprocessing The decoded video frames are subjected to brightness normalization and low-light enhancement processing. Combined with a frame-level quality assessment method based on gradient and information entropy, the degree of image degradation is quantified to generate a visibility score, which is used to enhance the robustness of subsequent detection and tracking processes. Step 3, Head detection in multi-stage structured pruning A head detection model is built based on a lightweight object detection network, and domain fine-tuning is performed for high-density occlusion and fisheye distortion scenarios in railway stations. On this basis, a multi-stage structured pruning strategy is introduced. Through regularized modeling and channel importance assessment, the model structure is gradually compressed, reducing the model parameter scale and computational complexity while ensuring detection accuracy. Step 4, Multi-target tracking Based on head detection results and combined with visibility scoring, a detection-based multi-target tracking method is adopted to perform cross-frame association of head targets. The consistency of target identity is maintained through motion prediction and association cost constraints, reducing identity switching under occlusion and illumination changes. Step 5, fisheye calibration of ground projection The monitoring camera is calibrated using a fisheye camera imaging model to establish a mapping relationship between pixel coordinates and ground physical coordinates, thereby achieving high-precision positioning of the head target on the ground plane and providing a reliable geometric basis for subsequent physical quantity calculations. Step 6: Calculation of Crowd Status Indicators and Privacy-Enabled Output Based on the target ground coordinates and tracking trajectory, safety status indicators such as crowd density, walking speed, directional flow, and area occupancy rate are calculated. Only non-identifiable numerical metadata is output, without transmitting original video frames or identifiable image information, thereby meeting the requirements of railway stations for privacy protection and data compliance.
[0022] Example 1 A compression-optimized edge visual crowd safety monitoring method for railway stations, wherein step 1 specifically involves: accessing the wide-angle or fisheye CCTV video stream of the railway station, transmitting the video stream via the RTSP protocol, and using NVDEC or V4L2 accelerated decoding modules for decoding to reduce CPU usage and ensure the real-time performance of the video stream input.
[0023] Assume the input video stream frame rate is The output frame rate after decoding is Then we have: (1) in, The maximum processing frame rate that the edge device can stably support is set to 30 FPS. The default resolution of the decoded output video frame is 1920×1080 pixels, and it can adaptively downsample according to the device's computing power.
[0024] Example 2 A compressed and optimized edge visual crowd safety monitoring method for railway stations, wherein in step 2, to quantify the visual degradation degree of video frames in the complex environment of railway stations, a frame-level quality measurement model based on gradient and information entropy is introduced. First, the Laplacian operator is used to calculate the image blur degree, and its gradient variance is defined as: (2) in, This represents the input video frame image. Represents the Laplace operator. This represents the statistical variance of the Laplacian response value over the entire image space. A smaller value indicates a higher degree of image blur. Simultaneously, the global information entropy of the image is calculated to measure the richness of detail, and it is defined as: (3) in, The probability of a pixel's grayscale value being k is represented by this value. Combining the aforementioned blurriness and entropy values, a frame-level visibility score is defined. for: (4) in, These are weighting coefficients used to balance the contributions of ambiguity and detailed information. This represents the maximum reference value of the Laplace gradient variance under given system settings and data distribution conditions, used to normalize ambiguity indices. This represents the maximum possible value of image information entropy under the same conditions, and is used to normalize the information entropy index. The visibility score is used to guide the temporal smoothing strategy in the subsequent multi-target tracking stage, improving the stable perception capability of distant head targets and targets in occluded scenes while ensuring computational efficiency.
[0025] Example 3 A compressed and optimized edge-vision crowd safety monitoring method for railway stations, in step 3, constructs a head detection model based on a lightweight object detection network, performs domain fine-tuning for high-density occlusion and fisheye distortion scenarios in railway stations, and achieves model compression and optimization through multi-stage structured pruning, specifically: Step 3.1, Hybrid Regularization Bootstrapping During model training, a hybrid nonconvex sparse constraint consisting of transformation L1 regularization and differentiable Lq regularization is introduced, and its regularization loss term is defined as: (5) in, The regularization intensity coefficient is . The transformed L1 or differentiable Lq regularization function. The scaling parameter corresponding to this channel in the Batch Normalization layer is used to apply sparse constraints to low-importance channels using the above formula, guiding the network to generate structured redundancy. Step 3.2, Channel Importance Measurement For the network Each channel has a structural importance score, defined as the absolute value of the corresponding BatchNormalization scaling parameter: (6) in: Indicates the first The structural importance score of each channel.
[0026] This reflects the degree to which the channel contributes to the network output; the smaller the value, the weaker the influence of the channel on feature representation. Step 3.3, Determining the Multi-Stage Pruning Threshold In the Each pruning phase involves setting a set of structural importance scores for all pruningable channels in the network. Perform a global sort and determine the pruning threshold based on the preset stage pruning ratio. Its definition is: (7) in, Describes the quantile operator. Indicates the first The target structured pruning ratio for each stage, with a value range of [value range missing]. ; Through gradual increments or adaptive adjustments This enables a multi-stage, gradual model compression process, thereby avoiding performance degradation caused by a one-time large-scale pruning. Step 3.4, Channel Pruning Criteria and Structured Pruning Operators Based on pruning threshold , define the first The channel pruning criterion function for each stage is as follows: (8) in, Indicates the first The first channel in the The stage was identified as a low-importance channel and pruning was performed; This indicates that the channel is reserved for the current stage; when At this time, the convolutional kernels, feature maps, and their connections in subsequent network layers for the corresponding channels are all subjected to structured removal operations. This achieves the physical removal of redundant channels while maintaining network topology consistency. Figure 2 As shown; Step 3.5, Explanation of Multi-Stage Pruning and Model Stability By repeatedly performing the above channel importance assessment and pruning criterion calculation process in multiple training, pruning, and fine-tuning stages, the model structure is gradually compressed, so that the network can significantly reduce the parameter size and computational complexity while ensuring head detection accuracy and geometric consistency. Step 3.6, Domain Adaptation Fine-tuning The pruned model was fine-tuned using a dataset containing visual features of railway stations to recover the accuracy loss caused by pruning. Step 3.7, Memory Layout Reorganization and Inference Optimization The pruned model is restructured using tensors and adapted to TensorRT or ONNX Runtime inference acceleration frameworks to reduce inference latency and improve execution efficiency on edge devices.
[0027] Example 4 A compressed-optimized edge visual crowd safety monitoring method for railway stations, wherein in step 4, during the multi-target tracking stage, a state prediction model based on Kalman filtering is adopted, and its state transition equation is: (9) in, Indicates the target at time. The state vector, Here is the state transition matrix. For process noise, Indicates the target at time. The state vector of the previous moment; The data association cost between the detection results and the predicted trajectory is defined as: (10) Where IoU represents the intersection-union ratio of bounding boxes. To test the confidence level, These are weighting coefficients. This ensures the continuity of the target's identity even in complex scenarios such as occlusion and changes in lighting.
[0028] Example 5 A compressed-optimized edge-vision crowd safety monitoring method for railway stations, wherein step 5 employs an isometric or polynomial fisheye imaging model, obtains camera intrinsic and extrinsic parameters and distortion coefficients through a calibration plate, establishes a precise mapping relationship from pixel coordinates to ground physical coordinates, achieves high-precision positioning of head targets on the ground plane, and ensures that the calculation errors of physical quantities such as velocity and density are within a controllable range. The forward projection model of the fisheye camera is defined as follows: (11) in, For pixel coordinates, As a world coordinate system, For the camera intrinsic parameter matrix, and These are the rotation matrix and the translation vector, respectively. For fisheye distortion parameters, For isometric or polynomial fisheye distortion mapping functions; The corresponding inverse projection model is: (12) By utilizing the aforementioned forward and backward projection relationships, a high-precision transformation from the center pixel coordinates of the head detection box to the physical coordinates on the ground is achieved, providing a reliable geometric basis for subsequent calculations of crowd state indicators, such as... Figure 3 As shown.
[0029] Example 6 A compressed and optimized edge visual crowd safety monitoring method for railway stations includes a step 6 method that calculates core safety indicators such as crowd density, walking speed, directional flow, and area occupancy rate based on ground physical coordinates and target trajectories. Only non-identifiable numerical metadata is output, without transmitting original video frames or identifiable target information, to meet the privacy protection and data governance requirements of railway stations. Population density was calculated using kernel density estimation. (13) in, For the first Ground coordinates of the head target For the kernel bandwidth, h = 0.5m. The kernel function is Gaussian, and N represents the total number of head targets successfully detected and projected onto the ground plane within the current time frame or time window. This method can obtain the spatial distribution characteristics of crowds in different areas of a railway station without relying on any individual identity information. Walking speed is calculated based on the change in ground physical coordinates of the same head target in adjacent video frames, and is defined as: (14) in, This represents the time interval between two adjacent frames, determined by the video frame rate, Δt = 1 / 30s. Indicates the target at time. Ground plane coordinates, Indicates the same target at the next moment. The ground plane coordinates are used to perform a moving average on the velocity values of multiple consecutive frames (e.g., 3 frames) in practical applications to suppress detection noise and improve the stability of velocity estimation. Targeted traffic flow calculation: Virtual gate areas are set up at ticket gates, stairwells, or passageway entrances in railway stations. These virtual gates can be in the form of line segments or rectangular areas. The number of people crossing these virtual gates per unit time (1 minute) is counted, and the inbound and outbound traffic flows are distinguished based on the direction of passage, thus obtaining targeted traffic flow information for the corresponding locations. Area occupancy rate calculation: Safety monitoring zones are pre-defined within railway stations, including but not limited to platform edge restricted areas, equipment maintenance areas, or emergency evacuation routes. The number of persistent target trajectories within each zone is counted, and this number is compared to the maximum capacity allowed in that zone to calculate the area occupancy rate. This rate is used to help determine the level of congestion and potential safety risks within the area.
[0030] The privacy-preserving output is as follows: all population status indicators generated in step 6 are output only in the form of non-identifiable numerical metadata. The metadata includes at least: area identifier, timestamp, population density value, walking speed statistics, directional traffic value, area occupancy rate, and corresponding security event type.
[0031] Throughout the entire processing, no raw video frames, target bounding boxes, or any image information that could be used to identify individuals are output, stored, or transmitted. This ensures that while meeting the safety monitoring needs of crowds at railway stations, it fully complies with privacy protection and data governance regulations. The detection effectiveness varies across different scenarios within railway stations. Figure 4As shown in the figure, several typical scenarios from railway station surveillance videos are selected, including scenarios of walking in passageways, high-density gatherings at ticket gates, and station hall scenarios under low light or nighttime conditions. Each row in the figure corresponds to a different monitoring environment, and each column sequentially displays the uncompressed YOLOv8n detection results, the structured and pruned YOLOv8n-Pruned detection results, and the detection results of the method of this invention. The detection box represents the head target position identified by the system, and the arrows in the figure are used to mark the positions where detection is missed or unstable. The comparison results show that under conditions of high-density occlusion, crowd gathering, and complex lighting, the uncompressed YOLOv8n model has some missed detections of distant small-scale targets and locally occluded areas. While the YOLOv8n-Pruned model achieves improved inference efficiency after compression, its detection stability in occluded or low-contrast areas decreases in some scenarios. In contrast, the compressed and optimized edge-visual crowd safety monitoring method proposed in this invention maintains relatively stable detection results under different density levels and lighting conditions. It can still detect a significant number of head targets in high-density crowd areas, distant small targets, and low-light environments, resulting in a more continuous spatial distribution of detection results. This demonstrates that the method of this invention achieves improved model compression and edge deployment efficiency while maintaining good detection accuracy and stability in complex railway station scenarios.
[0032] Example 7 Hardware deployment environment The monitoring method of this invention is deployed on an edge computing platform. The core hardware uses an NVIDIA Jetson XavierNX embedded device, which is equipped with 8GB LPDDR4X memory and 512 CUDA cores, and supports the TensorRT 8.6 inference acceleration framework and NVDEC hardware video decoding module.
[0033] The railway station monitoring terminal uses fisheye or ultra-wide-angle CCTV cameras with a field of view of no less than 120° and a video resolution of 1920×1080 pixels. The cameras are connected to the edge devices via gigabit Ethernet, and the video stream is transmitted using the RTSP protocol, with an end-to-end transmission latency of no more than 10ms.
[0034] Dataset processing and model training settings Dataset preparation: Training and validation data are constructed using the following public and private datasets: RPEE-Heads: A dataset specifically for railway station scenes, containing 18,000 images and 300,000 header annotations; SCUT-HEAD: A general head detection dataset containing 111,000 head instances; NWPU-Crowd: A high-density crowd dataset containing 5,109 images and 2.13 million head point annotations.
[0035] To enhance the model's adaptability to complex railway station scenes, the following augmentation operations were performed on the training data: fisheye distortion simulation; illumination intensity adjustment, with brightness ranging from 0.3 to 1.5 times that of the original image; occlusion mask generation, with occlusion ratios from 0% to 60%; and geometric transformations, including rotation ±15° and scaling from 0.8 to 1.2 times. The training and validation sets were divided in an 8:2 ratio. Basic Model Training: A basic head detection model was built based on YOLOv8n, with the following training settings: Optimizer: AdamW; Initial learning rate: 1×10⁻⁶. -3 Learning rate decay strategy: cosine annealing; Number of training epochs: 700; Batch size: 16; Loss function: a weighted combination of CIoU loss and cross-entropy loss; Multi-stage structured pruning parameter settings: After the basic model training is completed, a multi-stage structured pruning mechanism is introduced. The relevant parameter settings are as follows: Regularization intensity: λ=1×10 -3 Target structured pruning ratio: θ≈62%; Total number of pruning-fine-tuning iterations: 750; Learning rate adjusted to 1×10⁻⁶ during the domain adaptation fine-tuning phase after pruning. -4 The number of fine-tuning rounds is 300. During the pruning process, channel importance is evaluated based on the scaling parameters of the Batch Normalization layer, and structured channel removal is performed in stages to ensure model stability. Fisheye calibration process: A 12×9 checkerboard calibration board (each grid 20mm) was used to collect no fewer than 25 sets of calibration images at different poses and distances in the actual monitoring area of the railway station. The camera intrinsic matrix K, distortion coefficients d, and extrinsic parameters (R,t) were solved using OpenCV's fisheye module. After calibration, the camera reprojection error was controlled within 0.5 pixels.
[0036] Method execution process Step 1: The edge device receives the RTSP video stream output by the camera through the NVDEC hardware decoding module. The decoding frame rate is fixed at 30FPS, and the output is a BGR format image with a resolution of 1920×1080. Step 2: Adaptive histogram equalization (CLAHE, cropping limit of 0.02) is first performed on the decoded image, followed by enhancement of low-light region details using the Retinex algorithm. Then, the Laplacian gradient variance and image information entropy are calculated and fused to generate a frame-level visibility score, ranging from 0 to 1. When the visibility score is below 0.3, the frame is marked as a low-quality frame, and its temporal smoothing weight is increased in subsequent tracking processes. Step 3: Input the preprocessed image into the pruned and optimized YOLO head detection model, and output the bounding box coordinates (x1, y1, x2, y2) and confidence score of the head target. The detection confidence threshold is set to 0.5. Redundant detection boxes are removed by non-maximum suppression (NMS, IoU threshold 0.45). Step 4: Use the ByteTrack multi-target tracking algorithm combined with Kalman filtering to predict the target state. The process noise covariance is set to [1e...]. - ², 1e - ², 1e - ², 1e - ², 1e -4 , 1e -4 The observation noise covariance is set to [1e]. - ¹, 1e - ¹, 1e - ¹, 1e - ¹, 1e - ³, 1e - ³]. The trajectory interruption tolerance time is set to 3 frames; Step 5: Using the pre-calibrated camera parameters, perform fisheye inverse projection calculation on the center pixel coordinates (u,v) of the head detection box to obtain the ground physical coordinates (X,Y). The coordinate unit is meters, and the projection error is controlled within ±0.1m. Step 6: Population density calculation: Using a 5×5m area as the statistical unit, the kernel density estimation method was used for calculation, with the density threshold set at 5 people / m². 2 ; Walking speed calculation: Instantaneous speed is calculated based on the Euclidean distance of ground coordinates in consecutive frames, and smoothed by a 3-frame moving average; Targeted flow calculation: Set up a virtual gate with a length of 2m at the ticket gate, and count the number and direction of targets passing through per unit time; Area occupancy rate calculation: based on the maximum capacity of the restricted area (2 people / m²). 2 Based on calculations, an alert is triggered when the occupancy rate is not lower than 80%. All calculation results are output in the form of non-identifiable numerical metadata in JSON format and transmitted to the monitoring center at a frequency of 1Hz via the MQTT protocol. No original video or recognizable image information is transmitted.
[0037] Security Incident Judgment Rules Crowding incidents: Density ≥ 5 people / m² 2 And the average speed is ≤0.3m / s, lasting for 5 frames; Stasis event: Single target speed ≤ 0.1m / s, lasting 8 frames; Restricted Area Invasion Event: The target enters the restricted area and remains there for 2 frames; Traffic anomaly event: The fluctuation of targeted traffic within a unit of time exceeds 50% of the historical average; Alarm suppression: All alarms must undergo 10 frames of continuous verification to avoid false alarms caused by momentary interference, and the false alarm rate is controlled within 0.5%.
[0038] All alarms must undergo continuous verification for no less than 10 frames, and the false alarm rate must be controlled within 0.5%.
[0039] Example 8 To verify the effectiveness of the proposed compressed and optimized edge visual crowd safety monitoring method for railway stations in practical applications, a comparative experiment was conducted between the proposed method and traditional crowd monitoring methods based on real railway station surveillance video data, under the same dataset, fisheye camera parameters, and edge computing hardware environment.
[0040] 1. Comparison methods and experimental conditions The comparison methods include: 1) Traditional uncompressed detection method: The target detection model is deployed directly on edge devices without structured pruning; 2) Traditional lightweight or one-time pruning methods: a compressed detection model that does not incorporate multi-stage structured pruning and degradation sensing mechanisms.
[0041] All methods were tested on the same railway station crowd dataset, with test videos at a resolution of 1920×1080 and a frame rate of 30FPS. The test platform was the same model of edge computing device.
[0042] 2. Comparison of detection accuracy and timing stability In high-density crowd scenarios at railway stations: The head detection mAP@0.5 of the method of this invention reaches about 83%, which is an improvement of about 4% to 7% compared with the traditional one-time pruning model; In low-light and motion-blurred scenes, the detection accuracy of the method of this invention is controlled to decrease by less than 5%, while the accuracy of traditional methods can decrease by more than 10% in the same scene. In multi-target tracking tasks, the MOTA index of the method of this invention reaches about 79%, and the number of identity switching is reduced by about 20% to 30% compared with traditional methods.
[0043] The above results demonstrate that the method of the present invention has higher detection stability and timing consistency in complex railway station environments.
[0044] 3. Comparison of model compression effect and inference efficiency Under the same edge computing platform conditions, the model size and inference efficiency were tested, and the results show: Through multi-stage structured pruning, the method of this invention reduces the number of model parameters by approximately 58% and the computational cost (FLOPs) by approximately 60%. Under 1080P input conditions, the single-frame inference latency of the method of the present invention is stable within the range of 30 to 35 ms, which can meet the requirements of real-time processing. Compared to traditional unpruned models, this invention improves inference speed by more than 1.5 times under the same hardware conditions and can support parallel processing of multiple cameras.
[0045] Experimental results show that the method of the present invention has significant deployment advantages on edge devices with limited computing power.
[0046] 4. Comparison of the accuracy of geometric index calculations A comparative analysis of the geometric calculation errors of different methods was conducted for key safety indicators such as crowd density and walking speed. The results show that: This invention uses a fisheye imaging model and ground projection calibration to control the average error of crowd density calculation within the range of approximately 3% to 5%. The average error of walking speed estimation is about 0.1 m / s, which is significantly lower than that of the traditional scheme using a simple planar mapping method; In high-density and edge regions, the method of the present invention can effectively reduce systematic deviations caused by fisheye distortion.
[0047] The above results demonstrate that the present invention has higher accuracy and reliability in calculating geometric indices.
[0048] 5. Comprehensive Comparison Conclusion The results of comprehensive testing accuracy, operational efficiency, and geometric index calculations show that: This invention achieves significant model compression and inference acceleration while maintaining high detection accuracy; Compared with traditional methods, this invention demonstrates stronger stability and engineering applicability in complex railway station scenarios; This method can provide reliable data support for safety monitoring and risk warning of people in railway stations while meeting the requirements of real-time performance and privacy protection.
[0049] Therefore, the compressed and optimized edge visual crowd safety monitoring method for railway stations proposed in this invention has significant technical advantages and application value in practical railway station applications.
[0050] This invention provides a compressed and optimized edge visual crowd safety monitoring method for railway stations. It addresses the common problems of insufficient detection accuracy, poor temporal stability, large geometric calibration errors, and insufficient privacy compliance in existing railway station crowd monitoring systems under conditions of high-density occlusion, fisheye distortion, and limited edge computing power. The method achieves synergistic optimization of detection accuracy, operational efficiency, and privacy security.
Claims
1. A compressed and optimized edge visual crowd safety monitoring method for railway stations, characterized in that, The specific steps are as follows: Step 1, Video input and hardware-accelerated decoding; Step 2, Degradation perception preprocessing; Step 3, head detection of multi-stage structured pruning; Step 4, multi-target tracking; Step 5: Fisheye calibration of ground projection; Step 6: Calculation and privacy-preserving output of crowd status indicators.
2. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 1, characterized in that, Step 1 specifically involves: accessing the wide-angle or fisheye CCTV video stream from the railway station, decoding it using an NVDEC or V4L2 hardware acceleration decoding module, and outputting original video frames adapted to the processing capabilities of edge devices.
3. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 1, characterized in that, In step 2, to quantify the visual degradation degree of video frames in the complex environment of railway stations, a frame-level quality measurement model based on gradient and information entropy is introduced. First, the Laplacian operator is used to calculate the image blur degree, and its gradient variance is defined as: （1） in, This represents the input video frame image. Represents the Laplace operator. This represents the statistical variance of the Laplacian response value over the entire image space. A smaller value indicates a higher degree of image blur. Simultaneously, the global information entropy of the image is calculated to measure the richness of detail, and it is defined as: （2） in, The probability of a pixel's grayscale value being k is represented by this value. Combining the aforementioned blurriness and entropy values, a frame-level visibility score is defined. for: （3） in, These are weighting coefficients used to balance the contributions of ambiguity and detailed information. This represents the maximum reference value of the Laplace gradient variance under given system settings and data distribution conditions, used to normalize ambiguity indices. This represents the maximum possible value of image information entropy under the same conditions, and is used to normalize the information entropy index.
4. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 1, characterized in that, In step 3, a head detection model is constructed based on a lightweight object detection network. Domain fine-tuning is performed for high-density occlusion and fisheye distortion scenarios in railway stations. Model compression optimization is achieved through multi-stage structured pruning.
5. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 4, characterized in that, Step 3 involves multi-stage structured pruning to achieve model compression optimization, specifically as follows: Step 3.1, Hybrid Regularization Bootstrapping During model training, a hybrid nonconvex sparse constraint consisting of transform L1 regularization and differentiable Lq regularization is introduced, and its regularization loss term is defined as: （4） in, The regularization intensity coefficient is . The transformed L1 or differentiable Lq regularization function. The scaling parameter corresponding to the channel in the Batch Normalization layer is used to apply sparse constraints to the low importance channel through formula (4) to guide the network to generate structured redundancy; Step 3.2, Channel Importance Measurement For the network For each channel, its structural importance score is defined as the absolute value of the Batch Normalization scaling parameter corresponding to that channel: （5） in: Indicates the first Structural importance score for each channel; This reflects the degree to which the channel contributes to the network output; the smaller the value, the weaker the influence of the channel on feature representation. Step 3.3, Determining the Multi-Stage Pruning Threshold In the Each pruning phase involves setting a set of structural importance scores for all pruningable channels in the network. Perform a global sort and determine the pruning threshold based on the preset stage pruning ratio. Its definition is: （6） in, Represents the quantile operator. Indicates the first The target structured pruning ratio for each stage, with a value range of [value range missing]. ; Through gradual increments or adaptive adjustments This enables a multi-stage, gradual model compression process, thereby avoiding performance degradation caused by a one-time large-scale pruning. Step 3.4, Channel Pruning Criteria and Structured Pruning Operators Based on pruning threshold , define the first The channel pruning criterion function for each stage is as follows: （7） in, Indicates the first The first channel in the The stage was identified as a low-importance channel and pruning was performed; This indicates that the channel is reserved for the current stage; when At the same time, the convolution kernels, feature maps, and their connections in subsequent network layers of the corresponding channels are all removed in a structured manner, thereby achieving the physical removal of redundant channels while maintaining the consistency of network topology. Step 3.5, Explanation of Multi-stage Pruning and Model Stability By repeatedly performing the above channel importance assessment and pruning criterion calculation process in multiple training, pruning, and fine-tuning stages, the model structure is gradually compressed, so that the network can significantly reduce the parameter size and computational complexity while ensuring head detection accuracy and geometric consistency. Step 3.6, Domain Adaptation Fine-tuning The pruned model was fine-tuned and trained using a dataset containing visual features of railway stations to recover the accuracy loss caused by pruning. Step 3.7, Memory Layout Reorganization and Inference Optimization The pruned model is restructured using tensors and adapted to TensorRT or ONNX Runtime inference acceleration frameworks to reduce inference latency and improve execution efficiency on edge devices.
6. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 1, characterized in that, In step 4, during the multi-target tracking stage, a state prediction model based on Kalman filtering is adopted, and its state transition equation is: （8） in, Indicates the target at time. The state vector, Here is the state transition matrix. For process noise, Indicates the target at time. The state vector of the previous moment; The data association cost between the detection results and the predicted trajectory is defined as: （9） Where IoU represents the intersection-union ratio of bounding boxes. To test the confidence level, These are the weighting coefficients.
7. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 1, characterized in that, In step 5, an isometric or polynomial fisheye imaging model is used. Camera intrinsic and extrinsic parameters and distortion coefficients are obtained through a calibration plate. A precise mapping relationship between pixel coordinates and ground physical coordinates is established to achieve high-precision positioning of the head target on the ground plane. This ensures that calculation errors for physical quantities such as velocity and density are within a controllable range. The forward projection model of the fisheye camera is defined as follows: （10） in, For pixel coordinates, As a world coordinate system, For the camera intrinsic parameter matrix, and These are the rotation matrix and the translation vector, respectively. For fisheye distortion parameters, For isometric or polynomial fisheye distortion mapping functions; The corresponding inverse projection model is: （11）。 8. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 1, characterized in that, In step 6, based on ground physical coordinates and target trajectory, core safety indicators such as crowd density, walking speed, directional flow, and area occupancy rate are calculated. Only non-identifiable numerical metadata is output, without transmitting original video frames or identifiable target information, to meet the privacy protection and data governance requirements of railway stations. Population density was calculated using kernel density estimation. （12） in, For the first Ground coordinates of the head target For core bandwidth, is a Gaussian kernel function, and N represents the total number of head targets successfully detected and projected onto the ground plane within the current time frame or time window; Walking speed is defined as: （13） in, Indicates the target at time. Ground plane coordinates, Indicates the same target at the next moment. Ground plane coordinates, This represents the time interval between two adjacent frames, determined by the video frame rate.
9. The compressed-optimized edge visual crowd safety monitoring method for railway stations according to claim 8, characterized in that, In step 6, the following content will be used as the rule for determining the safe time: Crowding incidents: Density ≥ 5 people / m² 2 And the average speed is ≤0.3m / s, lasting for 5 frames; Stasis event: Single target speed ≤ 0.1m / s, lasting 8 frames; Restricted Area Invasion Event: The target enters the restricted area and remains there for 2 frames; Traffic anomaly event: The fluctuation of targeted traffic within a unit of time exceeds 50% of the historical average; Alarm suppression: All alarms must undergo 10 frames of continuous verification.