Rail transit hub video analysis system and method based on lightweight vision

Abstract

The application discloses a rail transit hub video analysis system and method based on lightweight vision, and belongs to the technical field of video intelligent analysis. The system constructs a shared backbone and a multi-task head network through knowledge distillation, dynamically adapts the head network activation and reasoning depth according to the edge device computing power by using a double-layer dynamic gate network, realizes unified modeling of multiple scenes and self-adaptive deployment of computing power, and realizes cross-layer continuous positioning by combining BIM partition calibration in the application service layer. The application is suitable for passenger flow monitoring, safety early warning, behavior analysis and equipment operation and maintenance of the rail transit hub.

Description

Technical Field

[0001] This invention belongs to the technical field of intelligent video analysis, and in particular, it is a video analysis system and method for rail transit hubs based on lightweight vision. Background Technology

[0002] As core nodes in urban transportation networks, rail transit hubs typically handle hundreds of thousands of passengers daily, with peak hours seeing 3-5 times the usual volume. Equipped with hundreds of high-definition cameras, they generate massive amounts of video data. Traditional manual monitoring methods struggle to detect all anomalies in real time. While artificial intelligence technology has led to the application of computer vision-based intelligent video analysis in rail transit scenarios, three major technical challenges remain.

[0003] First, it is difficult to balance the contradiction between model accuracy and real-time performance. The scenarios in rail transit hubs are complex and ever-changing, requiring video analysis models to have high generalization capabilities. While using lightweight models can improve speed, accuracy is significantly reduced. Traditional cloud-based centralized processing requires uploading video streams to backend servers. Affected by network transmission latency (usually 200-500ms), end-to-end event response latency often exceeds 1 second, which cannot meet the timeliness requirements of safety warnings.

[0004] Second, the system suffers from poor versatility across multiple scenarios and high adaptation costs for new scenarios. Rail transit hubs have diverse application scenarios, including dozens of sub-scenarios such as passenger flow monitoring, intrusion detection, fall detection, and abandoned object detection. Traditional methods train independent models for each scenario. Assuming a station covers 20 scenarios and deploys 10 cameras, this requires 200 model instances, making model uploading, updating, and scheduling management complex. For new scenarios or new stations, a large amount of labeled data and long training time are often required, with adaptation cycles lasting several weeks.

[0005] Third, edge computing resources are limited, making model deployment difficult. Front-end devices have limited computing resources; the computing power of domestically produced AI chips built into cameras is typically 4-8 TOPS, while edge gateways have a computing power of approximately 16-64 TOPS, and are also limited by power consumption, heat dissipation, and size. How to achieve high-precision multi-scene analysis under limited computing resources is the main obstacle to technical implementation.

[0006] Current video intelligent analysis technologies mainly adopt a single-scene independent model design approach. Existing patent CN111914937A proposes a lightweight improved target detection method, which involves lightweight model design but does not address a multi-task adaptive head network architecture. CN111144209B proposes a head detection method for surveillance videos based on heterogeneous multi-branch deep convolutional neural networks, which involves surveillance video analysis but does not employ knowledge distillation techniques for model compression. CN121585536A proposes a cloud-edge collaborative operation and maintenance system and method, which enables the deployment and application of video models at the edge, but does not disclose a multi-task architecture combining a shared backbone network and a scene-adaptive head network, nor does it disclose a dynamic task routing mechanism.

[0007] In summary, there is an urgent need for a video intelligent analysis technology that can achieve high-precision event detection, unified modeling across multiple scenarios, and efficient deployment on the edge, in order to improve the intelligent management level of rail transit hubs. Summary of the Invention

[0008] To address the aforementioned problems, this invention proposes a lightweight vision-based video analysis system and method for rail transit hubs, the specific solution of which is as follows:

[0009] A lightweight vision-based video analysis system for rail transit hubs includes:

[0010] The video acquisition layer is used to collect video stream data of the rail transit hub. The video stream data is collected by AI cameras deployed in various areas of the hub. The encoding format is H.265, the resolution is not less than 1920×1080, and the frame rate is 25fps.

[0011] The cloud-based training layer is used to train a lightweight visual model. Knowledge distillation is employed to transfer knowledge from the teacher model to the lightweight student model. The student model uses an improved version of MobileNetV3-Large as its shared backbone network. The improved version of MobileNetV3-Large is an improved lightweight backbone network formed by increasing the basic channel width to enhance feature dimensions, introducing a cross-stage feature fusion module to enhance multi-scale information interaction, and optimizing activation function configuration to reduce computational overhead, based on the original MobileNetV3-Large.

[0012] The edge inference layer deploys a multi-task architecture consisting of the shared backbone network and multiple scene-adaptive head networks. The scene-adaptive head networks include passenger flow monitoring, security early warning, behavior analysis, and equipment operation and maintenance head networks, each containing an independent decoupled detection head. The edge inference layer includes a two-layer dynamic gating network. The first layer is scene gating, which generates scene activation vectors based on the feature statistics of the current frame to control the sleep and wake-up of the head networks. The second layer is adaptive depth gating, which dynamically adjusts the inference depth of the head networks based on the feature L2 norm.

[0013] The application service layer provides corresponding scene detection capabilities based on the head network, including cross-layer continuous positioning based on a partitioning calibration strategy of a dynamic control point library.

[0014] The edge inference layer also includes a device computing power reading module and a model combination package management module; the device computing power reading module is used to obtain the AI ​​computing power level of the edge device.

[0015] The system adaptively downloads and activates a header network combination packet that matches the current computing power level. The computing power level is divided into three levels based on INT8 integer arithmetic capability: low computing power (<8 TOPS), medium computing power (8-64 TOPS), and high computing power (>64 TOPS).

[0016] The application service layer includes a BIM joint calibration module, which uses the BIM model to automatically extract structural key points of multiple elevation planes of the rail transit hub and establish a zoned control point library; for each camera coverage area, control points are selected from the zoned control point library for calibration, and when the detected target moves between different elevation planes, the corresponding mapping matrix is ​​automatically switched to achieve cross-layer continuous positioning.

[0017] The BIM joint calibration module is designed for multi-elevation, large-span spatial structures in rail transit hubs. It adopts a partitioned calibration strategy based on a dynamic control point library. After each elevation plane is independently calibrated, the homography matrix is ​​automatically switched by the target longitudinal position change rate to achieve continuous cross-layer positioning.

[0018] The BIM joint calibration module also introduces cross-layer consistency loss, which is used to constrain the spatial continuity of the ground trajectory point sequence obtained by mapping the corresponding matrices of the two planes when the target switches from the first elevation plane to the second elevation plane. This loss is used as a component of the joint loss function in the model fine-tuning stage, and participates in the optimization together with the detection loss and classification loss.

[0019] The edge inference layer also includes a cross-camera tracking module, which adopts a lightweight ReID feature extractor based on Ghost Bottleneck, introduces multi-scale feature fusion FPN, and adopts a joint training strategy of triple loss and ID loss. The cross-camera tracking module adds a channel attention mechanism and feature reuse branch to GhostNet (CVPR 2020) proposed by Huawei Noah's Ark Lab, forming an improved Ghost Bottleneck structure.

[0020] The safety warning head network and the passenger flow monitoring head network adopt a differentiated weighted Focal Loss design. The two head networks have opposite positive and negative sample emphasis directions, forming a complementary loss function pair. Among them, the loss function of the safety warning head network focuses on improving the recall rate, while the loss function of the passenger flow monitoring head network focuses on improving the precision rate.

[0021] The loss function of the safety warning head network introduces recall preference weighting on the basis of standard Focal Loss, w_rec : w_prec = 2 : 1; the loss function of the passenger flow monitoring head network adopts precision preference weighting, w_prec : w_rec = 2 : 1.

[0022] The cloud-based training layer employs a combination of feature-level distillation and logit-level distillation, with the total training loss function being a weighted sum of hard-label loss and soft-label loss.

[0023] This invention also provides a lightweight vision-based video analysis method for rail transit hubs. The application of the aforementioned lightweight vision-based rail transit hub video analysis system includes the following steps:

[0024] Step 1: Collect videos of typical scenes in rail transit hubs and construct a multi-label labeled dataset;

[0025] Step 2: Obtain a pre-trained teacher model, which is YOLOv8x trained on the Ascend MindSpore framework. A lightweight student model is then trained using this teacher model through knowledge distillation.

[0026] Step 3: Freeze the backbone network parameters of the trained student model and fine-tune only the parameters of the scene adaptation head network. The loss weights in its multi-task loss function satisfy λ1 + λ2 = 1.5, where λ1 = 1.0 and λ2 = 0.5.

[0027] Step 4: During deployment, dynamic task routing is executed. Based on the AI ​​computing power level of the edge device and the deployment location of the camera, the corresponding head network is automatically activated through a two-layer dynamic gating network and the inference depth is dynamically adjusted.

[0028] Step 5: Link the alarm information with the BIM model and use a partition calibration strategy to map the image coordinates to three-dimensional world coordinates.

[0029] Preferably, the step four of dynamically activating the corresponding head network based on the AI ​​computing power level of the edge device specifically involves: high-computing-power devices activating all scene head networks; medium-computing-power devices prioritizing the activation of security warning head networks and enabling adaptive depth gating; and low-computing-power devices prioritizing the activation of head networks with high alarm priority among the security warning head networks.

[0030] Preferably, in step three, the multi-task loss function in the fine-tuning stage includes detection loss, classification loss, and cross-layer consistency loss, wherein the sum of the weights of detection loss and classification loss is a fixed value of 1.5, and the weight of cross-layer consistency loss is 0.1.

[0031] The core beneficial effects of this invention include:

[0032] 1. A dual-layer dynamic gating network enables adaptive deployment based on computing power. The first layer, scene gating, generates activation vectors based on feature statistics to control the sleep and wake-up of the head network. The second layer, adaptive depth gating, dynamically adjusts the inference depth based on feature complexity. Working together, these two gating layers can reduce the average inference latency from 120ms to 85ms (-29%) and peak memory usage from 320MB to 195MB (-39%) under typical configurations. This allows a full-featured model that could only run on high-performance computing devices to be deployed on medium-performance computing devices for eight types of scene analysis tasks.

[0033] 2. A unified multi-scenario architecture combining a shared backbone network and a multi-task head network is adopted to integrate scenarios such as passenger flow monitoring, security early warning, behavior analysis, and equipment operation and maintenance into a single model. Compared to a single-scenario independent model solution (20 scenarios require the deployment of 20 independent models), this solution only requires one unified model, and the adaptation time for adding new scenarios is shortened from 2-4 weeks to less than 3 days (only a new head network needs to be trained).

[0034] 3. Knowledge from the teacher model is transferred to a lightweight student model via cloud-based knowledge distillation and then deployed to the edge. The teacher model is trained on YOLOv8x using the Ascend MindSpore framework. After knowledge distillation, the student model improves its target detection mAP@0.5:0.95 metric from 35.2% to 44.8% (+9.6%), while keeping the model size under 47.7MB (FP32), meeting the deployment constraint of ≤50MB for edge devices.

[0035] 4. A zoned calibration strategy based on a dynamic control point database is adopted to achieve continuous cross-layer positioning. In a field test at a large railway hub, the cross-layer tracking accuracy reached ±15cm, meeting the accuracy requirements for three-dimensional spatial positioning in the safety and emergency command of the rail transit hub. Cross-layer consistency loss constraints ensure that the target does not experience positional jumps when switching elevation planes. Attached Figure Description

[0036] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. These drawings are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0037] Figure 1 This is a diagram of the overall system architecture of the present invention;

[0038] Figure 2 This is a schematic diagram of the lightweight visual model structure of the present invention;

[0039] Figure 3 This is a diagram of the improved target detection network structure of the present invention;

[0040] Figure 4 This is a schematic diagram of the distributed collaborative architecture of the present invention;

[0041] Figure 5 This is a schematic diagram of the cross-camera tracking process of the present invention;

[0042] Figure 6 This is a schematic diagram illustrating the working principle of the dual-layer dynamic gating network of the present invention. Detailed Implementation

[0043] The specific structure, features, and advantages of the present invention will be described in detail below by way of examples. However, all descriptions are for illustrative purposes only and should not be construed as limiting the present invention in any way.

[0044] The following is in conjunction with the appendix Figure 1 -Appendix Figure 6 This invention will be described in detail.

[0045] Example 1:

[0046] like Figure 1 As shown, the lightweight vision-based video analysis system for rail transit hubs of the present invention adopts a layered collaborative architecture, consisting of four parts: video acquisition layer, edge inference layer, cloud training layer, and application service layer.

[0047] like Figure 2 As shown, the lightweight visual large model architecture adopted in this invention includes: after the input image is preprocessed, it is first fed into a shared backbone network composed of the MobileNetV3-Large improved version to extract multi-scale features;

[0048] Subsequently, a two-layer dynamic gating network dynamically determines which scene head networks are activated based on the complexity of the input features.

[0049] And inference depth; finally, the data is fed into a multi-task output head to generate passenger flow monitoring, safety warnings, and behavior reports, respectively.

[0050] Analysis results for scenarios such as data analysis and equipment maintenance.

[0051] The video acquisition layer is deployed in various areas of the rail transit hub, responsible for collecting video stream data. The cameras are domestically produced AI cameras, equipped with high-pixel CMOS sensors and built-in domestic AI inference chips (such as the Allwinner T527 NPU, 2 TOPS), supporting edge-side preprocessing. The cameras support various intelligent functions such as motion detection, area intrusion alarm, line crossing detection, loitering detection, and object leaving / taking detection. The video encoding uses H.265 format, with a resolution of 1920×1080 and a frame rate of 25fps.

[0052] The edge inference layer is deployed in edge servers and AI cameras within the rail transit hub. The edge servers utilize domestically produced equipment, such as the Ascend Atlas 500 Pro edge server (approximately 275 TOPS INT8 computing power, with two Atlas300I Pro inference cards) or the Cambricon Sigen 220 edge server (16 TOPS INT8 computing power). The edge inference engine is based on the Ascend CANN computing architecture and supports INT8 quantized inference.

[0053] The cloud-based training layer is deployed in a regional data center, using the Ascend MindSpore framework for model training, optimization, and management. The training platform supports distributed training, automatic hyperparameter search, and a model compression toolchain. The teacher model uses YOLOv8x pre-trained on Ascend MindSpore, and after knowledge distillation, generates a lightweight student model adapted for edge devices.

[0054] At the application service level, standardized APIs and visual interfaces are provided for various application scenarios in rail transit hubs, including functions such as event alarm subscription mechanisms, statistical report generation, video clip playback, and digital twin mapping. The digital twin 3D map is compatible with domestic GIS platforms (SuperMap and MapGIS).

[0055] like Figure 4 As shown, the system adopts a distributed architecture with cloud-edge collaboration: the cloud training module is responsible for the teacher...

[0056] The model is pre-trained and knowledge distilled to generate a lightweight student model, which is then distributed to the edge nodes; each edge node...

[0057] Each node downloads the appropriate head network combination package based on its own AI computing power level and independently executes inference tasks;

[0058] The video capture layer uploads the video stream to the edge nodes via a communication transmission channel (wired / 5G); the edge...

[0059] The nodes report the detection results and alarm information to the application service layer, forming a complete distributed collaborative processing.

[0060] assembly line.

[0061] Regarding two-layer dynamic gating networks:

[0062] like Figure 6 As shown, the dual-layer dynamic gating network is one of the core technical features of this invention, and its specific implementation is as follows:

[0063] The first layer is the Scene Gate. For the input feature map f∈R^(H×W×C), the channel statistics are first calculated: the channel mean μ = GlobalAvgPool(f)∈R^C, and the channel standard deviation σ = sqrt(GlobalAvgPool(f^2) - μ^2)∈R^C. μ and σ are concatenated to obtain the statistical vector [μ; σ]∈R^(2C). For the i-th head network, the activation vector a_i = sigmoid(W_i · [μ; σ] + b_i), where W_i∈R^(2C×1) and b_i∈R are learnable parameters, mapped to the (0,1) interval by the Sigmoid function. When a_i < τ (τ=0.3), the i-th head network enters a sleep state, its parameters are retained in the CPU memory pool and mapped to disk files, and are unloaded from the NPU device memory through CANN's almdlUnload interface, with an unloading delay of about 20ms; when a_i ≥ τ, the head network enters an active state, and its parameters are loaded into the NPU device memory through CANN's almdlLoadFromMem interface, with a loading delay of about 30ms.

[0064] The second layer is an Adaptive Depth Gate. For a head network that has entered an active state, the network inference depth is dynamically determined according to the complexity of input features. Specifically, the L2 norm of the feature is calculated as ||f||_2 = sqrt(sum(f^2)), then the inference depth d = min(D, ceil(κ · ||f||_2)) is determined, where D is the maximum depth of the head network (D=5 in this embodiment), and κ=0.15 is a scaling coefficient. When the input feature is simple (||f||_2 is small), d < D, and only the first d bneck blocks need to be executed to achieve sufficient detection accuracy, the subsequent D-d bneck blocks skip computation, saving approximately 15-25% of the computation amount; when the input feature is complex (||f||_2 is large), d = D, and the complete network is executed to obtain the highest accuracy.

[0065] Ablation experiments show that the computational efficiency improvement effect of the two-layer gating network on typical rail transit video streams is shown in Table 1.

[0066] Table 1 Performance Comparison of Ablation Experiments for Two-Layer Dynamic Gating Network

[0067] Average inference latency (ms) 120 95 105 85 Peak memory usage (MB) 320 240 280 195 Number of scenarios suitable for medium computing power devices 4 6 5 8 Multi-scenario overall accuracy (%) 94.4 94.2 94.0 93.8

[0068] Note: The test platform is Ascend Atlas 500 A2 (20 TOPS INT8), with batch size=1 and input resolution of 640×640. A 0.6 percentage point drop in accuracy is within the acceptable range, in exchange for a 29% reduction in latency and a 39% saving in memory usage.

[0069] Regarding the cross-layer consistency loss:

[0070] The cross-layer consistency loss is a further innovation of the present invention on the basis of partition calibration. When a detection target moves between different elevation planes (for example, a passenger moves from the concourse level to the platform level via an escalator), the cross-layer tracking unit needs to automatically determine the elevation plane where the target is located and switch the corresponding homography matrix according to the longitudinal position change rate of the target bounding box.

[0071] To prevent positional jumps during switching, a cross-layer consistency loss is introduced: L_consist = max(0, d_Frechet(Tr_A, Tr_B) - δ). Here, Tr_A and Tr_B are the ground trajectory point sequences obtained by mapping the homography matrices of planes A and B, respectively, d_Frechet is the Frechet distance, and δ = 0.5m is the threshold. d_Frechet is implemented using a differentiable approximation based on soft sorting to ensure gradient backpropagation. This loss, along with the detection loss L_det and classification loss L_cls, forms a joint loss function during the head network fine-tuning stage: L_total = λ1 × L_det + λ2 × L_cls + λ3 × L_consist, where λ3 = 0.1.

[0072] Ablation experiments show that cross-layer consistency loss can reduce cross-layer positioning error from ±35cm to ±15cm (an improvement of 57%), and meets the accuracy requirements of three-dimensional spatial positioning for safety emergency command in actual tests at a large railway hub.

[0073] Regarding complementary loss function pairs:

[0074] The design of the complementary loss function pair was rigorously verified through ablation experiments. Four configurations were set up for the experiment: standard FocalLoss (w_rec : w_prec = 1 : 1), safety preference (w_rec : w_prec = 2 : 1), passenger flow preference (w_prec : w_rec = 2 : 1), and the complementary pair configuration of this invention. The experimental results are shown in Table 2.

[0075] Table 2 Complementary loss function on ablation experiment results

[0076] Standard Focal Loss (1:1) 89.2 92.1 90.5 87.5 Safety preference only (2:1) 94.8 82.1 88.3 85.6 Passenger flow preference only (1:2) 85.3 93.5 94.1 93.5 Complementary pairs (in this invention) 94.5 91.8 89.8 93.2

[0077] Experimental results show that while a single safety preference configuration increases safety recall to 94.8%, it decreases precision to 82.1% (-10.0%), leading to an increase in false alarm rate. Similarly, while a single passenger flow preference configuration increases passenger flow precision to 93.5%, it decreases safety recall to 85.3% (-3.9%), increasing the risk of missed alarms. Only the complementary pair configuration achieves near-optimal levels for both safety recall (94.5%) and passenger flow precision (93.2%), demonstrating the effectiveness of the complementary loss function pair design.

[0078] Example 2: Knowledge Distillation Training Method

[0079] Knowledge distillation employs a YOLOv8x pre-trained on the Ascend MindSpore framework as the teacher model. After pre-training on the COCO dataset, YOLOv8x is fine-tuned on a rail transit hub labeled dataset (approximately 500,000 images, 1.2 million bounding boxes) and used as the teacher model. The student model uses an improved version of MobileNetV3-Large as a shared backbone network, coupled with four-class scene adaptive head networks. For example... Figure 3 As shown, the improved object detection network is based on MobileNetV3-Large. In the backbone network, some standard 3×3 convolutions are replaced with Ghost modules, and the penultimate bneck block is replaced with an improved Ghost Bottleneck. Channel attention mechanism and feature reuse branch are introduced. In the neck network, a bidirectional feature pyramid network (BiFPN) is used to achieve multi-scale feature fusion. The detection head adopts a decoupled structure, separating the classification and regression branches. Each branch contains two 3×3 convolutional layers.

[0080] Distillation employs a combination of feature-level and logit-level distillation. Feature-level distillation is performed at the feature map level: feature-level distillation alignment is performed between the student and teacher models, and an adaptive projection layer is used to map student features to the teacher feature space before calculating the L2 loss. Logit-level distillation is performed at the prediction level: the teacher model generates soft labels (temperature parameter T=4) on the training data, while the student model simultaneously learns both hard labels (true labels) and soft labels.

[0081] The distillation loss function is defined as:

[0082]

[0083] Where KL_Divergence is the Kullback-Leibler divergence loss, and the T² term is used to compensate for the effect of temperature scaling on the gradient magnitude. The total training loss function is defined as Equation (4):

[0084]

[0085] Where L_CE is the cross-entropy loss between the student model output and the true label, and α is the weighting coefficient balancing the hard label loss and distillation loss, α∈[0.5,0.8]. When T=4, grid search verification shows that α=0.7 is a better value. Key training hyperparameters for the distillation stage: the optimizer uses AdamW, with an initial learning rate of 1×10⁻³, batch size=32, epoch=100, and the learning rate decay strategy is cosine annealing. Data augmentation uses Mosaic, MixUp, random horizontal flipping, and color dithering.

[0086] The training hardware environment was an Ascend Atlas 800T training server (equipped with 8 Atlas 300T Pro training cards, with a total computing power of approximately 2.24 PFLOPS FP16), based on the Ascend MindSpore 2.0 framework, and accelerated by operators using the CANN 7.0 computing architecture. The training time was approximately 48 hours (4 cards in parallel). The final student model achieved a target detection mAP@0.5:0.95 of 44.8% on the validation set, an improvement of 9.6 percentage points compared to the baseline model without distillation (35.2%).

[0087] Example 3: Cross-camera tracking

[0088] like Figure 5 As shown, the cross-camera tracking process includes the following steps:

[0089] Step S1, Data Acquisition: Acquire target video streams by deploying cameras in different areas;

[0090] Step S2, Object Detection: Detect objects and extract bounding boxes using a lightweight vision model;

[0091] Step S3, Feature Extraction: An improved Ghost Bottleneck structure is used as a lightweight ReID feature extractor, which combines channel attention mechanism and feature reuse branch to extract discriminative appearance features of the target.

[0092] Step S4, Cross-camera matching: A two-layer dynamic gating network is used to coordinate feature matching between multiple cameras.

[0093] Calculate the feature similarity matrix of the target;

[0094] Step S5, Trajectory Association: Based on spatiotemporal constraints and feature similarity, the detection results from different cameras are associated as continuous motion trajectories.

[0095] The following descriptions are based on Examples 1, 2, and 3, and further elaborated in conjunction with application scenarios.

[0096] Application Example 1: Passenger Flow Density Monitoring

[0097] This case study focuses on real-time monitoring of passenger flow density in waiting areas and passageways. Test environment: A large railway hub, with a test period of 6 months. The test hardware environment consisted of a hybrid deployment of Ascend Atlas 500 Pro edge servers (equipped with two Atlas 300IPro inference cards, total computing power approximately 275 TOPS INT8) and AI cameras (with a built-in Allwinner T527 NPU, 2 TOPS). The inference framework was Ascend CANN 7.0, with an input resolution of 640×480 and a batch size of 1. The following test data are laboratory measurements taken under a high-performance edge server configuration to verify the system's technical feasibility.

[0098] Test data: 23 cameras (passenger flow monitoring deployment, with a total of 177 cameras planned for the entire station), with a cumulative video processing time of 1760 hours. The system uses an LSTM time series model to predict the density data of the previous 10 minutes and outputs the density prediction value for the next 10 minutes. MAPE is used to evaluate the prediction accuracy.

[0099] Table 3 Performance Indicators of Passenger Flow Density Monitoring

[0100] Personnel detection accuracy >95% 96.8%±0.5% Meets standards Density calculation error ±0.1 person / m² ±0.08 people / m² Meets standards Real-time detection time <200ms 150ms Meets standards Predicting MAPE <15% 11.5% Meets standards Regional coverage Full coverage Meets standards Meets standards

[0101] Note: The test value "96.8%±0.5%" represents the statistical result at a 95% confidence interval, based on a sliding window of 6 months of continuous monitoring data. The density error of ±0.08 people / m² was obtained by comparing the number of people detected with a laser passenger flow counter.

[0102] Application Example 2: Intrusion Detection and Border Crossing Warning. This case study focuses on intrusion detection in non-public areas and key protected areas. The video acquisition layer deploys cameras in locations such as equipment rooms, power distribution rooms, and control centers, setting up virtual warning zones and warning lines. The edge inference layer runs the intrusion detection algorithm, including a scene configuration module, a target detection module, a trajectory tracking module, and a decision module.

[0103] Table 4 Intrusion Detection Performance Data

[0104] Intrusion detection accuracy 95% 97.2%±0.4% Meets standards False alarm rate <1% 0.6% Meets standards underreporting rate <0.5% 0.2% Meets standards Alarm response delay <3 seconds 1.8 seconds Meets standards Cross-line detection accuracy ±10cm ±8cm Meets standards

[0105] Application Example 3: Fall Detection and Emergency Response This case study focuses on the automatic detection and emergency response to passenger falls within a station. Falls often occur in areas such as stairs, elevators, and platforms. The edge inference layer runs a fall detection algorithm, employing a spatiotemporal feature fusion method: continuous frames are processed by a temporal segmentation network (TSN) to extract temporal features; keyframes are processed by a human pose estimation model to extract key points of the human body (head, shoulders, elbows, wrists, hips, knees, and ankles, a total of 17 points); human pose features are calculated based on the key point locations; and a fall classification network determines whether a fall has occurred. The model is trained on the Ascend MindSpore framework, using the publicly available fall dataset (UR Fall Dataset) and on-site collected data, totaling 5000 samples, with a fall-to-normal ratio of 1:10.

[0106] Table 5. Fall Recognition Rate Performance Data

[0107] Staircase area 89.8 87.5 91.8 2.5 1080P elevator area 91.2 89.3 93.2 2.1 1080P Platform area 90.5 88.7 92.5 2.3 1080P Passage Area 88.7 86.9 90.5 2.8 1080P Overall Average 90.1 88.1 92.0 2.4 1080P

[0108] Note: F1-Weighted is the F1 score calculated by weighting the sample size of each class. Given the severe class imbalance in the dataset, data augmentation strategies were used to expand the positive samples to approximately 2000 segments.

[0109] Application Example 4: Comprehensive Application in Multiple Scenarios. This implementation case demonstrates the method's ability to be applied across all scenarios in rail transit hubs. The system covers multiple typical scenarios, forming a complete video intelligent analysis application system.

[0110] Passenger flow monitoring (5 categories): passenger flow density monitoring, passenger flow direction analysis, queue length detection, lingering area identification, and space occupancy analysis.

[0111] Security alerts (8 categories): intrusion detection, line crossing detection, loitering detection, fall detection, fighting detection, running detection, abandoned object detection, and smoke detection.

[0112] Behavioral analysis (6 categories): Smoking detection, phone call detection, mask wearing detection, leaning against escalator, going against the flow of traffic detection, and abnormal gathering identification.

[0113] Equipment maintenance (4 categories): elevator anomaly identification, equipment status identification, cleaning effect evaluation, facility anomaly detection.

[0114] Table 6 Scene Recognition Indicators

[0115] Passenger flow monitoring 5 4.5 120 96.5 23 Security Alert 8 6.2 180 94.2 78 Behavioral Analysis 6 4.1 150 92.8 44 Equipment maintenance 4 4.9 200 95.6 32 total 23 19.7 Maximum 200 94.4 177

[0116] Note: The head network is 19.7MB, the shared backbone network is approximately 28MB, and the total model size is approximately 47.7MB (FP32 accuracy, meeting the ≤50MB constraint). Inference latency is the measured value of a single-task head network on an Ascend Atlas 500 A2 (INT8 inference, Ascend CANN, batch size=1). The overall accuracy of 94.4% (without gating configuration) is calculated as a weighted average based on the sample size of each scene category.

[0117] Model ablation experiment

[0118] To verify the contribution of each technical module in this invention, an ablation experiment was designed. Using the lightweight YOLOv8n detection model as a baseline, each technical module was gradually added, and performance changes were observed. The experiment was conducted on a large railway hub test dataset, and the evaluation metrics were target detection mAP@0.5:0.95 and multi-scenario comprehensive accuracy. The experimental results are shown in Table 7.

[0119] Table 7 Comparison of Model Ablation Experiments

[0120] YOLOv8n (benchmark) 12.8 80 35.2 76.5 Knowledge distillation only (no multi-task head) 12.8 85 38.5 80.2 Multi-task head only (no knowledge distillation) 47.7 120 42.1 88.6 Complete solution for this method 47.7 85 44.8 93.8 This method + pruning quantification 38.2 95 43.5 91.2

[0121] Experimental results show that: the knowledge distillation-only configuration improves mAP by 3.3% compared to the benchmark, verifying the role of teacher model soft labels in promoting custom scene feature learning; the multi-task head network-only configuration improves mAP by 6.9% compared to the benchmark, verifying the feature reuse advantage of the shared backbone and scene-decoupled head architecture in multi-task scenarios; the complete solution (knowledge distillation + multi-task head network) improves mAP by 9.6% compared to the benchmark, and an additional 2.7% compared to "multi-task head only", indicating that knowledge distillation pre-training provides a better initial feature representation for the multi-task head network; adding a double-layer gating reduces latency by 29% and multi-scene comprehensive accuracy by 0.6 percentage points, verifying the effectiveness of the gating mechanism in optimizing computational efficiency; the pruning quantization scheme reduces model size by 19.9% ​​and inference latency by 20.8%, while mAP only decreases by 1.3 percentage points, verifying the engineering feasibility of model compression on edge devices; among them, INT8 quantization is performed on the standard convolutional layer on the Ascend CANN platform, while the Ghost module and attention structure maintain FP16 accuracy.

[0122] Compared with existing technologies:

[0123] To comprehensively evaluate the technological advancement of this invention, a quantitative comparison was conducted with existing mainstream technical solutions. The comparison dimensions included the number of models, the number of model parameters and storage size, adaptation time to new scenarios, edge deployment flexibility, cross-layer positioning capability, and computational adaptability. The comparison results are shown in Table 8.

[0124] Table 8 Performance Comparison of Different Technical Solutions

[0125] Number of models (20 scenarios) 20 1 1 Model size ~200MB ~80MB 47.7MB New scene adaptation time 2-4 weeks 1-2 weeks <3 days Edge deployment flexibility none Fix all heads Dual-layer gating dynamic adaptation Cross-layer positioning capability none none Partition calibration + cross-layer tracking Adaptive computation none none Scene gating + depth gating

[0126] The above embodiments and application examples have provided a detailed description of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A video analysis system for rail transit hubs based on lightweight vision, characterized in that, include: The video capture layer is used to collect video stream data from various areas of the rail transit hub. The cloud training layer is used to transfer knowledge from the teacher model to the lightweight student model through knowledge distillation. The student model uses an improved version of MobileNetV3-Large as a shared backbone network. The edge inference layer deploys a multi-task architecture consisting of the shared backbone network and multiple scene-adaptive head networks. The scenario-adaptive head network includes a passenger flow monitoring head network, a safety early warning head network, a behavior analysis head network, and an equipment operation and maintenance head network; The edge inference layer includes a two-layer dynamic gating network. The first layer of the two-layer dynamic gating network is a scene gating network, which is used to dynamically control the sleep and wake-up of different scene head networks based on the feature statistics of the input features. The second layer is an adaptive depth gating network, which is used to dynamically adjust the inference depth of the head network based on the L2 norm of the input features. The application service layer is used to provide corresponding scene analysis services based on the output of the activated head network.

2. The lightweight vision-based video analysis system for rail transit hubs according to claim 1, characterized in that: The edge inference layer also includes a device computing power reading module, used to obtain the AI ​​computing power level of the edge device; the system adaptively downloads and activates a head network combination package that matches the current computing power based on the computing power level.

3. The lightweight vision-based video analysis system for rail transit hubs according to claim 1, characterized in that: The application service layer includes a BIM joint calibration module, which is used to: automatically extract structural key points of multiple elevation planes of the rail transit hub using the BIM model, and establish a zoned control point library; for each camera coverage area, select control points from the zoned control point library for calibration, and automatically switch the corresponding mapping matrix when the detected target moves between different elevation planes to achieve cross-layer continuous positioning.

4. The lightweight vision-based video analysis system for rail transit hubs according to claim 3, characterized in that: The BIM joint calibration module also introduces cross-layer consistency loss, which is used to constrain the spatial continuity of the ground trajectory point sequence obtained by mapping the corresponding matrices of the two planes when the target switches from the first elevation plane to the second elevation plane. This loss is used as a component of the joint loss function in the model fine-tuning stage, and participates in the optimization together with the detection loss and classification loss.

5. The lightweight vision-based video analysis system for rail transit hubs according to claim 1, characterized in that: The detection loss of the safety warning head network and the passenger flow monitoring head network adopts differentiated weighted Focal Loss as a complementary loss function pair; among them, the loss function of the safety warning head network focuses on improving the recall rate, while the loss function of the passenger flow monitoring head network focuses on improving the precision rate.

6. The lightweight vision-based video analysis system for rail transit hubs according to claim 1, characterized in that: The edge inference layer also includes a cross-camera tracking module, which uses an improved Ghost Bottleneck structure as a lightweight ReID feature extractor and introduces a channel attention mechanism and a feature reuse branch.

7. The lightweight vision-based video analysis system for rail transit hubs according to claim 1, characterized in that: The cloud-based training layer employs a combination of feature-level distillation and logit-level distillation, with the total training loss function being a weighted sum of hard-label loss and soft-label loss.

8. A lightweight vision-based video analysis method for rail transit hubs, applied to the lightweight vision-based video analysis system for rail transit hubs as described in claim 1, characterized in that, Includes the following steps: Step 1: Collect videos of typical scenes in rail transit hubs and construct an labeled dataset; Step 2: In the cloud, a lightweight student model is obtained by using a pre-trained teacher model and training it through knowledge distillation. The student model uses an improved version of MobileNetV3-Large as the shared backbone network. Step 3: Freeze the backbone network parameters of the trained student model and only fine-tune the parameters of the scene adaptation head network; Step four: During deployment, based on the AI ​​computing power of the edge device, the corresponding head network is dynamically activated and the inference depth is adjusted through the dual-layer dynamic gating network to execute dynamic task routing.

9. The video analysis method for rail transit hubs based on lightweight vision according to claim 8, characterized in that: Step four describes dynamically activating the corresponding head network based on the AI ​​computing power level of the edge device. Specifically, high-computing-power devices activate all scene head networks; medium-computing-power devices prioritize activating security warning head networks and enable adaptive depth gating; and low-computing-power devices prioritize activating head networks with high alarm priority among the security warning head networks.

10. The video analysis method for rail transit hubs based on lightweight vision according to claim 8, characterized in that, In step three, the multi-task loss function in the fine-tuning stage includes detection loss, classification loss, and cross-layer consistency loss. The sum of the weights of detection loss and classification loss is a fixed value of 1.5, and the weight of cross-layer consistency loss is 0.1.

Images