Railway operation area personnel multi-target cross-mirror tracking alarm method and system fusing spatial-temporal characteristics

Abstract

The invention provides a railway operation area personnel multi-target cross-mirror tracking alarm method and system fusing spatial-temporal characteristics, and belongs to the technical field of target recognition. According to the method, a multi-thread heterogeneous reasoning structure and a mercuration AI reasoning platform are introduced, so that full-process intelligentization and real-time performance of railway perimeter intrusion personnel detection, tracking and re-identification, and line passing counting and alarming are realized. According to the method, YOLOv8 detection and a multi-target tracking and re-recognition algorithm are combined, so that the processing speed is remarkably increased while high detection precision is ensured; cooperative operation of video reading, reasoning and post-processing is achieved through a thread parallel mechanism, and system delay is effectively reduced. And meanwhile, intelligent early warning of excess intrusion is realized in combination with information of field operating personnel, so that high precision, high real-time performance, low false alarm rate and good adaptability are achieved, and the safety protection level of the railway perimeter can be remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, specifically to a method and system for multi-target cross-camera tracking and alarm of personnel in railway operation areas that integrates spatiotemporal features. It can be applied to intelligent video monitoring systems along railway lines to achieve real-time detection, tracking and re-identification of intruding targets, counting of crossings, and safety warnings. Background Technology

[0002] The railway perimeter is a crucial security boundary for the railway transportation system; intrusion can lead to serious safety incidents and operational disruptions. Traditional intrusion detection methods primarily rely on infrared, vibration fiber optic cables, or manual monitoring, which suffer from low recognition rates, high false alarm rates, high costs, and complex deployments. With the rapid development of video intelligent analysis technology, target detection and behavior analysis based on video streams have become key means for intelligent security of railway perimeters. However, existing video detection systems have the following shortcomings: limited multi-target tracking: traditional tracking algorithms cannot achieve accurate multi-target tracking and cross-camera tracking in complex railway environments, and their accuracy is not high; insufficient inference performance: conventional CPU / GPU deployments have high inference latency, making it difficult to achieve real-time processing of multiple video streams; task serialization issues: tasks such as detection, tracking, counting, and alarming are usually executed sequentially, lacking efficient parallel mechanisms; poor counting accuracy: repeated entry and exit of targets or occlusion can easily lead to double counting or missed counting. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for multi-target cross-camera tracking and alarm of personnel in railway operation areas that integrates spatiotemporal features. By deploying an improved YOLOv8 target detection model on the Ascend AI inference platform, and combining multi-target tracking and re-identification and line crossing judgment mechanisms, it achieves high-precision detection, real-time tracking and directional counting of personnel intruding into the railway perimeter. Furthermore, the system's real-time performance and stability are significantly improved through a multi-threaded heterogeneous inference mechanism, thus solving at least one of the technical problems existing in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for multi-target cross-camera tracking and alarm of personnel in railway operation areas that integrates spatiotemporal features, including: Step 1: Obtain the monitoring video stream of the railway operation area and the handheld terminal information of the workers. Use the improved YOLOv8 target detection model to identify the workers in the monitoring video stream and output the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. Step 2: Use a multi-target tracking model to track the workers, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories; when the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. Step 3: Compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

[0005] As a further limitation of the first aspect of the present invention, the multi-target tracking model performs post-processing on the output of the target detection model, and achieves continuous tracking of the target trajectory through detection box confidence filtering, motion prediction and IOU matching strategies, and retains the running trajectory.

[0006] As a further limitation of the first aspect of the present invention, the cross-camera re-identification model uses a feature encoding network based on multi-scale appearance representation and spatiotemporal attention fusion to extract deep feature vectors of the appearance information of the workers, and combines cosine similarity, cross-view spatiotemporal consistency constraints and trajectory continuity principles to associate the identity of targets from different cameras; when the target detection algorithm misses detection for a long time or the person moves across the camera, it automatically supplements the spatiotemporal trajectory and calculates the appearance similarity of the person under possible spatiotemporal distribution before re-tracking.

[0007] As a further limitation of the first aspect of the present invention, the counting rule for the number of railway workers is as follows: when the center point of the target trajectory moves from the outer area to the encroachment area, it is counted as a personnel entering the work area event, and when it moves from the inner area to the outer area, it is counted as a personnel leaving the work area event; if the same target triggers the IN or OUT event repeatedly, it will not be counted repeatedly to prevent duplicate statistics.

[0008] As a further limitation of the first aspect of the present invention, when personnel enter or leave the work area, a screenshot file of the corresponding target in the current video frame is automatically saved, and a record file is generated in the same directory. The record file includes the following fields: time, category, target ID, confidence level, and bounding box pixel coordinate information.

[0009] As a further limitation of the first aspect of the present invention, when the number of personnel in the work area is greater than the number of authorized personnel, an alarm will be triggered and the detection information of the target will be associated, including pixel coordinates, GPS coordinates, track number and time information.

[0010] Secondly, the present invention provides a multi-target cross-camera tracking and alarm system for personnel in railway operation areas that integrates spatiotemporal features, comprising: The acquisition module is used to acquire the monitoring video stream of the railway operation area and the handheld terminal information of the workers. It uses an improved YOLOv8 target detection model to identify the workers in the monitoring video stream and outputs the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. The tracking and identification module is used to track workers using a multi-target tracking model, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories. When the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. The comparison output module is used to compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

[0011] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they realize the multi-target cross-view tracking alarm method for personnel in railway operation areas that integrates spatiotemporal features as described in the first aspect.

[0012] Fourthly, the present invention provides a computer device including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the multi-target cross-view tracking alarm method for railway operation areas that integrates spatiotemporal features as described in the first aspect.

[0013] Fifthly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions to implement the multi-target cross-view tracking alarm method for railway operation areas that integrates spatiotemporal features as described in the first aspect.

[0014] The beneficial effects of this invention are as follows: By introducing a multi-threaded heterogeneous inference structure and the Ascend AI inference platform, the entire process of detecting, tracking, and re-identifying intruders at railway perimeter intrusion sites, as well as counting and alarming, is made intelligent and real-time. This method combines YOLOv8 detection with multi-target tracking and re-identification algorithms, significantly improving processing speed while ensuring high detection accuracy. It utilizes a thread parallel mechanism to achieve collaborative operation of video reading, inference, and post-processing, effectively reducing system latency. Simultaneously, it combines on-site personnel information to achieve intelligent early warning of excessive intrusions, possessing high accuracy, high real-time performance, low false alarm rate, and good adaptability, thus significantly improving the level of railway perimeter security.

[0015] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the multi-target cross-camera tracking and alarm method for personnel in railway operation areas that integrates spatiotemporal features, as described in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the target crossing line according to an embodiment of the present invention.

[0019] Figure 3 This is a visualization diagram of multi-target tracking as described in an embodiment of the present invention.

[0020] Figure 4 This is a structural diagram of the improved YOLOv8 model described in an embodiment of the present invention.

[0021] Figure 5 This is a flowchart illustrating the workflow of the multi-target cross-camera tracking and alarm system for railway personnel in a railway operation area, which integrates spatiotemporal features, as described in an embodiment of the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0023] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.

[0025] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0026] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0027] To facilitate understanding of the present invention, the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0028] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.

[0029] Example 1

[0030] In this embodiment 1, a multi-target cross-camera tracking alarm system for personnel in railway operation areas, incorporating spatiotemporal features, is provided. The system includes: an acquisition module for acquiring monitoring video streams of the railway operation area and handheld terminal information of the personnel; an improved YOLOv8 target detection model for identifying personnel in the monitoring video stream; and outputting the bounding box coordinates and center point coordinates of the targets. The improved YOLOv8 target detection model includes a newly added small target detection layer. A tracking and identification module is used to track the personnel using a multi-target tracking model, determining whether they have entered the railway operation area based on the tracking information, and outputting the number of personnel entering the railway operation area and their respective trajectories. When the same personnel enters different monitoring videos, a cross-camera re-identification model re-identifies the personnel and confirms their movement route. A comparison and output module compares the number of personnel in the operation area with the handheld terminal information, outputting the actual number and location of railway personnel.

[0031] In this embodiment, using the above-described system, a multi-target cross-camera tracking and alarm method for personnel in railway operation areas, incorporating spatiotemporal features, is implemented, including: Step 1: Obtain the monitoring video stream of the railway operation area and the handheld terminal information of the workers. Use the improved YOLOv8 target detection algorithm to identify the workers in the monitoring video stream and output the bounding box coordinates and center point coordinates of the targets.

[0032] Step 2: Use multi-target tracking and cross-camera re-identification algorithms to track the workers, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories; when the same worker enters different monitoring videos, the algorithm can re-identify the worker and confirm the same worker's movement route.

[0033] Step 3: Compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

[0034] The improved YOLOv8 algorithm in step 1 is an algorithm model designed for detecting railway intruders. It adds a P2 small target detection layer to the model structure and uses a dataset of railway intruders that is different from other YOLOv8 algorithms in terms of training set.

[0035] like Figure 4 As shown in the figure, the red box indicates the structure of the P2 small target detection layer. In the YOLO11 model, the neck structure achieves feature fusion by combining the C3k2 module and two upsampling operations, ultimately outputting three sets of feature maps, corresponding to detection layers with spatial sizes of 80×80, 40×40, and 20×20, respectively. This multi-scale feature fusion strategy aims to simultaneously address the detection needs of small, medium, and large targets. However, in the original structure, while deeper detection layers (such as the 20×20 P5 layer) possess strong semantic expressive power, they also suffer from insufficient accuracy in small target detection. To address this issue, this paper makes targeted adjustments to the YOLO11 detection head structure: introducing a shallow P2 detection layer (output size 160×160). The P2 layer is located in a shallower layer of the backbone network, preserving more spatial detail information. Its feature maps have higher resolution and can more effectively capture local features such as edges and textures of small targets, helping to improve detection accuracy and reduce missed detections.

[0036] Specifically, in this embodiment, the YOLOv8 algorithm used in step 1 is a lightweight multi-threaded algorithm adapted to multiple platforms, and it has been optimized specifically for the Huawei Ascend inference platform. This algorithm achieves multi-threaded, multi-stream collaborative execution by constructing a parallel pipeline architecture, significantly improving task throughput and real-time performance. Simultaneously, it combines the asynchronous scheduling mechanism of the Ascend CANN toolchain with MindSpore Lite compilation optimization, fully utilizing the parallel computing power of AI Core and the DVPP hardware acceleration module. At the model level, it introduces operator fusion, mixed-precision computation, and memory reuse strategies, and optimizes the layout for Ascend's unique FRACTAL_NZ data format, thus achieving a balance between lightweight design and high performance.

[0037] By decomposing the task into multiple independent stages for parallel processing, the system's throughput and real-time performance are significantly improved. In this architecture, steps such as video stream reading, object detection model inference, multi-object tracking and re-identification, and line crossing counting are performed simultaneously in different threads, thereby reducing processing time and improving the utilization of hardware resources.

[0038] Operator fusion is a technique that combines multiple computational operations into a single operation, aiming to reduce memory and computational resource consumption during computation. For deep learning models, many common operations (such as convolution, activation, and pooling) are typically executed sequentially. If each operation were to be computed and its intermediate results stored separately, it would incur significant memory access and computational overhead. Operator fusion allows multiple operations to be combined into a more complex operation within the computation graph, reducing the storage of intermediate results and improving computational efficiency. For example, convolution operations and activation functions (such as ReLU) can be fused into a single operation, thereby reducing memory usage and execution time. In the Ascend platform, operator fusion not only reduces memory bandwidth consumption but also better utilizes the parallel computing capabilities of the AI ​​Core.

[0039] Mixed-precision computing refers to using data types with different precisions during training or inference to balance computational accuracy and execution efficiency. In deep learning, common data precisions include 32-bit floating-point numbers (FP32), 16-bit floating-point numbers (FP16), and 8-bit integers (INT8). FP32 offers high computational precision but is relatively slow. FP16 and INT8 can accelerate the computation process, especially during inference, significantly improving performance, particularly when supported by the hardware. Mixed-precision computing selects appropriate precision for different layers or operations without compromising accuracy. For example, the first few convolutional layers can use FP16, while subsequent fully connected layers can use FP32, ensuring an optimal balance between speed and accuracy. On the Ascend platform, mixed-precision computing combined with hardware acceleration can significantly improve inference speed and reduce computational resource consumption.

[0040] Memory Reuse Strategy: Memory usage is a bottleneck in deep learning inference. Intermediate results from each layer need to be stored in memory. If each operation requires independent memory space, it leads to significant memory waste. Memory reuse strategies aim to reuse memory space as much as possible, reducing redundant memory allocation and deallocation. During inference, some intermediate results can be reused during computation without needing to allocate new memory blocks for each operation. This reduces memory bandwidth usage and lowers memory access latency. The Ascend platform optimizes memory usage based on the characteristics of the computation graph and hardware architecture when performing memory reuse, ensuring that memory bottlenecks do not occur during model inference, achieving higher efficiency.

[0041] In this embodiment, the dataset used to train the improved YOLOv8 was created by simulating foreign object intrusion scenarios in suburban areas, railway crossings, and other locations, capturing high-definition videos, and then extracting frames from the videos to convert them into images. The images were then labeled using the Labellmg software to complete the dataset construction. The dataset contains 5060 images of intruding objects, including pedestrians, motorcycles, bicycles, trains, dogs, stones, cardboard boxes, cars, trucks, and buses, all with a resolution of 1920 pixels x 1080 pixels. The dataset also includes blurry images with shaking, railway images under different weather conditions, and images of foreign objects at varying scales and distances, reflecting real-world usage scenarios.

[0042] In this embodiment, the multi-target tracking algorithm in step 2 performs post-processing on the output of the target detection algorithm. It achieves continuous tracking of the target trajectory through detection box confidence filtering, motion prediction, and IOU matching strategies, and retains the running trajectory for two consecutive seconds.

[0043] For confidence filtering, candidate targets are screened based on the confidence score output by the detection model. Detection boxes below a set threshold are removed, thereby reducing erroneous target inputs caused by noise, occlusion, or false detections, and ensuring that only reliable targets participate in subsequent association. For motion prediction, Kalman filtering uses the target's previous position and velocity as state inputs to predict its spatial position at the next moment, thus maintaining trajectory continuity even in cases of short-term target occlusion or detection loss. The IOU matching strategy associates the "predicted box" with the "current detection box," calculates their overlap, constructs a cost matrix, and prioritizes matching candidate boxes with higher IOUs, ensuring that the same target maintains a consistent ID across different frames, achieving stable target tracking.

[0044] The cross-camera re-identification algorithm in step 2 adopts a feature encoding network based on multi-scale appearance representation and spatiotemporal attention fusion. It uses pre-built model parameters to extract deep feature vectors from the appearance information of workers, and combines cosine similarity, cross-view spatiotemporal consistency constraints and trajectory continuity principles to associate the identity of targets from different cameras. The algorithm has breakpoint re-identification and cross-border re-identification functions. When the target detection algorithm misses detection for a long time or the person moves across the camera, it can automatically supplement the spatiotemporal trajectory and calculate the appearance similarity of the person under possible spatiotemporal distribution before re-tracking.

[0045] In the feature encoding network that integrates multi-scale appearance representation and spatiotemporal attention, the multi-scale feature fusion form is as follows: ;in These are feature maps at different scales. Represents the weight vector of the science department; This indicates that the feature map is adjusted and unified in dimensions through 1×1 convolution. It is a multi-scale appearance feature map after fusion.

[0046] Multi-scale feature maps of consecutive frames Perform global pooling to obtain a token vector:

[0047] And construct the attention vector:

[0048] in , , It is a learnable matrix.

[0049] The spatiotemporal attention weights are calculated as follows: ; After attention mechanism, spatiotemporal augmentation features are obtained:

[0050] The final appearance feature vector is: ; The cosine similarity matching strategy includes: for two trajectories from different cameras, their depth feature vectors are respectively and Cosine similarity is used to measure appearance similarity:

[0051] when Exceeding the threshold At the time, it was believed that the two were highly similar in appearance.

[0052] Cross-perspective spatiotemporal consistency constraints include: to ensure that candidate targets with similar appearances are reachable in both time and physical space, the algorithm introduces constraints such as temporal consistency, spatial reachability, and speed rationality. Time consistency: ; Spatial accessibility: The trajectory at the last position of camera A is The trajectory is at the initial position of camera B. , ; Speed ​​rationality: .

[0053] The counting rules for the number of railway workers in step 2 are as follows: when the center point of the target trajectory moves from the outer area to the encroaching area, it is counted as a personnel entering the work area event; when it moves from the inner area to the outer area, it is counted as a personnel leaving the work area event. If the same target triggers the IN or OUT event repeatedly, it will not be counted repeatedly to prevent duplicate statistics.

[0054] In step 2, the railway operation area can be open within a single video stream. When the operation area involves multiple videos, the counting line supports cross-camera connection. For example, if an operation area involves camera 1 and camera 2, the operator is counted when entering the operation area from camera 1. If the operator re-enters the operation area from camera 2, the count is not repeated. Therefore, this method supports cross-camera multi-target tracking and statistics.

[0055] When personnel enter or leave the work area, a screenshot file of the corresponding target in the current video frame is automatically saved (named cls_id_in / out.jpg), and a record file (CSV format) is generated in the same directory. This file includes the following fields: time, category, target ID, confidence level, bounding box pixel coordinates, etc., for subsequent statistical analysis and source tracing.

[0056] When the number of personnel in the work area exceeds the authorized number of personnel, an alarm will be triggered and the detection information of the target will be linked, including pixel coordinates, GPS coordinates, track number and time information, so as to facilitate subsequent verification and linkage with the safety management system.

[0057] Example 2

[0058] like Figures 1 to 5 As shown in this embodiment 2, a method for multi-target cross-camera tracking and alarm of personnel in railway operation areas that integrates spatiotemporal features is provided. By deploying an improved YOLOv8 target detection model on the Ascend AI inference platform, combined with multi-target tracking and re-identification and line crossing judgment mechanism, high-precision detection, real-time tracking and directional counting of personnel intruding into the railway perimeter are achieved. The system's real-time performance and stability are significantly improved through a multi-threaded heterogeneous inference mechanism.

[0059] In this embodiment, the multi-target cross-camera tracking and alarm method for personnel in railway operation areas, which integrates spatiotemporal features, includes the following steps: Step 1, real-time acquisition of video streams or historical recordings from the railway monitoring system, and input of the number of authorized personnel registered in the on-site handheld terminal device as the baseline information for personnel in the operation area; Step 2, setting the pixel coordinates of the two endpoints of the counting line in the video interface, and defining the two endpoint pixel coordinates of the counting line to form a connection by input or clicking; Step 3, using the YOLOv8 target detection algorithm to infer the video frames, detecting intruding personnel targets, and outputting information such as the target category, confidence level, and bounding box coordinates; Step 4, inputting the YOLOv8 detection output into the multi-target tracking and cross-camera re-identification algorithm for target tracking and re-identification; Step 5, crossing direction. Judgment and Counting; Step 6: When an IN or OUT event is detected, the system automatically saves a screenshot file of the target crossing the line (named cls_id_in / out.jpg) and generates a CSV format record file in the same directory, containing information such as event timestamp, category, target ID, confidence level, bounding box coordinates, and center point coordinates, for subsequent statistical analysis and security backtracking; Step 7: When the number of intruders counted by the video surveillance system exceeds the input number of authorized personnel, the system automatically triggers an alarm, outputting information including: pixel coordinates of the excess personnel, geographic GPS location (obtained through coordinate transformation), track number, intrusion time, etc., for security response; Step 8: Based on the Ascend platform, multi-threaded heterogeneous inference separates and executes tasks such as video reading, model inference, and result post-processing in parallel. Data transfer between threads is achieved through a synchronization queue, and computation and I / O parallelism are achieved using the Ascend Stream mechanism, thereby significantly improving system throughput and real-time response capabilities.

[0060] In this embodiment, specifically, step 3 mainly includes the following steps: Step 3.1, the model is converted from .pt format to .onnx format, and then converted to .om format using Huawei Ascend CANN's ATC tool to adapt to the Ascend AI inference platform; Step 3.2, the inference process calls the InferSession interface to load the .om model file and perform forward inference.

[0061] Step 4 mainly includes the following steps: Step 4.1, the algorithm continuously tracks the target trajectory through detection box confidence, motion prediction, and IOU matching strategies, assigns a unique ID to each target, and retains the target's historical two-second motion trajectory information for subsequent line-crossing behavior judgment; Step 4.2, when the target detection algorithm misses a long period of time or a person moves across cameras, the cross-camera re-identification algorithm uses a feature encoding network based on multi-scale appearance representation and spatiotemporal attention fusion to extract the appearance features corresponding to the trajectory, and combines cosine similarity, cross-view spatiotemporal consistency constraints, and trajectory continuity principles to associate the candidate trajectory of the target from different cameras. This algorithm has the ability to re-identify at breakpoints and re-identify across borders, and can complete the trajectory according to the possible spatiotemporal distribution during the detection breakpoint, and calculate the similarity after fusing appearance features and inferred spatiotemporal position to achieve re-matching and continuous tracking of the target.

[0062] In the feature encoding network that integrates multi-scale appearance representation and spatiotemporal attention, the multi-scale feature fusion form is as follows: ;in These are feature maps at different scales. Represents the weight vector of the science department; This indicates that the feature map is adjusted and unified in dimensions through 1×1 convolution. It is a multi-scale appearance feature map after fusion.

[0063] Multi-scale feature maps of consecutive frames Perform global pooling to obtain a token vector:

[0064] And construct the attention vector:

[0065] in , , It is a learnable matrix.

[0066] The spatiotemporal attention weights are calculated as follows: Q, K, and V are parameters in the attention mechanism, representing query, key, and value, respectively. d is also a parameter in the attention mechanism, representing the dimensions of the query and key vectors. Their function is to stabilize training, prevent gradient vanishing, and ensure a more reasonable distribution of attention.

[0067] After attention mechanism, spatiotemporal augmentation features are obtained:

[0068] The final appearance feature vector is: GAP ( ) represents global average pooling, which is used to fuse multi-scale features with spatiotemporal enhancement features; This indicates spatiotemporal enhancement features.

[0069] The cosine similarity matching strategy includes: for two trajectories from different cameras, their depth feature vectors are respectively and Cosine similarity is used to measure appearance similarity:

[0070] when Exceeding the threshold At the time, it was believed that the two were highly similar in appearance.

[0071] Cross-perspective spatiotemporal consistency constraints include: to ensure that candidate targets with similar appearances are reachable in both time and physical space, the algorithm introduces constraints such as temporal consistency, spatial reachability, and speed rationality. Time consistency: ; Indicates the time when trajectory j (camera B) appears; Indicates the time when trajectory i (camera A) appears; This indicates the maximum reachable time estimated based on the distance between the two cameras.

[0072] Spatial accessibility: The trajectory at the last position of camera A is The trajectory is at the initial position of camera B. , ; Indicates distance; This describes the algorithm for converting pixel coordinates into actual geographic coordinates. This indicates the cross-camera reach distance set according to the camera layout.

[0073] Speed ​​rationality: ; This represents the theoretical maximum speed of a pedestrian (walking / running).

[0074] Step 5 mainly includes the following steps: Step 5.1, when the line connecting the center point of the target trajectory intersects the counting line between two adjacent frames, it is determined as a line crossing event; Step 5.2, when the target crosses the counting line from the outside to the inside, it is counted as an IN event; when it crosses the counting line from the inside to the outside, it is counted as an OUT event; Step 5.3, to prevent duplicate counting, when the same target repeatedly triggers events in the same direction, it will no longer be counted.

[0075] Step 8 mainly includes the following steps: Step 8.1, Video reading thread: Reads frames from the video source and sends them to the shared buffer; Step 8.2, Inference thread: Calls the CANN inference API to load the .om model compiled and optimized by MindSpore Lite, and uses the AI ​​Core parallel computing unit to perform heterogeneous inference tasks; Step 8.3, Post-processing thread: Receives the inference output results, performs tasks such as target tracking and re-identification, line crossing detection and visual counting, and combines the DVPP hardware acceleration module to optimize image decoding and rendering performance to achieve efficient real-time output.

[0076] In this embodiment, a perimeter monitoring scenario of a railway transportation section is used as an example to illustrate the above method. The system acquires real-time video streams through high-definition cameras deployed along the line, with an input frame rate of 25 fps and a resolution of 1920×1080. After decoding, the video data enters the multi-threaded heterogeneous inference pipeline of this method. The system mainly includes three types of threads: video reading thread, inference thread, and post-processing thread. Data is transferred between threads through a synchronization queue, realizing heterogeneous parallelism of data acquisition, inference, tracking, counting, and visualization.

[0077] (1) Video reading thread: This thread reads image frames from the video source (such as RTSP stream or local video file). Users can set the pixel coordinates at both ends of the counting line by clicking the mouse or inputting coordinates. After uniformly performing size scaling and normalization processing, the frames are placed in a shared buffer for the inference thread to call. To ensure system stability, a maximum length of the buffer queue is set. When the queue is close to saturation, old frames are automatically discarded to prioritize real-time performance.

[0078] (2) Inference Thread: This thread calls the Ascend AI CANN inference interface InferSession to load the YOLOv8 detection model, which has been converted to .om format, and performs forward inference. The inference output includes the detection box category (class_id), confidence, bounding box coordinates (x1, y1, x2, y2), and center point coordinates. The inference thread packages the results and sends them to the result queue for use by the post-processing thread.

[0079] (3) Post-processing thread: The post-processing thread is mainly responsible for target tracking and re-identification, line crossing judgment, count update and alarm.

[0080] 1) Target Tracking and Re-identification: The detection results are input into a multi-target tracking and cross-camera re-identification algorithm. Continuous tracking of the target trajectory is achieved through detection box confidence, motion prediction, and IOU matching strategies, retaining the trajectory for two consecutive seconds. When the target detection algorithm misses targets for an extended period or when people move across cameras, the cross-camera re-identification algorithm extracts deep appearance features using a feature encoding network based on multi-scale appearance representation and spatiotemporal attention fusion. It then combines cosine similarity, cross-view spatiotemporal consistency constraints, and trajectory continuity principles for target matching. This module can complete the possible spatiotemporal trajectories for missing time periods and re-associate the target after fusing appearance features and spatiotemporal estimation, achieving breakpoint re-identification and cross-camera re-identification.

[0081] 2) Line Crossing Detection: In the video interface, when the center point of the target's trajectory intersects the counting line between two adjacent frames, the system determines that the target has crossed the line, and distinguishes between IN and OUT events based on its direction of movement. Only one count is recorded for the same target in the same direction to prevent duplicate counting. When a line crossing event is detected, the system automatically saves a screenshot of the target and a CSV record file containing information such as timestamp, category, target ID, confidence level, bounding box coordinates, and center point coordinates.

[0082] 3) Count Update and Alarm: The system has a built-in interface for inputting authorized personnel information. Users can enter the number of authorized personnel via handheld terminals as a legal benchmark. The real-time detection module compares the current number of intruders with the benchmark number. When the number of intruders exceeds the authorized personnel threshold, the system triggers an alarm and outputs: intrusion time, target pixel coordinates, geo-registered GPS coordinates, track number, and video source number. The alarm signal can be sent to the dispatch center via the network for coordinated early warning.

[0083] In the above process, the inference thread allocates an independent Ascend Stream channel to each video stream, achieving parallel computation and I / O. For high frame rate videos, batch inference is used to reduce model loading overhead. The post-processing thread and the inference thread are decoupled through an asynchronous queue, thereby fully utilizing Ascend AI inference computing resources and achieving parallel execution of inference computation and post-processing logic. Under the same hardware conditions (Ascend 310P platform), the method in this embodiment reduces the average latency from 380ms in the traditional serial method to 97ms in scenarios where multiple video streams are processed simultaneously, and the frame processing rate is increased by approximately 3.9 times. At the same time, the target detection mAP@0.5 reaches 93.4%, meeting the requirements of real-time monitoring at railway sites.

[0084] Example 3

[0085] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they implement the multi-target cross-view tracking alarm method for personnel in railway operation areas that integrates spatiotemporal features, as described above. The method includes: Step 1: Obtain the monitoring video stream of the railway operation area and the handheld terminal information of the workers. Use the improved YOLOv8 target detection model to identify the workers in the monitoring video stream and output the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. Step 2: Use a multi-target tracking model to track the workers, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories; when the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. Step 3: Compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

[0086] Example 4

[0087] This embodiment 4 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, and the memory stores program instructions that can be executed by the processor. The processor calls the program instructions to execute the multi-target cross-view tracking alarm method for personnel in railway operation areas that integrates spatiotemporal features as described above. The method includes: Step 1: Obtain the monitoring video stream of the railway operation area and the handheld terminal information of the workers. Use the improved YOLOv8 target detection model to identify the workers in the monitoring video stream and output the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. Step 2: Use a multi-target tracking model to track the workers, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories; when the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. Step 3: Compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

[0088] Example 5

[0089] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions to implement the multi-target cross-view tracking alarm method for personnel in railway operation areas that integrates spatiotemporal features as described above. The method includes: Step 1: Obtain the monitoring video stream of the railway operation area and the handheld terminal information of the workers. Use the improved YOLOv8 target detection model to identify the workers in the monitoring video stream and output the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. Step 2: Use a multi-target tracking model to track the workers, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories; when the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. Step 3: Compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. A method for multi-target cross-camera tracking and alarm of personnel in railway operation areas that integrates spatiotemporal features, characterized in that, include: Step 1: Obtain the monitoring video stream of the railway operation area and the handheld terminal information of the workers. Use the improved YOLOv8 target detection model to identify the workers in the monitoring video stream and output the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. Step 2: Use a multi-target tracking model to track the workers, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories; when the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. Step 3: Compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

2. The method for multi-target cross-camera tracking and alarm of personnel in railway operation areas by integrating spatiotemporal features as described in claim 1, characterized in that, The multi-target tracking model performs post-processing on the output of the target detection model. It achieves continuous tracking of the target trajectory through detection box confidence filtering, motion prediction, and IOU matching strategies, and retains the running trajectory.

3. The multi-target cross-camera tracking and alarm method for personnel in railway operation areas that integrates spatiotemporal features according to claim 1, characterized in that, The cross-camera re-identification model uses a feature encoding network based on multi-scale appearance representation and spatiotemporal attention fusion to extract deep feature vectors of the appearance information of workers. It also combines cosine similarity, cross-view spatiotemporal consistency constraints and trajectory continuity principles to associate the identity of targets from different cameras. When the target detection algorithm misses a target for a long time or the person moves across the camera, it automatically supplements the spatiotemporal trajectory and calculates the appearance similarity of the person under possible spatiotemporal distribution before re-tracking.

4. The method for multi-target cross-camera tracking and alarm of personnel in railway operation areas by integrating spatiotemporal features as described in claim 1, characterized in that, The counting rules for railway workers are as follows: when the center point of the target trajectory moves from the outer area to the encroaching area, it is counted as a personnel entering the work area event; when it moves from the inner area to the outer area, it is counted as a personnel leaving the work area event. If the same target triggers the IN or OUT event repeatedly, it will not be counted repeatedly to prevent duplicate statistics.

5. The method for multi-target cross-camera tracking and alarm of personnel in railway operation areas by integrating spatiotemporal features according to claim 1, characterized in that, When personnel enter or leave the work area, a screenshot of the corresponding target in the current video frame is automatically saved, and a log file is generated in the same directory. This file includes the following fields: time, category, target ID, confidence level, and bounding box pixel coordinates.

6. The method for multi-target cross-camera tracking and alarm of personnel in railway operation areas by integrating spatiotemporal features according to claim 1, characterized in that, When the number of personnel in the work area exceeds the authorized number of personnel, an alarm will be triggered and the detection information of the target will be associated, including pixel coordinates, GPS coordinates, track number, and time information.

7. A multi-target cross-camera tracking and alarm system for personnel in railway operation areas that integrates spatiotemporal characteristics, characterized in that, include: The acquisition module is used to acquire the monitoring video stream of the railway operation area and the handheld terminal information of the workers. It uses an improved YOLOv8 target detection model to identify the workers in the monitoring video stream and outputs the bounding box coordinates and center point coordinates of the targets. Among them, the improved YOLOv8 target detection model adds a small target detection layer. The tracking and identification module is used to track workers using a multi-target tracking model, determine whether they have entered the railway work area based on the tracking information, and output the number of people who have entered the railway work area and their respective trajectories. When the same worker enters different surveillance videos, the cross-camera re-identification model re-identifies the worker and confirms the same worker's movement route. The comparison output module is used to compare the number of personnel in the work area with the information on the handheld terminal, and output the actual number and location of railway workers.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the multi-target cross-view tracking alarm method for personnel in railway operation areas that integrates spatiotemporal features as described in any one of claims 1-6.

9. A computer device, characterized in that, The system includes a memory and a processor, which communicate with each other. The memory stores program instructions that can be executed by the processor. The processor calls the program instructions to execute the multi-target cross-view tracking alarm method for personnel in railway operation areas that integrates spatiotemporal features as described in any one of claims 1-6.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions to implement the multi-target cross-view tracking alarm method for railway operation areas that integrates spatiotemporal features as described in any one of claims 1-6.

Images