A substation cross-camera personnel trajectory global unification method, system, device, medium and product

Abstract

The application discloses a substation cross-camera personnel track global unification method, system, device, medium and product, relates to the technical field of video analysis, obtains a plurality of to-be-measured video frames and a plurality of active historical tracks of a substation, carries out target detection on each to-be-measured video frame through a pre-trained target detection model, obtains a plurality of personnel detection frames, carries out confidence level stratified screening on each personnel detection frame, obtains a plurality of first target detection frames and a plurality of second target detection frames, carries out association matching on each first target detection frame through each active historical track, obtains a plurality of matching results, re-associates each matching result according to each second target detection frame, and obtains corresponding global personnel tracks. The application overcomes the technical problem that the existing personnel tracking technology lacks a feature compensation mechanism in a shielding state, and a Kalman filter predicted track is prone to drift and divergence after a target is shielded, and reduces the accuracy of cross-camera personnel track unification.

Description

Technical Field

[0001] This invention relates to the field of video analytics, and in particular to a method, system, device, medium, and product for globally unified personnel trajectories across cameras in a substation. Background Technology

[0002] With the continuous implementation of new power system construction, substations, as key hubs for power grid energy transformation and transmission, face increasingly stringent regulatory requirements regarding personnel safety management, violation prevention, and risk control from unauthorized personnel. Traditional substation security models heavily rely on manual on-site inspections and post-event retrieval of surveillance footage. This not only requires a significant investment of maintenance manpower but also suffers from incomplete coverage of blind spots in on-site monitoring, delayed response to safety risks, and low efficiency in accident tracing. Dangerous behaviors such as personnel accidentally entering high-voltage energized areas, illegally crossing safety fences, or unauthorized intrusion are difficult to detect and address in a timely manner, easily leading to personal injury, equipment failure, or even regional power grid outages. Against this backdrop, intelligent video analytics based on computer vision is gradually becoming a core component of intelligent auxiliary control systems for substations. Multi-camera cross-scene personnel tracking and pedestrian re-identification technologies are the underlying foundation for achieving full-area personnel positioning, continuous trajectory reconstruction, and real-time anomaly alarms.

[0003] Currently, mainstream cross-camera personnel tracking technologies in substations fall into two categories. One is a centralized, heavy-duty deep learning solution, which relies on a large-parameter detection network and a deep feature extraction model to aggregate multiple video streams to a backend GPU server for unified computation, achieving personnel matching through brute-force comparison of global features. The other is a spatiotemporally assisted solution that introduces graph neural networks and clustering algorithms. This solution constructs camera topology, sets fixed time windows to optimize matching logic, and simply splices positional and appearance features to reduce matching confusion. However, substations are densely packed with transformers, high-voltage structures, insulators, and other equipment, making it easy for personnel to experience prolonged and large-area occlusion during movement. Existing personnel tracking technologies lack feature compensation mechanisms under occlusion conditions. When a target is occluded, the Kalman filter-predicted trajectory easily drifts and diverges. When a person reappears in the frame, pure appearance features are difficult to correlate with historical trajectories, and the complete movement path is fragmented into unconnected trajectory islands, reducing the accuracy of unified personnel trajectories across cameras. Summary of the Invention

[0004] This invention provides a method, system, device, medium, and product for globally unifying the trajectory of personnel across cameras in substations. It solves the technical problems of existing personnel tracking technologies lacking feature compensation mechanisms under occlusion conditions, Kalman filter prediction trajectories easily drifting and diverging after the target is occluded, and when personnel reappear in the frame, pure appearance features are difficult to associate with historical trajectories, and complete movement lines are cut into unconnected trajectory islands, reducing the accuracy of unifying the trajectory of personnel across cameras.

[0005] The first aspect of this invention provides a method for globally unifying personnel trajectories across cameras in a substation, comprising: Multiple video frames to be tested and multiple active historical trajectories of the substation are acquired. Target detection is performed on each of the video frames to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes. Each of the aforementioned personnel detection boxes is subjected to confidence-based stratified filtering to obtain multiple first target detection boxes and multiple second target detection boxes; Each of the aforementioned active historical trajectories is used to perform association matching on each of the first target detection boxes to obtain multiple matching results; Based on each of the second target detection boxes, the matching results are re-associated to obtain the corresponding global personnel trajectory.

[0006] Optionally, the step of performing confidence-based hierarchical filtering on each of the person detection boxes to obtain multiple first target detection boxes and multiple second target detection boxes includes: When the confidence level of the person detection box is greater than or equal to the preset first confidence threshold, the person detection box is determined as the first target detection box. When the confidence level of the personnel detection box is less than the first confidence level threshold, it is determined whether the confidence level of the personnel detection box is greater than or equal to the preset second confidence level threshold. When the confidence level of the person detection box is greater than or equal to the second confidence threshold, the person detection box is determined as the second target detection box.

[0007] Optionally, the step of using each of the active historical trajectories to perform association matching on each of the first target detection boxes to obtain multiple matching results includes: Each of the active historical trajectories is predicted using Kalman filtering to obtain multiple predicted trajectories; Calculate the intersection-union ratio between each of the first target detection boxes and each of the predicted trajectories; The center offset direction corresponding to each of the first target detection boxes and the motion direction corresponding to each of the predicted trajectories are respectively processed by difference to obtain multiple first differences, and the absolute value of each first difference is determined as a consistency measure. When the cross-union ratio is greater than the preset cross-union ratio threshold, it is determined whether the consistency metric corresponding to the cross-union ratio is less than the preset angle threshold. When the consistency metric corresponding to the intersection-union ratio is less than the angle threshold, the first target detection box and the predicted trajectory corresponding to the intersection-union ratio are determined as a cost candidate pair. A corresponding cost matrix is ​​constructed using each of the aforementioned cost candidate pairs, each of the first target detection boxes, and each of the aforementioned predicted trajectories; The Hungarian algorithm is used to optimally assign the cost matrix, resulting in multiple matching results.

[0008] Optionally, the step of re-associating each of the matching results based on each of the second target detection boxes to obtain the corresponding global personnel trajectory includes: When the matching result is a successful match, the successful match is determined as an associated match. When any of the matching results is the trajectory to be recovered, the appearance features of each of the second target detection boxes are extracted to obtain multiple appearance feature vectors. Based on each of the second target detection boxes and each of the appearance feature vectors, appearance feature matching is performed on each of the trajectories to be recovered to obtain corresponding association matching pairs; When any of the matching results is an initial detection trajectory, cross-camera matching is performed on each initial detection trajectory based on the pre-acquired cross-camera feature library to obtain the corresponding associated matching pair; Each of the aforementioned associated matching pairs is used as the corresponding global personnel trajectory.

[0009] Optionally, the step of performing appearance feature matching on each of the trajectories to be recovered based on each of the second target detection boxes and each of the appearance feature vectors to obtain corresponding association matching pairs includes: Each of the second target detection boxes and each of the trajectories to be recovered are paired up to obtain multiple candidate matching pairs; Each candidate matching pair is input into a preset cosine distance function along with its corresponding appearance feature vector and the trajectory to be recovered, to obtain multiple cosine distances. Based on a preset dual-threshold determination strategy, each candidate matching pair is filtered according to each cosine distance to obtain the corresponding associated matching pair and the reset trajectory to be recovered; When the lifetime count value of the reset trajectory to be restored is less than the preset lifetime threshold, the lifetime count value of the reset trajectory to be restored is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each of the active historical trajectories to perform association matching on each of the first target detection boxes to obtain multiple matching results.

[0010] Optionally, the step of performing cross-camera matching on each of the initial detection trajectories based on a pre-acquired cross-camera feature library to obtain corresponding associated matching pairs includes: The initial detection trajectories are filtered based on a preset time window to obtain the corresponding first detection trajectory; Based on the preset topological adjacency constraints and each of the first detection trajectories, the cross-camera historical trajectories in the pre-acquired cross-camera feature library are filtered to obtain the corresponding second detection trajectory. Each of the first detection trajectories and each of the second detection trajectories are input into a preset cost function to obtain multiple cost function values; Construct corresponding candidate cost matrices using each of the aforementioned cost function values; The Hungarian algorithm is used to perform global optimal matching on the candidate cost matrix to obtain the corresponding initial matching results; When the initial matching result satisfies the preset adaptive threshold relaxation condition, the initial matching result is determined as the corresponding associated matching pair. When the initial matching result does not meet the adaptive threshold relaxation condition, the lifetime count value of the first detection trajectory in the initial matching result is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each of the active historical trajectories to perform association matching on each of the first target detection boxes to obtain multiple matching results.

[0011] The second aspect of this invention provides a globally unified system for personnel tracing across cameras in a substation, comprising: The acquisition module is used to acquire multiple video frames to be tested and multiple active historical trajectories of the substation. It performs target detection on each of the video frames to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes. The filtering module is used to perform confidence-based stratified filtering on each of the personnel detection boxes to obtain multiple first target detection boxes and multiple second target detection boxes; The matching module is used to perform association matching on each of the first target detection boxes using each of the active historical trajectories to obtain multiple matching results; The association module is used to re-associate each of the matching results based on each of the second target detection boxes to obtain the corresponding global personnel trajectory.

[0012] The third aspect of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the substation cross-camera personnel trajectory global unification method as described above.

[0013] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the above-described method for globally unified personnel trajectories across cameras in substations.

[0014] The fifth aspect of the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein, when the program instructions are executed by a computer, the computer performs the substation cross-camera personnel trajectory global unification method as described above.

[0015] As can be seen from the above technical solutions, the present invention has the following advantages: This invention acquires multiple video frames and active historical trajectories from a substation. A pre-trained target detection model performs target detection on each video frame, resulting in multiple personnel detection boxes. These boxes are then layered and filtered based on confidence levels to obtain multiple first and second target detection boxes. The first target detection boxes are then correlated with each active historical trajectory to obtain multiple matching results. Finally, the matching results are re-correlated based on each second target detection box to obtain the corresponding global personnel trajectory. This invention overcomes the technical problem of existing personnel tracking technologies lacking feature compensation mechanisms for occlusion states. When a target is occluded, the Kalman filter-predicted trajectory easily drifts and diverges. When a person reappears in the frame, purely visual features are difficult to correlate with historical trajectories, and the complete movement path is fragmented into unconnected trajectory islands, reducing the accuracy of unified personnel trajectories across cameras. Compared with traditional personnel tracking technologies, this invention prioritizes the use of high-confidence first target detection boxes and active historical trajectories for the first round of association matching. This enables stable tracking of personnel within a single camera based on clear and complete personnel image information, maximizing trajectory matching accuracy under normal operating conditions and reducing unnecessary computational overhead. For trajectories that fail to match in the first round due to occlusion, dim lighting, or blurred distant views, a second re-association repair is performed using low-confidence second target detection boxes, effectively recovering lost or interrupted tracking trajectories. This avoids the numerous mismatches caused by using low-confidence detection boxes throughout the process and compensates for the shortcomings of a single high-confidence matching mode, which is prone to trajectory breaks and personnel tracking loss. It is suitable for complex monitoring environments in substations with numerous high-voltage areas, many obstructed scenes, and dispersed personnel movement routes, improving the accuracy of unified personnel trajectories across cameras. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a flowchart illustrating the steps of a method for globally unifying personnel trajectories across cameras in a substation, as provided in Embodiment 1 of the present invention. Figure 2 This is a flowchart illustrating the steps of a method for globally unifying personnel trajectories across cameras in a substation, as provided in Embodiment 2 of the present invention. Figure 3 This is a structural block diagram of a substation cross-camera personnel trajectory global unified system provided in Embodiment 3 of the present invention; Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0018] This invention provides a solution to the problem that existing personnel tracking technologies lack a feature compensation mechanism for occlusion states. When a target is occluded, the Kalman filter predicts a trajectory that is prone to drift and diverge. When a person reappears in the frame, it is difficult to associate pure appearance features with historical trajectories. The complete movement line is cut into unconnected trajectory islands, which reduces the accuracy of unifying personnel trajectories across cameras.

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a method for globally unifying the personnel trajectory across cameras in a substation, as provided in Embodiment 1 of the present invention.

[0021] This invention provides a method for globally unifying personnel trajectories across cameras in a substation, comprising: Step 101: Obtain multiple video frames to be tested and multiple active historical trajectories of the substation. Perform target detection on each video frame to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes.

[0022] The video frame to be tested refers to a static image frame extracted from the original video stream collected by each monitoring camera in the substation according to a fixed sampling strategy.

[0023] Active historical trajectories refer to the movement trajectories of personnel that have been established before the current moment and have not yet been marked as closed. Their status is continuously updated to match with the new detection results of the current frame and maintain the continuity of tracking.

[0024] A person detection box refers to a rectangular bounding box output by an object detection model. It is defined by coordinates (x, y), width w, height h, and confidence score, and identifies the estimated location and confidence level of a person in the image.

[0025] In this embodiment of the invention, multiple video frames to be tested and multiple active historical trajectories of a substation are acquired. A pre-trained target detection model is used to perform target detection on each video frame to obtain multiple personnel detection boxes. The target detection model is a lightweight YOLOv8-n network fine-tuned using a substation on-site personnel dataset.

[0026] It should be noted that the training process of the object detection model is as follows: A1. Construct a self-built dataset covering typical substation operating scenarios. The dataset is generated by collecting raw videos from multiple cameras deployed in different substations (including indoor switch rooms, outdoor high-voltage areas, transformer areas, etc.), and extracting frames to select image samples containing personnel targets. The samples need to cover various postures (walking, standing, bending, climbing, etc.), multiple scales (small targets at a distance and large targets at close range), diverse lighting conditions (sunny days, cloudy days, low light at night, backlight), and different degrees of equipment occlusion. For the collected image samples, professional annotation tools are used to accurately annotate the personnel targets with rectangular boxes. The annotation boxes are close to the personnel outline (including accessories such as insulated poles, safety helmets, etc.), and each box is assigned a personnel category label. To enhance the model's generalization ability, online data augmentation is performed on the annotated dataset before training. Augmentation strategies include random horizontal flipping, random brightness and contrast adjustment, random cropping and scaling, mosaic enhancement, and hybrid enhancement to simulate the complex and ever-changing visual environment in actual substation operation. A2. Based on the augmented dataset, a lightweight YOLOv8-n network was used as the basic architecture for training. This network employs a C2f-ELAN high-efficiency backbone network for feature extraction, combined with a P6-level FPN-PAN feature pyramid structure to achieve full fusion of multi-scale features, improving the detection capability of small-scale people and occluded targets. During training, an SGD optimizer was used with a cosine annealing learning rate scheduling strategy. The initial learning rate was set to 0.01, the momentum parameter to 0.937, and the weight decay coefficient to 0.0005. The batch size was set to 16 or 32 depending on the memory capacity of the edge device. The loss function consisted of a weighted sum of three terms: bounding box regression loss (using CIoU loss), classification loss (using binary cross-entropy loss), and confidence loss (using binary cross-entropy loss), with the total loss being the sum of these three. The model was iterated on the training set for a preset number of rounds (e.g., 300 rounds) until the loss converged, and its mean precision and recall were evaluated on the validation set. The weight file with the best performance was selected as the final model. A3. After training, the model weights are converted from FP32 precision to FP16 half precision format, and TensorRT is used to optimize the inference engine to generate an optimized deployment model for the target edge device (such as JetsonOrin) to achieve the millisecond-level latency target in the inference stage.

[0027] Step 102: Stratify and filter the detection boxes for each person based on their confidence level to obtain multiple first target detection boxes and multiple second target detection boxes.

[0028] The first target detection box refers to the detection box with a confidence score greater than or equal to the first confidence threshold (0.6). It has high positioning accuracy and reliability, and directly participates in the initial matching association to achieve accurate trajectory tracking and updating.

[0029] The second target detection box refers to the detection box with a confidence score between the second confidence threshold (0.35) and the first confidence threshold (0.6). It has a medium confidence score but contains valid target information. It is temporarily stored in the low confidence pool and will participate in the second matching after the first matching is completed. It is used to recover the trajectory that may be broken under occlusion or blur.

[0030] In this embodiment of the invention, for any person detection box, the confidence level of the person detection box is compared with a preset first confidence threshold. If the confidence level of the person detection box is greater than or equal to the first confidence threshold, the person detection box is determined as a first target detection box. If the confidence level of the person detection box is less than the first confidence threshold, it is determined whether the confidence level of the person detection box is greater than or equal to a preset second confidence threshold. If the confidence level of the person detection box is greater than or equal to the second confidence threshold, the person detection box is determined as a second target detection box. If the confidence level of the person detection box is less than the second confidence threshold, it is determined to be a noise or false alarm box and is directly removed without participating in any subsequent association processing.

[0031] Step 103: Use each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0032] In this embodiment of the invention, each active historical trajectory is input into a Kalman filter for prediction, resulting in multiple predicted trajectories. The intersection-union ratio (IUR) between each first target detection box and each predicted trajectory is calculated. The center offset direction corresponding to each first target detection box and the motion direction corresponding to each predicted trajectory are subtracted to obtain multiple first differences, and the absolute value of each first difference is determined as a consistency metric. Combinations of first target detection boxes with an IUR greater than a preset IU threshold and a consistency metric less than an angle threshold are determined as cost candidate pairs. A corresponding cost matrix is ​​constructed using each cost candidate pair, each first target detection box, and each predicted trajectory. The Hungarian algorithm is used to optimally assign the cost matrix, resulting in multiple matching results.

[0033] It should be noted that the Kalman filter is a recursive state estimation algorithm for target tracking. Its core function is to optimally estimate the target's motion state at the next moment through an iterative prediction-update process in noisy observation data. In this invention, the motion state of each active historical trajectory is modeled and predicted using an 8-dimensional parameterized state vector, specifically defined as X = [x, y, w, h, vx, vy, vw, vh]^T, where x and y represent the horizontal and vertical coordinates of the target bounding box center point on the image plane (unit: pixels), w and h represent the width and height of the bounding box (unit: pixels), and vx, vy, vw, and vh represent the first derivatives of the above four quantities, namely the center horizontal velocity, center vertical velocity, width change rate, and height change rate (all in pixels / frame). These eight state components together constitute a complete description of the target's kinematic features. In the prediction phase, the Kalman filter calculates the prior state estimate and prior estimate covariance matrix for the current frame based on the updated state estimate and state transition matrix from the previous frame, combined with the process noise covariance matrix. It outputs the predicted bounding box of the target in the current frame, along with its velocity and scale change trends. These prediction results constitute the aforementioned predicted trajectory set T_pred. In the update phase, after successfully associating a detection box with a trajectory, the Kalman filter corrects the prior estimate using the observed values ​​of that detection box (i.e., the x, y, w, and h values). By calculating the Kalman gain and weighted fusing the observed and predicted values, it obtains the posterior state estimate for the current frame, thereby updating the trajectory's position and velocity information and reducing the state covariance matrix to reflect the reduced uncertainty. For trajectories that fail to match a detection box, the Kalman filter only performs the prediction step and skips the update step. In this case, the trajectory's state covariance increases over time, reflecting the gradual increase in prediction uncertainty. When the number of consecutive unmatched frames exceeds a preset lifetime threshold (e.g., 30 frames), the trajectory is marked as terminated and recycled.

[0034] Step 104: Reassociate each matching result based on each second target detection box to obtain the corresponding global personnel trajectory.

[0035] In this embodiment of the invention, when the matching result is a successful match, the successful match is determined as an associated match. When any matching result is a trajectory to be recovered, each second target detection box is preprocessed, and appearance features are extracted from each preprocessed second target detection box to obtain multiple appearance feature vectors. Appearance feature matching is performed on each trajectory to be recovered based on each second target detection box and each appearance feature vector to obtain the corresponding associated match. When any matching result is an initial detection trajectory, cross-camera matching is performed on each initial detection trajectory based on a pre-acquired cross-camera feature library to obtain the corresponding associated match.

[0036] In this embodiment of the invention, multiple video frames to be tested and multiple active historical trajectories from a substation are acquired. A pre-trained target detection model is used to detect targets in each video frame, resulting in multiple personnel detection boxes. These personnel detection boxes are then layered and filtered based on confidence levels to obtain multiple first target detection boxes and multiple second target detection boxes. Each active historical trajectory is used to correlate and match the first target detection boxes, resulting in multiple matching results. Finally, each second target detection box is used to re-correlate the matching results to obtain the corresponding global personnel trajectory. This overcomes the technical problem of existing personnel tracking technologies lacking a feature compensation mechanism under occlusion conditions. When a target is occluded, the Kalman filter-predicted trajectory easily drifts and diverges. When a person reappears in the frame, pure appearance features are difficult to correlate with historical trajectories, and the complete movement line is fragmented into unconnected trajectory islands, reducing the accuracy of cross-camera personnel trajectory unification. Compared with traditional personnel tracking technologies, this invention prioritizes the use of high-confidence first target detection boxes and active historical trajectories for the first round of association matching. This enables stable tracking of personnel within a single camera based on clear and complete personnel image information, maximizing trajectory matching accuracy under normal operating conditions and reducing unnecessary computational overhead. For trajectories that fail to match in the first round due to occlusion, dim lighting, or blurred distant views, a second re-association repair is performed using low-confidence second target detection boxes, effectively recovering lost or interrupted tracking trajectories. This avoids the numerous mismatches caused by using low-confidence detection boxes throughout the process and compensates for the shortcomings of a single high-confidence matching mode, which is prone to trajectory breaks and personnel tracking loss. It is suitable for complex monitoring environments in substations with numerous high-voltage areas, many obstructed scenes, and dispersed personnel movement routes, improving the accuracy of unified personnel trajectories across cameras.

[0037] Please see Figure 2 , Figure 2 The flowchart illustrates the steps of a method for globally unifying the personnel trajectory across cameras in a substation, as provided in Embodiment 2 of the present invention.

[0038] This invention provides a method for globally unifying personnel trajectories across cameras in a substation, comprising: Step 201: Obtain multiple video frames to be tested and multiple active historical trajectories of the substation. Perform target detection on each video frame to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes.

[0039] In this embodiment of the invention, multiple video frames to be tested and multiple active historical trajectories of a substation are acquired. Each video frame to be tested is input into a pre-trained target detection model for forward inference to obtain multiple personnel detection boxes.

[0040] Step 202: Stratify and filter the detection boxes for each person based on their confidence level to obtain multiple first target detection boxes and multiple second target detection boxes.

[0041] Further, step 202 includes the following sub-steps: S11. When the confidence level of the personnel detection box is greater than or equal to the preset first confidence threshold, the personnel detection box is determined as the first target detection box.

[0042] The first confidence threshold refers to the preset confidence threshold value, set to 0.6, which is used to divide the detection boxes into high reliability level and medium-low reliability level. Detection boxes with a confidence level greater than or equal to this value are considered to be high-quality detection results with accurate positioning and high target confidence.

[0043] In this embodiment of the invention, when the confidence level of the personnel detection box is greater than or equal to the preset first confidence level threshold, the detection box is determined to have high reliability, its bounding box is accurately located and the target has a high degree of certainty, and the personnel detection box is determined as the first target detection box.

[0044] S12. When the confidence level of the personnel detection box is less than the first confidence threshold, determine whether the confidence level of the personnel detection box is greater than or equal to the preset second confidence threshold.

[0045] The second confidence threshold refers to the preset lower confidence limit, set to 0.35, which serves as the critical value for distinguishing between potentially valid detection boxes and invalid noise boxes. Detection boxes with a confidence level between 0.35 and 0.6 are considered to have potential value, while detection boxes with a confidence level below 0.35 are discarded.

[0046] In this embodiment of the invention, when the confidence level of the personnel detection box is less than 0.6, it is determined whether the confidence level of the personnel detection box is greater than or equal to 0.35.

[0047] S13. When the confidence level of the personnel detection box is greater than or equal to the second confidence threshold, the personnel detection box is determined as the second target detection box.

[0048] In this embodiment of the invention, when the confidence level of the personnel detection box is greater than or equal to 0.35, it indicates that although the confidence level of the detection box does not reach the high score standard, the certainty of the existence of the target is still within an acceptable range. Such situations often arise from challenging scenarios such as the target being at the edge of the device occlusion, long-distance small-scale imaging, backlighting, or low illumination. The detection model has a certain degree of uncertainty in its judgment, but it does not completely deny the possibility of the target's existence. The personnel detection box is then determined as the second target detection box.

[0049] Step 203: Use each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0050] Furthermore, step 203 includes the following sub-steps: S21. Perform Kalman filtering on each active historical trajectory to predict multiple predicted trajectories.

[0051] The predicted trajectory refers to the predicted bounding box and its motion state output after the active historical trajectory is predicted by Kalman filtering.

[0052] In this embodiment of the invention, each active historical trajectory is input into a Kalman filter for prediction stage operation to obtain multiple predicted trajectories.

[0053] S22. Calculate the intersection-union ratio between each first target detection box and each predicted trajectory.

[0054] The Intersection over Union (IoU) ratio is a metric that measures the degree of spatial overlap between two rectangular bounding boxes. It is defined as the ratio of the intersection area to the union area, and the value ranges from [0,1]. The higher the value, the greater the overlap between the two boxes and the more closely they match in spatial position.

[0055] In this embodiment of the invention, the intersection-union ratio (IoU) between each first target detection box and each predicted trajectory is calculated. The IoU is the ratio of the area of ​​the intersection region to the area of ​​the union region of the two rectangles, i.e., IoU = (intersection area) / (union area).

[0056] S23. Perform difference processing on the center offset direction corresponding to each first target detection box and the motion direction corresponding to each predicted trajectory to obtain multiple first differences, and determine the absolute value of each first difference as the consistency measure.

[0057] The center offset direction refers to the directional vector angle from the center of the predicted trajectory to the center of the first target detection box.

[0058] The direction of motion refers to the direction angle calculated from the velocity components vx and vy in the prior state estimation of the Kalman filter. It represents the direction of the target's movement in the historical process and is an important basis for judging the continuity of the motion trend.

[0059] The consistency metric refers to the angle difference within the range of 0° to 180° obtained after normalizing the absolute value of the first difference. The smaller the value, the more consistent the offset direction of the detection box is with the movement direction of the trajectory, and the higher the confidence of the match.

[0060] In this embodiment of the invention, the difference between the center offset direction corresponding to each first target detection box and the motion direction corresponding to each predicted trajectory is calculated to obtain multiple first differences, and the absolute value of each first difference is taken as a consistency measure.

[0061] S24. When the crossover-union ratio is greater than the preset crossover-union ratio threshold, determine whether the consistency metric corresponding to the crossover-union ratio is less than the preset angle threshold.

[0062] The intersection-union ratio (IUGR) threshold refers to a preset spatial overlap threshold, set to 0.3, used to filter whether there is sufficient positional overlap between the detection box and the predicted trajectory.

[0063] The angle threshold refers to the preset motion direction consistency judgment limit value, set to 45°, which is used to measure whether the deviation between the detection box offset direction and the historical motion direction of the trajectory is within an acceptable range. If it is less than this threshold, it means that the motion logic matches.

[0064] In this embodiment of the invention, when the crossover ratio is greater than 0.3, it is determined that the first target detection box and the predicted trajectory have a preliminary association condition in the spatial dimension, and it is determined whether the consistency metric corresponding to the crossover ratio is less than a preset angle threshold.

[0065] S25. When the consistency metric corresponding to the crossover-union ratio is less than the angle threshold, the first target detection box and the predicted trajectory corresponding to the crossover-union ratio are determined as cost candidate pairs.

[0066] In this embodiment of the invention, when the consistency metric corresponding to the cross-union ratio is less than the angle threshold, it is determined that the first target detection box and the predicted trajectory corresponding to the cross-union ratio meet the requirements in both spatial overlap and motion logic consistency dimensions, and the first target detection box and the predicted trajectory corresponding to the cross-union ratio are determined as cost candidate pairs.

[0067] S26. Construct the corresponding cost matrix using each cost candidate pair, each first target detection box, and each predicted trajectory.

[0068] The cost matrix is ​​a two-dimensional numerical matrix constructed with the first object detection box as the rows and the predicted trajectory as the columns. The matrix elements represent the matching cost required to assign the corresponding detection box to the corresponding trajectory.

[0069] In this embodiment of the invention, a cost matrix is ​​constructed using the first target detection box as the row and the predicted trajectory as the column, with the matching cost required to assign the first target detection box to the corresponding predicted trajectory as the matrix element. When the first target detection box and the predicted trajectory corresponding to a matrix element do not belong to a cost candidate pair, the matrix element is set to a very large positive number (such as 9999), representing that the matching between the two is physically and logically unacceptable.

[0070] S27. The Hungarian algorithm is used to optimally assign the cost matrix, resulting in multiple matching results.

[0071] It should be noted that the Hungarian algorithm is a combinatorial optimization algorithm for solving the linear assignment problem. Its core objective is to assign a unique column to each row and a unique row to each column in a cost matrix, such that the sum of all selected elements (i.e., matching costs) is globally minimized. In this embodiment of the invention, the Hungarian algorithm is deployed in the final step of the initial matching process to solve for the globally optimal matching of the cost matrix constructed in the preceding steps. Its input is an m-row, n-column cost matrix (where m is the number of first target detection boxes in the high-resolution detection pool D_high, and n is the number of predicted trajectories in the predicted trajectory set T_pred), and the matrix elements are the matching costs between the corresponding detection boxes and predicted trajectories. The specific execution flow of the algorithm is as follows: First, the cost matrix is ​​checked for validity. If the number of detection boxes is inconsistent with the number of trajectories, the matrix is ​​expanded into a square matrix (i.e., the number of rows and columns are equal) by adding virtual rows or columns. Each element of the virtual row or column is filled with a maximum value (e.g., 9999) to ensure that the algorithm can handle unbalanced scenarios where m ≠ n. Subsequently, the algorithm performs row and column reduction operations sequentially on the expanded matrix—subtracting the smallest element of each row to ensure that each row contains at least one zero value; and subtracting the smallest element of each column to ensure that each column also contains at least one zero value. Based on this, the algorithm searches for a minimum set of rows and columns in the reduced matrix that covers all zero elements. By repeatedly performing checks to ensure zero coverage and searching for augmenting paths, the algorithm gradually adjusts the current matching scheme until it finds an optimal assignment with a matching number equal to the matrix dimension. At this point, the sum of the original costs corresponding to the selected elements is the global minimum total cost.

[0072] In this embodiment of the invention, the Hungarian algorithm is used to optimize the cost matrix in combination, so as to find the optimal assignment scheme of detection box-trajectory with the minimum total matching cost in the global scope, and obtain multiple matching results.

[0073] Step 204: When the matching result is a successful match, the successful match is determined as an associated match.

[0074] A successful match refers to a match result in the optimal assignment scheme output by the Hungarian algorithm where the detection box and the predicted trajectory are successfully paired and the corresponding cost is a non-maximum finite value. This means that the high-confidence detection box and the active trajectory have passed the dual threshold tests of spatial overlap and motion consistency, and have been confirmed as the optimal association through global optimization.

[0075] A correlated matching pair refers to a successfully matched pair that is determined to be a reliable match.

[0076] In this embodiment of the invention, when the matching result is a successful match, the successful match is used as the corresponding associated match.

[0077] Step 205: When any matching result is the trajectory to be recovered, the appearance features of each second target detection box are extracted to obtain multiple appearance feature vectors.

[0078] The trajectory to be recovered refers to the predicted trajectory that failed to match any first target detection box in the first match. It represents the target that is in an occluded or low-visibility state but has not yet reached the end of its lifetime condition, and is waiting for a second match to attempt recovery.

[0079] In this embodiment of the invention, when any matching result is the trajectory to be recovered, for each second target detection box, the center of the second target detection box is expanded outward by 20% pixels. Then, the cropped image patch is scaled to a standard resolution of 256×128 using a bilinear interpolation algorithm, resulting in multiple second target detection boxes with preprocessed geometric features. Each second target detection box with preprocessed geometric features is input into a preset appearance feature extraction network to obtain multiple original depth feature vectors. The appearance feature extraction network is a pre-trained FastReID-R50 backbone network. The original depth feature vectors, i.e., 2048-dimensional features, are reduced to 256-dimensional using a PCA projection matrix pre-trained on a self-built dataset in a substation. Finally, L2 normalization is performed to normalize the feature vector magnitude to 1, eliminating the influence of feature magnitude changes caused by differences in image brightness and contrast on subsequent distance calculations, resulting in a 256-dimensional appearance feature vector.

[0080] It should be noted that the FastReID-R50 backbone network refers to the FastReID re-identification network based on the ResNet-50 architecture. The final fully connected classification layer is removed, and a 2048-dimensional deep feature vector is output using global average pooling. It is fine-tuned and trained using a self-built dataset from a substation, and has the ability to extract discriminative appearance features of personnel in complex industrial scenarios.

[0081] The PCA projection matrix refers to the optimal projection matrix obtained by performing covariance matrix eigenvalue decomposition on the features of the substation's self-built dataset in the Principal Component Analysis algorithm. It is used to reduce the dimensionality of the 2048-dimensional high-dimensional features to a 256-dimensional principal component space, retaining the most important discriminative information while reducing storage and computational overhead.

[0082] L2 normalization refers to the operation of normalizing the magnitude of the eigenvector by dividing each element of the vector by the L2 norm (i.e., the Euclidean length) of the vector itself, so that the magnitude of the normalized eigenvector is 1, ensuring that the subsequent cosine distance calculation depends only on the direction of the vector rather than the magnitude of the magnitude.

[0083] Step 206: Perform appearance feature matching on each trajectory to be recovered based on each second target detection box and each appearance feature vector to obtain the corresponding association matching pair.

[0084] Furthermore, step 206 includes the following sub-steps: S31. Pair each second target detection box with each trajectory to be recovered to obtain multiple candidate matching pairs.

[0085] In this embodiment of the invention, each second target detection box and each trajectory to be recovered are combined in pairs to obtain multiple candidate matching pairs.

[0086] S32. Input the corresponding appearance feature vector and the trajectory to be restored for each candidate matching pair into the preset cosine distance function to obtain multiple cosine distances.

[0087] In this embodiment of the invention, the appearance feature vector of the second target detection box and the appearance feature vector of the trajectory to be recovered in each candidate matching pair are respectively input into a preset cosine distance function to obtain multiple cosine distances.

[0088] It should be noted that the cosine distance function is specifically as follows: ; in, Cosine distance For the first i The appearance feature vector of a second object detection box. For the first j One appearance feature vector of the trajectory to be recovered. i 1 represents the index of the second target detection box. j 1 is the index of the trajectory to be recovered.

[0089] S33. Based on the preset dual threshold judgment strategy, each candidate matching pair is filtered according to each cosine distance to obtain the corresponding associated matching pair and the reset trajectory to be restored.

[0090] The dual-threshold judgment strategy refers to a matching judgment rule composed of two conditions: an absolute distance threshold and a relative distance ratio threshold. It requires that the minimum cosine distance be less than a preset benchmark value (0.35) and the ratio of the second smallest distance to the minimum distance be greater than 1.5 for the match to be considered valid. This aims to ensure both the sufficiency and uniqueness of the similarity of the match.

[0091] In this embodiment of the invention, based on a preset dual-threshold determination strategy, each candidate matching pair is filtered according to each cosine distance to obtain the corresponding associated matching pair and the trajectory to be reset and recovered. For example, for each trajectory to be recovered, when the minimum value of the cosine distances corresponding to the trajectory to be recovered is less than 0.35 and the second smallest distance (i.e., the cosine distance preceding the minimum value) is greater than 1.5 × (the minimum value of the cosine distances corresponding to the trajectory to be recovered), then the candidate matching pair associated with the minimum value of the cosine distances corresponding to the trajectory to be recovered is determined as an associated matching pair. When there is no corresponding associated matching pair for the trajectory to be recovered, the trajectory to be recovered is determined as a trajectory to be reset and recovered.

[0092] S34. When the lifetime count value of the reset trajectory to be restored is less than the preset lifetime threshold, the lifetime count value of the reset trajectory to be restored is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0093] The lifetime count value refers to the cumulative number of frames in which the trajectory to be recovered has failed to match a detection box since the last successful match. It increments by 1 for each frame that fails to match.

[0094] The lifespan threshold refers to the maximum number of unmatched frames that the preset trajectory can tolerate, set to 30 frames (approximately 3 seconds).

[0095] In this embodiment of the invention, when the lifetime count value of the reset trajectory to be restored is less than 30 frames, the lifetime count value of the reset trajectory to be restored is adjusted to obtain a new active historical trajectory (i.e., the lifetime count value of the new active historical trajectory = the lifetime count value of the reset trajectory to be restored + 1), and the process jumps to execute the step of using each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0096] Step 207: When any matching result is an initial detection trajectory, cross-camera matching is performed on each initial detection trajectory based on the pre-acquired cross-camera feature library to obtain the corresponding associated matching pair.

[0097] Furthermore, step 207 includes the following sub-steps: S41. Based on a preset time window, each initial detection trajectory is filtered to obtain the corresponding first detection trajectory.

[0098] The time window refers to a preset time filtering range used to limit the maximum allowable time interval between the last observation time of a historical trajectory and the generation time of the current new trajectory during cross-scope matching. Historical trajectories exceeding this range are considered expired and discarded. The value is 30 minutes.

[0099] The first detection trajectory refers to the set of historical trajectories retrieved from the cross-camera feature database after the initial detection trajectory has been filtered through a time window. The trajectories have last observation times within the allowed range and include metadata such as trajectory ID, camera ID, last observation timestamp, appearance feature template, and complete bounding box sequence.

[0100] In this embodiment of the invention, the frame acquisition timestamp of each initial detection trajectory is extracted and the end time of all historical trajectories in the cross-camera feature library is used to perform a difference operation to obtain multiple time intervals. Initial detection trajectories with time intervals exceeding the time window are removed to obtain the corresponding first detection trajectory.

[0101] S42. Based on the preset topological adjacency constraints and each first detection trajectory, the cross-camera historical trajectories in the pre-acquired cross-camera feature library are filtered to obtain the corresponding second detection trajectory.

[0102] Topological adjacency constraint refers to a spatial constraint that requires a connected path between two cameras whose trajectories are matched across cameras, based on the physical layout diagram of cameras in a substation. This constraint is used to exclude cross-camera pairings that are spatially inaccessible.

[0103] In this embodiment of the invention, based on preset topological adjacency constraints and each first detection trajectory, the cross-camera historical trajectories in the pre-acquired cross-camera feature library are filtered to obtain the corresponding second detection trajectories. For example, each first detection trajectory corresponds to a timestamp t_det and a camera ID number c_det, and each cross-camera historical trajectory in the cross-camera feature library corresponds to a last observation timestamp t_traj and a last observation camera ID number c_traj. For each cross-camera historical trajectory, a time continuity check is first performed: the time interval Δt = |t_det - t_traj| between the first detection trajectory and the cross-camera historical trajectory is calculated. If the minimum time interval corresponding to the cross-camera historical trajectory exceeds the preset maximum allowable time difference threshold (e.g., 30 seconds), the cross-camera historical trajectory is determined to be discontinuous in time logic (i.e., the time when the trajectory disappears is too far from the time when the first detection trajectory appears, and it is impossible for it to belong to the continuous movement of the same target), and the cross-camera historical trajectory is directly removed. For cross-camera historical trajectories that pass the time continuity check, a topological adjacency graph of all cameras in the substation is pre-constructed and stored. This graph uses each camera as a node and camera pairs with physical paths as edges. Each edge records the physical shortest path distance (in meters) between two cameras. The camera node to which the cross-camera historical trajectory belongs is extracted. The camera numbers corresponding to each first detection trajectory are traversed. If the camera node to which the cross-camera historical trajectory belongs is not adjacent to any camera number corresponding to a first detection trajectory, the cross-camera historical trajectory is removed. If the cross-camera historical trajectory is adjacent to any camera number corresponding to a first detection trajectory, the physical shortest path distance between the cross-camera historical trajectory and the adjacent camera is further extracted. When the physical shortest path distance does not exceed a preset spatial threshold (e.g., 200 meters), the cross-camera historical trajectory corresponding to the physical shortest path distance is determined as the second detection trajectory. When the physical shortest path distance exceeds the spatial threshold, it is determined whether the time interval corresponding to the physical shortest path distance is within a preset theoretical time threshold range (e.g., the theoretical time threshold range is [shortest path time, shortest path time + 90s]). When the time interval corresponding to the physical shortest path distance is within the theoretical time threshold range, the cross-mirror historical trajectory corresponding to the physical shortest path distance is determined as the second detection trajectory.

[0104] S43. Input each first detection trajectory and each second detection trajectory into a preset cost function to obtain multiple cost function values.

[0105] In this embodiment of the invention, each first detection trajectory and each second detection trajectory are paired up, and each pair is input into a preset cost function to obtain multiple cost function values.

[0106] It should be noted that the cost function is as follows: ; in, The cost function value, This is the first cost weighting coefficient, with a value of 0.8. This is the second cost weighting coefficient, with a value of 0.15. This is the third cost weighting coefficient, with a value of 0.05. For the first i The appearance feature vector of each trajectory For the first j The appearance feature vector of each trajectory For the first i The time of each trajectory For the first j The time of each trajectory For time window, For the first i Each trajectory corresponds to a camera. For the first j Each trajectory corresponds to a camera. This is the maximum spatial normalization factor, used to normalize physical distances to the [0, 1] interval. For the first i The camera corresponding to the trajectory and the first j The actual straight-line distance in Euclidean space between the cameras corresponding to each trajectory. i As the first index of the trajectory, j This is the second index of the trajectory.

[0107] S44. Construct the corresponding candidate cost matrix using the values ​​of each cost function.

[0108] In this embodiment of the invention, a corresponding candidate cost matrix is ​​constructed using various cost function values. The candidate cost matrix has the first detection trajectory as rows, the second detection trajectory as columns, and the cost function values ​​as matrix elements.

[0109] S45. The Hungarian algorithm is used to perform global optimal matching on the candidate cost matrix to obtain the corresponding initial matching results.

[0110] In this embodiment of the invention, the Hungarian algorithm is used to perform a global optimal assignment operation on the candidate cost matrix to obtain the corresponding initial matching result.

[0111] S46. When the initial matching result meets the preset adaptive threshold relaxation condition, the initial matching result is determined as the corresponding associated matching pair.

[0112] Adaptive threshold relaxation refers to the judgment rule in cross-camera matching that dynamically adjusts the matching threshold based on the time interval. The shorter the time interval, the stricter the threshold (close to 0.35), and the longer the time interval, the more relaxed the threshold (which can be relaxed to 0.30), in order to balance the precision of short-term matching with the recall of long-term matching.

[0113] In this embodiment of the invention, when the initial matching result satisfies the preset adaptive threshold relaxation condition, the initial matching result is determined as the corresponding associated matching pair. For example, for each initial matching result (i.e., the first and second detection trajectories that are successfully matched), the corresponding appearance similarity cost (i.e., the cosine distance between the appearance feature vectors of the first and second detection trajectories, denoted as d_app) and the time interval Δt between the two trajectories (the absolute value of the difference between the generation time of the first detection trajectory and the last observation time of the second detection trajectory) are extracted. Then, the current matching threshold τ_adaptive(Δt) = 0.35 - 0.05 × (1 - Δt / 1800) is dynamically calculated based on the time interval, where Δt is in seconds. When the appearance similarity cost is less than or equal to the matching threshold τ_adaptive(Δt), the initial matching result satisfies the preset adaptive threshold relaxation condition, and the initial matching result is determined as the corresponding associated matching pair.

[0114] S47. When the initial matching result does not meet the adaptive threshold relaxation condition, the lifetime count value of the first detection trajectory in the initial matching result is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0115] In this embodiment of the invention, when the initial matching result does not meet the adaptive threshold relaxation condition, the lifetime count value of the first detection trajectory in the initial matching result is adjusted to obtain a new active historical trajectory (i.e., the lifetime count value of the new active historical trajectory = the lifetime count value of the first detection trajectory + 1), and the process jumps to execute the step of using each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0116] Step 208: Use each associated matching pair as the corresponding global personnel trajectory.

[0117] In this embodiment of the invention, the local trajectory data of each set of associated matching pairs is time-series normalized after continuous internal updates, and a unique global identity is uniformly assigned to replace the local trajectory ID originally used within the range of a single camera. The personnel coordinates, acquisition time series, appearance feature templates after multi-frame iterative optimization, and camera number information of all frames of the trajectory are integrated synchronously. Discrete noise coordinates caused by instantaneous interference from the equipment in the trajectory are removed to complete trajectory smoothing correction. After normalization, the coherent and complete motion line is the formed global personnel trajectory.

[0118] In this embodiment of the invention, multiple video frames to be tested and multiple active historical trajectories from a substation are acquired. A pre-trained target detection model is used to detect targets in each video frame, resulting in multiple personnel detection boxes. These personnel detection boxes are then layered and filtered based on confidence levels to obtain multiple first target detection boxes and multiple second target detection boxes. Each active historical trajectory is used to correlate and match the first target detection boxes, resulting in multiple matching results. Finally, each second target detection box is used to re-correlate the matching results to obtain the corresponding global personnel trajectory. This overcomes the technical problem of existing personnel tracking technologies lacking a feature compensation mechanism under occlusion conditions. When a target is occluded, the Kalman filter-predicted trajectory easily drifts and diverges. When a person reappears in the frame, pure appearance features are difficult to correlate with historical trajectories, and the complete movement line is fragmented into unconnected trajectory islands, reducing the accuracy of cross-camera personnel trajectory unification. Compared with traditional personnel tracking technologies, this invention prioritizes the use of high-confidence first target detection boxes and active historical trajectories for the first round of association matching. This enables stable tracking of personnel within a single camera based on clear and complete personnel image information, maximizing trajectory matching accuracy under normal operating conditions and reducing unnecessary computational overhead. For trajectories that fail to match in the first round due to occlusion, dim lighting, or blurred distant views, a second re-association repair is performed using low-confidence second target detection boxes, effectively recovering lost or interrupted tracking trajectories. This avoids the numerous mismatches caused by using low-confidence detection boxes throughout the process and compensates for the shortcomings of a single high-confidence matching mode, which is prone to trajectory breaks and personnel tracking loss. It is suitable for complex monitoring environments in substations with numerous high-voltage areas, many obstructed scenes, and dispersed personnel movement routes, improving the accuracy of unified personnel trajectories across cameras.

[0119] Please see Figure 3 , Figure 3 This is a structural block diagram of a substation cross-camera personnel trajectory global unified system provided in Embodiment 3 of the present invention.

[0120] This invention provides a globally unified system for personnel tracing across cameras in a substation, comprising: The acquisition module 301 is used to acquire multiple video frames to be tested and multiple active historical trajectories of the substation. It performs target detection on each video frame to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes. The filtering module 302 is used to perform confidence-based stratified filtering on each person detection box to obtain multiple first target detection boxes and multiple second target detection boxes. Matching module 303 is used to perform association matching on each first target detection box using each active historical trajectory to obtain multiple matching results; The association module 304 is used to re-associate each matching result based on each second target detection box to obtain the corresponding global personnel trajectory.

[0121] Furthermore, the filtering module 302 includes: The first classification submodule is used to determine the person detection box as the first target detection box when the confidence of the person detection box is greater than or equal to the preset first confidence threshold. The second classification submodule is used to determine whether the confidence level of the personnel detection box is greater than or equal to the preset second confidence threshold when the confidence level of the personnel detection box is less than the first confidence threshold. When the person detection box is greater than or equal to the second confidence threshold, the person detection box is determined as the second target detection box.

[0122] Furthermore, the matching module 303 includes: The first matching submodule is used to perform Kalman filter prediction on each active historical trajectory to obtain multiple predicted trajectories. The second matching submodule is used to calculate the intersection-union ratio between each first target detection box and each predicted trajectory; The center offset direction corresponding to each first target detection box and the motion direction corresponding to each predicted trajectory are respectively processed by difference to obtain multiple first differences, and the absolute value of each first difference is determined as the consistency measure. When the crossover ratio is greater than the preset crossover ratio threshold, it is determined whether the consistency metric corresponding to the crossover ratio is less than the preset angle threshold. When the consistency metric corresponding to the crossover union ratio is less than the angle threshold, the first target detection box and the predicted trajectory corresponding to the crossover union ratio are determined as cost candidate pairs. A corresponding cost matrix is ​​constructed using each cost candidate pair, each first target detection box, and each predicted trajectory; The Hungarian algorithm is used to optimally assign the cost matrix, resulting in multiple matching outcomes.

[0123] Furthermore, the associated module 304 includes: The first association submodule is used to determine the successful match as the associated match when the matching result is a successful match; The second association submodule is used to extract the appearance features of each second target detection box when any matching result is the trajectory to be recovered, and obtain multiple appearance feature vectors. The third association submodule is used to perform appearance feature matching on each trajectory to be recovered based on each second target detection box and each appearance feature vector to obtain the corresponding association matching pair. The fourth association submodule is used to perform cross-camera matching on each initial detection trajectory based on the pre-acquired cross-camera feature library when any matching result is an initial detection trajectory, so as to obtain the corresponding association matching pair. The fifth association submodule is used to use each association matching pair as the corresponding global personnel trajectory.

[0124] Furthermore, the third associated submodule includes: The first pairing unit is used to pair each second target detection box with each trajectory to be recovered to obtain multiple candidate matching pairs; The second pairing unit is used to input the corresponding appearance feature vector and the trajectory to be recovered of each candidate matching pair into a preset cosine distance function to obtain multiple cosine distances; Based on the preset dual threshold judgment strategy, each candidate matching pair is filtered according to each cosine distance to obtain the corresponding associated matching pair and the reset trajectory to be restored; When the lifetime count value of the trajectory to be restored is less than the preset lifetime threshold, the lifetime count value of the trajectory to be restored is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0125] Furthermore, the fourth associated submodule includes: The third pairing unit is used to filter each initial detection trajectory based on a preset time window to obtain the corresponding first detection trajectory; Based on the preset topological adjacency constraints and each first detection trajectory, the cross-camera historical trajectories in the pre-acquired cross-camera feature library are filtered to obtain the corresponding second detection trajectories; The fourth pairing unit is used to input each first detection trajectory and each second detection trajectory into a preset cost function to obtain multiple cost function values; Construct the corresponding candidate cost matrix using the values ​​of each cost function; The Hungarian algorithm is used to perform global optimal matching on the candidate cost matrix to obtain the corresponding initial matching results; When the initial matching result meets the preset adaptive threshold relaxation condition, the initial matching result is determined as the corresponding associated matching pair. If the initial matching result does not meet the adaptive threshold relaxation condition, the lifetime count value of the first detection trajectory in the initial matching result is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each active historical trajectory to perform association matching on each first target detection box to obtain multiple matching results.

[0126] Please see Figure 4 , Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 4 of the present invention.

[0127] An electronic device according to an embodiment of the present invention includes: a memory 401 and a processor 402. The memory 401 stores a computer program. When the computer program is executed by the processor 402, the processor 402 executes the substation cross-camera personnel trajectory global unification method as described in any of the above embodiments.

[0128] Memory 401 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 401 has storage space 403 for program code 413 for performing any of the method steps described above. For example, storage space 403 for program code may include individual program codes 413 for implementing the various steps in the methods described above. This program code may be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When run by a computing processing device, this code causes the computing processing device to perform the various steps in the methods described above. This program code may be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When this code is run by a computing device, it causes the device to execute the steps in the globally unified method for cross-camera personnel trajectories in substations described above.

[0129] Embodiment 5 of the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the substation cross-camera personnel trajectory global unification method as described in any of the above embodiments.

[0130] Embodiment 6 of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the substation cross-camera personnel trajectory global unification method as described in any of the above embodiments.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for globally unifying personnel trajectories across cameras in a substation, characterized in that, include: Multiple video frames to be tested and multiple active historical trajectories of the substation are acquired. Target detection is performed on each of the video frames to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes. Each of the aforementioned personnel detection boxes is subjected to confidence-based stratified filtering to obtain multiple first target detection boxes and multiple second target detection boxes; Each of the aforementioned active historical trajectories is used to perform association matching on each of the first target detection boxes to obtain multiple matching results; Based on each of the second target detection boxes, the matching results are re-associated to obtain the corresponding global personnel trajectory.

2. The method for globally unifying personnel trajectories across cameras in substations according to claim 1, characterized in that, The step of performing confidence-based hierarchical filtering on each of the person detection boxes to obtain multiple first target detection boxes and multiple second target detection boxes includes: When the confidence level of the person detection box is greater than or equal to the preset first confidence threshold, the person detection box is determined as the first target detection box. When the confidence level of the personnel detection box is less than the first confidence level threshold, it is determined whether the confidence level of the personnel detection box is greater than or equal to the preset second confidence level threshold. When the confidence level of the person detection box is greater than or equal to the second confidence threshold, the person detection box is determined as the second target detection box.

3. The method for globally unifying personnel trajectories across cameras in substations according to claim 1, characterized in that, The step of associating and matching each of the first target detection boxes with each of the active historical trajectories to obtain multiple matching results includes: Each of the active historical trajectories is predicted using Kalman filtering to obtain multiple predicted trajectories; Calculate the intersection-union ratio between each of the first target detection boxes and each of the predicted trajectories; The center offset direction corresponding to each of the first target detection boxes and the motion direction corresponding to each of the predicted trajectories are respectively processed by difference to obtain multiple first differences, and the absolute value of each first difference is determined as a consistency measure. When the cross-union ratio is greater than the preset cross-union ratio threshold, it is determined whether the consistency metric corresponding to the cross-union ratio is less than the preset angle threshold. When the consistency metric corresponding to the intersection-union ratio is less than the angle threshold, the first target detection box and the predicted trajectory corresponding to the intersection-union ratio are determined as a cost candidate pair. A corresponding cost matrix is ​​constructed using each of the aforementioned cost candidate pairs, each of the first target detection boxes, and each of the aforementioned predicted trajectories; The Hungarian algorithm is used to optimally assign the cost matrix, resulting in multiple matching results.

4. The method for globally unifying personnel trajectories across cameras in substations according to claim 1, characterized in that, The step of re-associating each matching result based on each of the second target detection boxes to obtain the corresponding global personnel trajectory includes: When the matching result is a successful match, the successful match is determined as an associated match. When any of the matching results is the trajectory to be recovered, the appearance features of each of the second target detection boxes are extracted to obtain multiple appearance feature vectors. Based on each of the second target detection boxes and each of the appearance feature vectors, appearance feature matching is performed on each of the trajectories to be recovered to obtain corresponding association matching pairs; When any of the matching results is an initial detection trajectory, cross-camera matching is performed on each initial detection trajectory based on the pre-acquired cross-camera feature library to obtain the corresponding associated matching pair; Each of the aforementioned associated matching pairs is used as the corresponding global personnel trajectory.

5. The method for globally unifying personnel trajectories across cameras in substations according to claim 4, characterized in that, The step of performing appearance feature matching on each of the trajectories to be recovered based on each of the second target detection boxes and each of the appearance feature vectors to obtain corresponding associated matching pairs includes: Each of the second target detection boxes and each of the trajectories to be recovered are paired up to obtain multiple candidate matching pairs; Each candidate matching pair is input into a preset cosine distance function along with its corresponding appearance feature vector and the trajectory to be recovered, to obtain multiple cosine distances. Based on a preset dual-threshold determination strategy, each candidate matching pair is filtered according to each cosine distance to obtain the corresponding associated matching pair and the reset trajectory to be recovered; When the lifetime count value of the reset trajectory to be restored is less than the preset lifetime threshold, the lifetime count value of the reset trajectory to be restored is adjusted to obtain a new active historical trajectory, and the process jumps to execute the step of using each of the active historical trajectories to perform association matching on each of the first target detection boxes to obtain multiple matching results.

6. The method for globally unifying personnel trajectories across cameras in substations according to claim 4, characterized in that, The step of performing cross-camera matching on each of the initial detection trajectories based on a pre-acquired cross-camera feature library to obtain corresponding associated matching pairs includes: The initial detection trajectories are filtered based on a preset time window to obtain the corresponding first detection trajectory; Based on the preset topological adjacency constraints and each of the first detection trajectories, the cross-camera historical trajectories in the pre-acquired cross-camera feature library are filtered to obtain the corresponding second detection trajectory. Each of the first detection trajectories and each of the second detection trajectories are input into a preset cost function to obtain multiple cost function values; Construct corresponding candidate cost matrices using each of the aforementioned cost function values; The Hungarian algorithm is used to perform global optimal matching on the candidate cost matrix to obtain the corresponding initial matching results; When the initial matching result satisfies the preset adaptive threshold relaxation condition, the initial matching result is determined as the corresponding associated matching pair. When the initial matching result does not meet the adaptive threshold relaxation condition, the lifetime count value of the first detection trajectory in the initial matching result is adjusted to obtain a new active historical trajectory, and the process jumps to the step of using each of the active historical trajectories to perform association matching on each of the first target detection boxes to obtain multiple matching results.

7. A globally unified system for tracking personnel across cameras in a substation, characterized in that, include: The acquisition module is used to acquire multiple video frames to be tested and multiple active historical trajectories of the substation. It performs target detection on each of the video frames to be tested using a pre-trained target detection model to obtain multiple personnel detection boxes. The filtering module is used to perform confidence-based stratified filtering on each of the personnel detection boxes to obtain multiple first target detection boxes and multiple second target detection boxes; The matching module is used to perform association matching on each of the first target detection boxes using each of the active historical trajectories to obtain multiple matching results; The association module is used to re-associate each of the matching results based on each of the second target detection boxes to obtain the corresponding global personnel trajectory.

8. An electronic device, characterized in that, The device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the substation cross-camera personnel trajectory global unification method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the global unified method for personnel trajectory across cameras in substations as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein when the program instructions are executed by a computer, the computer performs the substation cross-camera personnel trajectory global unification method as described in any one of claims 1-6.

Images