Multi-camera combined pedestrian tracking method

Abstract

The invention relates to the technical field of computer vision, in particular to a multi-camera combined pedestrian tracking method, which comprises the following steps: performing feature processing in a detection area containing pedestrians to obtain pedestrian face identity information of the pedestrians, and extracting re-recognition features of the pedestrians; based on a reconfigurable space-time diagram model, constructing pedestrian face identity information from different cameras at different moments into a unified space diagram structure; based on a graph neural network, carrying out message transmission and aggregation on graph nodes in the spatial graph structure in a spatial dimension, and aggregating and reconstructing the graph nodes belonging to the same pedestrian into super nodes; obtaining global trajectory information of the pedestrian based on the timestamp and the motion trend of the pedestrian; and determining the identity of the pedestrian based on the global trajectory information and the pedestrian face identity information. The method can effectively solve the identity information confirmation problem in the pedestrian tracking process, can be applied to various general scenes, and reduces the equipment cost and the information acquisition cost.

Description

Technical Field

[0001] The embodiments of this application relate to the field of computer vision technology, and in particular to a pedestrian tracking method using multiple cameras. Background Technology

[0002] With the large-scale deployment of video surveillance equipment, multi-camera multi-target tracking (MCMT) has become a key technology in applications such as smart security, smart parks, and intelligent transportation. Traditional single-camera target tracking methods can only detect and track pedestrians within a fixed field of view, and cannot handle complex scenarios such as pedestrians moving across cameras, severe occlusion, or appearing in different locations, thus failing to meet the actual needs of large-scale monitoring.

[0003] Currently, deep learning has driven performance improvements in object detection and person re-identification (ReID) technologies. However, methods relying solely on pedestrian appearance features remain unstable under conditions of varying lighting, viewing angles, and resolution inconsistencies. Furthermore, in multi-camera systems, the spatial location, temporal synchronization, and degree of overlap between the images from each camera prevent simple feature matching algorithms from accurately establishing cross-camera pedestrian correspondences. Existing methods typically require complex and difficult-to-collect data (such as absolute position information), lacking versatility and ease of use. Therefore, a robust and high-precision cross-camera pedestrian tracking method is urgently needed to address issues such as unstable associations, insufficient identity matching, and low efficiency. Summary of the Invention

[0004] In view of this, embodiments of this application propose a multi-camera joint pedestrian tracking method, which effectively solves the problem of identity information verification in the pedestrian tracking process, and can be applied to various general scenarios, reducing equipment costs and information collection costs.

[0005] To achieve the above objectives, embodiments of this application propose a multi-camera combined pedestrian tracking method, the method comprising: Pedestrian detection is performed on multiple target images to obtain detection regions containing pedestrians; the target images are acquired by different cameras. Feature processing is performed within the detection area to obtain the pedestrian's facial identity information, and at the same time, the pedestrian's re-identification features are extracted; the target image is acquired by different cameras, and the pedestrian's facial identity information includes facial feature vectors and candidate identities; Based on a reconfigurable spatiotemporal graph model, pedestrian facial identity information from different cameras and at different times is constructed into a unified spatial graph structure. The spatial graph structure includes graph nodes corresponding to each pedestrian facial identity information, and the node attributes include camera identifier, re-identification features, and location information. Based on graph neural networks, message passing and aggregation are performed on graph nodes in the spatial graph structure in the spatial dimension, and graph nodes belonging to the same pedestrian are aggregated and reconstructed into super nodes; Based on timestamps and pedestrian movement trends, supernodes at different times are connected, and graph neural network inference is performed in the time dimension to obtain global trajectory information of pedestrians; The identity of a pedestrian is determined based on global trajectory information and pedestrian facial identity information.

[0006] To achieve the above objectives, embodiments of this application also propose a multi-camera combined pedestrian tracking device, the device comprising: The pedestrian detection module is used to perform pedestrian detection on multiple target images to obtain detection regions containing pedestrians; the target images are acquired by different cameras. The feature extraction module is used to perform feature processing within the detection area to obtain the pedestrian's facial identity information and extract the pedestrian's re-identification features. The target image is acquired by different cameras, and the pedestrian's facial identity information includes facial feature vectors and candidate identities. The module is used to construct a unified spatial graph structure based on a reconfigurable spatiotemporal graph model, which uses pedestrian facial identity information from different cameras and at different times. The spatial graph structure includes graph nodes corresponding to each pedestrian facial identity information, and the node attributes include camera identifier, re-identification features and location information. The node aggregation module is used to perform message passing and aggregation of graph nodes in the spatial graph structure based on graph neural networks, and to aggregate and reconstruct graph nodes belonging to the same person into super nodes. The trajectory generation module is used to connect super nodes at different times based on timestamps and pedestrian movement trends, and perform graph neural network inference in the time dimension to obtain the global trajectory information of pedestrians; The identity verification module is used to determine the identity of a pedestrian based on global trajectory information and pedestrian facial identity information.

[0007] To achieve the above objectives, embodiments of this application also propose an electronic device, including: a processor and a memory, wherein the memory stores instructions executable by the processor, and the processor is configured to execute the instructions such that the electronic device can implement a multi-camera combined pedestrian tracking method as described above.

[0008] To achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program that, when executed by a processor, enables a multi-camera combined pedestrian tracking method as described above.

[0009] Optionally, feature processing is performed within the detection area to obtain the pedestrian's facial identity information, including: performing face detection on the detection area of ​​each frame of the target image to locate the face region; inputting the face region into a pre-trained face feature vector extraction network to generate a fixed-dimensional face feature vector; and comparing the face feature vector with a face feature database to obtain the Top-K candidate identity identifiers and their corresponding confidence scores.

[0010] Optionally, the identity of a pedestrian is determined based on global trajectory information and pedestrian facial identity information, including: based on the facial feature vectors of each frame in the global trajectory information, a temporal weighted fusion strategy is adopted to fuse and weight the candidate identity identifiers and their corresponding confidence levels of each frame to obtain a trajectory-level identity probability distribution, and output an identity candidate list; wherein, the identity candidate list includes multiple candidate identity identifiers and their corresponding confidence levels; and the candidate identity identifiers in the identity candidate list with confidence levels higher than a preset confidence level are taken as the identity identifiers of the pedestrians.

[0011] Optionally, based on the facial feature vectors of each frame in the global trajectory information, a temporal weighted fusion strategy is adopted to fuse and weight the candidate identities and their corresponding confidence scores in each frame, resulting in a trajectory-level identity probability distribution, including: The global trajectory information includes The detection node; the first in the global trajectory information The face recognition results of the frame detection node are determined by the candidate identity identifier. and their corresponding confidence levels Composition; Trajectory-level identity probability distribution This can be expressed by the following formula: ; in, The weights are dynamic and are calculated dynamically based on the face quality of each frame. Face quality includes at least one of facial clarity, facial orientation, face area, and occlusion ratio.

[0012] Optionally, before performing pedestrian detection on multiple target images to obtain a detection region containing pedestrians, the method further includes: performing size normalization processing on the original images captured by multiple cameras to generate target images; using the target position of each camera as the origin of the local ground coordinate system, and based on sparse manual annotation and a nonlinear fitting model, generating a pedestrian ground coordinate sequence corresponding to each camera; wherein, the target position is the installation position or the vertical projection point of the ground.

[0013] Optionally, based on a reconfigurable spatiotemporal graph model, pedestrian facial identity information from different cameras and at different times is constructed into a unified spatial graph structure, including: using the reconfigurable spatiotemporal graph model, pedestrian facial identity information from different cameras within the current time window is used as graph nodes, and camera identifiers, pedestrian location information, and pedestrian re-identification features are used as node attributes of the graph nodes to construct a unified spatial graph structure.

[0014] Optionally, based on a graph neural network, message passing and aggregation are performed on graph nodes in the spatial graph structure in the spatial dimension, and graph nodes belonging to the same pedestrian are aggregated and reconstructed into a super node, including: based on a graph neural network, message passing is performed in the spatial dimension, the spatial correlation between graph nodes is calculated, graph nodes with a correlation higher than a preset threshold are determined as the same pedestrian, and merged into a super node; wherein, the spatial correlation is determined based on the ground position distance between nodes, the similarity of re-identification features, and the topological relationship of the camera.

[0015] This application proposes a multi-camera joint pedestrian tracking method. It performs pedestrian detection on multiple target images to obtain detection regions containing pedestrians; performs feature processing within these regions to obtain pedestrian facial identity information and extract re-identification features; based on a reconfigurable spatiotemporal graph model, it constructs a unified spatial graph structure from pedestrian facial identity information from different cameras and at different times; based on a graph neural network, it performs message passing and aggregation on graph nodes in the spatial graph structure in the spatial dimension, aggregating and reconstructing graph nodes belonging to the same pedestrian into supernodes; based on timestamps and pedestrian movement trends, it connects supernodes at different times and performs graph neural network inference in the temporal dimension to obtain global trajectory information of the pedestrian; based on the global trajectory information and pedestrian movement trends... Facial identity information is used to determine the identity of pedestrians. Since the target images are acquired through different cameras, the pedestrian facial identity information includes facial feature vectors and candidate identities. The spatial graph structure includes graph nodes corresponding to each pedestrian's facial identity information. Node attributes include camera identifiers, re-identification features, and location information. In this way, a reconfigurable spatiotemporal graph model can be used to construct a unified graph structure for pedestrian facial identity information from different cameras and at different times. Message passing in the spatial and temporal dimensions is achieved through graph neural networks to generate globally consistent pedestrian trajectories, and finally, the pedestrian's identity is determined. Based on this, the embodiments of this application can effectively solve the problem of identity information confirmation in the pedestrian tracking process while ensuring the accuracy of trajectory association. At the same time, it can be applied to various general scenarios, reducing equipment costs and information collection costs. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies of this application will be briefly introduced below. Obviously, the following drawings are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings described herein are only used to explain this application and are not intended to limit this application.

[0017] Figure 1 This is a flowchart of a multi-camera combined pedestrian tracking method provided in one embodiment of this application; Figure 2 This is a flowchart of a pedestrian detection method provided in one embodiment of this application; Figure 3 This is a schematic diagram of a geographic coordinate location calibration provided in one embodiment of this application; Figure 4 This is a flowchart of a multi-camera pedestrian tracking algorithm provided in one embodiment of this application; Figure 5 This is a schematic diagram of a pedestrian trajectory face recognition result provided in one embodiment of this application; Figure 6 This is an overall flowchart of a multi-camera combined pedestrian tracking method provided in one embodiment of this application; Figure 7 This is a schematic diagram of the structure of a multi-camera combined pedestrian tracking device provided in another embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. Those skilled in the art will understand that many technical details have been presented in the embodiments of this application to facilitate better understanding. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of this application. The following embodiments can be combined with and referenced by each other without contradiction.

[0019] With the large-scale deployment of video surveillance equipment, multi-camera collaborative pedestrian tracking has become a key technology in applications such as smart security, smart parks, and intelligent transportation. Traditional single-camera target tracking methods can only detect and track pedestrians within a fixed field of view, and cannot handle complex scenarios such as pedestrians moving across cameras, severe obstruction, or appearing in different locations, thus failing to meet the actual needs of large-scale monitoring.

[0020] Currently, deep learning has driven performance improvements in object detection and pedestrian re-identification technologies. However, methods relying solely on pedestrian appearance features remain unstable under conditions of varying lighting, viewing angles, and resolution inconsistencies. Furthermore, in densely populated environments with frequent occlusion, the clothing features of different pedestrians may be highly similar, further increasing the difficulty of cross-camera correlation.

[0021] Furthermore, in multi-camera systems, the images captured by each camera exhibit spatial differences, temporal asynchrony, and inconsistent levels of field-of-view overlap. This makes it difficult for simple feature matching algorithms to accurately establish pedestrian correspondence across cameras. In some scenarios, pedestrian appearance features are insufficient to distinguish identities, necessitating the use of more stable biometric information, such as facial feature vectors and pedestrian speed, to enhance the consistency of target identification.

[0022] Graph Neural Networks (GNNs) have significant advantages in modeling relationships between entities and processing non-Euclidean structured data. Applying them to cross-camera matching can effectively fuse the appearance features, spatial relationships, and motion trajectory information of the same pedestrian at different times and under different cameras, thereby improving the accuracy of cross-camera matching. However, existing methods often require overly complex and difficult-to-collect data, such as the absolute position information of pedestrian targets on the ground, and lack universality and ease of use for camera devices.

[0023] Therefore, there is an urgent need for a robust and high-precision cross-camera pedestrian tracking method to solve problems such as unstable association, insufficient identity matching, and low efficiency.

[0024] In view of this, embodiments of this application propose a multi-camera joint pedestrian tracking method, which can effectively solve the problem of identity information verification in the pedestrian tracking process, and can be applied to various general scenarios, reducing equipment costs and information collection costs.

[0025] One embodiment of this application proposes a multi-camera combined pedestrian tracking method applied to an electronic device, wherein the electronic device can be a terminal or a server. This embodiment and the following embodiments will use a server as an example for description. The implementation details of the multi-camera combined pedestrian tracking method proposed in this embodiment will be described in detail below. The following implementation details are provided for ease of understanding and are not necessary for implementing this solution.

[0026] The specific process of the multi-camera combined pedestrian tracking method proposed in this embodiment can be described as follows: Figure 1 As shown, it includes: Step 101: Perform pedestrian detection on multiple target images to obtain detection regions containing pedestrians.

[0027] The target images were captured by different cameras.

[0028] For example, a deep learning-based pedestrian detector can be used to analyze video streams captured by multiple cameras frame by frame. Taking the YOLO series detection model as an example, pedestrian detection is performed on the target image to obtain target detection box information containing pedestrians, i.e., the detection region; the target detection box information includes location coordinates, confidence score, and category label. The target detector can achieve robust pedestrian localization under complex backgrounds, lighting changes, and occlusion conditions, providing basic spatial location data for subsequent face region acquisition and cross-camera trajectory association. The detected pedestrian region image will be cropped and used as input for the subsequent face analysis module.

[0029] In one possible embodiment, prior to step 101, the method proposed in this application further includes: performing size normalization processing on the original images captured by multiple cameras to generate target images; and using the target position of each camera as the origin of the local ground coordinate system, generating a pedestrian ground coordinate sequence corresponding to each camera based on sparse manual annotation and a nonlinear fitting model.

[0030] The target location is either the installation location or a vertical projection point on the ground.

[0031] For example, after normalizing the size of the original images captured by multiple cameras, the resulting target image is an image adapted to the detection model.

[0032] For example, a letterbox scaling strategy consistent with that used in object detection models can be employed. This method scales input images of arbitrary size proportionally to the model's fixed input size (e.g., 640x640 pixels) while maintaining the original aspect ratio. By calculating the scaling factor *r* and the padding offset, any pixel in the original image is mapped to the scaled image coordinate space. Ultimately, any pixel in the original image is accurately mapped to the scaled image coordinate space, avoiding image distortion and ensuring that the object detection model can stably handle inputs of different resolutions.

[0033] For example, after preprocessing to generate the target image, sparse manual annotation and a nonlinear fitting model can be used to generate real-world ground coordinates for each camera. For instance, using the installation point or projection point of each camera as the origin of the local ground coordinate system, two-dimensional planar coordinates (x, y) can be used to represent the actual position of the pedestrian on the ground, thereby supporting cross-camera trajectory association.

[0034] For example, such as Figure 3 As shown, Figure 3 This diagram illustrates a geographic coordinate location calibration method provided for an embodiment of this application. A local ground coordinate system is defined for each camera. The camera's installation point (e.g., the camera's physical location) or its projection point (e.g., the projection of the camera's field of view center onto the ground) is used as the origin of the coordinate system. This allows the pixel coordinates in the image to be converted into unified ground spatial coordinates, facilitating spatial association between multiple cameras. For example, in security scenarios, cameras may be installed at different heights and angles; using the projection point as the origin simplifies coordinate transformation calculations. Since full-density annotation for all pedestrian targets in the target image is costly, a sparse sampling nonlinear modeling completion strategy can be adopted. Specifically, for each pedestrian target, only a small amount of sparse manual annotation is performed on keyframes or salient locations (e.g., when a pedestrian enters the camera's field of view, at a turning point, or when there is minimal occlusion). The annotation content is the pedestrian's two-dimensional coordinates (x, y) on the ground, estimated based on the image detection box and camera parameters. The frequency of sparse annotation can be dynamically adjusted according to the scene; for example, adding annotation points in densely populated areas can improve fitting accuracy. This method reduces the annotation workload from full frame-by-frame annotation to key point sampling, significantly reducing labor costs and time expenditure.

[0035] After obtaining sparsely labeled points, a nonlinear regression model is used to fit the labeled data, generating high-precision, temporally continuous comprehensive ground location labels. The document mentions fitting models including spline interpolation, polynomial fitting, or trajectory regression models using lightweight neural networks. Specifically, the model takes timestamps and sparsely labeled coordinates as input and reconstructs the continuous movement trajectory of pedestrians throughout the entire time series using mathematical fitting algorithms (such as cubic spline interpolation to ensure smoothness, or polynomial regression to capture trends). The fitting process considers motion consistency; for example, it interpolates coordinates based on pedestrian speed and time intervals to avoid abrupt jumps. The final output is a continuous ground coordinate sequence for each pedestrian trajectory, which can be used for spatial correlation in multi-camera tracking. Approximating real-world geographic locations using sparse 2D point labels solves the technical problem of difficult absolute location information acquisition, improving the algorithm's versatility.

[0036] Step 102: Perform feature processing within the detection area to obtain the pedestrian's facial identity information, and simultaneously extract the pedestrian's re-identification features.

[0037] Among them, pedestrian facial identity information includes facial feature vectors and candidate identities.

[0038] In one possible embodiment, feature processing is performed within the detection area to obtain the pedestrian's facial identity information, including: performing face detection on the detection area of ​​each frame of the target image to locate the face region; inputting the face region into a pre-trained face feature vector extraction network to generate a fixed-dimensional face feature vector; and comparing the face feature vector with a face feature database to obtain the Top-K candidate identity identifiers and their corresponding confidence scores.

[0039] For example, after obtaining the target detection bounding box information of pedestrians, i.e., the detection region, facial information of pedestrians within the detection region can be further extracted. First, a lightweight or high-precision face detection model, such as SCRFD, is used to locate faces in the cropped detection region, obtaining the detection bounding box of the facial region (i.e., the face region). Then, the detection bounding box of the facial region is fed into a pre-trained face feature vector extraction network (e.g., a CNN model based on ArcFace, FaceNet, or a Transformer model) to calculate a fixed-dimensional face feature vector. Subsequently, this vector is compared with the face feature vector library pre-built by the system, and identity matching is achieved through cosine similarity or Euclidean distance to obtain Top-K face information. Finally, each pedestrian in the target image is assigned a corresponding face recognition result (i.e., pedestrian face identity information) for the confirmation of face identity information of pedestrian trajectories.

[0040] For example, such as Figure 2 As shown, Figure 2This document provides a flowchart for pedestrian detection in an embodiment of this application. First, raw video frames from various cameras are processed using the YOLO object detection algorithm to output pedestrian images, i.e., target images. Then, SCRFD (an efficient and accurate face detector) is used to further locate face regions in the pedestrian images. Based on the detected face bounding boxes, standardized face images are cropped from the pedestrian images. Using a face recognition model trained on architectures such as ResNet, the face image is encoded into a fixed-dimensional, discriminative feature vector (i.e., a face feature embedding). This vector represents the deep features of the face. This vector is then quickly compared and retrieved against a pre-built face feature database (e.g., using the Faiss vector library). The Faiss vector library can quickly find the most similar entries in a massive number of vectors, thus determining whether the face has appeared before. Simultaneously, a ResNet-specific model is used to process the target image to obtain re-identification features representing the overall appearance of the pedestrian.

[0041] Understandably, to achieve globally consistent pedestrian trajectory fusion in a multi-camera environment, the method provided in this application introduces a reconfigurable spatial-temporal graph model (ReST). The Reconfigurable spatial-temporal graph model can construct a unified graph structure from detection results at different times from different cameras, and perform cross-viewpoint and cross-time correlation reasoning through graph neural networks, ultimately generating a unique global trajectory ID. After generating the global trajectory ID, the trajectory face identity is generated using the weights of face information from all historical trajectories. See steps 103 to 105 below for details.

[0042] Step 103: Based on the reconfigurable spatiotemporal graph model, construct a unified spatial graph structure from pedestrian facial identity information from different cameras and at different times.

[0043] The spatial graph structure includes graph nodes corresponding to the facial identity information of each pedestrian, and the node attributes include camera identification, re-identification features, and location information.

[0044] Understandably, integrating pedestrian facial identity information from different cameras into a coherent graph model, through spatial and temporal correlation, can improve the accuracy and robustness of cross-camera tracking.

[0045] In one possible embodiment, step 103 includes: using a reconfigurable spatiotemporal graph model, taking the pedestrian face identity information of different cameras within the current time window as graph nodes, and taking the camera identifier, pedestrian location information, and pedestrian re-identification features as node attributes of the graph nodes, in order to construct a unified spatial graph structure.

[0046] For example, the ReST model constructs a spatial graph based on pedestrian detection instances from all cameras within the current time window (e.g., video frames within a few seconds). Each detection instance is defined as a graph node, and the node attributes integrate information from multiple sources to ensure the reliability of subsequent associations.

[0047] For example, node attributes may include: a camera identifier, representing the camera number or unique identifier of the detection source, used to distinguish data sources from different viewpoints. Pedestrian location information includes the geometric coordinates of the pedestrian's position in a unified ground coordinate system (e.g., two-dimensional plane coordinates). Pedestrian re-identification features, i.e., pedestrian appearance embeddings, are extracted through a pre-trained pedestrian re-identification model (e.g., a ReID network based on ResNet or Transformer), representing visual features such as clothing, color, and posture, enhancing the ability to distinguish identities under cross-camera lighting changes and occlusion.

[0048] Step 104: Based on the graph neural network, perform message passing and aggregation on the graph nodes in the spatial graph structure in the spatial dimension, and aggregate and reconstruct the graph nodes belonging to the same pedestrian into super nodes; In one possible embodiment, step 104 includes: performing message passing in the spatial dimension based on the graph neural network, calculating the spatial correlation between graph nodes, determining graph nodes with a correlation higher than a preset threshold as the same pedestrian, and merging them into a super node; Spatial correlation is determined based on the ground location distance between nodes, the similarity of re-identified features, and the topological relationship of cameras.

[0049] For example, the edges of the spatial graph are initialized as potential associations within or across cameras. Spatial consistency between nodes can be learned through the message passing mechanism of a graph neural network (GNN). For instance, different camera detection nodes belonging to the same pedestrian will gradually aggregate.

[0050] For example, based on the spatial graph, the ReST model performs a graph reconfiguration step to optimize the graph structure and reduce redundancy.

[0051] By reasoning through the spatial graph using a Generative Neural Network (GNN), the model clusters spatially adjacent, visually similar nodes from different cameras, classifying them as instances of the same pedestrian. These nodes are then merged into a single supernode, and a more compact, higher-level graph structure is reconstructed based on these supernodes, providing a more stable representation of the target for subsequent temporal modeling. The reconstructed graph, composed of supernodes with reconnected edges to reflect higher-level spatial relationships, reduces graph complexity and lays the foundation for temporal expansion. The reconstruction process enhances the model's robustness to occlusion and viewpoint changes because the supernodes integrate multi-viewpoint information.

[0052] Step 105: Based on the timestamp and the pedestrian's movement trend, connect the super nodes at different times, and perform graph neural network inference in the time dimension to obtain the pedestrian's global trajectory information; For example, supernodes with different timestamps can be connected, with edge weights based on information such as timestamp differences, motion trends (e.g., velocity vectors), and similarity of appearance features. For instance, supernodes with smooth motion trajectories in consecutive frames will be preferentially associated.

[0053] By passing messages again through a GNN on the time map, the ReST model learns to establish continuous cross-temporal associations, integrates tracking information across cameras and time periods, and automatically fills in trajectory gaps caused by brief occlusions. Finally, through iterative optimization, the system assigns a unique global trajectory ID (i.e., the pedestrian's global trajectory information) to each pedestrian, ensuring the continuity of the trajectory in cross-camera scenarios.

[0054] For example, such as Figure 4 As shown, Figure 4 This document presents a flowchart of a multi-camera pedestrian tracking algorithm provided in an embodiment of this application. First, target images from three cameras are acquired, and a spatial graph neural network (GNN) is used to construct a global spatial graph from the detected targets (such as pedestrians) from all cameras. The initial association results output by the spatial GNN are then optimized, i.e., post-processing. This typically includes threshold filtering (removing edges with low confidence) and clustering (forming target clusters across cameras) to obtain a cleaner and more reliable association set. The multiple "target clusters" or association groups obtained from the previous post-processing step are abstracted and merged into new high-level nodes, thereby constructing a new and simpler graph. The nodes representing the same target at different time points (generated from consecutive video frames) in the reconstructed graph are connected by temporal edges to construct a temporal graph. The temporal GNN learns the target's motion pattern over time to complete cross-frame tracking of the target, generating a continuous and complete trajectory. Finally, the output image should typically contain a tracking box with a unique ID number, indicating that the system has successfully identified and associated the same target at different cameras and different times, achieving continuous tracking across cameras and across time.

[0055] Step 106: Based on global trajectory information and pedestrian facial identity information, determine the pedestrian's identity.

[0056] In one possible embodiment, step 106 includes: based on the face feature vectors of each frame in the global trajectory information, using a temporal weighted fusion strategy, fusing and weighting the candidate identity identifiers and their corresponding confidence levels of each frame to obtain a trajectory-level identity probability distribution, and outputting an identity candidate list; and using the candidate identity identifiers in the identity candidate list whose confidence level is higher than a preset confidence level as the identity identifiers of the pedestrian.

[0057] The candidate identity list includes multiple candidate identity identifiers and their corresponding confidence levels; For example, after the ReST model completes the global trajectory generation under multiple cameras, the method provided in this application proposes a trajectory-level face information weighted fusion method to generate a face candidate list for each global trajectory. This method does not use face features as input for multi-camera association, but rather as supplementary information for trajectory attributes, improving the stability and reliability of identity prediction through multi-frame fusion.

[0058] At each detection node of the trajectory, a single-frame face recognition result is generated by the face recognition module. For a given frame in the trajectory, if a face is successfully detected, a list containing the Top-K candidate identities and their corresponding matching confidence scores is generated. The confidence score is typically obtained by calculating the cosine similarity between the extracted face feature vector and registered features in the face feature database, and then normalizing the result.

[0059] In one possible embodiment, based on the facial feature vectors of each frame in the global trajectory information, a temporal weighted fusion strategy is used to fuse and weight the candidate identities and their corresponding confidence scores in each frame to obtain a trajectory-level identity probability distribution, including: The global trajectory information includes The detection node; the first in the global trajectory information The face recognition results of the frame detection node are determined by the candidate identity identifier. and their corresponding confidence levels Composition; Trajectory-level identity probability distribution This can be expressed by the following formula: ; in, The weights are dynamic and are calculated dynamically based on the face quality of each frame. Face quality includes at least one of facial clarity, facial orientation, face area, and occlusion ratio.

[0060] For example, dynamic weights Instead of a simple average, the weighting is dynamically calculated based on the quality and reliability of the face image in each frame. Images with clearer images and less noise contain more reliable facial features and therefore receive higher weights. Frontal or near-frontal face images have a higher match rate with registered frontal templates in the database, thus receiving higher weights than images with side profiles or excessively large tilt angles. Faces occupying a large pixel area in the image and not severely obscured by other objects (such as masks, glasses, hands, or other pedestrians) are more accurately detected and their feature extraction is more reliable. Frames in the middle of the trajectory are generally more stable than those at the beginning or end. This is because when a target first enters or is about to leave the field of view, it may be at the edge, and its image quality or integrity may be poor. Therefore, frames in the middle of the trajectory can be assigned slightly higher weights.

[0061] For example, after calculating the weighted probabilities for all candidate identities, the system can output a probability calculated based on the final fusion probability for each global trajectory. A list of Top-K identity candidates arranged in descending order, such as Figure 5 As shown. Each candidate identity in the list is defined by attributes. Composition, in which It is a facial recognition ID. This is the corresponding trajectory-level similarity probability. The system will traverse this candidate list and officially identify the highest-ranking candidate identity with a confidence level higher than a preset confidence threshold (e.g., 0.9) as the identity of the pedestrian (i.e., the trajectory).

[0062] Optionally, if the probability of no candidate identity exceeds the threshold, the identity of the trajectory may be marked as "unknown" and updated after a subsequent frame with higher confidence appears.

[0063] like Figure 6 As shown, Figure 6 This is an overall flowchart of a multi-camera joint pedestrian tracking method provided in an embodiment of this application. First, the data is preprocessed. Then, pedestrian detection is performed based on the YOLO algorithm, and the pedestrian's re-identification ID is obtained, followed by face comparison. Pedestrian trajectories are tracked using multiple cameras, and finally, face post-processing is performed based on the trajectories. From spatial graph initialization to temporal graph fusion, the ReST model dynamically adjusts the graph topology to adapt to real-time monitoring scenarios. The model output includes a global ID for each trajectory, and a weighted fusion result of facial information used for identity enhancement. This method not only improves tracking accuracy but also reduces computational overhead through reconfigurable design.

[0064] This application proposes a multi-camera joint pedestrian tracking method. It performs pedestrian detection on multiple target images to obtain detection regions containing pedestrians; performs feature processing within these regions to obtain pedestrian facial identity information and extract re-identification features; based on a reconfigurable spatiotemporal graph model, it constructs a unified spatial graph structure from pedestrian facial identity information from different cameras and at different times; based on a graph neural network, it performs message passing and aggregation on graph nodes in the spatial graph structure in the spatial dimension, aggregating and reconstructing graph nodes belonging to the same pedestrian into supernodes; based on timestamps and pedestrian movement trends, it connects supernodes at different times and performs graph neural network inference in the temporal dimension to obtain global trajectory information of the pedestrian; based on the global trajectory information and pedestrian movement trends... Facial identity information is used to determine the identity of pedestrians. Since the target images are acquired through different cameras, the pedestrian facial identity information includes facial feature vectors and candidate identities. The spatial graph structure includes graph nodes corresponding to each pedestrian's facial identity information. Node attributes include camera identifiers, re-identification features, and location information. In this way, a reconfigurable spatiotemporal graph model can be used to construct a unified graph structure for pedestrian facial identity information from different cameras and at different times. Message passing in the spatial and temporal dimensions is achieved through graph neural networks to generate globally consistent pedestrian trajectories, and finally, the pedestrian's identity is determined. Based on this, the embodiments of this application can effectively solve the problem of identity information confirmation in the pedestrian tracking process while ensuring the accuracy of trajectory association. At the same time, it can be applied to various general scenarios, reducing equipment costs and information collection costs.

[0065] The steps described above are for clarity only. In implementation, they can be combined into one step, or some steps can be broken down into multiple steps, as long as they involve the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0066] Another embodiment of this application proposes a multi-camera combined pedestrian tracking device. The details of this multi-camera combined pedestrian tracking device are described below. The following content is for ease of understanding and is not essential for implementing this example. Figure 7 This is a schematic diagram of the structure of a multi-camera combined pedestrian tracking device proposed in this embodiment, including: The pedestrian detection module 710 is used to perform pedestrian detection on multiple target images to obtain detection regions containing pedestrians; wherein, the target images are acquired by different cameras; The feature extraction module 720 is used to perform feature processing within the detection area to obtain the pedestrian's facial identity information and extract the pedestrian's re-identification features. The target image is acquired by different cameras, and the pedestrian's facial identity information includes facial feature vectors and candidate identities. Module 730 is used to construct a unified spatial graph structure based on a reconfigurable spatiotemporal graph model, using pedestrian facial identity information from different cameras and at different times. The spatial graph structure includes graph nodes corresponding to each pedestrian facial identity information, and the node attributes include camera identifier, re-identification features, and location information. The node aggregation module 740 is used to perform message passing and aggregation of graph nodes in the spatial graph structure in the spatial dimension based on graph neural network, and to aggregate and reconstruct graph nodes belonging to the same person into super nodes; The trajectory generation module 750 is used to connect super nodes at different times based on timestamps and pedestrian movement trends, and perform graph neural network inference in the time dimension to obtain global trajectory information of pedestrians; The identity verification module 760 is used to determine the identity of a pedestrian based on global trajectory information and pedestrian facial identity information. It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and effects mentioned in the above method embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.

[0067] It is worth mentioning that all modules and units involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units do not exist in this embodiment.

[0068] Another embodiment of this application provides an electronic device, such as Figure 8 As shown, it includes a processor 81 and a memory 82. The memory 82 stores instructions that the processor 81 can execute. When the processor 81 is configured to execute the instructions, the electronic device can implement a multi-camera combined pedestrian tracking method as described in the above method embodiment.

[0069] The memory and processor are connected via a bus, which includes any number of interconnecting buses and bridges, connecting various circuits of one or more processors and the memory. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0070] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0071] Another embodiment of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, can implement a multi-camera combined pedestrian tracking method as described in the above method embodiments.

[0072] That is, those skilled in the art will understand that all or part of the steps in the above method embodiments can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (such as a microcontroller, chip, etc.) or processor to execute all or part of the steps of the method described in the method embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0073] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application. For those skilled in the art, several improvements and modifications can be made without departing from the principles of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A method for pedestrian tracking using multiple cameras, characterized in that, The method includes: Pedestrian detection is performed on multiple target images to obtain detection regions containing pedestrians; the target images are acquired by different cameras. Feature processing is performed within the detection area to obtain pedestrian facial identity information, and pedestrian re-identification features are extracted at the same time; among which, pedestrian facial identity information includes facial feature vector and candidate identity; Based on a reconfigurable spatiotemporal graph model, pedestrian facial identity information from different cameras and at different times is constructed into a unified spatial graph structure. The spatial graph structure includes graph nodes corresponding to each pedestrian facial identity information, and the node attributes include camera identifier, re-identification features, and location information. Based on graph neural networks, message passing and aggregation are performed on graph nodes in the spatial graph structure in the spatial dimension, and graph nodes belonging to the same pedestrian are aggregated and reconstructed into super nodes; Based on timestamps and pedestrian movement trends, supernodes at different times are connected, and graph neural network inference is performed in the time dimension to obtain global trajectory information of pedestrians; The identity of a pedestrian is determined based on global trajectory information and pedestrian facial identity information.

2. The method according to claim 1, characterized in that, The step of performing feature processing within the detection area to obtain pedestrian facial identity information includes: Face detection is performed on the detection area of ​​each frame of the target image to locate the face region; The face region is input into a pre-trained face feature vector extraction network to generate a fixed-dimensional face feature vector; The facial feature vector is compared with the facial feature database to obtain the Top-K candidate identity identifiers and their corresponding confidence scores.

3. The method according to claim 1, characterized in that, The process of determining a pedestrian's identity based on global trajectory information and pedestrian facial recognition information includes: Based on the facial feature vectors of each frame in the global trajectory information, a temporal weighted fusion strategy is adopted to fuse and weight the candidate identity identifiers and their corresponding confidence scores of each frame to obtain the trajectory-level identity probability distribution and output the identity candidate list; wherein, the identity candidate list includes multiple candidate identity identifiers and their corresponding confidence scores; Candidate identity identifiers with a confidence level higher than the preset confidence level in the candidate identity list will be used as the identity identifiers of pedestrians.

4. The method according to claim 3, characterized in that, The facial feature vectors of each frame based on global trajectory information are fused using a temporal weighted fusion strategy to fuse and weight the candidate identities and their corresponding confidence levels in each frame, resulting in a trajectory-level identity probability distribution, including: The global trajectory information includes The detection node; the first in the global trajectory information The face recognition results of the frame detection node are determined by the candidate identity identifier. and their corresponding confidence levels Composition; Trajectory-level identity probability distribution This can be expressed by the following formula: ； in, The weights are dynamic and are calculated dynamically based on the face quality of each frame. Face quality includes at least one of facial clarity, facial orientation, face area, and occlusion ratio.

5. The method according to claim 1, characterized in that, Before performing pedestrian detection on multiple target images to obtain detection regions containing pedestrians, the method further includes: The original images captured by multiple cameras are normalized in size to generate the target image; Using the target position of each camera as the origin of the local ground coordinate system, a pedestrian ground coordinate sequence corresponding to each camera is generated based on sparse manual annotation and nonlinear fitting model; where the target position is the installation position or the vertical projection point of the ground.

6. The method according to claim 1, characterized in that, The reconfigurable spatiotemporal graph model constructs a unified spatial graph structure from pedestrian facial identity information from different cameras and at different times, including: By using a reconfigurable spatiotemporal graph model, pedestrian facial identity information from different cameras within the current time window is used as graph nodes, and camera identifiers, pedestrian location information, and pedestrian re-identification features are used as node attributes of the graph nodes to construct a unified spatial graph structure.

7. The method according to claim 6, characterized in that, The method based on graph neural networks performs message passing and aggregation on graph nodes in a spatial graph structure in the spatial dimension, and aggregates and reconstructs graph nodes belonging to the same pedestrian into super nodes, including: Based on graph neural networks, message passing is performed in the spatial dimension, the spatial correlation between graph nodes is calculated, and graph nodes with a correlation higher than a preset threshold are identified as the same pedestrian and merged into a super node. Spatial correlation is determined based on the ground location distance between nodes, the similarity of re-identified features, and the topological relationship of cameras.

8. A pedestrian tracking device using multiple cameras, characterized in that, The device includes: The pedestrian detection module is used to perform pedestrian detection on multiple target images to obtain detection regions containing pedestrians; the target images are acquired by different cameras. The feature extraction module is used to perform feature processing within the detection area to obtain the pedestrian's facial identity information and extract the pedestrian's re-identification features; wherein, the pedestrian's facial identity information includes facial feature vectors and candidate identities; The module is used to construct a unified spatial graph structure based on a reconfigurable spatiotemporal graph model, which uses pedestrian facial identity information from different cameras and at different times. The spatial graph structure includes graph nodes corresponding to each pedestrian facial identity information, and the node attributes include camera identifier, re-identification features and location information. The node aggregation module is used to perform message passing and aggregation of graph nodes in the spatial graph structure based on graph neural networks, and to aggregate and reconstruct graph nodes belonging to the same person into super nodes. The trajectory generation module is used to connect super nodes at different times based on timestamps and pedestrian movement trends, and perform graph neural network inference in the time dimension to obtain the global trajectory information of pedestrians; The identity verification module is used to determine the identity of a pedestrian based on global trajectory information and pedestrian facial identity information.

9. An electronic device, characterized in that, include: The processor and memory, wherein the memory stores instructions executable by the processor, and the processor is configured to, when executing the instructions, enable the electronic device to implement the multi-camera combined pedestrian tracking method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement the multi-camera combined pedestrian tracking method as described in any one of claims 1 to 7.

Images