A personnel matching and cooperative tracking system across ball machine cameras

Abstract

The application discloses a personnel matching and cooperative tracking system across ball machine cameras and belongs to the technical field of video monitoring and intelligent security and protection. The personnel matching and cooperative tracking system comprises a video access module, a target detection and feature extraction module, a zooming self-adaption and target region maintaining module, a cross-camera matching module and a cooperative tracking control module. The video access module is used for acquiring multiple ball machine video streams and pan-tilt attitude information and transmitting the same to the target detection and feature extraction module to extract target detection results, human key point information and appearance re-identification features. The zooming self-adaption and target region maintaining module compensates and adjusts a target region scale according to a current zooming state, so that the target region is kept stable in the ball machine movement and zooming process. The target information processed is transmitted to the cross-camera matching module, the cross-camera mapping relationship is calculated according to calibration information, the candidate matching pairs are generated in combination with dynamic offset compensation, the cross-ball machine personnel matching scores are calculated, and the personnel corresponding relationship between the cross-ball machines is determined. Finally, the cooperative tracking control module allocates the ball machine camera tracking authority, controls the corresponding ball machine pan-tilt to execute target personnel tracking, and coordinates the cooperative work of multiple ball machines.

Description

Technical Field

[0001] This invention relates to the field of video surveillance and intelligent security technology, and in particular to a personnel matching and collaborative tracking system for PTZ cameras. Background Technology

[0002] Existing video surveillance systems, in multi-camera collaborative scenarios, typically rely on independent detection and tracking by a single camera, making it difficult to accurately identify the same person across different cameras. Furthermore, most existing personnel matching and collaborative tracking technologies are applied to bullet camera scenarios. Because bullet cameras have relatively fixed installation positions and shooting directions, the field-of-view relationships, target imaging scales, and spatial position mappings between cameras are usually quite stable, allowing for cross-camera association based on appearance features, spatial position, or trajectory information. However, in actual security monitoring, PTZ cameras are widely used due to their pan-tilt-zoom (PTZ) rotation, zoom, and dynamic viewing angle adjustment capabilities. But compared to bullet cameras, PTZ cameras experience continuous changes in their field of view, observation angle, target scale, and background environment during operation, leading to significantly increased differences in target imaging between different cameras at different times. This makes traditional cross-camera personnel matching and collaborative tracking methods, designed for fixed viewing angles, difficult to directly adapt to PTZ camera scenarios.

[0003] In existing technologies, one type of method mainly relies on the target's appearance features for cross-camera re-identification, but its accuracy is limited under conditions of low resolution, pose changes, occlusion, or lighting changes; another type of method mainly relies on spatial position constraints, but it is prone to mismatches when there are PTZ camera rotations, calibration errors, and distortions at the edge of the field of view.

[0004] Therefore, there is an urgent need for a cross-PTZ personnel matching and collaborative tracking scheme that can adapt to the dynamic changes in the field of view of the PTZ camera, while taking into account geometric relationships, human body structure information and temporal stability.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a cross-PTZ personnel matching and collaborative tracking scheme that can adapt to the dynamic field of view changes of the PTZ camera while taking into account geometric relationships, human body structure information and temporal stability.

[0007] To achieve the above objectives, the present invention provides the following solution: A system for matching and collaboratively tracking people across PTZ cameras includes: The video access module is used to acquire video streams from multiple PTZ cameras and gimbal attitude information; The target detection and feature extraction module is used to perform target detection and feature extraction on the video stream of the PTZ camera to obtain target detection results, human key point information and appearance re-identification features; The zoom adaptation and target area preservation module is used to compensate and adjust the scale of the target area according to the zoom state of the PTZ camera, and to keep the target area stable during PTZ camera movement and zooming. The cross-camera matching module is used to calculate the cross-camera mapping relationship based on the calibration information, project the target position in the first PTZ camera to the coordinate domain of the second PTZ camera, generate candidate matching pairs by combining dynamic offset compensation, calculate the cross-PTZ personnel matching score, and determine the personnel correspondence between the cross-PTZ cameras. The collaborative tracking control module is used to allocate tracking permissions for PTZ cameras based on the personnel correspondence between the PTZ cameras, control the corresponding PTZ cameras to perform target personnel tracking, and coordinate the collaborative work of multiple PTZ cameras.

[0008] Optionally, the video access module is further configured to: The system acquires video stream data from at least two PTZ cameras and parses the video stream data frame by frame to obtain the image frame corresponding to the current moment. The video access module is also used to acquire the gimbal attitude information of the accessed PTZ cameras. The gimbal attitude information includes at least the current horizontal rotation angle information and the current gimbal focal length parameter. The video access module can also acquire the preset point information and inspection status information corresponding to the PTZ cameras. The video access module uses the image frames and gimbal attitude information to form input data for the target detection and feature extraction module, and uses the preset point information and inspection status information to plan the camera inspection logic of the collaborative tracking control module.

[0009] Optionally, the target detection and feature extraction module is further used for: This is used to receive image frames output by the video access module, associate the same target in consecutive frames, and assign corresponding target identifier IDs based on the principle of minimizing the Euclidean distance between the center points of the target detection box and the temporal continuity constraint, thereby obtaining the target detection results and corresponding target identifier information; and extract human key point information and appearance re-identification features of each target based on the target detection results to form target feature data for processing by the zoom adaptation and target region preservation module and the cross-camera matching module.

[0010] Optionally, the zoom adaptation and target area preservation module is further configured to: Obtain the current pan-tilt focal length parameter or zoom status information of the PTZ camera; Based on the reference target area, the scale change of the target area is compensated and adjusted according to the current gimbal focal length parameter or zoom status information; During the movement and zooming of the PTZ camera, the position and range of the target area are updated and maintained to keep the target area stable; The target area information, after compensation adjustment and area preservation, is output to the cross-camera matching module for subsequent candidate matching pair generation and cross-PTZ personnel matching.

[0011] Optionally, the compensation adjustment for scale changes in the target region includes: A center region scaling estimation method based on log-polar transform and phase correlation is used to obtain the scaling factor of the target region before and after zooming; The width and height of the target area are proportionally adjusted according to the scaling factor; The target region is reprojected based on the projection transformation relationship between the reference target region and the current video frame, and the position of the target region is corrected by combining local translation compensation. The target region is updated, smoothed, and preserved by combining the target tracking results in consecutive frames in order to maintain the stability of the target region.

[0012] Optionally, the cross-camera matching module is further used for: A cross-camera mapping model is constructed based on the corresponding calibration point information in the common field of view of the first and second PTZ cameras. The cross-camera mapping model includes a homography transformation matrix. Extract the ground position anchor points of each target in the first PTZ camera, and project the ground position anchor points to the coordinate domain of the second PTZ camera based on the cross-camera mapping model to obtain the corresponding mapped positions; The dynamic offset compensation amount at the current moment is estimated based on the positional offset relationship between the ground position of each target in the second PTZ camera and the mapped position, and the mapped position is compensated and corrected in combination with the zoom state compensation result to generate candidate matching pairs; Based on the candidate matching pairs, the similarity of human posture, consistency of spatial mapping, similarity of appearance re-identification, and relative order of targets are fused, the cross-PTZ personnel matching score is calculated, and the correspondence between personnel across the cross-PTZ cameras is determined according to the cross-PTZ personnel matching score.

[0013] Optionally, the calculation of the cross-PTZ personnel matching score includes: A multi-feature weighted fusion model is adopted to fuse human pose similarity, spatial mapping consistency, appearance re-identification similarity, and target relative order relationship to obtain the initial matching confidence of candidate matching pairs. The initial matching confidence in consecutive frames is then smoothed by exponential moving average (EMA) to obtain the smoothed matching confidence. When the smoothed matching confidence meets a preset threshold condition within a preset number of consecutive frames, the corresponding candidate matching pair is confirmed as a cross-PTZ personnel correspondence.

[0014] Optionally, the cooperative tracking control module is used for: Under normal circumstances, the PTZ camera is inspected and controlled based on preset point information and inspection status information. During the personnel entering the target area, collaborative tracking is performed based on the personnel correspondence between the PTZ cameras; Based on the cross-camera personnel correspondence, determine whether the target personnel in the current PTZ camera have established a matching relationship with the active tracking targets in other PTZ cameras; When the target person in the current PTZ camera has established a matching relationship with the active tracking target in other PTZ cameras, the current PTZ camera is prohibited from repeatedly performing tracking control on the target person according to the determination result of the main tracking PTZ camera; When the target person in the current PTZ camera has not established a matching relationship with the active tracking target in other PTZ cameras, the tracking permission of the target person is assigned to the current PTZ camera; Control the pan-tilt head of the PTZ camera that has been granted tracking permission to rotate toward the location of the target person, so that the target person remains within the field of view of the PTZ camera; Based on the activity tracking status of each PTZ camera and the personnel matching results across PTZ cameras, the tracking permissions between PTZ cameras are dynamically adjusted to achieve multi-PTZ camera collaborative tracking.

[0015] Optionally, the determination of the main tracking PTZ camera in the collaborative tracking control module includes: When multiple PTZ cameras simultaneously establish a cross-PTZ camera correspondence with the same target person, the activity tracking status of each PTZ camera is obtained. The primary tracking camera is determined based on the tracking status, continuous tracking performance, and preset priority rules of each PTZ camera on the target person. The main tracking PTZ camera is identified as the only authorized tracking camera for the target person, and the corresponding main tracking identification information is output.

[0016] Optionally, the cooperative tracking control module is further configured to: When the tracking status of the main tracking PTZ camera on the target person is lost, mismatched, out of the field of view, or the conditions for continuous tracking are not met, the tracking permission of the main tracking PTZ camera on the target person is released. Based on the activity tracking status of the remaining PTZ cameras and the cross-PTZ personnel matching results, select the PTZ camera that meets the tracking conditions as the new main tracking PTZ camera and assign it tracking permissions. Control the pan-tilt head of the newly acquired tracking camera to rotate toward the location of the target person in order to maintain continuous tracking of the target person.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a cross-camera personnel matching and collaborative tracking system. Addressing the issues of insufficient accuracy in matching the same person across multiple cameras, poor target area stability under dynamic camera rotation and zoom conditions, and repetitive tracking by multiple cameras in existing PTZ video surveillance scenarios, the system proposes a series of improved methods and collaborative control measures, effectively enhancing cross-camera personnel matching, target maintenance, and PTZ collaborative tracking capabilities. The system acquires video streams from multiple PTZ cameras and PTZ attitude information through a video access module, and combines this with a target detection and feature extraction module to extract target detection results, human key point information, and appearance re-identification features, providing stable input for subsequent cross-camera matching. To address the issue of target area scaling and position drift under PTZ movement and zoom scenarios, this invention incorporates a zoom adaptation and target area maintenance module. Employing techniques such as log-polar transform and phase-related scale estimation, projection reprojection, and local translation compensation, the system compensates for and continuously maintains the target area, significantly enhancing the stability of the target area under zoom conditions and the continuity of cross-camera matching. In cross-camera personnel matching, this invention constructs cross-camera mapping relationships, integrates human posture similarity, spatial mapping consistency, appearance re-identification similarity, and target relative order relationships to calculate cross-PTZ personnel matching scores, and combines exponential moving average smoothing and continuous threshold confirmation mechanisms to improve the accuracy and robustness of cross-PTZ personnel correspondence. In collaborative tracking, this invention achieves unique authorized tracking and continuous PTZ control of the same target personnel through primary tracking PTZ camera determination, tracking permission allocation, and dynamic relay tracking control. This effectively avoids repeated tracking and control by multiple PTZ cameras, improving collaborative tracking efficiency and continuous target tracking capabilities in PTZ camera scenarios.

[0018] This invention not only technically improves the matching accuracy and collaborative tracking stability of cross-camera systems in dynamic rotation and zoom scenarios, but also reduces the risks of repeated and missed tracking, thereby enhancing the intelligence and practical value of multi-PTZ video surveillance systems. It has promising application prospects and significant potential for widespread adoption in areas such as park security, key area monitoring, and personnel tracking in complex scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is a schematic diagram of the overall process of the personnel matching and collaborative tracking system using a PTZ camera provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram illustrating the method for calculating the personnel matching score across PTZ cameras, as provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The purpose of this invention is to provide a cross-PTZ personnel matching and collaborative tracking scheme that can adapt to the dynamic field of view changes of the PTZ camera while taking into account geometric relationships, human body structure information and temporal stability.

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Example 1: This embodiment proposes a personnel matching and collaborative tracking system that spans multiple PTZ cameras, such as... Figure 1 As shown, the personnel matching and collaborative tracking system for the PTZ camera includes: The system comprises the following modules: a video access module for acquiring video streams from multiple PTZ cameras and gimbal attitude information; a target detection and feature extraction module for performing target detection and feature extraction on the PTZ camera video streams to obtain target detection results, human key point information, and appearance re-identification features; a zoom adaptation and target area preservation module for compensating and adjusting the target area scale according to the zoom state of the PTZ cameras and maintaining target area stability during PTZ camera movement and zooming; a cross-camera matching module for calculating cross-camera mapping relationships based on calibration information, projecting the target position in the first PTZ camera to the coordinate domain of the second PTZ camera, generating candidate matching pairs by combining dynamic offset compensation, calculating cross-PTZ personnel matching scores, and determining the personnel correspondence between PTZ cameras; and a collaborative tracking control module for allocating PTZ camera tracking permissions according to the cross-PTZ personnel correspondence, controlling the corresponding PTZ camera gimbal to perform target personnel tracking, and coordinating the collaborative work of multiple PTZ cameras. The overall system flowchart is as follows: Figure 1 As shown. The specific process includes: Step 1: Connect the PTZ camera: The present invention realizes cross-camera matching and cooperative tracking of persons using pan-tilt-zoom cameras. Compared with fixed cameras, pan-tilt-zoom cameras have the characteristics of being rotatable, zoomable, and dynamically adjustable in field of view, can cover a larger monitoring range and flexibly adjust the field of view according to the motion state of a target, and are therefore more suitable for multi-camera collaborative tracking in complex monitoring scenarios.

[0026] In this embodiment, the video access module is responsible for accessing the video stream data of at least two pan-tilt-zoom cameras and corresponding pan-tilt attitude information. The video stream data is used to provide image input for subsequent target detection, feature extraction and cross-camera matching; the pan-tilt attitude information is used to characterize the current observation state of the pan-tilt-zoom camera, which preferably includes current horizontal rotation angle information, as well as focal length parameters or zoom state information, and in some implementations may further include preset position information, patrol state information and corresponding reference rotation angle information. By acquiring the video stream data and the pan-tilt attitude information synchronously, basic input can be provided for subsequent cross-pan-tilt-zoom camera mapping relationship calculation, region state determination, zoom compensation and cooperative tracking control.

[0027] Further, the pan-tilt-zoom camera supports patrol parameter configuration and mode switching to adapt to different monitoring task requirements. The patrol mode at least includes a normal patrol mode and a zoom patrol mode. In the normal patrol mode, the pan-tilt-zoom camera performs field-of-view scanning according to a preset patrol path, rotation period, rotation angle range or preset position sequence; in the zoom patrol mode, while performing field-of-view scanning, the pan-tilt-zoom camera performs multi-scale patrol on the monitoring area according to a preset zoom factor, focal length interval or zoom gear, so as to improve the ability to cover far and near targets and acquire details.

[0028] In the actual operation process, the video access module may perform frame-by-frame parsing on the accessed multi-pan-tilt-zoom camera video streams to obtain the image frame corresponding to the current moment, and perform time alignment on the image frame and the synchronously acquired pan-tilt attitude information before outputting the result to the target detection and feature extraction module, so as to ensure that the data processed by the subsequent modules have a consistent time sequence correspondence.

[0029] Step 2: Zoom adaptation and target region maintenance: In this embodiment, after the system completes the access of the pan-tilt-zoom cameras, the cooperative tracking control module performs patrol control according to the preset point information, patrol state information and pan-tilt attitude information of the pan-tilt-zoom cameras. The patrol control may include normal patrol and zoom patrol. In the normal patrol mode, the pan-tilt-zoom camera performs field-of-view scanning according to a preset preset point sequence, rotation period or scanning angle range; in the zoom patrol mode, the pan-tilt-zoom camera further performs multi-scale field-of-view switching according to a preset zoom factor, focal length interval or zoom gear during the patrol process. Through the above manner, the system can switch between large-range coverage and local detail observation, and provide continuous video input for subsequent target region maintenance and cross-pan-tilt-zoom camera matching.

[0030] In this embodiment, the target area is a reference target area used for subsequent target maintenance, candidate matching pair generation, and cross-PTZ personnel matching. The reference target area can be preset according to the application scenario, preferably a hazardous area in the workshop, a key monitoring area, a restricted area, or other areas requiring continuous monitoring. The target area is obtained during the initialization phase, preferably determined in the following way: First, the region of interest of the target is manually marked in the current frame of the first PTZ camera to obtain the reference target area R. The reference target areas of the other PTZ cameras can also be set using the same method. Second, corresponding common point sets are marked within the common field of view of the first and second PTZ cameras, and a cross-PTZ geometric mapping relationship is established based on these corresponding common points. Let the first PTZ common point set be... The second set of common points for the PTZ cameras is ,in Then, the homography transformation matrix from the first PTZ camera to the second PTZ camera can be solved by finding the corresponding common points. ,satisfy:

[0031] in, As a scaling factor, This is the cross-PTZ camera mapping matrix. This allows the reference target region in the first PTZ camera to be projected onto the coordinate domain of the second PTZ camera, providing spatial constraints for the subsequent generation of cross-PTZ camera candidate matching pairs.

[0032] During normal operation, when the PTZ camera is in a state of field of view change during routine inspection rotation or tracking of personnel, the system adopts a target area preservation method based on projection transformation. Specifically, the set of vertices of the reference target area obtained in the initialization phase is represented as... The projection transformation matrix of the current frame relative to the reference frame. Reproject the target region to obtain the target region in the current frame. :

[0033] in, This represents the k-th vertex of the reference target region. This represents the projected vertex corresponding to the current frame. This method helps maintain consistency between the target area and the real background area during slow changes in the PTZ camera's field of view or routine inspections.

[0034] When the PTZ camera zooms, the target area undergoes not only a change in position but also a significant change in scale. Therefore, this embodiment introduces a zoom adaptive compensation mechanism. The system first obtains the current gimbal focal length parameter or zoom status information and establishes the scale change relationship of the target area before and after zooming. Let the focal length parameter in the reference state be... The focal length parameter in the current state is The scale compensation ratio of the target region It can be represented as .

[0035] Based on this, the width and height of the target area are proportionally adjusted. Let the center of the reference target area be... Any point within the reference area is Then the points after scale compensation satisfy:

[0036] This means scaling the target area with the center of the target area as a reference to accommodate changes in area size caused by zooming.

[0037] Furthermore, to improve the robustness of scale estimation under zoom conditions, this embodiment employs a center region scaling estimation technique based on log-polar transform and phase correlation to estimate the scaling factor. The basic idea is: after mapping the image center region to the log-polar coordinate domain, the scaling change can be transformed into a translation change in a certain direction. Then, the translation amount is calculated through phase correlation, thereby obtaining the scaling factor. Let the radial translation amount in the log-polar domain be... The corresponding logarithmic scaling factor is Then the scaling factor s can be expressed as:

[0038] This method allows for a relatively stable estimation of target area scale changes under significant zoom conditions, serving as a supplement or correction to the aforementioned focal length ratio compensation.

[0039] After completing the scaling estimation, to eliminate the small-scale field-of-view shift that may accompany zooming, this embodiment further introduces local translation compensation. Let the center of the target area after scaling compensation be... The local translation compensation amount is The corrected target region center is:

[0040] This allows for the translation and updating of the overall position of the target area, thereby improving the consistency between the target area and the actual target position in the current video frame.

[0041] Furthermore, to avoid target region jitter caused by single-frame estimation errors, this embodiment combines target tracking results from consecutive frames to smoothly preserve the target region. Let the target region obtained after transformation in the current frame be... The target area after the previous frame is The smoothed target area Exponential smoothing can be used for updating:

[0042] in, ∈[0,1] represents the smoothing coefficient. By smoothly updating the vertex coordinates of the target region, the jump in the target region caused by homography estimation error, local translation error, or short-term image disturbance can be effectively suppressed.

[0043] During the target area preservation process, the system can also modify the output results by combining target area visibility constraints and continuous frame state constraints. When the predicted position of the target area completely leaves the current field of view, or when the visibility ratio of the target area in the current frame is lower than a preset threshold, the system can mark the target area as invisible; when the repositioning is successful in subsequent frames and the continuous confirmation condition is met, the corresponding target area output is restored. Through the above-mentioned update, scaling compensation, reprojection, translation correction, and smooth preservation mechanisms, the system can maintain the stability of the target area during normal and zoom inspections of the PTZ camera, providing reliable input for subsequent cross-PTZ candidate matching pair generation and personnel matching.

[0044] Step 3: Intrusion detection in the target area: In this embodiment, the PTZ camera inspects the monitored area according to preset point information, inspection status information, or a custom inspection strategy. After acquiring the current video frame, the target detection and feature extraction module detects and tracks target personnel in the video frame locally, and performs area intrusion detection based on the target area preserved in the aforementioned steps.

[0045] Specifically, the system prioritizes using the ground location of the target person as the basis for intrusion determination; when the ground location is not obtained, the center point of the target detection box's foot is used as a substitute determination point. Let the determination point of the target person in the current frame be... The target region polygon is Then when the decision point When a person falls inside the polygon of the target area, it is determined that the person has entered the target area. This can be represented as:

[0046] Among them, when a ground location point exists, the determination point The ground location of the target person is determined; if no ground location point exists, the determination point is the center point of the target detection frame's foot, i.e.:

[0047] in, The coordinates of the target detection box.

[0048] In this embodiment, to reduce the impact of single-frame false detections or momentary jitter on the intrusion determination result, the system further employs a continuous frame confirmation mechanism to perform temporal verification of intrusion events in the target area. For the same target identifier ID, if the number of times it enters the target area in the most recent N detection frames is not less than the preset hit count M, the target person is confirmed as a valid intrusion target, where M≤N. This can be expressed as:

[0049] in, This indicates the determination result of whether the target person has entered the target area in the k-th frame. The value is 1 if the person has entered, and 0 otherwise.

[0050] Once a target person is confirmed to have entered the target area, the system extracts the target detection results, human body key point information, appearance re-identification features, and target identification information for that person, and then calls the cross-camera matching module to prepare for cross-PTZ camera personnel matching. Through this method, the system can achieve stable detection and event triggering of personnel entering the target area during PTZ camera inspections, and provide triggering conditions for subsequent cross-PTZ camera matching and collaborative tracking.

[0051] Step 4: Matching personnel across PTZ cameras: In this embodiment, after step 4 confirms that the target person has entered the target area, the system calls the cross-camera matching module to perform cross-PTZ camera personnel matching processing. The purpose of cross-PTZ camera personnel matching is to determine whether the target in the different PTZ camera images is the same person when at least two PTZ cameras simultaneously observe the target person, and to output the matching result to the collaborative tracking control module for executing whether to allocate tracking permissions and whether to trigger PTZ tracking control commands.

[0052] Specifically, cross-PTZ personnel matching first uses the static mapping relationship established during the initialization phase and the dynamic transformation relationship obtained during the zoom adaptation and target area preservation phases to uniformly transform the target positions in different PTZ cameras. Let the static mapping matrix between the first and second PTZ cameras, obtained by calibrating common points, be... The dynamic zoom matrix of the current frame, estimated by the zoom adaptation and target region preservation stage, is: Then, the comprehensive mapping matrix from the target point in the first PTZ camera to the coordinate domain of the second PTZ camera can be expressed as:

[0053] in, Used to characterize the cross-camera geometric mapping relationship between different PTZ cameras under a reference state. This is used to characterize the dynamic correction relationship caused by the current zoom state and local field of view changes. Through the comprehensive mapping matrix H, the target ground position anchor point, target detection box center point, or reference target area in the first PTZ camera can be projected to the coordinate domain of the second PTZ camera, thereby completing the viewpoint transformation and spatial alignment.

[0054] Furthermore, the system combines the target positions currently detected by the second PTZ camera with the spatial proximity relationship between the target positions and the projected positions using a greedy algorithm to generate candidate matching pairs.

[0055] After generating candidate matching pairs, the system calculates a cross-camera personnel matching score for each pair. See also Figure 2 Targeting the first PTZ camera Target in the second PTZ camera The system calculates the similarity of human poses respectively. Spatial mapping consistency Appearance re-identification similarity and the relative order of the targets The initial matching confidence score was calculated using a multi-feature weighted fusion model. Its expression can be represented as: =

[0056] in, Let be the weight coefficients corresponding to each feature term, and satisfy:

[0057] Among them, human posture similarity Used to measure the consistency of key point structures between two target human bodies; spatial mapping consistency. Used to measure the closeness between the projected position of the target after viewpoint transformation and the true position of the candidate target; appearance re-identification similarity. To measure the similarity between two targets in appearance feature space; the relative order of the targets. This describes whether the relative order of the current target with surrounding targets within the field of view of the PTZ camera is consistent.

[0058] To reduce the impact of jitter in single-frame matching results on the confirmation of the same person, this embodiment further uses exponential moving average (EMA) to smooth the initial matching confidence in consecutive frames, obtaining a smoothed matching confidence. The calculation method is as follows:

[0059] Where α∈[0,1] is the smoothing coefficient. This represents the smoothed matching confidence from the previous time step. This represents the initial matching confidence score for the current frame. The exponential moving average processing suppresses the impact of single-frame detection errors, short-term occlusion, and local pose changes on the matching results.

[0060] After obtaining the smoothed matching confidence score, the system further employs a continuous threshold confirmation mechanism to determine whether to establish a cross-camera correspondence for the same person. Specifically, if a candidate matching pair's smoothed matching confidence score remains higher than a preset matching threshold for a consecutive preset number of frames N, the system will proceed as follows: At that time, it is confirmed that the two targets corresponding to the candidate matching pair are related as the same person across the PTZ cameras. This can be represented as:

[0061] When the above conditions are met, the target is confirmed. With the goal If the person is the same person across multiple PTZ cameras, no matching relationship is established; otherwise, no matching relationship is established. Thus, the system obtains stable personnel matching results across multiple PTZ cameras. Furthermore, the system outputs the personnel matching results to the collaborative tracking control module for subsequent camera tracking control.

[0062] Step 5: PTZ camera collaborative tracking control: In this embodiment, after obtaining the cross-PTZ personnel matching result in step 4, the system outputs the matching result to the collaborative tracking control module. The collaborative tracking control module determines whether to issue a PTZ tracking control command to the current PTZ camera based on the matching result, the current PTZ camera activity tracking status, and whether the target personnel have been tracked by other PTZ cameras, thereby realizing collaborative tracking control in multi-PTZ camera scenarios.

[0063] Specifically, when the current PTZ camera detects a target person entering the target area, the system first determines, based on the cross-PTZ camera matching results, whether the target person has established a cross-PTZ camera personnel correspondence with targets in other PTZ cameras. If no correspondence has been established, it means that the target person has not yet been included in the cross-PTZ camera collaborative tracking link by other PTZ cameras. At this time, the system determines the current PTZ camera as the main tracking PTZ camera for the target person, assigns tracking permissions to the current PTZ camera, and controls its pan / tilt head to rotate toward the location of the target person, thereby performing tracking on the target person.

[0064] If the target person in the current PTZ camera has established a cross-PTZ personnel correspondence with targets in other PTZ cameras, the system further queries whether the target person is currently being tracked by other PTZ cameras. If the query result shows that the target person is being tracked by other PTZ cameras, the current PTZ camera will no longer repeatedly perform tracking control on the target person, but will continue to maintain its current inspection state to avoid multiple PTZ cameras repeatedly rotating, tracking, and controlling the same target person. If the query result shows that although the target person has established a cross-PTZ personnel correspondence, it is not currently being actually tracked by other PTZ cameras, the system assigns tracking permissions for the target person to the current PTZ camera and controls the current PTZ camera as the primary tracking PTZ camera to perform pan-tilt tracking.

[0065] In this embodiment, the determination of "whether it is being tracked by other PTZ cameras" can be based on a joint judgment of the activity tracking status of other PTZ cameras, the current tracking target identification information, and the cross-PTZ camera matching results. Let the target person identification be... The current PTZ camera is Other PTZ cameras are grouped together Then, if any PTZ camera exists... satisfy:

[0066] At that time, the target personnel were identified. It is currently being tracked by other PTZ cameras; among them, Indicates target personnel PTZ camera A cross-site personnel correspondence has been established between the targets. Indicates PTZ camera Current target personnel It is currently under activity tracking.

[0067] Furthermore, once the system identifies the current PTZ camera as the primary tracking camera, the collaborative tracking control module sends PTZ control commands to the PTZ controller, including but not limited to horizontal rotation commands, pitch adjustment commands, and zoom control commands, to keep the target person within the current PTZ camera's field of view or a preset tracking area as much as possible. In some implementations, the system can also dynamically adjust the PTZ rotation speed or zoom ratio based on the target person's offset within the frame to improve the continuous tracking effect.

[0068] When a target person leaves the target area, leaves the current PTZ camera's field of view, is lost for more than a preset time, or is determined by cross-PTZ camera matching results to be tracked by another PTZ camera, the collaborative tracking control module releases the current PTZ camera's tracking authority for that target person and controls the current PTZ camera to exit active tracking mode, reverting to normal inspection mode, zoom inspection mode, or custom inspection mode. Let the current tracking state be... Then, when the target personnel leave the target area or the conditions for continuous tracking are no longer met, we have:

[0069] At this point, the current PTZ camera returns to the inspection process, continuously scanning the monitored area, and waiting for new target personnel to enter the target area before triggering step 3 (target area intrusion detection) and step 4 (cross-PTZ personnel matching) again, thus completing the entire system business loop from area detection, cross-PTZ matching, main tracking PTZ camera determination, PTZ-coordinated tracking to resuming inspection.

[0070] Through the above method, this embodiment realizes multi-camera collaborative tracking control based on cross-camera personnel matching results. It can not only trigger tracking in time when the target personnel first enter the target area, but also avoid repeated tracking when multiple cameras observe the same target personnel at the same time. It can also automatically resume the inspection state when the target personnel leave the area or after the tracking ends, thereby improving the intelligence level and target tracking continuity of multi-camera collaborative work in PTZ camera scenarios.

[0071] This embodiment is applicable to multi-PTZ video surveillance applications, including but not limited to monitoring hazardous areas in workshops, inspecting key areas, issuing warnings for restricted areas, and continuous personnel tracking across cameras in complex scenarios. Traditional multi-camera monitoring systems mainly rely on fixed-view cameras or independent tracking by a single camera, which suffers from problems such as insufficient accuracy in matching the same person across cameras, poor stability of the target area under dynamic camera rotation and zoom conditions, and low efficiency in repeated tracking and collaboration among multiple PTZ cameras. This embodiment, by jointly processing the video streams from multiple PTZ cameras, PTZ attitude information, and target detection and feature data, and combining target area preservation, zoom adaptive compensation, cross-PTZ personnel matching, and collaborative tracking control mechanisms, not only improves the accuracy and stability of cross-camera personnel matching in scenarios with dynamic field of view changes, but also enables the determination of the primary tracking PTZ camera, the allocation of tracking permissions, and PTZ collaborative control among multiple PTZ cameras, thus providing a more intelligent and reliable personnel matching and collaborative tracking solution for multi-PTZ video surveillance scenarios.

[0072] Since the system described in the above embodiments of the present invention is a system used to implement the method of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the system based on the method described in the above embodiments of the present invention, and therefore will not be described again here. All systems used in the method of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0073] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations. The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0074] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A personnel matching and collaborative tracking system using a PTZ camera, characterized in that, include: The video access module is used to acquire video streams from multiple PTZ cameras and gimbal attitude information; The target detection and feature extraction module is used to perform target detection and feature extraction on the video stream of the PTZ camera to obtain target detection results, human key point information and appearance re-identification features; The zoom adaptation and target area preservation module is used to compensate and adjust the scale of the target area according to the zoom state of the PTZ camera, and to keep the target area stable during PTZ camera movement and zooming. The cross-camera matching module is used to calculate the cross-camera mapping relationship based on the calibration information, project the target position in the first PTZ camera to the coordinate domain of the second PTZ camera, generate candidate matching pairs by combining dynamic offset compensation, calculate the cross-PTZ personnel matching score, and determine the personnel correspondence between the cross-PTZ cameras. The collaborative tracking control module is used to allocate tracking permissions for PTZ cameras based on the personnel correspondence between the PTZ cameras, control the corresponding PTZ cameras to perform target personnel tracking, and coordinate the collaborative work of multiple PTZ cameras.

2. The personnel matching and collaborative tracking system for cross-camera cameras according to claim 1, characterized in that, The video access module is also used for: The system acquires video stream data from at least two PTZ cameras and parses the video stream data frame by frame to obtain the image frame corresponding to the current moment. The video access module is also used to acquire the gimbal attitude information of the accessed PTZ cameras. The gimbal attitude information includes at least the current horizontal rotation angle information and the current gimbal focal length parameter. The video access module can also acquire the preset point information and inspection status information corresponding to the PTZ cameras. The video access module uses the image frames and gimbal attitude information to form input data for the target detection and feature extraction module, and uses the preset point information and inspection status information to plan the camera inspection logic of the collaborative tracking control module.

3. The personnel matching and collaborative tracking system for PTZ cameras according to claim 1, characterized in that, The target detection and feature extraction module is also used for: It is used to receive image frames output by the video access module, associate the same target in consecutive frames, and assign the corresponding target identifier ID based on the principle of minimizing the Euclidean distance between the center points of the target detection box and the temporal continuity constraint, so as to obtain the target detection result and the corresponding target identifier information. Based on the target detection results, human key point information and appearance re-identification features of each target are extracted to form target feature data for processing by the zoom adaptation and target region preservation module and the cross-camera matching module.

4. The personnel matching and collaborative tracking system for PTZ cameras according to claim 1, characterized in that, The zoom adaptation and target area preservation module is also used for: Obtain the current pan-tilt focal length parameter or zoom status information of the PTZ camera; Based on the reference target area, the scale change of the target area is compensated and adjusted according to the current gimbal focal length parameter or zoom status information; During the movement and zooming of the PTZ camera, the position and range of the target area are updated and maintained to keep the target area stable; The target area information, after compensation adjustment and area preservation, is output to the cross-camera matching module for subsequent candidate matching pair generation and cross-PTZ personnel matching.

5. The personnel matching and collaborative tracking system for cross-camera cameras according to claim 4, characterized in that, The compensation adjustment for scale changes in the target area includes: A center region scaling estimation method based on log-polar transform and phase correlation is adopted to obtain the scaling factor of the target region before and after zooming; The width and height of the target area are proportionally adjusted according to the scaling factor; The target region is reprojected based on the projection transformation relationship between the reference target region and the current video frame, and the position of the target region is corrected by combining local translation compensation. The target region is updated, smoothed, and preserved by combining the target tracking results in consecutive frames in order to maintain the stability of the target region.

6. The personnel matching and collaborative tracking system for PTZ cameras according to claim 1, characterized in that, The cross-camera matching module is also used for: A cross-camera mapping model is constructed based on the corresponding calibration point information in the common field of view of the first and second PTZ cameras. The cross-camera mapping model includes a homography transformation matrix. Extract the ground position anchor points of each target in the first PTZ camera, and project the ground position anchor points to the coordinate domain of the second PTZ camera based on the cross-camera mapping model to obtain the corresponding mapped positions; The dynamic offset compensation amount at the current moment is estimated based on the positional offset relationship between the ground position of each target in the second PTZ camera and the mapped position, and the mapped position is compensated and corrected in combination with the zoom state compensation result to generate candidate matching pairs; Based on the candidate matching pairs, the similarity of human posture, consistency of spatial mapping, similarity of appearance re-identification, and relative order of targets are fused, the cross-PTZ personnel matching score is calculated, and the correspondence between personnel across the cross-PTZ cameras is determined according to the cross-PTZ personnel matching score.

7. The personnel matching and collaborative tracking system for PTZ cameras according to claim 6, characterized in that, The calculation of the cross-PTZ personnel matching score includes: A multi-feature weighted fusion model is adopted to fuse human pose similarity, spatial mapping consistency, appearance re-identification similarity, and target relative order relationship to obtain the initial matching confidence of candidate matching pairs. The initial matching confidence in consecutive frames is then smoothed by exponential moving average (EMA) to obtain the smoothed matching confidence. When the smoothed matching confidence meets a preset threshold condition within a preset number of consecutive frames, the corresponding candidate matching pair is confirmed as a cross-PTZ personnel correspondence.

8. The personnel matching and collaborative tracking system for PTZ cameras according to claim 6, characterized in that, The collaborative tracking control module is used for: Under normal circumstances, the PTZ camera is inspected and controlled based on preset point information and inspection status information. During the personnel entering the target area, collaborative tracking is performed based on the personnel correspondence between the PTZ cameras; Based on the cross-camera personnel correspondence, determine whether the target personnel in the current PTZ camera have established a matching relationship with the active tracking targets in other PTZ cameras; When the target person in the current PTZ camera has established a matching relationship with the active tracking target in other PTZ cameras, the current PTZ camera is prohibited from repeatedly performing tracking control on the target person according to the determination result of the main tracking PTZ camera; When the target person in the current PTZ camera has not established a matching relationship with the active tracking target in other PTZ cameras, the tracking permission of the target person is assigned to the current PTZ camera; Control the pan-tilt head of the PTZ camera that has been granted tracking permission to rotate toward the location of the target person, so that the target person remains within the field of view of the PTZ camera; Based on the activity tracking status of each PTZ camera and the personnel matching results across PTZ cameras, the tracking permissions between PTZ cameras are dynamically adjusted to achieve multi-PTZ camera collaborative tracking.

9. The personnel matching and collaborative tracking system for PTZ cameras according to claim 6, characterized in that, The determination of the main tracking PTZ camera in the collaborative tracking control module includes: When multiple PTZ cameras simultaneously establish a cross-PTZ camera correspondence with the same target person, the activity tracking status of each PTZ camera is obtained. The primary tracking camera is determined based on the tracking status, continuous tracking performance, and preset priority rules of each PTZ camera on the target person. The main tracking PTZ camera is identified as the only authorized tracking camera for the target person, and the corresponding main tracking identification information is output.

10. The personnel matching and collaborative tracking system for PTZ cameras according to claim 8, characterized in that, The cooperative tracking control module is also used for: When the tracking status of the main tracking PTZ camera on the target person is lost, mismatched, out of the field of view, or the conditions for continuous tracking are not met, the tracking permission of the main tracking PTZ camera on the target person is released. Based on the activity tracking status of the remaining PTZ cameras and the cross-PTZ personnel matching results, select the PTZ camera that meets the tracking conditions as the new main tracking PTZ camera and assign it tracking permissions. Control the pan-tilt head of the newly acquired tracking camera to rotate toward the location of the target person in order to maintain continuous tracking of the target person.

Images