An active tracking method for PTZ cameras based on yolov5 and deepsort
By combining the yolov5 and deepsort algorithms, the preset box anchor tracking range and zoom strategy are designed, and the problem of target loss and error in long-distance target tracking is solved, achieving a more stable and accurate target tracking effect.
Patent Information
- Application Number
- CN202211479451.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-11-24
AI Technical Summary
Existing PTZ cameras are prone to target loss or error in long-distance target tracking, and the lack of zoom control leads to missed detection of small targets, making the tracking distance difficult to expand.
Using the PTZ camera active tracking method based on yolov5 and deepsort, by establishing a tracking system of data acquisition, perception guidance control module and execution control instruction module, using the yolov5 object detection algorithm and deepsort object tracking method, the preset box anchor tracking range and Zoom strategy, output PTZ camera motion direction and focal length control, achieving continuous follow-up of the target.
It effectively improves the target active follow-up capability of the PTZ camera under partial occlusion conditions and the continuous follow-up accuracy of far and near targets, and enhances the stability and accuracy of target tracking.
Smart Images

Figure CN115988321B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image analysis, and in particular relates to an active tracking method for a PTZ camera based on yolov5 and deepsort. Background Art
[0002] The unmanned vehicle is equipped with multiple sensors such as laser radar and camera, which can complete the close-range perception around the unmanned vehicle. However, in the long-distance perception of the unmanned vehicle, the laser radar point cloud is very sparse, and the short-focus camera with a large field of view has very few target pixels. Although the long-focus camera can guarantee the target pixel volume, the field of view is small and it is impossible to form a comprehensive understanding of the target area. The PTZ camera can achieve continuous measurement of long- and short-distance targets by adjusting the rotation, pitch and focal length parameters, making up for the shortcomings of the conventional environmental perception sensors of the unmanned vehicle in detecting information distance and characteristics, and expanding the target visual tracking range.
[0003] The technology of actively tracking targets using PTZ cameras has made significant progress. However, due to the fact that long-distance target detection and tracking are prone to missed detection and false detection, target occlusion, diverse target categories and other situations can easily lead to target tracking failure. Therefore, using PTZ cameras to stably follow diverse targets in various environments is a hot topic in current research. The patent document with the patent application publication number CN 115174861 A discloses a method for automatically tracking moving targets with a PTZ camera, including: obtaining a motion detection result; judging the current state: if the current state is a tracking state, then judging whether the image of the monitoring screen is stable when the motion detection result is obtained, if it is not stable, discarding the current motion detection result, if it is stable, updating the current state to a non-tracking state and processing the motion detection result, and then deciding whether to execute the tracking of the moving target according to the processing result; if the current state is a non-tracking state, then processing the motion detection result and then deciding whether to execute the tracking of the moving target according to the processing result; when executing the tracking of the moving target, updating the current state to a tracking state. The technical solutions of the prior art generally have the following problems: 1. For target occlusion: the target is easily lost or the target ID changes, resulting in the PTZ camera actively following the wrong target; 2. For long-distance targets: due to the lack of zoom control, small targets are easily missed and the tracking distance is difficult to expand. Summary of the invention
[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned technology and provide a PTZ camera active tracking method based on yolov5 and deepsort, which can keep the target actively following under partial occlusion conditions and realize continuous following of long and short distance targets through zoom control.
[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a PTZ camera active tracking method based on yolov5 and deepsort, characterized in that: a tracking system of a data acquisition module, a data receiving and perception module, a perception guidance control module and an execution control instruction module is established, and on the basis of image and PTZ parameter information acquisition, the yolov5 target detection algorithm and the deepsort target tracking method are used to detect and track the target, a tracking target locking strategy is established, a preset frame anchoring tracking range is designed, the movement direction of the PTZ camera is output, and the PTZ camera is controlled in combination with the zoom strategy design to improve the target active tracking effect. The specific steps are as follows:
[0006] Step 1: collecting image data and PTZ parameters;
[0007] Step 2 uses the yolov5 target detection algorithm to obtain the target detection information of the current frame;
[0008] Step 3: Use the deepsort target tracking method to achieve multi-target detection and tracking;
[0009] Step 4: Establish a tracking target locking strategy to lock and follow a single designated target;
[0010] Step 5: Establish a perception guidance control following system of the preset frame anchor tracking range and output the movement direction of the PTZ camera;
[0011] Step 6 The PTZ camera actively follows the target zoom control;
[0012] Step 7: Implement motion control on the camera to actively follow the moving target;
[0013] Among them: Step 5 is specifically as follows:
[0014] 1) Set the preset frame anchor tracking range: Set the preset frame rect1 = (x r1 ,y r1 ,w r1 ,h r1 ),satisfy:
[0015]
[0016] Where W and H are the width and height of the image. The image can be divided into nine-square grid areas through the preset frame, which are numbered 1, 2, 3, 4, 5, 6, 7, 8, and 9 from left to right and from top to bottom; the area inside the preset frame is numbered 5, which is the PTZ camera motion stop area, and the remaining areas are numbered outside the preset frame, which are the PTZ camera motion moving areas;
[0017] 2) Tracking direction determination: Determine the moving direction of the PTZ camera based on the position of the target in the PTZ camera's motion zone:
[0018] (1) When in area 1, the PTZ camera moves to the upper left;
[0019] (2) When in area 2, the PTZ camera moves upward;
[0020] (3) When in area 3, the PTZ camera moves to the upper right;
[0021] (4) When in area 4, the PTZ camera moves to the left;
[0022] (5) When the PTZ camera does not move in area 5, proceed to step 6;
[0023] (6) When in area 6, the PTZ camera moves to the right;
[0024] (7) When in area 7, the PTZ camera moves to the lower left;
[0025] (8) When in area 8, the PTZ camera moves downward;
[0026] (9) When in area 9, the PTZ camera moves to the lower right.
[0027] Furthermore, step 2) of obtaining the current frame target detection information is specifically as follows:
[0028] 1) Data preprocessing: Mosaic data enhancement, adaptive image filling and other processing are performed on the image to complete data enhancement and ensure that the image size is the same as the network input size;
[0029] 2) Feature extraction network: Use the CSP1_x module and CBL module containing multiple residual components to extract features from the image to obtain multi-layer downsampling features and build a feature pyramid;
[0030] 3) Neck layer network: extracts and concatenates the feature pyramid through multiple layers of upsampling and downsampling, and outputs the extraction results of different layers;
[0031] 4) Prediction network: The output results of different layers are passed through the convolution module to obtain the confidence scores and regression coefficients under different sampling multiples, and the target detection results are output.
[0032] Furthermore, in step 3), the deepsort target tracking method is used to realize multi-target detection and tracking, specifically:
[0033] 1) Use Kalman filtering to predict the target tracking trajectory, match the target detection result with the target tracking prediction result through a cascade matcher, and use the Mahalanobis distance between the detection box and the tracking trajectory to describe the matching degree;
[0034] d(i,j)=(d i -y j )S i,j -1 (d i -y j )
[0035] Among them, d i Indicates the position of the i-th detection target box, y j represents the predicted position of the jth target in the trajectory, S i,j Represents the covariance matrix between the target detection position and the trajectory position;
[0036] 2) If the match is successful, the Kalman filter is used to update the tracking trajectory state; if the match is not successful, if the tracking trajectory is not matched, it is determined whether the tracking result needs to be deleted according to the tracking loss threshold;
[0037] 3) When the target detection result and the target tracking prediction result do not match successfully, and the detection result does not match, the intersection-over-union matching method is used to match again. If the match is successful, the Kalman filter is used to update the tracking trajectory state; if the match fails, a new tracking trajectory is generated.
[0038] Furthermore, in step 4), a tracking target locking strategy is established to lock and follow a single designated target, specifically:
[0039] 1) Set the initial target following conditions and the continuous target following conditions respectively, establish the tracking target locking strategy, set the follow_lock_flag tag value, and in the initial state, follow_lock_flag = false, indicating that the system has failed to lock the following target at this time;
[0040] 2) After the system selects a target to follow, the initial target following condition is met. At this time, follow_lock_flag = true, and the system begins to enter the state of continuous target following. As long as the previous algorithm can output the state information of the target to follow, the system maintains the active following state, and the target following sequence keeps increasing;
[0041] 3) When the current sequence algorithm fails to output the target state, the system stops immediately and remains in a waiting state, and the target follow sequence keeps decreasing. When the target follow sequence decreases to 0, follow_lock_flag is set to false.
[0042] Furthermore, in step 6), the PTZ camera actively follows the target zoom control, specifically:
[0043] 1) The relationship between the target pixel position and the focal length is:
[0044]
[0045] in, f x1 is the focal length before zooming, f x2 is the focal length after zooming, (c x ,c y ) is the camera principal point, (x l ,y l ,1) T ,(x r ,y r ,1) T are the homogeneous coordinates of the two points on the left and right of the target point, (u 1l ,v 1l ) T ,(u 1r ,v 1r ) T The two points on the left and right of the target are at a focal length of f x1 The pixel coordinates under (u 1l ,v 1l ) T ,(u 2r ,v 2r ) T The two points on the left and right of the target are at a focal length of f x2 The pixel coordinates below.
[0046] 2) The relationship between the length and width of the pixel value and the focal length before and after the target zoom is:
[0047]
[0048]
[0049] Among them, w1, w2, h1, and h2 represent the length and width of the target before and after zooming, respectively. In actual testing, you can set f x1 ≈f y1 =f1,f x2 ≈f y2 =f2
[0050] 3) In summary, in order to ensure that the target size is within the preset standard frame, the zoom strategy is set as follows:
[0051]
[0052] 4) Set a preset frame rect2 in the center of the image with the same proportion as rect1. r2 ,y r2 ,w r2 ,h r2 ),satisfy:
[0053]
[0054] The zoom operation stops when the size of the perceived target following box (w1, h1) satisfies the following equation:
[0055] w r2 ≤w1≤w r1 or r2 ≤h1≤h r1 .
[0056] Beneficial effect: The PTZ camera active tracking method designed by the present invention takes advantage of deep learning perception, uses a preset frame to anchor the tracking range, and combines the zoom strategy design to control the PTZ camera, effectively improving the PTZ camera's active tracking target accuracy and distance. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 This is the design diagram of the PTZ camera active tracking system based on yolov5 and deepsort;
[0058] Figure 2 This is the network structure diagram of yolov5;
[0059] Figure 3 It is the deepsort algorithm flow chart;
[0060] Figure 4 It is a target locking flow chart;
[0061] Figure 5 It is a perception-guided control diagram based on the Nine-square grid;
[0062] Figure 6 It is the change of PTZ camera zoom value with frame rate;
[0063] Figure 7 It is the change of the pitch angle (tilt value) of the PTZ camera with the frame number;
[0064] Figure 8 It is the change of PTZ camera rotation angle (pan value) with the frame number;
[0065] Fig. 9 It is the x-coordinate motion statistics of the target center;
[0066] Fig.10 It is the y-coordinate motion statistics of the target center;
[0067] Fig.11 It is the success rate statistical curve;
[0068] Fig.12 It is the accuracy statistical curve. DETAILED DESCRIPTION
[0069] In order to be able to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth in order to fully understand the present invention, and the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0070] In the various embodiments of the present invention, for the convenience of description and not to limit the present invention, the term "connection" used in the patent application specification and claims of the present invention is not limited to physical or mechanical connection, but may include electrical connection, whether direct or indirect. "Up", "down", "below", "left", "right", etc. are only used to indicate relative position relationship. When the absolute position of the described object changes, the relative position relationship also changes accordingly.
[0071] See the attached drawings for details. This embodiment provides a PTZ camera active tracking method based on yolov5 and deepsort, which is characterized by: establishing a tracking system of a data acquisition module, a data receiving and perception module, a perception guidance control module and an execution control instruction module, and based on the image and PTZ parameter information collection, using the yolov5 target detection algorithm and the deepsort target tracking method to detect and track the target, establish a tracking target locking strategy, design a preset frame anchor tracking range, output the movement direction of the PTZ camera, and combine the zoom strategy design to control the PTZ camera to improve the target active tracking effect. The specific steps are as follows:
[0072] Step 1: collecting image data and PTZ parameters;
[0073] Step 2 uses the yolov5 target detection algorithm to obtain the target detection information of the current frame;
[0074] Step 3: Use the deepsort target tracking method to achieve multi-target detection and tracking;
[0075] Step 4: Establish a tracking target locking strategy to lock and follow a single designated target;
[0076] Step 5: Establish a perception guidance control following system of the preset frame anchor tracking range and output the movement direction of the PTZ camera;
[0077] Step 6 The PTZ camera actively follows the target zoom control;
[0078] Step 7: Implement motion control on the camera to actively follow the moving target;
[0079] Among them: Step 5 is specifically as follows:
[0080] 1) Set the preset frame anchor tracking range: Set the preset frame rect1 = (x r1 ,y r1 ,w r1 ,h r1 ),satisfy:
[0081]
[0082] Where W and H are the width and height of the image. The image can be divided into nine-square grid areas through the preset frame, which are numbered 1, 2, 3, 4, 5, 6, 7, 8, and 9 from left to right and from top to bottom; the area inside the preset frame is numbered 5, which is the PTZ camera motion stop area, and the remaining areas are numbered outside the preset frame, which are the PTZ camera motion moving areas;
[0083] 2) Tracking direction determination: Determine the moving direction of the PTZ camera based on the position of the target in the PTZ camera's motion zone:
[0084] (1) When in area 1, the PTZ camera moves to the upper left;
[0085] (2) When in area 2, the PTZ camera moves upward;
[0086] (3) When in area 3, the PTZ camera moves to the upper right;
[0087] (4) When in area 4, the PTZ camera moves to the left;
[0088] (5) When the PTZ camera does not move in area 5, proceed to step 6;
[0089] (6) When in area 6, the PTZ camera moves to the right;
[0090] (7) When in area 7, the PTZ camera moves to the lower left;
[0091] (8) When in area 8, the PTZ camera moves downward;
[0092] (9) When in area 9, the PTZ camera moves to the lower right.
[0093] The preferred solution of this embodiment is, see the attached Figure 2 , step 2) obtaining the current frame target detection information, specifically:
[0094] 1) Data preprocessing: Mosaic data enhancement, adaptive image filling and other processing are performed on the image to complete data enhancement and ensure that the image size is the same as the network input size;
[0095] 2) Feature extraction network: Use the CSP1_x module and CBL module containing multiple residual components to extract features from the image to obtain multi-layer downsampling features and build a feature pyramid;
[0096] 3) Neck layer network: extracts and concatenates the feature pyramid through multiple layers of upsampling and downsampling, and outputs the extraction results of different layers;
[0097] 4) Prediction network: The output results of different layers are passed through the convolution module to obtain the confidence scores and regression coefficients under different sampling multiples, and the target detection results are output.
[0098] The preferred solution of this embodiment is, see the attached Figure 3 , step 3) uses the deepsort target tracking method to achieve multi-target detection and tracking, specifically:
[0099] 1) Use Kalman filtering to predict the target tracking trajectory, match the target detection result with the target tracking prediction result through a cascade matcher, and use the Mahalanobis distance between the detection box and the tracking trajectory to describe the matching degree;
[0100] d(i,j)=(d i -y j )S i,j -1 (d i -y j )
[0101] Among them, d i Indicates the position of the i-th detection target box, y j represents the predicted position of the jth target in the trajectory, S i,j Represents the covariance matrix between the target detection position and the trajectory position;
[0102] 2) If the match is successful, the Kalman filter is used to update the tracking trajectory state; if the match is not successful, if the tracking trajectory is not matched, it is determined whether the tracking result needs to be deleted according to the tracking loss threshold;
[0103] 3) When the target detection result and the target tracking prediction result do not match successfully, and the detection result does not match, the intersection-over-union matching method is used to match again. If the match is successful, the Kalman filter is used to update the tracking trajectory state; if the match fails, a new tracking trajectory is generated.
[0104] The preferred solution of this embodiment is, see the attached Figure 4 , step 4) establishes a tracking target locking strategy to lock and follow a single designated target, specifically:
[0105] 1) Set the initial target following conditions and the continuous target following conditions respectively, establish the tracking target locking strategy, set the follow_lock_flag tag value, and in the initial state, follow_lock_flag = false, indicating that the system has failed to lock the following target at this time;
[0106] 2) After the system selects a target to follow, the initial target following condition is met. At this time, follow_lock_flag = true, and the system begins to enter the state of continuous target following. As long as the previous algorithm can output the state information of the target to follow, the system maintains the active following state, and the target following sequence keeps increasing;
[0107] 3) When the current sequence algorithm fails to output the target state, the system stops immediately and remains in a waiting state, and the target follow sequence keeps decreasing. When the target follow sequence decreases to 0, follow_lock_flag is set to false.
[0108] The preferred solution of this embodiment is, see the attached Figure 5 , step 6) the PTZ camera actively follows the target zoom control, specifically:
[0109] 1) The relationship between the target pixel position and the focal length is:
[0110]
[0111] in, f x1 is the focal length before zooming, f x2 is the focal length after zooming, (c x ,c y ) is the camera principal point, (x l ,y l ,1) T ,(x r ,y r ,1) T are the homogeneous coordinates of the two points on the left and right of the target point, (u 1l ,v 1l ) T ,(u 1r ,v 1r ) T The two points on the left and right of the target are at a focal length of f x1 The pixel coordinates under (u 1l ,v 1l ) T ,(u2r ,v 2r ) T The two points on the left and right of the target are at a focal length of f x2 The pixel coordinates below.
[0112] 2) The relationship between the length and width of the pixel value and the focal length before and after the target zoom is:
[0113]
[0114]
[0115] Among them, w1, w2, h1, and h2 represent the length and width of the target before and after zooming, respectively. In actual testing, you can set f x1 ≈f y1 =f1,f x2 ≈f y2 =f2
[0116] 3) In summary, in order to ensure that the target size is within the preset standard frame, the zoom strategy is set as follows:
[0117]
[0118] 4) Set a preset frame rect2 in the center of the image with the same proportion as rect1. r2 ,y r2 ,w r2 ,h r2 ),satisfy:
[0119]
[0120] The zoom operation stops when the size of the perceived target following box (w1, h1) satisfies the following equation:
[0121] w r2 ≤w1≤w r1 or r2 ≤h1≤h r1 .
[0122] Example
[0123] The active following system designed by the present invention is divided into four modules, namely, a data acquisition module, a data receiving and sensing module, a sensing guidance control module and an execution control instruction module. First, the operator selects the target to be followed in the current detection and tracking target results by using the mouse. The data acquisition module collects images and PTZ parameters. The data receiving and sensing module completes target locking according to the selected target and outputs the position of the target in the image. The sensing guidance control module outputs the camera movement direction according to the position information. When the camera rotation and pitch movement stops, the focal length value is adjusted autonomously according to the size of the target in the image to ensure that the target is within the preset frame range in the image. The target can remain stationary or in motion throughout the implementation process. After zooming, the PTZ camera continues to track the locked target. The present invention uses C++ to design an active following system on Ubuntu and conducts a target following experiment outdoors.
[0124] 1. Model parameters and experimental scenarios
[0125] (1) The parameters of the active following system are as follows:
[0126]
[0127] (2) Experimental scene setting:
[0128] Outdoor open space, target type is cyclist, moving speed is 10km / h;
[0129] 2. Experimental Results
[0130] (1) Keep the rider moving and use the algorithm proposed in this invention to actively follow the vehicle. After testing, the target can always appear in the image, and the total number of tracking frames is 1116. During the tracking process, in order to keep the target in the preset frame, the rotation, pitch and zoom values of the PTZ camera are always adjusted. The parameter changes are as follows: Figure 6 , Figure 7 and Figure 8 After statistics, the x-coordinate and y-coordinate statistics of the target center position are shown in Fig. 9 and Fig.10 As shown. Since the coordinates of the upper left corner of the preset frame are [480, 270] and the coordinates of the lower right corner are [800, 450], in order to keep the target in the preset frame, the x coordinate of the center position of the target needs to be kept in the interval [480, 800] as much as possible, and the y coordinate needs to be kept in the interval [270, 450] as much as possible. Fig. 9 and Fig.10 In the figure, the statistical probability that the x-coordinate of the target center is in the interval [480,800] is 84.28%, and the statistical probability that the y-coordinate of the target center is in the interval [270,450] is 93.80%, and the distribution is concentrated inside the preset box.
[0131] (2) Although the algorithm proposed in the present invention can keep tracking the moving vehicle target and keep the target within the image frame, in order to further study the performance of the algorithm proposed in the present invention, the experimental results are statistically analyzed in terms of active tracking success rate and accuracy, such as Fig.11 and Fig.12 The tracking success rate here is determined by the IOU value, and a value greater than 0.5 is considered a successful tracking. According to statistics, the active tracking success rate is 74.65% and the accuracy rate is 73.52%. The active tracking success rate and accuracy results are relatively good.
[0132] The above-mentioned detailed description of a PTZ camera active tracking method based on yolov5 and deepsort with reference to the embodiment is illustrative rather than restrictive, and several embodiments can be listed according to the limited scope. Therefore, changes and modifications without departing from the overall concept of the present invention should fall within the scope of protection of the present invention.
Claims
1. A PTZ camera active tracking method based on yolov5 and deepsort, characterized by: establishing The tracking system of the data acquisition module, data receiving and perception module, perception guidance control module and execution control instruction module, based on the image and PTZ parameter information collection, uses the yolov5 target detection algorithm and deepsort target tracking method to detect and track the target, establishes a tracking target locking strategy, designs a preset frame to anchor the tracking range, outputs the movement direction of the PTZ camera, and controls the PTZ camera in combination with the zoom strategy design to improve the target active tracking effect. The specific steps are as follows: Step 1) collecting image data and PTZ parameters; Step 2) Use the yolov5 target detection algorithm to obtain the target detection information of the current frame; Step 3) Use the deepsort target tracking method to achieve multi-target detection and tracking; Step 4) Establish a tracking target locking strategy to lock and follow a single designated target; Step 5) Establish a perception guidance control following system of the preset frame anchor tracking range and output the PTZ camera movement direction; Step 6) The PTZ camera actively follows the target zoom control; Step 7) Implement motion control on the camera to actively follow the moving target; Wherein: Step 5) is specifically as follows: 1) Set the preset frame anchor tracking range: Set the preset frame rect1 = (x r1 ,y r1 ,w r1 ,h r1 ),satisfy: Where W and H are the width and height of the image. The image can be divided into nine-square grid areas through the preset frame, which are numbered 1, 2, 3, 4, 5, 6, 7, 8, and 9 from left to right and from top to bottom. The area inside the preset frame is numbered 5, which is the PTZ camera motion stop area, and the remaining areas are numbered outside the preset frame, which are the PTZ camera motion moving areas. 2) Tracking direction determination: Determine the moving direction of the PTZ camera based on the position of the target in the PTZ camera's motion zone: (1) When in area 1, the PTZ camera moves to the upper left; (2) When in area 2, the PTZ camera moves upward; (3) When in area 3, the PTZ camera moves to the upper right; (4) When in area 4, the PTZ camera moves to the left; (5) When in area 5, the PTZ camera does not move, proceed to step 6); (6) When in area 6, the PTZ camera moves to the right; (7) When in area 7, the PTZ camera moves to the lower left; (8) When in area 8, the PTZ camera moves downward; (9) When in area 9, the PTZ camera moves to the lower right; Step 4) establishes a tracking target locking strategy to lock and follow a single designated target, specifically: 1) Set the initial target following conditions and the continuous target following conditions respectively, establish the tracking target locking strategy, set the follow_lock_flag tag value, and in the initial state, follow_lock_flag = false, indicating that the system has failed to lock the following target at this time; 2) After the system selects a target to follow, the initial target following condition is met. At this time, follow_lock_flag = true, and the system begins to enter the state of continuous target following. As long as the previous algorithm can output the state information of the target to follow, the system maintains the active following state, and the target following sequence keeps increasing; 3) If the current sequence algorithm fails to output the target state, the system stops immediately and remains in a waiting state, and the target follow sequence keeps decreasing. When the target follow sequence decreases to 0, follow_lock_flag is set to false; Step 6) The PTZ camera actively follows the target zoom control, specifically: 1) The relationship between the target pixel position and the focal length is: in, f x1 is the focal length before zooming, f x2 is the focal length after zooming, (c x ,c y ) is the camera principal point coordinate, (x l ,y l ,1) T ,(x r ,y r ,1) T are the homogeneous coordinates of the left and right points of the target, respectively, (u 1l ,v 1l ) T ,(u 1r ,v 1r ) T The two points on the left and right of the target are at a focal length of f x1 The pixel coordinates under (u 2l ,v 2l ) T ,(u 2r ,v 2r ) T The two points on the left and right of the target are at a focal length of f x2 The pixel coordinates below; 2) The relationship between the pixel value width and height and focal length before and after the target zoom is: Among them, w1 and w2 represent the width of the target before and after zooming, h1 and h2 represent the height of the target before and after zooming. In actual testing, you can set f x1 ≈f y1 =f1,f x2 ≈f y2 =f2 3) In summary, in order to ensure that the target size is within the preset standard frame, the zoom strategy is set as follows: 4) Set a preset frame rect2 in the center of the image with the same proportion as rect1. r2 ,y r2 ,w r2 ,h r2 ),satisfy: The zoom operation stops when the size of the perceived target following box (w1, h1) satisfies the following equation: w r2 ≤w1≤w r1 or r2 ≤h1≤h r1 .
2. The PTZ camera active tracking method based on yolov5 and deepsort according to claim 1 is characterized in that: Step 2) obtaining the current frame target detection information is specifically as follows: 1) Perform Mosaic data enhancement and adaptive image filling processing on the image to complete data enhancement and ensure that the image size is the same as the network input size; 2) Use the CSP1_x module and CBL module containing multiple residual components to extract features from the image to obtain multi-layer downsampling features and construct a feature pyramid; 3) Extract and concatenate the feature pyramid through multi-layer upsampling and downsampling features, and output the extraction results of different layers; 4) The output results of different layers are passed through the convolution module to obtain the confidence scores and regression coefficients under different sampling multiples, and the target detection results are output.
3. The PTZ camera active tracking method based on yolov5 and deepsort according to claim 1 is characterized in that: Step 3) uses the deepsort target tracking method to achieve multi-target detection and tracking, specifically: 1) Use Kalman filtering to predict the target tracking trajectory, match the target detection result with the target tracking prediction result through a cascade matcher, and use the Mahalanobis distance between the detection box and the tracking trajectory to describe the matching degree; d(i,j)=(d i -y j )S i,j -1 (d i -y j ) Among them, d i Indicates the position of the i-th detection target box, y j represents the predicted position of the jth target in the trajectory, S i,j Represents the covariance matrix between the target detection position and the trajectory position; 2) If the match is successful, the Kalman filter is used to update the tracking trajectory state; if the match is not successful, if the tracking trajectory is not matched, it is determined whether the tracking result needs to be deleted according to the tracking loss threshold; 3) When the target detection result and the target tracking prediction result do not match successfully, and the detection result does not match, the intersection-over-union matching method is used to match again. If the match is successful, the Kalman filter is used to update the tracking trajectory state; if the match fails, a new tracking trajectory is generated.
