A variable speed moving target adaptive active following method based on spherical coordinates

Abstract

The application relates to a variable-speed motion target adaptive active following method based on spherical coordinates, a spherical coordinate system is established with the optical center of a PTZ camera as the origin, a perception module and a control module are respectively established, the perception module realizes acquisition of position, category and id information of a target by using a yolov5 target detection algorithm, after the target is locked, a target re-identification algorithm is used to improve the target locking effect and acquire target historical information; the locked target pixel coordinates are converted into spherical coordinates to acquire longitude, latitude and angular velocity of the target; the control module comprises a speed control equation which is established by comprehensively considering system start-up delay, target motion angular velocity and distance deviation between the target center and the image center, and the PTZ camera is controlled by using an incremental PID control algorithm to complete active following of the target. Beneficial effects: the variable-speed motion target adaptive active following method based on spherical coordinates can improve the active following target performance of the PTZ camera, and effectively improves the active tracking target precision and stability of the PTZ camera.

Description

Technical Field

[0001] This invention belongs to the field of image analysis technology, and in particular relates to an adaptive active following method for a variable-speed moving target based on spherical coordinates. Background Technology

[0002] The PTZ camera is an integrated system that combines optical information acquisition, data image processing, electromechanical control, digital signal processing, and computer applications. Through the electromechanical control of the turntable, it keeps the turntable's line of sight moving synchronously with the target. It features variable viewing angle and adjustable focal length, enabling dynamic target measurement in large spaces, over long distances, and without contact. It has unique advantages in compensating for the shortcomings of conventional environmental perception sensors in unmanned vehicles in terms of detection distance and features, expanding the target visual tracking range, and achieving continuous target tracking.

[0003] In complex and ever-changing natural scenes, active tracking technology using PTZ cameras still faces many challenges. For example, changes in target speed, the need to track distant targets, target occlusion, and diverse target types can easily lead to target tracking failure. Therefore, using PTZ cameras to stably track targets in various environments and under various motion states is a current research hotspot. For instance, patent document CN 112637550 A discloses a PTZ moving target tracking method using multi-channel 4K near real-time stitched video. The method includes the following steps: Step 1: Real-time acquisition of monitoring video via PTZ camera and transmission of the monitoring video to a processor; the processor includes a video stitching module and a data storage module, and the processor is connected to at least one PTZ camera; Step 2: Quality analysis of the monitoring video by the processor, and stitching together high-quality monitoring videos to obtain a stitched video; Step 3: Target identification of the stitched video, and adjustment of the PTZ camera angle based on the identification results; Step 4: Determination of whether the target enters the field of view of an adjacent PTZ camera. When the target enters the field of view of an adjacent PTZ camera, tracking is performed using that adjacent PTZ camera.

[0004] One common problem with existing technologies for actively tracking targets using PTZ cameras is that, for targets with varying speeds, the target's position in the PTZ camera changes rapidly due to the inconsistent speed, which can affect the target detection effect and thus the active tracking result. Another problem is that existing technologies often consider pixel coordinates when calculating PTZ camera motion, which can easily lead to overshoot when adjusting the speed, causing instability in the system during tracking. This problem is particularly pronounced when the focal length increases.

[0005] To address the common problem of PTZ cameras actively tracking targets, an adaptive control method for PTZ cameras with variable speed moving targets based on PID control is proposed. According to the target's motion state, the PID control algorithm is used to control the PTZ camera, thereby improving the active target tracking effect. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the above-mentioned technologies and provide an adaptive active tracking method for variable-speed moving targets based on spherical coordinates. This method can fully utilize the motion characteristics of the target's spherical coordinates and the rotation and pitch of the PTZ camera, and establish a velocity control equation by combining the start-up delay, thereby effectively improving the accuracy and stability of the PTZ camera's active target tracking.

[0007] To achieve the above objectives, this invention employs the following technical solution: an adaptive active tracking method for variable-speed moving targets based on spherical coordinates, characterized by: establishing a spherical coordinate system with the optical center of a PTZ camera as the origin; establishing a sensing module and a control module respectively; acquiring the latitude, longitude, and angular velocity of the target using the spherical coordinates; establishing a velocity control equation using the target's angular velocity and the distance deviation between the target center and the image center; and using an incremental PID control algorithm to control the PTZ camera to achieve active target tracking.

[0008] The specific steps are as follows:

[0009] I. Establishing a perception module

[0010] Step 1) Use the YOLOv5 target detection algorithm to output the target's location, category, and ID information;

[0011] Step 2) Establish a target tracking strategy, use a target re-identification algorithm to lock onto a single following target, and obtain historical information about the target;

[0012] Step 3) Establish a spherical coordinate system with the camera optical center as the origin, and combine the camera intrinsic and extrinsic parameters and PTZ parameters to convert the target pixel coordinates into spherical coordinates;

[0013] II. Establishing a Control Module

[0014] Step 4) Use the target tracking information to solve for the angular velocity of the target motion, and establish a velocity control equation by comprehensively considering the target motion angular velocity and the distance deviation between the target center and the image center;

[0015] Step 5) Calibrate the gear speed and start-up time delay, and output control parameters using incremental PID control method for start-up and gear shifting situations;

[0016] Step 6) Implement motion control on the PTZ camera to achieve active tracking of moving targets.

[0017] Further, the location, category, and ID of the target mentioned in step 1) are output, wherein:

[0018] ① Data preprocessing: Perform data augmentation and adaptive image filling on the image to complete the data augmentation and ensure that the image size is the same as the network input size;

[0019] ② Feature extraction network; The image is input into the target detection network framework, which consists of the Backbone network, Neck network, and Head network in sequence. The Backbone network includes a Focus module and a CSP module to obtain multi-layer image features; the Neck network mainly consists of an FPN module and a PAN structure to fuse multi-layer sampling information; the Head network is mainly used to predict image features, using GIOU_loss as the loss function for bounding boxes, and using weighted NMS to filter anchor boxes;

[0020] ③ Classify and regress the detected target information, and output the target information including:

[0021] x = (id, t, x, y, w, h, cid, conf)

[0022] Where id represents the target's identification number, t represents the target's image acquisition time, (x,y,w,h) represents the target bounding box's position information in the image, cid represents the target's category information, and conf represents the target's category confidence level.

[0023] Further, step 2) involves locking onto and following a single specified target based on the target re-identification algorithm and obtaining the target's historical information, specifically as follows:

[0024] ① Select a target to track by clicking the mouse or setting a preset target, list the target as a query image, and create a historical query image set for the target with a set size of N;

[0025] ② Extracting target features based on target re-identification algorithm: Establishing a candidate image set g using the targets output by the target detection algorithm. i Given a set of size M, a convolutional neural network is used to extract a candidate image set g. i and query image q j Based on the image features, generate a feature vector r. i ,r j ,

[0026] r i =f(g i ),r j =f(q) j )

[0027] Where f(x) represents the feature extraction of the input image x using a convolutional neural network f;

[0028] ③ Calculate the cosine distance between the historical query image set and the feature vectors of all images in the candidate set for the target, and generate a distance matrix D, where the elements of D are:

[0029] D i,j =1-r j T r i

[0030] Where D is of size M×N

[0031] ④ Calculate the average distance between the candidate set and the query image row by row:

[0032]

[0033] When d k When the distance is less than the threshold, the two targets can be considered to belong to the same ID. For images less than the threshold, sort them by distance, find the candidate set with the minimum distance relative to the query image, which is the matching target, update the locked target status, and obtain the target's historical information.

[0034] Furthermore, step 3) of establishing and transforming the spherical coordinate system specifically involves:

[0035] ① The pixel coordinate system is transformed into the camera coordinate system as follows:

[0036]

[0037] Among them, f x f y Indicates focal length; (c x ,c y (x1, y1, z1) represents the principal point position of the camera, (u, v) represents the pixel coordinates, and (x1, y1, z1) represents the pixel coordinates. T These are the homogeneous coordinates of the target point in the camera coordinate system.

[0038] ② Normalize the coordinates in the camera coordinate system to (x2, y2, y3). T Satisfy the following formula:

[0039]

[0040] ③ Calculate the spherical coordinates (β, α) of the point. T It can be calculated using the following formula:

[0041]

[0042] ④ World spherical coordinates of a point at time t (β) w (t),α w (t)) TThat is, (p(0),t(0)) T The target coordinates in the spherical coordinate system with the initial position can be calculated using the following formula:

[0043]

[0044] Where (p(0),t(0)) and (p(t),t(t)) represent the poses of the PTZ camera at the initial position t=0 and time t=t, respectively, (β c (t),α c (t)) T Here are the spherical coordinates of a point on the PTZ camera.

[0045] Furthermore, step 4) involves solving for the angular velocity of the target motion and establishing the velocity control equations, specifically as follows:

[0046] ①The formula for calculating the angular velocity of the target motion is as follows:

[0047]

[0048] Where T(t) and T(t-1) represent the timestamps at times t and t-1, respectively, and β w (t),β w (t-1) represents the latitude coordinates at times t and t-1, α w (t),α w (t-1) represents the longitude coordinates at times t and t-1.

[0049] ② Image center pixel coordinates (u c ,v c The condition (W / 2, H / 2) corresponds to the spherical coordinates (β). cc ,α cc )

[0050] ③ Set a preset frame in the center area of ​​the PTZ camera. The area inside the preset frame is the stationary zone, and the area outside the preset frame is the rotation and pitch zone. To ensure that the target always moves within the center area of ​​the PTZ camera, establish an error equation:

[0051]

[0052] Where (β(t), α(t)) are the target latitude and longitude at time t.

[0053] ④ Assuming the target's position in the image is in the same direction as its motion, the rotation and pitch angular velocities of the PTZ camera at time t must satisfy the following equation:

[0054]

[0055] Where Vt is the actual rotation time of the PTZ camera, and τβi , τ αi The delay for the rotation and pitch motion of the PTZ camera is i (1≤i≤N);

[0056] ⑤The equation expressing the motion delay τ of the PTZ camera is as follows:

[0057]

[0058] Where δ is α or β

[0059] ⑥ Regarding the PTZ camera's active target tracking start-up control speed requirement, i.e., satisfying condition ④, Vt≥0, therefore satisfying the expression:

[0060]

[0061] Furthermore, the control parameters output by the calibration start-up time delay and the incremental PID control method described in step 5) are specifically as follows:

[0062] ①Calibrate the start-up time delay and calculate the time delay τ from command start to movement of the PTZ camera at different speed levels. δ Where δ is α or β;

[0063] ②Establishment of incremental PID control equations:

[0064] Vω(t)=k p (er(t)-er(t-1))+k I er(t)+k d (er(t)-2er(t-1)+er(t-2))

[0065] ③ Since a PID control strategy is used during the motion, τ = 0 at this time, therefore ω βc (t+1) and ω αc (t+1) can be represented as:

[0066]

[0067] ④ Since the PTZ camera has N speed settings, select speed settings i,j (1≤i,j≤N) such that i,j satisfy:

[0068]

[0069] Wherein, ω is set β0 =ω α0 =0

[0070] ⑤ Select a gear that satisfies the minimum value of the gear, i.e., min{i,j}, and then output the speed control gear.

[0071] Beneficial effects: The adaptive active tracking method for variable-speed moving targets based on spherical coordinates in this invention can improve the active target tracking performance of PTZ cameras, and effectively improve the accuracy and stability of active target tracking by PTZ cameras. Attached Figure Description

[0072] Figure 1 This is a flowchart of the adaptive active following method of the present invention;

[0073] Figure 2 This is a diagram showing the establishment of a spherical coordinate system;

[0074] Figure 3 It is a graph of point motion in a spherical coordinate system;

[0075] Figure 4 This is a flowchart of the target re-identification algorithm;

[0076] Figure 5 This is a graph showing the change of the start / stop control angle of the 7 gears under the rotation direction over time;

[0077] Figure 6 This is a graph showing the change of control angle over time under the commands to stop in 6th gear and start in 7th gear in the direction of rotation.

[0078] Figure 7 These are the curves showing the changes in speed and start / stop delay for the seven gears in the direction of rotation.

[0079] Figure 8 This is a speed control flowchart;

[0080] Figure 9 This shows how the rotation angle (pan value) of the PTZ camera changes with the number of frames.

[0081] Figure 10 This shows how the PTZ camera's tilt angle (tilt value) changes with the number of frames.

[0082] Figure 11 This shows how the zoom value of a PTZ camera changes with the frame rate.

[0083] Figure 12 This is a statistical chart of the x-coordinate of the target center location;

[0084] Figure 13 This is a statistical chart of the y-coordinate of the target center position;

[0085] Figure 14 It is a success rate statistical comparison curve;

[0086] Figure 15 It is an accuracy statistical comparison curve. Detailed Implementation

[0087] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. The described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present invention.

[0088] In the various embodiments of the present invention, for ease of description and not limitation of the invention, the term "connection" used in the present invention patent application specification and claims is not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Above," "below," "underneath," "left," "right," etc., are only used to indicate relative positional relationships, and when the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0089] As shown in the attached diagram, this embodiment provides an adaptive active tracking method for variable-speed moving targets based on spherical coordinates. A spherical coordinate system is established with the optical center of the PTZ camera as the origin, and a perception module and a control module are established respectively. The perception module mainly uses the YOLOv5 target detection algorithm to output the target's position, category, ID, and historical information. The operator selects the target to be followed from the current detection and tracking results using the mouse, and locks onto the selected target. The target re-identification algorithm is used to improve the target locking effect. The pixel coordinates of the locked target are converted into spherical coordinates to obtain the target's latitude, longitude, and angular velocity. The control module mainly includes a speed control equation established by comprehensively considering the system startup delay and using the target's motion angular velocity and the distance deviation between the target center and the image center. An incremental PID control algorithm is used to control the PTZ camera to complete the active target tracking.

[0090] The specific steps are as follows:

[0091] Step 1): Use a target detection algorithm to output the target's location, category, and ID information, where:

[0092] S21: Data preprocessing: Perform data augmentation, adaptive image filling, and other processing on the image to complete the data augmentation and ensure that the image size is the same as the network input size;

[0093] S22: Feature Extraction Network; The image is input into the object detection network framework, consisting of the Backbone network, Neck network, and Head network in sequence. The Backbone network mainly includes a Focus module and a CSP module, used to acquire multi-layer image features; the Neck network mainly consists of an FPN module and a PAN structure, mainly used to fuse multi-layer sampling information; the Head network is mainly used to predict image features, using GIOU_loss as the loss function for bounding boxes, and using weighted NMS to filter anchor boxes;

[0094] S23: Classify and regress the detected target information, and output the target information including:

[0095] x = (id, t, x, y, w, h, c_id, conf)

[0096] Where id represents the target's identification number, t represents the target's image acquisition time, (x,y,w,h) represents the target bounding box's position information in the image, cid represents the target's category information, and conf represents the target's category confidence level;

[0097] See appendix for details Figure 2 Step 2): Establish a target tracking strategy, use a target re-identification algorithm to lock onto a single following target, and obtain historical information about the target; specifically:

[0098] S31: Select a target to track by clicking the mouse or setting a preset target, list the target as a query image, and create a historical query image set for the target with a set size of N;

[0099] S32: Extracting target features based on target re-identification algorithm: Establishing a candidate image set g using the target output from the target detection algorithm. i Given a set of size M, a convolutional neural network is used to extract a candidate image set g. i and query image q j Based on the image features, generate a feature vector r. i ,r j ,

[0100] r i =f(g i ),r j =f(q) j )

[0101] Here, f(x) represents the feature extraction of the input image x using a convolutional neural network f.

[0102] S33: Calculate the cosine distance between the historical query image set and the feature vectors of all images in the candidate set for the target, and generate a distance matrix D, where the elements of D are:

[0103] D i,j =1-r j T r i

[0104] Where D is of size M×N

[0105] S34: Calculate the average distance between the candidate set and the query image row by row:

[0106]

[0107] When d k If the distance is less than the threshold, the two targets can be considered to belong to the same ID. Images with distances less than the threshold are sorted by distance, and the minimum distance between the candidate set and the query image is found. This minimum distance is the matching target, the target status is updated, and the target's historical information is obtained.

[0108] Step 3): Establish a spherical coordinate system with the camera optical center as the origin, and combine the camera intrinsic and extrinsic parameters and PTZ parameters to convert the target pixel coordinates into spherical coordinates;

[0109] See appendix for details Figure 3 Specifically:

[0110] S41: The pixel coordinate system is transformed into the camera coordinate system as follows:

[0111]

[0112] Among them, f x f y Indicates focal length; (c x ,c y (x1, y1, z1) represents the principal point position of the camera, (u, v) represents the pixel coordinates, and (x1, y1, z1) represents the pixel coordinates. T These are the homogeneous coordinates of the target point in the camera coordinate system.

[0113] S42: Normalize the coordinates in the camera coordinate system to (x2, y2, y3). T Satisfy the following formula:

[0114]

[0115] S43: Calculate the spherical coordinates (β, α) of the point. T It can be calculated using the following formula:

[0116]

[0117] S44: World spherical coordinates of a point at time t (β) w (t),α w (t)) T That is, (p(0),t(0))T The target coordinates in the spherical coordinate system with the initial position can be calculated using the following formula:

[0118]

[0119] Where (p(0),t(0)) and (p(t),t(t)) represent the poses of the PTZ camera at the initial position t=0 and time t=t, respectively, (β c (t),α c (t)) T The spherical coordinates of a point on the PTZ camera;

[0120] Step 4): Use the target tracking information to solve for the angular velocity of the target motion, and establish a velocity control equation by comprehensively considering the target motion angular velocity and the distance deviation between the target center and the image center;

[0121] Specifically:

[0122] S51: See appendix for details Figure 4 The formula for calculating the angular velocity of the target motion is as follows:

[0123]

[0124] Where T(t) and T(t-1) represent the timestamps at times t and t-1, respectively, and β w (t),β w (t-1) represents the latitude coordinates at times t and t-1, α w (t),α w (t-1) represents the longitude coordinates at times t and t-1.

[0125] S52: Image center pixel coordinates (u c ,v c The condition (W2, H2) satisfies the condition (β), and the corresponding spherical coordinates are (β). cc ,α cc ), can be calculated using the S54 method.

[0126] S53: A preset frame is set in the central region of the PTZ camera. The area inside the preset frame is the stationary zone, and the area outside the preset frame is the rotation and pitch zone. To ensure that the target always moves within the central region of the PTZ camera, an error equation is established:

[0127]

[0128] Where (β(t), α(t)) are the target latitude and longitude at time t.

[0129] S54: Assuming the target's position in the image is in the same direction as its motion, the rotation and pitch angular velocities of the PTZ camera at time t must satisfy the following equation:

[0130]

[0131] Where Vt is the actual rotation time of the PTZ camera, and τ βi , τ αi The delay for the rotation and pitch motion of the PTZ camera is i (1≤i≤N).

[0132] S55: The equation expressing the motion delay τ of the PTZ camera is as follows:

[0133]

[0134] Where δ is α or β

[0135] S56: Regarding the PTZ camera's active target tracking start-up control speed requirement, i.e., satisfying S64, Vt≥0, therefore satisfying the expression:

[0136]

[0137] Step 5): Calibrate the gear speed and start-up time delay, and output control parameters using incremental PID control method for start-up and gear shifting situations;

[0138] See appendix for details Figure 8 Specifically:

[0139] S61: If the system starts from a standstill, the startup time delay needs to be considered. During the estimated startup time delay, change the speed gear every 20 seconds, run for 15 seconds, stop, and then change to the next speed gear after 5 seconds. Simultaneously record the PTZ parameters at different times. See the appendix for details on the change in rotation direction angle (pan) over time. Figure 5 . (Regarding the appendix) Figure 5 Six levels of magnification, see attached. Figure 6 The dotted box indicates the stop command, the line box indicates the stop point, the dotted line box indicates the start command, and the solid line box indicates the point where the speed at each gear is maintained. By calculating the slope of the speed change at each gear, the angular velocity of rotation at different gears can be calculated. Simultaneously, the time delay τ from command initiation to movement of the PTZ camera at different speed gears can be calculated. δ Where δ is α or β. The statistical results of the speed variation curves with start-stop delay for the seven gears in the direction of rotation are shown in the appendix. Figure 7 The solid line represents the curve relating rotational angular velocity to gear position, the dashed line represents the curve relating start-up time delay to gear position, and the dotted line represents the curve relating stop-stop time delay to gear position.

[0140] S62: Establishment of Incremental PID Control Equations:

[0141] Vω(t)=k p(er(t)-er(t-1))+k I er(t)+k d (er(t)-2er(t-1)+er(t-2))

[0142] S63: Since a PID control strategy is used during gear shifting, τ = 0 at this time, therefore ω βc (t+1) and ω αc (t+1) can be represented as:

[0143]

[0144] S64: Since the PTZ camera has a motion speed of N levels, select levels i and j (1 ≤ i, j ≤ N) such that i and j satisfy:

[0145]

[0146] Wherein, ω is set β0 =ω α0 =0

[0147] S65: Select a gear that satisfies the minimum value of the gear, i.e., min{i,j}, and then output the speed control gear.

[0148] Step 6): Implement motion control on the camera to achieve active tracking of the moving target.

[0149] Example

[0150] The active tracking method designed in this invention is mainly divided into a perception module and a control module. First, the perception module establishes a spherical coordinate system with the optical center of the PTZ camera as the origin, and uses the YOLOv5 target detection algorithm to acquire the target's position, category, and ID. The operator selects the target to be followed from the currently detected and tracked target results using the mouse, locks onto the selected target, and uses a target re-identification algorithm to improve the target locking effect, acquires historical target information, converts the target pixel coordinates into spherical coordinates, and obtains the target's latitude, longitude, and angular velocity. The control module comprehensively considers system startup delay, sets a preset bounding box, establishes velocity control equations for the target's angular velocity and the distance deviation between the target center and the image center, and uses a PID control algorithm to control the PTZ camera to complete the active target tracking. This invention utilizes C++ to design the active tracking system on Ubuntu, and outdoor target tracking experiments were conducted.

[0151] I. Model Parameters and Experimental Scenarios

[0152] (1) The parameters of the active follow system are as follows:

[0153]

[0154] (2) The experimental scenario is as follows:

[0155] In an open outdoor area, follow a vehicle as the target type, with a movement speed of 5-20 km / h;

[0156] II. Test Results

[0157] (1) To maintain the vehicle target's motion, the algorithm proposed in this invention was used to actively follow the vehicle target. Testing showed that the target consistently appeared in the image, with a total tracking frame count of 5766. During tracking, to keep the target within the preset frame, the rotation, pitch, and zoom values ​​of the PTZ camera were continuously adjusted. The parameter changes are as follows: Figure 9 , Figure 10 and Figure 11 As shown in the figure. The statistical analysis of the x and y coordinates of the target center position is shown in the following graph. Figure 12 and Figure 13 As shown, since the coordinates of the upper left corner of the preset box are [480, 270] and the coordinates of the lower right corner are [800, 450], in order to keep the target within the preset box, the x-coordinate of the target's center position needs to be kept as close as possible to the interval [480, 800] and the y-coordinate as close as possible to the interval [270, 450]. Figure 12 and Figure 13 In the sample, the statistical probability that the x-coordinate of the target center is in the interval [480, 800] is 75.79%, and the statistical probability that the y-coordinate of the target center is in the interval [270, 450] is 94.66%, with the distribution concentrated inside the preset box.

[0158] (2) The methods used in this invention, based on spherical coordinates and image coordinates, are compared to actively track targets moving at varying speeds. The same target motion path and initial state of the PTZ camera are set, and the success rate and accuracy of active tracking are statistically analyzed as follows: Figure 14 and Figure 15 As shown, the tracking success rate and accuracy based on the spherical coordinate system method are represented by solid lines, at 79.69% and 73.27%, respectively. The tracking success rate and accuracy based on the image coordinate system method are represented by dashed lines, at 66.4% and 70.15%, respectively. It is worth noting that the tracking success rate here is determined based on the Intersection over Union (IOU) value; a value greater than 0.5 indicates successful tracking. Although both methods can actively track the target, the spherical coordinate system method effectively calculates the distance of the target relative to the image center, significantly improving both the active tracking success rate and accuracy.

[0159] The above detailed description of an adaptive active following method for a variable-speed moving target based on spherical coordinates, with reference to the embodiments described above, is illustrative rather than limiting. Several embodiments may be listed within the defined scope. Therefore, variations and modifications that do not depart from the overall concept of the present invention should be within the protection scope of the present invention.

Claims

1. A method for adaptive active following of a variable speed moving target based on spherical coordinates, characterized in that: A spherical coordinate system is established with the optical center of the PTZ camera as the origin, a perception module and a control module are established respectively, the latitude and longitude and angular velocity of the target are obtained through the spherical coordinate, a speed control equation is established by using the angular velocity of the target motion and the distance deviation between the target center and the image center, the PTZ camera is controlled by using the incremental PID control algorithm, and the active following of the target is completed, The specific steps are as follows: I. Establishing a perception module Step 1) The position, category and id information of the target are output by using the yolov5 target detection algorithm; Step 2) A tracking target locking strategy is established, a single target is locked by using a target re-identification algorithm, and historical information of the target is obtained; Step 3) A spherical coordinate system is established with the optical center of the camera as the origin, and the pixel coordinates of the locked target are converted into spherical coordinates in combination with the camera internal and external parameters and the PTZ parameters; II. Establishing a control module Step 4) The angular velocity of the target motion is solved by using the target tracking information, and a speed control equation is established by comprehensively considering the angular velocity of the target motion and the distance deviation between the target center and the image center; the solving of the angular velocity of the target motion and the establishment of the speed control equation are specifically as follows: ① The solving formula of the angular velocity of the target motion is as follows: Wherein, T(t), T(t-1) represent time stamps at t and t-1 time, respectively w (t), T(t-1) represent time stamps at t and t-1 time, respectively w (t), T(t-1) represent time stamps at t and t-1 time, respectively w (t), T(t-1) represent time stamps at t and t-1 time, respectively w (t), T(t-1) represent time stamps at t and t-1 time, respectively ②Image center pixel coordinates (u c ,v c ) satisfy (W / 2, H / 2), corresponding spherical coordinates are (β cc ,α cc ); ③ A preset box is set in the central region of the PTZ camera, the preset box is a static area, and the preset box is a rotating and pitching area, in order to make the target always move in the central region of the PTZ camera, an error equation is established: Wherein, (β(t), α(t)) is the latitude and longitude of the target at t time, ④ Assuming that the position of the target in the image is the same as the motion direction of the target, the rotation and pitching angular velocity of the PTZ camera at t time needs to satisfy the equation: wherein, Δt is the actual rotation time of the PTZ camera, τ βi , τ αi is the rotation and pitch motion delay of the PTZ camera, and the gear is i (1≤i≤N). ⑤ The motion delay τ of the PTZ camera is expressed by the equation Wherein, δ is α or β ⑥ In order to meet the requirements of the control speed of the PTZ camera active tracking target, that is, to meet the case of ④, Δt≥0 is required, so the expression is satisfied: Step 5) The gear speed and start time delay are calibrated, and the incremental PID control method is used to output control parameters in the start and gear shifting conditions; Step 6) The motion control of the PTZ camera is implemented, and the active following of the moving target is completed.

2. The method of claim 1, wherein the method further comprises: Step 1) The position, category and id output of the target, wherein: ① Data preprocessing: data enhancement and adaptive image padding processing are performed on the image, data enhancement is completed, and the image size and network input size are ensured to be the same; ② Feature extraction network; the image is input into the target detection network framework, which is Backbone main network, Neck network and Head network in turn, the Backbone main network includes Focus module and CSP module, which is used to obtain multi-layer image features; the Neck network mainly consists of FPN module and PAN structure, which is used to fuse multi-layer sampling information; the Head network is mainly used for predicting image features, GIOU_loss is used as the loss function of the bounding box, and weighted NMS is used to screen the anchor box; ③ The detection target information is classified and regressed, and the target information output includes: x=(id, t, x, y, w, h, cid, conf) Wherein, id represents the identity number of the target, t represents the image acquisition time of the target, (x, y, w, h) represents the position information of the target frame in the image, cid represents the category information of the target, and conf represents the category confidence of the target.

3. The method of claim 1, wherein the method further comprises: Step 2) The target re-identification algorithm is used to lock and follow a single specified target, and the historical information of the target is obtained, specifically: ① The mouse clicks or the preset target mode selects the tracking target, lists the target as the query image, and establishes the historical query image set of the target, and the size of the set is N; (ii) extracting target features based on a target re-identification algorithm: using the target output by the target detection algorithm to establish a candidate image set g i , the size of the set is M, using a convolutional neural network to extract image features of the candidate image set g i and the query image q j , generating a feature vector r i ,r j , r i = f(g i ), r j = f(q j ) Wherein, f(x) represents the feature extraction of the input image x by using the convolutional neural network f; ③ Calculate the cosine distance between the historical query image set of the target and all image feature vectors in the candidate set to generate a distance matrix D, and the elements in D are: D i,j ＝1-r j T r i Wherein, D is M×N ④ Calculate the average distance between the candidate set and the query image by row: When d k If the distance is less than the threshold, it is considered that the two targets may belong to the same ID; the images less than the threshold are sorted according to the distance, the minimum distance of the candidate set relative to the query image is found, that is, the matching target, the locked target state is updated, and the target historical information is obtained.

4. The method of claim 1, wherein the method further comprises: Step 3) The spherical coordinate system is established and the spherical coordinate is converted, specifically: ① The pixel coordinate system is converted into the camera coordinate system as follows: where f x , f y denote the focal length; (c x , c y ) denote the camera principal point position, (u, v) denote the pixel coordinates, (x1 y1 z1) T is the homogeneous coordinates of the target point in the camera coordinate system; ii) normalizing the coordinates in the camera coordinate system, (x2, y2, y3) T satisfies the following equation: ③ Calculate the spherical coordinates (β, α) of the point T This can be calculated from the following equation: (β w (t),α w (t)) T , i.e. the target coordinates in the spherical coordinate system with (p(0), t(0)) T as the initial position can be calculated by the following formula: where (p(0), t(0)) and (p(t), t(t)) represent the pose of the PTZ camera at the initial position t = 0 and t = t, respectively, (β c (t), a c (t)) T are the spherical coordinates of a point on the PTZ camera.

5. The method of claim 1, wherein the method further comprises: Step 5) The gear speed and start time delay are calibrated, and the incremental PID control method is used to output the control parameters under the conditions of starting and gear shifting, specifically:

1. Calibration start time delay, calculate the time delay τ of PTZ camera from the command start to movement at different speed gears δ wherein δ is either a or β; ② The incremental PID control equation is established: Δω(t) = k p (er(t) - er(t-1)) + k I er(t) + k d (er(t) - 2er(t-1) + er(t-2)) iii. Since PID control strategy is used during the movement, at this time τ = 0, thus ω βc (t+1) and ω αc (t+1) can be expressed as: ④ Since the PTZ camera movement speed is N gears, select gears i, j (1≤i, j≤N) so that i, j satisfy: where ω β0 = ω α0 = 0 ⑤ Select the gear to satisfy the minimum gear value, that is, min{i, j}, at this time the output speed control gear.

Images