An object tracking method without updating the Euclidean signed distance field

Abstract

The present invention belongs to the fields of robotics, computer vision, and path planning, and discloses a target tracking method without updating the Euclidean signed distance field. The steps are as follows: pre-construct a Euclidean signed distance field that does not need to be updated; detect, segment, and solve the target position; predict the future position of the target; calculate the visible area of the target and find the optimal unobstructed observation point sequence; optimize the observation point sequence to obtain a smooth tracking trajectory. The tracking algorithm proposed by the present invention for unmanned aerial vehicles can effectively evaluate the target visibility through the pre-constructed FOV-ESDF, and effectively avoid occlusion and collision during the tracking process. The path search algorithm and trajectory optimization algorithm proposed by the present invention have strong real-time performance. Since there is no need to construct and update the environmental ESDF, the computational amount is reduced by about 10 times.

Description

Technical Field

[0001] The present invention belongs to the fields of robotics, computer vision, and path planning, and relates to efficient path planning, field-of-view perception evaluation of a target tracking system, and motion planning optimization algorithms. Specifically, it is an aerial target tracking method that does not require updating the Euclidean signed distance field. Background Art

[0002] Target tracking is an important research topic in the fields of mobile robots and unmanned aerial vehicles. Its core task is to track a target in real time in a dynamic environment while ensuring path safety and field-of-view continuity. This technology has wide application value in many fields such as security monitoring, intelligent transportation, film production, and post-disaster rescue.

[0003] Real-time target tracking is one of the key technologies in the field of mobile robots such as unmanned aerial vehicles. Existing state-of-the-art methods (visibility perception planners, adaptive dynamic tracking methods, etc.) usually rely on the environmental Euclidean signed distance field (ESDF) for target visibility evaluation and path optimization. However, the online update of the environmental ESDF requires a large amount of computing resources. Especially in open scenarios with complex obstacles and frequent dynamic environmental changes, this computational overhead will increase significantly, becoming the main bottleneck of the real-time tracking system. In addition, most current tracking methods (fast trackers, elastic trackers, safety-first trackers, intention perception planners, etc.) lack specialized optimization for field-of-view perception, which easily leads to problems such as the target being occluded by obstacles and lost or the planned path being too conservative.

[0004] To overcome the above problems, the present invention proposes a target tracking method that does not require updating the ESDF. By designing a field-of-view ESDF (FOV-ESDF) and an efficient path search algorithm, the computational complexity is significantly reduced, and at the same time, the robustness and continuity of target tracking are improved. Summary of the Invention

[0005] The present invention aims to provide an efficient and low-computation-cost target tracking method that can achieve real-time target tracking in a dynamic open environment. The present invention can greatly increase the duration of the target within the tracking field of view, reduce the probability of occlusion, and avoid collisions between the tracker and obstacles in the environment. The path generated by the present invention meets the smoothness requirements, ensuring the stability of the tracker's movement. In addition, the present invention does not require constructing and updating the environmental ESDF, reducing the computational amount by about 10 times.

[0006] The technical solution of the present invention:

[0007] A target tracking method without updating the Euclidean signed distance field is as follows:

[0008] Step 1: Pre-build the Euclidean signed distance field that does not need to be updated;

[0009] Measure the field of view angle of the tracker camera and define the optimal observation distance from the tracker camera to the target , if the horizontal field of view angle of the tracker camera is and the vertical field of view angle is , then the field of view of the tracker camera is modeled by a quadrangular pyramid, and the formula is:

[0010]

[0011] where, is the coordinate in the tracker coordinate system, represents the transpose operator, is determined by the pre-defined optimal observation distance ;

[0012] For a point located within the field of view , the FOV-ESDF value is defined as the distance from the point to the nearest field of view boundary plane; for a point located outside the field of view , the FOV-ESDF value is defined as .

[0013] Step 2: Detect, segment and solve the target position;

[0014] Use the YOLOv8 model to perform instance segmentation on the target, then solve the relative position between the target and the tracker, and store it in the queue;

[0015] Step 3: Predict the future position of the target;

[0016] Based on the target position data of the nearest several frames to the current time, use the quadratic smoothing function to fit the motion trajectory of the target, and obtain the possible positions of the target at multiple future time steps according to the fitted motion trajectory;

[0017] Step 4: Calculate the visible area of the target and find the optimal observation point sequence without occlusion;

[0018] The specific implementation process is as follows:

[0019] For the th observation position of the target, the observation area at this position is defined as:

[0020]

[0021] where, ; Denote the th column of the third-order identity matrix, taking 1, 2, 3; denote the tracker position vector satisfying the above conditions; denote the maximum allowable observation distance error in the vertical direction, denote the maximum allowable observation distance error in the horizontal direction;

[0022] Considering the occlusion of the target by obstacles, the occlusion area is defined as:

[0023]

[0024] where, denote the local grid map, if and only if there is no obstacle; denote the integration variable;

[0025] The calculation formula for the visible area of the target is:

[0026]

[0027] For the tracker located at generate the candidate position of the tracker at the next moment, and the formula is:

[0028]

[0029] Finally, perform a breadth-first search with as the source point to obtain the position of the tracker at the next moment.

[0030] Step 5: Optimize the observation point sequence to obtain a smooth tracking trajectory.

[0031] The specific implementation process is as follows:

[0032] Use a third-order B-spline to parameterize the position and orientation of the tracker with respect to time, and the formula is:

[0033]

[0034]

[0035] where, denote the yaw angle corresponding to the tracker at position , denote the th position control point, denote the th angle control point;

[0036] Push the obstacles out of the field of view through occlusion penalties, and the objective function is:

[0037]

[0038] Among them, represents taking the value of FOV - ESDF, represents the number of obstacle points entering FOV - ESDF, which is calculated by the following formula:

[0039]

[0040] Among them, is the world coordinate of the th obstacle point, represents the rotation matrix; this occlusion penalty will generate gradients with respect to position and yaw angle, and is calculated as follows:

[0041]

[0042]

[0043] Among them, represents the Nabla operator;

[0044] By observing the occlusion penalty, the tracker is kept at an appropriate distance from the target, and the objective function is:

[0045]

[0046] Among them, is the maximum value of FOV - ESDF;

[0047] The gradient of this occlusion penalty function is:

[0048]

[0049]

[0050] The angular penalty is used to promote the convergence of the observation occlusion penalty, and the objective function is:

[0051]

[0052] The gradient of this penalty function is:

[0053]

[0054]

[0055] Among them, is the arctangent function with quadrant discrimination;

[0056] For obstacle avoidance, the robot-centered ESDF is the RC-ESDF, and the objective function is:

[0057]

[0058] where is the number of obstacles entering the RC-ESDF, is the evaluation value of the RC-ESDF;

[0059] By constraining the velocity and acceleration of the B-spline control points, the kinematic feasibility of the trajectory is fully guaranteed, and the objective function is:

[0060]

[0061]

[0062] where and represent the velocity control point and acceleration control point of the position B-spline curve, and represent the velocity control point and acceleration control point of the yaw angle B-spline curve, 、 、 、 respectively represent the limit velocity, acceleration, angular velocity and angular acceleration of the tracker, and the function ;

[0063] Benefiting from the convex hull property of the B-spline, smoothness is ensured by minimizing the higher-order derivatives of the control points, and the objective function is:

[0064]

[0065] Finally, the L-BFGS algorithm is used to jointly optimize these objective functions to obtain the final trajectory.

[0066] Advantages of the present invention:

[0067] (1) The tracking algorithm for unmanned aerial vehicles proposed by the present invention can effectively evaluate the target visibility through the pre-constructed FOV-ESDF, and effectively avoid occlusion and collision during the tracking process.

[0068] (2) The path search algorithm and trajectory optimization algorithm proposed by the present invention have strong real-time performance. Since there is no need to construct and update the environmental ESDF, the computational complexity is reduced by about 10 times. Description of the drawings

[0069] Figure 1 is the implementation flowchart of the present invention.

[0070] Figure 2 Schematic cross-section of FOV-ESDF proposed by the present invention.

[0071] Figure 3 Schematic diagram of optimizing the observation position using FOV-ESDF in the present invention.

[0072] Figure 4 Schematic diagram of the tracking trajectory for target tracking using the method of the present invention.

[0073] Figure 5 Tracking effect of the fast tracker.

[0074] Figure 6 Tracking effect of the visibility-aware planner.

[0075] Figure 7 Tracking effect of the safety-first two-stage tracker.

[0076] Figure 8 Tracking effect of the elastic tracker.

[0077] Figure 9 Tracking effect of the present invention. Detailed implementation manners

[0078] The following further illustrates the detailed implementation manners of the present invention in conjunction with the accompanying drawings and technical solutions.

[0079] Embodiment

[0080] An airborne target tracking method without updating the Euclidean signed distance field, the steps are as follows:

[0081] (1) The horizontal field of view angle of the tracker camera is measured to be 69.4° and the vertical field of view angle is 42.5°. The optimal observation distance from the camera to the target is defined as 2.2 m, and thus the field of view range of the tracker is determined. First, the distance transformation algorithm is used to calculate the values of most key points in the FOV-ESDF, and then the trilinear interpolation is used to obtain the FOV-ESDF value and gradient of any point. The constructed FOV-ESDF profile is as shown in Figure 2 The closer to the red area, the higher the FOV-ESDF value.

[0082] (2) Predict and model the target position and trajectory.

[0083] The camera image is input into the YOLOv8 model to obtain the instance segmentation result of the target, and the relative position between the target and the tracker is calculated using the camera internal and external parameters. Then, based on the position data of the target in the recent several frames, the quadratic smoothing function is used to fit the motion trajectory of the target, so as to obtain the positions of the target at multiple future time steps.

[0084] (3)Define the visible area based on the target position and perform path planning.

[0085] Determine the best observation range according to the distance between the target and the tracker. By restricting the offsets of the target in the horizontal and vertical directions, a region containing the best observation distance is generated. This region ensures that the target is always within the field of view of the tracker and at the best observation position. On the other hand, to avoid the target being blocked by obstacles, ray detection is used to evaluate whether there are obstacles between the target and the tracker. Any obstacle blocking the line of sight of the target is regarded as an occlusion area. The final visible area is obtained by removing the occlusion area from the observation area.

[0086] The present invention performs path planning based on the generated visible area. First, according to the predicted position of the target, an initial reference point is selected within the observation area. This point has the shortest distance from the current position of the tracker and is the best observation distance. Then, starting from the initial point, a path is gradually searched within the visible area. During the search process, the algorithm selects an unobstructed path to ensure that the generated reference path can maintain a clear observation of the target. Finally, a set of reference points is generated for each target position, and these points are connected into a continuous path to guide the next movement of the tracker.

[0087] (4)Optimize the trajectory of the result of path planning to obtain the final trajectory of the tracker.

[0088] To ensure that the target is always within the field of view of the tracker, during the optimization process, the present invention evaluates the visibility of the target according to the pre-constructed FOV-ESDF and adjusts the path. If the target is close to the edge of the field of view or blocked by obstacles, the optimization algorithm will adjust the position and heading angle of the tracker to relocate the target to the central area of the field of view. The present invention ensures that the obstacles within the field of view are excluded by optimizing the relative position of the tracker, while maintaining a clear observation of the target, as Figure 3 shown.

[0089] The tracking effect of the present invention is as Figure 4 shown. The green line is the trajectory of the tracker, the blue line is the trajectory of the target being tracked, and the red pyramid is the field of view of the tracker. The target passes through a narrow passage at point A and quickly turns. The tracker can follow the target into the narrow passage and maintain visibility. The target suddenly turns along the obstacle at points B and C, which is likely to cause occlusion. However, the tracker successfully avoids the occlusion caused by environmental obstacles by adjusting its own observation position. The comparison chart of the tracking effects between the present invention and other advanced methods is as Figures 5-9As shown. The heatmap shows the position of the target in the tracker's field of view throughout the tracking process. Compared with previous methods, this method can better keep the target at the exact center of the field of view and maintain a reasonable tracking distance. Real-time performance is another major feature of the present invention. Table 1 statistics the average time required for one path search and one trajectory optimization of the method of the present invention in a one-minute tracking task.

[0090] Table 1 Average calculation time for one planning

[0091]

[0092] Table 1 shows that thanks to the efficient calculation method, the present invention can achieve efficient optimization with extremely low computational overhead in complex scenarios.

[0093] In summary, whether the target moves rapidly or the environment undergoes sudden changes, the present invention can quickly respond and regenerate the optimal trajectory to ensure the continuity of tracking.

Claims

1. A target tracking method without updating the Euclidean signed distance field, characterized in that: Here are the steps: Step 1: Pre-construct the Euclidean signed distance field that does not need to be updated; Step 2: Detect, segment and calculate the target position; Use the YOLOv8 model to perform instance segmentation on the target, then solve the relative position of the target and the tracker and store it in a queue; Step 3: Predict the target's future observation position; Based on the target position data of the nearest several frames, the motion trajectory of the target is fitted using a quadratic smoothing function, and the possible position of the target in multiple future time steps is obtained according to the fitted motion trajectory; Step 4: Calculate the visible area of ​​the target and find the optimal sequence of observation points without occlusion; For the kth observation position of the target The observation area at this location is defined as: Among them, Π2=(e1,e2,0); e j represents the jth column of the third-order identity matrix, where j is 1, 2, or 3; p represents the position vector of the tracker that satisfies the above conditions; ε v Indicates the maximum observation distance error allowed in the vertical direction, ε h Indicates the maximum observation distance error allowed in the horizontal direction; Considering the occlusion of the target by obstacles, the occlusion area is defined as: Where G represents the local grid map, G(p) = 0 if and only if there is no obstacle at p; λ represents the integral variable; The calculation formula of the visible area of ​​the target is: In k =D k -O k For the position p k-1 The tracker generates the candidate position of the tracker at the next moment. The formula is: Finally, c k Perform a breadth-first search for the source point to get the position of the tracker at the next moment; Step 5: Optimize the observation point sequence to obtain a smooth tracking trajectory.

2. The target tracking method without updating the Euclidean signed distance field according to claim 1, characterized in that: The specific implementation process of step 1 is as follows: Measure the field of view of the tracker camera and define the optimal observation distance d from the tracker camera to the target. If the horizontal field of view of the tracker camera is α and the vertical field of view is β, the field of view of the tracker camera is modeled with a tetrahedron, and the formula is: Among them, (x, y, z) are the coordinates in the tracker coordinate system, represents the transposition operator, and D is determined by the predefined optimal observation distance d; For a point within the field of view F, the FOV-ESDF value is defined as the distance from the point to the nearest field of view boundary plane; for a point outside the field of view F, the FOV-ESDF value is defined as 0.

3. The target tracking method without updating the Euclidean signed distance field according to claim 1, characterized in that: The specific implementation process of step 5 is as follows: The position and orientation of the tracker are parameterized in time using a third-order B-spline, as follows: Among them, ψ k Indicates that the tracker is at position p k The corresponding yaw angle, Q k represents the kth position control point, Ψ k represents the kth angle control point; The obstacles are removed from the field of view through occlusion penalty, and the objective function is: Where K represents the number of tracker position vectors to be optimized, Ξ represents the value of FOV-ESDF, and N k represents the number of obstacle points entering the FOV-ESDF, r k,i Calculated by the following formula: r k,i =R k (w k,i -p k ) Among them, w k,i is the world coordinate of the ith obstacle point, R k Represents the rotation matrix; the occlusion penalty produces a gradient for the position and a gradient for the yaw angle, calculated as follows: in, represents the Nabla operator; By observing the occlusion penalty, the tracker maintains an appropriate distance from the target. The objective function is: Where M is the maximum value of FOV-ESDF; The gradient of the occlusion penalty function is: The angle penalty is used to promote the convergence of the observation occlusion penalty. The objective function is: The gradient of the penalty function is: Among them, atan2 is the inverse tangent function with quadrant discrimination; For obstacle avoidance, the robot-centered ESDF is used as RC-ESDF, and the objective function is: Among them, M k is the number of obstacles entering the RC-ESDF, H i,k is the evaluation value of RC-ESDF; The kinematic feasibility of the trajectory is fully guaranteed by constraining the velocity and acceleration of the B-spline control points. The objective function is: Among them, V k and A k Indicates the velocity control point and acceleration control point of the position B-spline curve, V k and A k represents the velocity control point and acceleration control point of the yaw angle B-spline curve, v, a, ω, a represent the limit velocity, acceleration, angular velocity and angular acceleration of the tracker respectively, and the function r(x) = max(x, 0); Thanks to the convex hull property of B-spline, smoothness is ensured by minimizing the high-order derivatives of the control points. The objective function is: Finally, the L-BFGS algorithm is used to jointly optimize these objective functions to obtain the final trajectory.

Images