Unmanned aerial vehicle target tracking method and system

Abstract

The invention provides an unmanned aerial vehicle target tracking method and system, and relates to the technical field of data processing, and the method comprises the steps: obtaining a plurality of video frames which have to-be-tracked target labeling information and have continuous collection moments; extracting a phase information structure descriptor of the to-be-tracked target from each video frame by using Zernike moments; according to the phase information structure descriptor, constructing a phase flow field describing motion characteristics of the to-be-tracked target; whether the to-be-tracked target is shielded or not is judged by combining the shielding factor based on the phase information structure descriptor; if yes, determining a plurality of prediction positions of the to-be-tracked target through the Lie group motion principle in combination with the phase flow field; collecting a plurality of prediction video frames by taking each prediction position as a focusing center of an unmanned aerial vehicle camera; taking minimization of the phase gradient cross product as a target, and capturing a to-be-tracked target from each prediction video frame; otherwise, capturing the target to be tracked according to the phase information structure descriptor; and tracking the to-be-tracked target according to the capture result. And the tracking anti-interference capability and the endurance capability are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a UAV target tracking method and system. BACKGROUND

[0002] A UAV is an autonomous flying vehicle, usually powered by batteries, and controlled through remote control or autonomous navigation systems for its flight and task execution. The application of UAVs covers military, agriculture, environmental monitoring, photography, express delivery and many other fields. They are widely used for their flexibility, cost-effectiveness and efficiency, especially when entering dangerous or hard-to-reach areas.

[0003] UAV target tracking is crucial in many applications, by tracking the target in real time, the UAV can accurately locate and track the movement of the target, ensuring continuous monitoring and data collection of the target. For automated tasks such as search and rescue, it can improve the efficiency and accuracy of operations and reduce the waste of human resources. At the same time, accurate target tracking can help the UAV avoid obstacles, optimize the flight path, ensure flight safety, and improve the success rate of tasks.

[0004] However, with the rapid development of artificial intelligence, current UAV target tracking largely applies artificial intelligence for target recognition and tracking, which consumes high computing resources, reduces the endurance of the UAV, and most of the image processing algorithms based on artificial intelligence are based on real-time pixel-by-pixel scanning to identify target color differences or local contour shape differences to track the target, which has insufficient response capability, further increasing the consumption of computing resources, and in the presence of background interference or occlusion, it is prone to target loss or tracking errors. SUMMARY

[0005] In view of the above deficiencies of the prior art, the purpose of the embodiments of the present application is to provide a UAV target tracking method, which can solve the technical problems of the prior art that with the rapid development of artificial intelligence, current UAV target tracking largely applies artificial intelligence for target recognition and tracking, which consumes high computing resources, reduces the endurance of the UAV, and most of the image processing algorithms based on artificial intelligence are based on real-time pixel-by-pixel scanning to identify target color differences or local contour shape differences to track the target, which has insufficient response capability, further increasing the consumption of computing resources, and in the presence of background interference or occlusion, it is prone to target loss or tracking errors.

[0006] The first aspect of the embodiments of the present application proposes a UAV target tracking method, comprising:

[0007] S1: acquiring a plurality of video frames with continuous acquisition time of target tracking label information;

[0008] S2: according to the to-be-tracked target annotation information, extracting a phase information structure descriptor of the to-be-tracked target from each video frame by using a Zernike matrix;

[0009] S3: constructing a phase flow field describing a motion feature of the to-be-tracked target according to the phase information structure descriptor;

[0010] S4: judging whether the to-be-tracked target is occluded or not by combining an occlusion factor based on the phase information structure descriptor, if yes, entering step S5, otherwise, entering step S8;

[0011] S5: determining a plurality of predicted positions of the to-be-tracked target by combining the phase flow field and a Lie group motion principle;

[0012] S6: collecting a plurality of predicted video frames by taking each predicted position as a focus center of the unmanned aerial vehicle camera;

[0013] S7: capturing the to-be-tracked target from each predicted video frame by taking minimizing a phase gradient cross product as a target;

[0014] S8: capturing the to-be-tracked target according to the phase information structure descriptor;

[0015] S9: tracking the to-be-tracked target according to the capturing result.

[0016] The second aspect of the embodiment of the present application provides an unmanned aerial vehicle target tracking system, comprising a processor and a memory.

[0017] The memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the unmanned aerial vehicle target tracking method according to the first aspect.

[0018] The third aspect of the embodiment of the present application provides a readable storage medium, and the readable storage medium stores programs or instructions, and the programs or instructions are executed by a processor to implement the steps of the unmanned aerial vehicle target tracking method according to the first aspect.

[0019] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0020] In the embodiment of the present application, the phase information structure descriptor is extracted from the video frame by the Zernike moment, which can effectively capture the phase characteristics of the target, thereby reducing the influence of background interference and avoiding target loss caused by complex background or target blur. In addition, by constructing the phase flow field, the motion characteristics of the target can be described, and the prediction process of the target position is optimized by combining the Lie group motion principle, thereby effectively reducing the calculation amount based on image pixel scanning, reducing the consumption of computing resources, and prolonging the endurance time of the unmanned aerial vehicle. Especially when the occlusion occurs, the method can predict multiple positions of the target by combining the phase flow field and the occlusion factor, and accurately track the target, solving the problem that the traditional method cannot stably track the target when the target is occluded. In addition, the minimization method based on the phase gradient cross product further improves the accuracy and response speed of target capture, enhances the real-time performance, ensures the continuous tracking of the target, and further improves the accurate long-time tracking ability of the unmanned aerial vehicle in complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0022] Figure 1 is a flow diagram of an unmanned aerial vehicle target tracking method provided by an embodiment of the present application;

[0023] Figure 2 is a structural diagram of an unmanned aerial vehicle target tracking system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order for those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0025] The unmanned aerial vehicle target tracking method provided by the embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and application scenarios.

[0026] Reference is made to the drawingsFigure 1 Figure 1 shows a flowchart of a method for tracking a target by a UAV according to an embodiment of the present application.

[0027] The method for tracking a target by a UAV according to an embodiment of the present application can comprise the following steps:

[0028] S1: acquiring a plurality of video frames with continuous time instants and target annotation information to be tracked.

[0029] The target annotation information to be tracked refers to the specific position, shape and other feature information of the target determined by pre-processing or manual annotation when the UAV first captures the target. The continuous time instants refer to the continuous shooting of a plurality of video frames by the UAV, which are continuous in time.

[0030] It should be noted that the video frames are at least two to capture the motion characteristics. Before determining the tracking, the UAV will continuously collect a plurality of video frames of the target to be tracked, and then return to mark to determine the target to be tracked. These frames are used to capture the initial position and motion characteristics of the target. By continuously collecting video frames, the dynamic behavior of the target can be more accurately described, providing accurate initial data for subsequent target recognition and tracking.

[0031] S2: extracting a phase information structure descriptor of the target to be tracked from each video frame using Zernike moments according to the target annotation information to be tracked.

[0032] The Zernike moment is a mathematical tool based on Zernike phase field, which is used to extract the phase information of the image. The Zernike moment can effectively describe the microstructure of the target by processing the phase of the image, and capture the detailed information in the image. The phase information structure descriptor is a mathematical description extracted from the image, which describes the phase structure of the target, especially the edge features and local shape of the target. The high-order component of the Zernike moment can capture the fine structure of the target, and the phase information has strong robustness to illumination changes, local occlusions and background noise, and is not easily disturbed by environmental changes, thereby enhancing the stability of target recognition.

[0033] It should be noted that by using Zernike moments to extract the phase information structure descriptor, the "structure fingerprint" of the target can be effectively constructed, which is not sensitive to local occlusion, illumination change and background noise. Compared with traditional image feature extraction methods, the Zernike moment can better maintain the stability of the target in complex environments, thereby improving the accuracy and reliability of target tracking, especially in the case of large occlusion or background interference, the target can be tracked accurately and continuously.

[0034] In one possible implementation, S2 specifically comprises:

[0035] S201: Locating an information area including a target to be tracked in a video frame according to the labeling information of the target to be tracked.

[0036] The information area is a marked area that is approximately circular, and the information area is the smallest circle that can completely include the target to be tracked.

[0037] S202: Calculating weighted integrals of grayscale images and Zernike polynomials under different order combinations in the information region to obtain Zernike moments describing the target to be tracked, wherein the order combination is a combination of radial and angular orders of the Zernike polynomials.

[0038] The radial order describes how the Zernike polynomials vary in the radial direction. The angular order describes how the Zernike polynomials vary in the angular direction. These two orders correspond to each other and can be set according to actual needs.

[0039] The calculation formula of Zernike moment is:

[0040]

[0041] Where n represents the radial order, m represents the angular order, represents the Zernike moment of the target to be tracked corresponding to the video frame at time t related to n and m, represents pi, represents the normalization coefficient that eliminates the influence of the information region size on the Zernike moment, Represents the information area corresponding to the video frame at time t, Represents the grayscale value of the pixel at position x in the video frame at time t, Represents the polar coordinates of each pixel in the information area, Represents , n, and m related Zernike polynomials.

[0042] Zernike polynomials are a type of orthogonal polynomial defined in polar coordinates, with their shape controlled by radial and angular orders. Zernike polynomials are effective in representing the geometric shape of objects in images, particularly within circular or near-circular regions. Their advantage lies in their orthogonality, enabling the extraction of independent features while avoiding interference between different features.

[0043] S203: Extracting phase components of Zernike moments under different order combinations respectively.

[0044] The extraction formula of the phase component is as follows:

[0045]

[0046] in, denotes a phase component corresponding to a video frame at time t related to n and m, denotes an argument function.

[0047] S204: combine the phase components corresponding to each video frame respectively to obtain the phase information structural descriptor of each video frame.

[0048] The combination formula of the phase information structural descriptor is specifically:

[0049]

[0050] wherein, denotes a phase information structural descriptor corresponding to a video frame at time t, denotes a preset maximum radial order.

[0051] It should be noted that the size of the preset maximum radial order can be set according to actual needs by those skilled in the art, and the present application does not limit it here.

[0052] Specifically, the core of the whole process is to describe and extract the phase information of the target through Zernike moments to improve the stability and accuracy of target tracking. First, the information area of the target in the video frame is located and labeled, and the area is labeled with the smallest circle to ensure that the target is completely contained. Then, by calculating the weighted integral of the gray image and the Zernike polynomial, the Zernike moments of the target are extracted, and the shape and structure information of the target are accurately modeled. Here, the combination of radial order and angular order is combined to capture the details of the target at different levels, and the normalized coefficient is used to eliminate the influence of the size of the information area. Step S203 extracts the phase component and obtains accurate phase information through the argument function, and finally, step S204 combines the phase components into a phase information structural descriptor to provide stable feature description for subsequent target tracking. This method has strong robustness to local occlusion, illumination change and background noise through phase information, and compared with traditional feature extraction methods based on color, shape or edge, it can better maintain the stability and accuracy of target tracking, especially in complex environment and fast motion, significantly improving the sustained tracking ability of the target.

[0053] S3: constructing a phase flow field describing the motion characteristics of the target to be tracked according to the phase information structural descriptor.

[0054] The phase flow field is constructed according to the extracted phase information structure descriptor, and is used to represent the motion characteristics of the target in the video frame. The phase flow field describes the motion trajectory of the target by tracking the changes of the phase information in the continuous video frames, thereby converting the static phase features into a dynamic motion field. This process can solve the problem of the separation between static features and dynamic motion estimation in traditional methods, and closely combines motion estimation and target tracking. Through the phase flow field, the motion trajectory of the target can still be predicted when the target is temporarily occluded, ensuring continuous tracking and occlusion detection of the target.

[0055] It can be understood that by establishing the motion continuity constraint, the method can maintain tracking by predicting the trajectory when the target is temporarily occluded, avoiding the problem of trajectory interruption in traditional methods. This motion continuity constraint significantly improves the stability and accuracy of tracking, especially in complex environments, and can effectively deal with occlusion and motion changes.

[0056] In a possible implementation, S3 specifically includes:

[0057] S301: determining the association relationship between each phase information structure descriptor and the phase flow field based on the phase energy conservation principle.

[0058] The association relationship is specifically:

[0059]

[0060] wherein, represents the spatial gradient of the phase information structure descriptor of the video frame at time t, represents the phase flow field, represents the partial derivative.

[0061] It should be noted that the association relationship based on the phase energy conservation principle can accurately describe the motion characteristics of the target by ensuring the conservation of phase information in space and time. The combination of the spatial gradient of the phase information and the phase flow field ensures the consistency of the phase in motion, which makes the motion estimation of the target more stable and accurate, especially in dealing with complex motion and occlusion, which can effectively reduce errors and improve the robustness and accuracy of target tracking.

[0062] S302: optimizing the phase flow field by minimizing an energy function with data items and regularization terms, wherein the data items are used to constrain the phase of the video frame pixels to satisfy the phase conservation in motion, and the regularization terms are used to constrain the adjacent pixels of the video frame to be in a smooth state in space.

[0063] The energy function is specifically:

[0064] ​

[0065] wherein, denotes the phase flow field to be optimized, denotes the energy function value of denotes the spatial gradient of denotes the data item, denotes the adjustment regularization term the weight coefficient of the importance degree.

[0066] It should be noted that by minimizing the energy function containing the data item and the regularization term, the phase flow field can be effectively optimized to ensure the continuity and smoothness of the phase in time and space. The data item ensures the conservation of the phase in motion, making the motion prediction in the target tracking process more accurate. The regularization term avoids errors caused by local noise or drastic changes by smoothing the phase flow field. The overall optimization process balances the data item and the regularization term, enabling stable tracking of the target in complex environments while maintaining high efficiency and accuracy of the calculation.

[0067] Specifically, the energy function can be solved by variational method or iterative algorithms such as Gauss-Newton method or conjugate gradient method to obtain a unique phase flow field.

[0068] S303: Output the optimized phase flow field.

[0069] Specifically, the entire process accurately estimates and optimizes the phase flow field through the principle of phase energy conservation and energy optimization method, thereby ensuring continuous tracking and stability of the target. More specifically, first, the principle of phase energy conservation is used to establish the correlation between the phase information structure descriptor and the phase flow field, ensuring the continuous stability of the phase information through the spatial gradient and the time variation term. Then, by minimizing the energy function containing the data item and the regularization term, the phase flow field is optimized. The data item ensures the conservation of the phase in motion, while the regularization term constrains the smoothness of the phase flow field, avoiding inaccurate tracking results caused by drastic changes. Through variational method or iterative algorithms (such as Gauss-Newton method or conjugate gradient method), the best phase flow field can be accurately solved. In step S303, the optimized phase flow field is output as the final result for subsequent target tracking. It can handle complex motion patterns and occlusion situations, accurately predict the target, reduce errors and computational burden in traditional methods, and improve tracking stability.

[0070] S4: Determine whether the target to be tracked is occluded based on the occlusion factor based on the phase information structure descriptor. If yes, go to step S5, otherwise go to step S8.

[0071] The occlusion factor is a measure based on the phase information descriptor, which is used to determine whether the target to be tracked is occluded. Specifically, the occlusion factor evaluates whether the target is partially or completely occluded by analyzing the change of the phase information of the target in the current frame. If the phase information of the target changes abnormally or the features of the target are lost, it is determined that the target may be occluded, thereby triggering the subsequent processing flow, such as entering step S5 for occlusion processing. The introduction of the occlusion factor can effectively detect whether the target is occluded, avoiding tracking errors caused by occlusion. The occlusion factor can accurately determine whether the target is occluded, which provides key information for subsequent target tracking. Through the occlusion judgment based on the phase information, compared with the traditional occlusion detection method based on appearance or edge features, the method has stronger robustness to light changes, background interference and target deformation, can more accurately identify whether the target is occluded, thereby reducing the tracking loss or error caused by occlusion, and enhancing the stability and reliability of target tracking.

[0072] In a possible implementation, S4 specifically includes:

[0073] S401: Under each order combination, the phase difference between adjacent video frames between different time instants is calculated.

[0074] The calculation formula of the phase difference between adjacent video frames is specifically:

[0075]

[0076] wherein, denotes the phase component of the video frame at time t-1 under the order combination of n and m, denotes the phase component of the video frame at time t under the order combination of n and m, denotes the phase difference between adjacent video frames.

[0077] S402: Each phase difference between adjacent video frames is converted into a unit vector.

[0078] The unit vector obtained after conversion is:

[0079]

[0080] wherein, denotes a natural constant, and i denotes an imaginary unit.

[0081] S403: The obtained unit vectors under different order combinations are summed to obtain a total vector describing the consistency of the structural changes of the target to be tracked.

[0082] S404: The total vector is normalized by dividing the modulus of the total vector by the number of order combination groups.

[0083] Wherein, the total vector module length reflects whether the change of the to-be-tracked target is consistent. If consistent, the total length is large, and if the direction is chaotic, the total length is small.

[0084] S405: Calculate the difference between 1 and the normalized total vector to obtain an occlusion factor.

[0085] The calculation method of the occlusion factor is specifically:

[0086]

[0087] Wherein, represents the occlusion factor corresponding to the video frame at t time.

[0088] It should be noted that the occlusion factor is obtained by calculating the difference between the module length of the normalized total vector and 1, and the phase consistency of the target between the continuous frames is judged by comparing the module length of the normalized total vector. If the target is not occluded, the phase is consistent, the total vector module length is close to 1, and the occlusion factor is close to 0; if the target is occluded or the phase change is inconsistent, the total vector module length decreases, and the occlusion factor increases. Strong anti-interference ability, and then based on the phase to accurately distinguish the occlusion and motion change of the target, especially in complex scenes and more background noise, can effectively reduce the misjudgment and improve the stability.

[0089] S406: In the case that the occlusion factor is greater than a preset occlusion factor, it is determined that the to-be-tracked target is occluded, otherwise, it is determined that the to-be-tracked target is not occluded.

[0090] It should be noted that the size of the preset occlusion factor can be set by the person skilled in the art according to actual needs, which is not limited in the present application.

[0091] Optionally, the occlusion factor can be set to 0.8 or 0.9.

[0092] Specifically, the whole process calculates the phase difference between adjacent video frames, uses a unit vector to describe the structural change consistency of the target, and then judges whether the target is occluded. In step S401, the change of adjacent video frames is obtained by calculating the phase difference. In step S402, the phase difference is converted into a unit vector to ensure the numerical stability of the calculation. Step S403 sums the unit vectors to obtain a total vector describing the structural change consistency of the target, and further normalizes the total vector in step S404 to measure the consistency of the target change. In step S405, the difference between the total vector module length and 1 is calculated to obtain an occlusion factor, and whether the target is occluded is judged by step S406. The advantage of this method is that by accurately calculating the occlusion factor, whether the target is occluded can be dynamically evaluated, avoiding the misjudgment in the traditional method.

[0093] The method accurately captures the consistency of the target under different order combinations by calculating the phase difference between video frames and converting it into a unit vector, thereby effectively evaluating whether the target is occluded. Through normalization processing and the calculation of the occlusion factor, the change of the target can be accurately distinguished from occlusion, avoiding tracking errors caused by background interference or partial occlusion of the target. Compared with traditional methods, this phase difference-based occlusion detection is more robust, can continuously track the target in complex scenes, and improves the accuracy and stability of tracking.

[0094] S5: In combination with the phase flow field, the multiple predicted positions of the target to be tracked are determined by the Lie group motion principle.

[0095] The Lie group motion principle is a mathematical method based on Lie group and Lie algebra theory, used to describe and process rigid body motion. Lie group is a group that describes rigid body motion (such as translation, rotation, etc. transformation), which combines geometric transformation and group theory, and can effectively model and predict motion. In target tracking, the Lie group motion principle can be used to predict the future positions of the target based on its current motion state (position and velocity, etc.), and thus predict the motion trajectory of the target. By combining the phase flow field, the Lie group motion principle can more accurately predict multiple positions of the target, enhancing the accuracy of motion estimation.

[0096] It should be noted that the Lie group motion principle is used to predict multiple positions of the target, thereby providing more accurate motion estimation for continuous tracking of the target. This method can accurately capture the motion law of the target, and is efficient in both translation and rotation. In the case of occlusion or rapid motion, by predicting multiple possible positions of the target, the situation of target loss or false tracking is avoided, enhancing the stability and accuracy of target tracking. The application of this Lie group-based motion prediction in complex environments significantly improves the robustness and reliability of tracking.

[0097] In one possible implementation, S5 specifically includes:

[0098] S501: Obtain the real-time position of the target to be tracked from the video frame at time t-1.

[0099] S502: According to the video frame at time t-1, determine the Lie algebra generator describing the motion law of the target to be tracked using the Lie algebra prior template, wherein the Lie algebra generator includes a translation generator, a rotation generator, and a scaling generator.

[0100] The Lie algebra prior template is a basic template or model used to represent target motion, containing the mathematical form of basic motions such as translation, rotation, and scaling. The Lie algebra generator is a set of basic operators that describe Lie group elements (such as translation, rotation, scaling, etc. transformation).

[0101] Optionally, the prior template of the translation generation element specifically includes an x-axis translation prior template and a y-axis translation prior template, wherein the formula form of the x-axis translation prior template is specifically:

[0102] .

[0103] The formula form of the y-axis translation prior template is specifically:

[0104] .

[0105] The formula form of the prior template of the rotation generation element is specifically:

[0106] .

[0107] The prior template of the scaling generation element specifically includes an isotropic scaling prior template and an anisotropic prior template, and the formula form of the isotropic scaling prior template is specifically:

[0108] .

[0109] The formula form of the anisotropic prior template is specifically:

[0110] .

[0111] In the formula form of each prior template, respectively represent the generation elements of the corresponding basic motion.

[0112] S503: Obtain an intensity coefficient describing the motion law of the target to be tracked according to the phase flow field fitting, wherein the intensity coefficient includes a translation velocity, a rotation angular velocity and a scaling velocity.

[0113] Specifically, the fitting process specifically includes decomposing the phase flow field to each basic motion to obtain the contribution on each basic motion (i.e. how many meters per second in the x direction, how many meters per second in the y direction, how many degrees per second of rotation, how much proportion of reduction per second and how much proportion of enlargement per second), and then matching or mapping the decomposition result to the corresponding prior template, that is, obtaining the corresponding intensity coefficient.

[0114] S504: Combine the intensity coefficient and the Lie algebra generation element into a predicted position through exponential mapping.

[0115] The calculation formula of the predicted position is specifically:

[0116]

[0117] Wherein, represents the real-time position of the target to be tracked at t-1 time, and exp represents an exponential function.​ a predicted position of the target at time t, a generating element of the k-th basis motion, an intensity coefficient of the k-th basis motion at time t, a number of basis motion types, wherein the basis motions include translation, rotation, and scaling.

[0118] It should be noted that by combining the Lie algebra generating element and the intensity coefficient through exponential mapping, the translation, rotation, and scaling motions of the target can be accurately unified into one prediction model, providing accurate target position prediction. This method can consider multiple motion types at the same time, adapt to the diverse changes of the target in complex dynamic environments, and more naturally handle nonlinear motion, improving the accuracy and robustness of target tracking, especially when the target has multiple motion methods (such as rotation or scaling), ensuring more stable tracking performance.

[0119] Specifically, the entire process is based on Lie algebra generating elements and phase flow fields to accurately predict the motion trajectory of the target. In S501, the position of the target to be tracked in the video frame at the last time is first obtained. Then, S502 uses the Lie algebra prior template to determine the motion law of the target, generating generating elements such as translation, rotation, and scaling to describe different types of motion. In S503, the intensity coefficients describing the motion of the target are calculated by fitting the phase flow field, which specifically includes translation velocity, rotation angular velocity, and scaling velocity. In S504, the intensity coefficients are combined with the corresponding Lie algebra generating elements through exponential mapping to obtain the predicted position of the target. The advantage of this method is that the Lie algebra generating element can effectively capture the translation, rotation, and scaling changes of the target, and accurate motion prediction is performed in combination with phase flow field fitting. Compared with traditional methods, this method can more accurately cope with complex changes in target motion and improve the accuracy and robustness of target tracking in complex environments.

[0120] S6: Collect multiple predicted video frames with each predicted position as the focus center of the UAV camera.

[0121] It should be noted that by collecting multiple predicted video frames with each predicted position as the focus center of the UAV camera, it can ensure that the UAV always accurately observes and captures the target. By focusing on the predicted position of the target, the UAV can quickly adjust the camera focus, reducing the risk of the target deviating from the lens and improving the probability of capturing the target. In addition, this method can effectively cope with fast-moving or short-occluded situations, ensuring stable tracking of the target while reducing image loss due to large target displacement, improving the real-time and reliability of tracking.

[0122] S7: Capture the target to be tracked from each predicted video frame by minimizing the phase gradient cross product.

[0123] Wherein, the minimization of phase gradient cross product is an optimization method for accurate capture of the target. The phase gradient cross product is a measure of the change of the target phase information, which represents the direction and size of the change of the phase information. When the phase information of the target changes in the continuous predicted video frames, the phase gradient is calculated and the cross product operation is performed to obtain a quantized change value. The goal of minimizing the phase gradient cross product is to adjust the parameters of the UAV camera so that the phase gradient change of the target in the predicted video frame is minimized, thereby accurately capturing the target position. This process ensures the minimum error capture of the target features, thereby improving the accuracy of tracking.

[0124] In one possible implementation, S7 specifically comprises:

[0125] S701: Calculate the predicted phase information structure descriptor in the preset neighborhood of each predicted position.

[0126] It should be noted that the calculation method of the phase information structure descriptor in step S701 is consistent with step S2, and the entire preset neighborhood in S701 is similar to the target tracking information.

[0127] S702: Calculate the phase gradient cross product between each predicted phase information structure descriptor and the phase information structure descriptor.

[0128] The calculation method of the phase gradient cross product is specifically:

[0129]

[0130] Wherein, represents the L1 norm, and respectively represent the spatial gradients of the predicted phase information structure descriptor and the phase information structure descriptor of the target to be tracked at time t-1, i.e. in the unoccluded state .

[0131] Wherein, the spatial gradient is a quantity that describes the rate of change of an image or function in space. It represents the direction and rate of change of the gray value at a certain point in the image, which is calculated by differentiating the image. In the image, the size of the spatial gradient reflects the degree of change of the gray value in the image, and the direction indicates the direction of the greatest change.

[0132] S703: Select the predicted video frame corresponding to the minimum phase gradient cross product.

[0133] S704: Capture the predicted position corresponding to the selected predicted video frame as the capture position of the target to be tracked.

[0134] Specifically, first, the phase information structure descriptor in the prediction position neighborhood is calculated. Then, the difference between the prediction phase information descriptor and the target phase descriptor is evaluated by calculating the phase gradient cross product, and the prediction video frame corresponding to the minimum phase gradient cross product is selected. Then, the optimal prediction position is captured as the capture position of the target. Through the calculation of the phase gradient cross product, the matching degree of the prediction position can be accurately evaluated, and the prediction frame that best matches the current state of the target is selected, thereby improving the accuracy and stability of target tracking. Especially when the target moves quickly or is occluded, the tracking error can be effectively reduced, and stable target capture is ensured.

[0135] S8: capturing the target to be tracked according to the phase information structure descriptor.

[0136] In a possible implementation, S8 specifically includes:

[0137] S801: obtaining a video frame at time t+1, and dividing the video frame at time t+1 into a plurality of sub-regions according to a preset number of regions.

[0138] It should be noted that a person skilled in the art can set the size of the preset number of regions according to actual needs, which is not limited in the present application.

[0139] S802: calculating a sub-region phase information structure descriptor of each sub-region.

[0140] S803: calculating a sub-region phase gradient cross product between the sub-region phase information structure descriptor and a phase information structure descriptor corresponding to the video frame at time t.

[0141] S804: selecting a sub-region corresponding to the minimum sub-region phase gradient cross product as a capture region.

[0142] S805: taking the midpoint of the capture region as the capture position of the target to be tracked.

[0143] S806: capturing the target to be tracked according to the capture position.

[0144] Specifically, the entire process improves the accuracy and stability of target tracking by refining the target capture area. First, the video frame at time t+1 is obtained and segmented into multiple sub-regions to analyze the details of the target more accurately. Next, the phase information structure descriptor of each sub-region is calculated to capture the features within the region. Then, the phase gradient cross product of each sub-region with the frame at time t is calculated to evaluate the matching degree, and the sub-region corresponding to the smallest phase gradient cross product is selected as the capture area. The midpoint of the capture area is selected as the capture position of the target, and the target is captured. The advantage of this method is that by subdividing the video frame and accurately calculating the phase information of each sub-region, the target position can be captured more accurately, especially when the target pose or background is complex, reducing errors and ensuring stable and efficient tracking.

[0145] After S8, it further includes:

[0146] In the case where the capture result is not obtained, a hovering traversal scan is performed to obtain multiple environmental video frames.

[0147] The phase information structure descriptor is used to capture the target to be tracked from each environmental video frame.

[0148] Specifically, when the target cannot be successfully captured, the system obtains multiple environmental video frames by performing a hovering traversal scan. These video frames provide more data sources for further target capture. Then, the phase information structure descriptor is used to extract target features from each environmental video frame to improve the accuracy and robustness of capturing the target. The advantage of this method is that by hovering and scanning the environment, the system can provide more perspectives and data when the target is not captured, enhancing the stability of target tracking in dynamic or complex environments, avoiding errors caused by local occlusion or environmental changes, and ensuring the accuracy of continuous tracking.

[0149] S9: Track the target to be tracked according to the capture result.

[0150] In practical applications, first, the system constructs the "structure fingerprint" of the target by continuously acquiring video frames and extracting the phase information of the target, which enables the target to maintain stable tracking in complex environments (such as occlusion and illumination changes). By combining the phase flow field and the Lie group motion principle, the system can accurately predict the motion trajectory of the target, further reducing tracking errors caused by occlusion or rapid target motion. By minimizing the phase gradient cross product, the matching degree of the target and the predicted frame is accurately evaluated to ensure the accuracy of the capture. By subdividing the video frame and calculating the phase descriptor of the sub-region, the accuracy and robustness of target capture are further improved. When the target is not successfully captured, the system performs hovering scanning to collect more environmental video frames, ensuring efficient and accurate target capture in dynamic environments. This method can maintain high-precision tracking in complex backgrounds, occlusions, and irregular motion environments, improving the stability and robustness of target tracking and ensuring the continuous and accurate operation of the UAV in changing environments.

[0151] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0152] In the embodiments of the present application, the phase information structure descriptor is extracted from the video frame by the Zernike matrix. This method can effectively capture the phase characteristics of the target, thereby reducing the influence of background interference and improving the recognition accuracy of the target in complex backgrounds or when the target is similar to the background. This method avoids target loss caused by complex backgrounds or target blurring. Secondly, by constructing a phase flow field, the motion characteristics of the target can be described, and the prediction is combined with the Lie group motion principle to optimize the prediction process of the target position, thereby effectively reducing the calculation amount based on image pixel scanning, reducing the consumption of computing resources, and prolonging the endurance time of the UAV. Especially when occlusion occurs, the method can predict multiple positions of the target by combining the phase flow field and the occlusion factor, and accurately track the target, solving the problem of unstable tracking of traditional methods when the target is occluded. In addition, the minimization method based on the phase gradient cross product further improves the accuracy and response speed of target capture, enhances the real-time performance, ensures continuous tracking of the target, and further improves the accurate long-time tracking ability of the UAV in complex environments.

[0153] The UAV target tracking method provided by the embodiments of the present application can be executed by a UAV target tracking device. In the embodiments of the present application, the UAV target tracking device is taken as an example to execute the UAV target tracking method, and the UAV target tracking device provided by the embodiments of the present application is described.

[0154] Reference is made to the accompanying drawings Figure 2 , which shows a structural schematic diagram of a UAV target tracking system provided by the embodiments of the present application.

[0155] The embodiment of the present application provides a UAV target tracking system 20, comprising a processor 201 and a memory 202;

[0156] The memory 202 stores programs or instructions which can run on the processor 201, the programs or instructions are executed by the processor 201 to realize the steps of the UAV target tracking method and achieve the same technical effects, for the sake of avoiding repetition, the present application will not be described again.

[0157] It should be understood that the processor 201 in the embodiment of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0158] It is also to be understood that the memory 202 in embodiments of the present application can be volatile or nonvolatile memory, or can include both volatile and nonvolatile memory. Nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as external cache. By way of example, and not limitation, many forms of random access memory can be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double-data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DR RAM).

[0159] The above-described embodiments can be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any combination of the three. When implemented in software, the above-described embodiments can be implemented in the form of one or more computer programs that are stored in a computer-readable storage medium. The computer-readable storage medium stores one or more computer instructions or computer programs that, when loaded into a computer, cause the computer to perform the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, such as from a website, a computer, a server, or a data center to another website, computer, server, or data center, via a wired (e.g., infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more collections of available media. The available media can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0160] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0161] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0163] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation. For example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0164] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0165] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0166] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts of the technical solutions that make contributions to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0167] The embodiments of the present application provide a readable storage medium, which includes: a program or instructions stored on the readable storage medium, the program or instructions are executed by a processor to implement the steps of the unmanned aerial vehicle target tracking method described above, and the same technical effects can be achieved. To avoid repetition, the present application will not be described again.

[0168] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application.

Claims

1. A method for tracking a target of an unmanned aerial vehicle, characterized in that: include: S1: Acquire multiple video frames with continuous acquisition time and labeled information of the target to be tracked; S2: extracting a phase information structure descriptor of the target to be tracked from each of the video frames using the Zernike moment according to the labeling information of the target to be tracked; S3: constructing a phase flow field describing the motion characteristics of the target to be tracked according to the phase information structure descriptor; S4: Determine whether the target to be tracked is occluded by combining the occlusion factor based on the phase information structure descriptor. If so, proceed to step S5; otherwise, proceed to step S8. S5: Determine multiple predicted positions of the target to be tracked by combining the phase flow field and using the Lie group motion principle; S6: Capturing multiple predicted video frames with each of the predicted positions as the focus center of the drone camera; S7: capturing the target to be tracked from each of the predicted video frames with the goal of minimizing the phase gradient cross product; S8: Capturing the target to be tracked according to the phase information structure descriptor; S9: Tracking the target to be tracked according to the capture result.

2. The UAV target tracking method according to claim 1, characterized in that: The S2 specifically includes: S201: Locating an information area including the target to be tracked in the video frame according to the labeling information of the target to be tracked; S202: Calculating weighted integrals of grayscale images and Zernike polynomials under different order combinations within the information region to obtain Zernike moments describing the target to be tracked, wherein the order combination is a combination of radial and angular orders of the Zernike polynomials; S203: extracting phase components of the Zernike moments under different order combinations respectively; S204: Combining phase components corresponding to the respective video frames to obtain a phase information structure descriptor of each video frame.

3. The UAV target tracking method according to claim 1, characterized in that: The S3 specifically includes: S301: Determine the association relationship between each phase information structure descriptor and the phase flow field based on the phase energy conservation principle; S302: Optimizing the phase flow field with the goal of minimizing an energy function having a data term and a regularization term, wherein the data term is used to constrain the phase of pixels in the video frame to satisfy phase conservation during motion, and the regularization term is used to constrain adjacent pixels in the video frame to be in a smooth state in space; S303: Output the optimized phase flow field.

4. The method for tracking a target by an unmanned aerial vehicle according to claim 2, wherein: The S4 specifically includes: S401: Under each order combination, calculate the phase difference between adjacent video frames at different times; S402: Converting the phase difference of each adjacent video frame into a unit vector; S403: summing the obtained unit vectors in different order combinations to obtain a total vector describing the consistency of the target structure change to be tracked; S404: dividing the total vector modulus by the number of order combination groups to normalize the total vector; S405: Calculate the difference between 1 and the normalized total vector to obtain the occlusion factor; S406: If the occlusion factor is greater than a preset occlusion factor, determine that the target to be tracked is occluded; otherwise, determine that the target to be tracked is not occluded.

5. The UAV target tracking method according to claim 1, characterized in that: The S5 specifically includes: S501: Obtaining the real-time position of the target to be tracked from the video frame at time t-1; S502: Determine, based on the video frame at time t-1, a Lie algebra generator describing the motion law of the target to be tracked using a Lie algebra prior template, wherein the Lie algebra generator includes a translation generator, a rotation generator, and a scaling generator; S503: Obtaining an intensity coefficient describing the motion law of the target to be tracked according to the phase flow field fitting, wherein the intensity coefficient includes translation velocity, rotation angular velocity, and scaling velocity; S504: Combining the intensity coefficient and the Lie algebra generator into the predicted position through exponential mapping.

6. The method for tracking a target by an unmanned aerial vehicle according to claim 1, wherein: The S7 specifically includes: S701: Calculating a predicted phase information structure descriptor within a preset neighborhood of each predicted position; S702: Calculating the phase gradient cross product between each of the predicted phase information structure descriptors and the phase information structure descriptor; S703: Selecting a predicted video frame corresponding to the minimum phase gradient cross product; S704: Capturing the predicted position corresponding to the selected predicted video frame as the capture position of the target to be tracked.

7. The UAV target tracking method according to claim 1, characterized in that: The S8 specifically includes: S801: Obtain a video frame at time t+1, and evenly divide the video frame at time t+1 according to a preset number of regions to obtain multiple sub-regions; S802: Calculate the sub-region phase information structure descriptor of each sub-region; S803: Calculating the sub-region phase gradient cross product between the sub-region phase information structure descriptor and the phase information structure descriptor corresponding to the video frame at time t; S804: Selecting the subregion corresponding to the minimum subregion phase gradient cross product as the capture region; S805: Taking the midpoint of the capture area as the capture position of the target to be tracked; S806: Capture the target to be tracked according to the capture position.

8. The method for tracking a target by an unmanned aerial vehicle according to claim 1, wherein: After S8, the method further includes: If no capture results are obtained, perform hovering traversal scanning to obtain multiple environment video frames; The target to be tracked is captured from each of the environment video frames using the phase information structure descriptor.

9. A UAV target tracking system, characterized in that: include: processor and memory; The memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the drone target tracking method according to any one of claims 1 to 8 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the drone target tracking method according to any one of claims 1 to 8 are implemented.

Images