A target tracking method and device based on drone pod

Abstract

The present invention discloses a target tracking method based on an unmanned aerial vehicle (UAV) pod, comprising the steps of: processing image data captured in real time by a shooting component to determine whether a moving target is currently within the shooting field of view of the shooting component; if so, obtaining motion information of the moving target from the image data, and predicting a first motion trajectory of the moving target within a first preset time period based on the current motion trajectory and the current motion state, and adjusting the shooting posture of the shooting component according to the first motion trajectory to ensure that the moving target is within the shooting field of view of the shooting component; otherwise, controlling the UAV to continue flying along the current path of the moving target for a second preset time, and after the second preset time has passed, determining again that the moving target is not currently within the shooting field of view of the shooting component, and controlling the onboard computer to adjust the flight heading and the shooting posture of the shooting component according to a pre-stored highest-priority alternative path to ensure that the moving target is within the shooting field of view of the shooting component.

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a target tracking method and device based on an unmanned aerial vehicle pod. Background Art

[0002] Aircraft, such as unmanned aerial vehicles (UAVs), can carry payloads to perform special functions, such as capturing images of the surrounding environment by carrying cameras. Therefore, they are often used to perform surveillance, reconnaissance, military and civilian exploration missions.

[0003] With the continuous development of drones in recent years, their application scenarios have become increasingly broad, playing a significant role in scenarios such as power line inspection, traffic rescue, and disaster relief. For example, in disaster relief, drones can not only obtain important ground or low-altitude information such as images, terrain, and moving objects, but also extract precise positioning information of the target to be rescued, facilitating tasks such as tracking the target. In some cases, drones are even required to continuously track the target, especially the direction of movement of the moving target.

[0004] Traditional tracking of moving targets is achieved through control commands issued by a user-operated remote control terminal or device, requiring significant manual intervention. On the one hand, manual tracking can be difficult in certain situations. For example, because both the moving target and the drone are in motion, especially when the moving target is moving quickly, it can often become difficult to track continuously due to its movement beyond the camera's field of view; or when the moving target is obscured. On the other hand, manual tracking requires the sustained attention of a dedicated person to control the drone's onboard camera module. However, the camera module is physically separated from the user controlling the drone, increasing the cost of drone aerial photography and other applications. Furthermore, rescue operations are often time-sensitive and often involve difficult routes in mountainous areas or other harsh geographical environments. Therefore, requiring the moving target to carry accessories such as the drone's remote control undoubtedly increases the workload of the moving target.

[0005] In view of this, there is an urgent need for a method that can automatically and continuously track moving targets. Summary of the Invention

[0006] In order to partially or to a certain extent overcome the above problems, the present invention provides a target tracking method and device based on a drone pod, which can continuously track moving targets in complex environments such as post-disaster rescue.

[0007] The present invention provides a target tracking method, which comprises the steps of:

[0008] S0, the onboard computer processes the image data captured in real time by the shooting component to determine whether the moving target to be tracked is currently within the shooting field of view of the shooting component. If so, execute step S1; otherwise, execute step S2;

[0009] S1, an onboard computer obtains motion information of the moving target from the image data, and predicts a first motion trajectory of the moving target within a first preset time period based on the current motion trajectory and current motion state in the motion information, and executes step S3;

[0010] S2, the onboard computer controls the drone to continue flying along the current path of the moving target for a second preset time, and when the second preset time has passed, it is again determined that the moving target is no longer within the shooting field of view of the shooting component, and step S4 is executed;

[0011] S3, the onboard computer adjusts the shooting posture of the shooting component according to the first motion trajectory so that the moving target is within the shooting field of view of the shooting component;

[0012] S4, the onboard computer controls the UAV to adjust the flight direction and the shooting posture of the shooting component according to the pre-stored highest priority alternative path, so that the moving target is within the shooting field of view of the shooting component.

[0013] In some embodiments, step S1 specifically includes the following steps:

[0014] The onboard computer constructs a two-dimensional coordinate system with the center of the image data as the origin to obtain a first relative position of the moving target in the image data;

[0015] The onboard computer calculates a second relative position and relative speed of the moving target relative to the drone based on the first relative position, the current actual position of the drone, and the current flight speed;

[0016] The current coordinate position, current speed and current motion trajectory of the moving target are calculated according to the second relative position and the relative speed.

[0017] In some embodiments, step S1 specifically includes the following steps:

[0018] The onboard computer constructs a two-dimensional coordinate system with the center of the image data as the origin to obtain a first relative position of the moving target in the image data;

[0019] The onboard computer calculates the current coordinate position of the moving target according to the first relative position;

[0020] The onboard computer calculates the current speed and current motion trajectory of the moving target based on the current coordinate position and the historical coordinate position of the moving target.

[0021] In some embodiments, the step of calculating the current coordinate position of the moving target specifically includes the steps of:

[0022] Obtaining a current shooting angle of the shooting component, and obtaining an azimuth angle of the drone based on the shooting angle and the first relative position;

[0023] Obtaining a pixel size of the moving target in the image data, and calculating a relative distance between the moving target and the UAV based on the pixel size;

[0024] The current coordinate position of the moving target is calculated based on the azimuth angle and the relative distance.

[0025] In some embodiments, the step of calculating the current speed of the moving target specifically includes the step of calculating the current speed of the moving target based on the flight speed and the relative speed.

[0026] In some embodiments, the step of calculating the current motion trajectory of the moving target specifically includes the steps of: calculating the relative motion trajectory of the moving target based on the first relative position of the moving target in each image data captured within a second preset time period.

[0027] In some embodiments, the step of predicting a first motion trajectory within a first preset time period based on the current motion trajectory and the current motion state specifically includes the steps of: dividing the speed of the moving target in the image data into two speed components, horizontal and vertical, based on the current motion trajectory and the current motion state of the moving target; and determining the motion trajectory based on the direction of maximum gradient change of the speed component.

[0028] In some embodiments, the method further includes the step of: an onboard computer correcting the shooting posture based on image data captured in real time by the shooting component.

[0029] In some embodiments, the method further includes the steps of: an onboard computer predicting a second motion trajectory of the moving target within a second preset time period based on the motion information and the latest shooting posture of the shooting component; and the onboard computer adjusting the flight route according to the second motion trajectory to ensure continuous tracking.

[0030] In some embodiments, the method further includes the step of: an onboard computer correcting the flight route of the drone in real time based on the drone's flight data, the image data captured by the shooting component in real time, and the latest shooting posture of the shooting component.

[0031] In some embodiments, the step of correcting the shooting posture of the shooting component specifically includes the following steps: the onboard computer calculates the current actual position of the moving target based on the image data captured in real time by the shooting component; the onboard computer calculates the deviation between the actual position and the predicted position in the first motion trajectory; the onboard computer readjusts the shooting posture of the shooting component according to the deviation.

[0032] In some embodiments, the step of adjusting the flight route of the UAV specifically includes the following steps: an onboard computer obtains the latest shooting posture of the shooting component; the onboard computer determines whether the attitude angle in the shooting posture is the maximum attitude angle; if the attitude angle is the maximum attitude angle, the onboard computer calculates the current actual position of the moving target based on the image data captured in real time by the shooting component; the onboard computer calculates the deviation between the actual position and the predicted position in the second motion trajectory; the onboard computer readjusts the flight route of the UAV according to the deviation.

[0033] The second aspect of the present invention is to provide a target tracking device, characterized in that it includes: a data acquisition module for acquiring image data captured by a shooting component in real time; a first data processing module for processing the image data acquired by the data acquisition module to determine whether the moving target to be tracked is currently within the shooting field of view of the shooting component; a second data processing module for acquiring motion information of the moving target from the image data when the first data processing module determines that the moving target is currently within the shooting field of view of the shooting component, and predicting a first motion trajectory of the moving target within a first preset time period based on the current motion trajectory and current motion state in the motion information; a third data processing module for predicting a first motion trajectory of the moving target within a first preset time period when the first data processing module determines that the moving target is currently within the shooting field of view of the shooting component When the moving target is not currently within the shooting field of view of the shooting component, the UAV is controlled to continue flying along the current path of the moving target for a second preset time; the fourth data processing module is used to adjust the shooting posture of the shooting component according to the first motion trajectory predicted by the second data processing module, so that the moving target is within the shooting field of view of the shooting component; the fifth data processing module is used to determine whether the moving target appears again within the shooting field of view of the shooting component after the second preset time has passed, and when it is determined that the moving target is not within the shooting field of view of the shooting component, the UAV is controlled to adjust the flight heading and the shooting posture of the shooting component according to the pre-stored highest priority alternative path, so that the moving target is within the shooting field of view of the shooting component.

[0034] Beneficial effects: For post-disaster rescue in scenarios with complex geographical environments, tracking moving targets through drones can record the current progress of the moving targets and the difficulties encountered in real time, and can also provide real-time feedback, thereby providing a strong reference for subsequent analysis or decision-making by the command center.

[0035] By pre-storing the moving target's path and alternative action path (i.e., alternative rescue path, or alternative tracking path) in the drone, the drone can continuously track according to the pre-stored path or alternative action path. Compared with the traditional method of remotely controlling the drone manually, there is no need for the operator to continuously focus on controlling the drone for continuous tracking; and compared with the method of controlling the drone tracking by the moving target, there is no need to configure special technicians to follow the moving target, so that the special technicians can perform other tasks. From a planning perspective, reasonable allocation can be achieved. In addition, there is no need for the moving target to carry accessories for controlling the drone, which reduces the workload of the moving target to a certain extent.

[0036] On the other hand, compared with using a neural network algorithm on an onboard computer carried by a drone to predict the motion trajectory of a moving target, or using a complex neural network algorithm to determine the path of a moving target, although the intelligence of the drone is improved, this requires the drone to be equipped with a high-configuration, large-volume onboard computer, which will not only increase the cost, weight and volume of the drone, but also affect the flight altitude and flight performance of the drone, making such drones unsuitable for continuous tracking of moving targets in complex post-disaster geographical environments such as mountainous areas and valleys. Because, in such a complex environment, it not only requires the drone to be able to continuously track, but also requires the drone to have a smaller volume and lighter weight, so that it can track moving targets in geographical environments such as valleys and forests; based on this, the present invention adopts a completely different inventive concept, that is, it does not use complex algorithms such as neural networks for prediction, but provides an algorithm that can be run even on a relatively low-configuration onboard computer for prediction.

[0037] Of course, the target tracking method of the present invention can also be applied to the continuous tracking of moving targets in other complex geographical environments, such as border patrols and the search and tracking of large-area high-altitude targets in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the embodiments or the description of the prior art. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the various elements or parts are not necessarily drawn according to the actual scale. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without inventive work.

[0039] Figure 1 A flowchart of a target tracking method according to an exemplary embodiment of the present invention is shown;

[0040] Figure 2 A flowchart of a target tracking method according to a specific embodiment of the present invention is shown;

[0041] Figure 3 Schematic diagrams identifying alternative pathways for reactions;

[0042] Figure 4 Schematic diagram of the label box and prediction box in the process of evaluating the target tracking algorithm;

[0043] Figure 5 Schematic diagram showing a comparison of the success rates of a target tracking algorithm according to an exemplary embodiment of the present invention and target tracking algorithms of nine control groups;

[0044] Figure 6 Schematic diagram showing a comparison of the accuracy of a target tracking algorithm according to an exemplary embodiment of the present invention and nine control groups of target tracking algorithms;

[0045] Figure 7 This is a table showing the success rate and accuracy test data of a target tracking algorithm of an exemplary embodiment of the present invention and nine groups of control group target tracking algorithms;

[0046] Figure 8 This is a functional module diagram of a landing guidance device in an exemplary target tracking device of the present invention;

[0047] Figure 9 The figure is a flow chart of a guided landing method in an exemplary target tracking method of the present invention. DETAILED DESCRIPTION

[0048] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Herein, suffixes such as "module," "component," or "unit" used to represent elements are only used to facilitate description of the present invention and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0050] See also Figure 1 , is a flow chart of a target tracking method according to an exemplary embodiment of the present invention. Specifically, the target tracking method includes the following steps:

[0051] S100, the onboard computer processes the image data captured in real time by the shooting component to determine whether all tracked moving targets are currently within the shooting field of view of the shooting component. If so, execute step S101, otherwise execute step S102.

[0052] In some embodiments, the onboard computer processes the image data captured in real time by the camera assembly, including image recognition, and determines whether the moving target is currently within the camera assembly's field of view based on the image recognition results. Specifically, if a moving target appears in the image data, the moving target is considered to be within the camera assembly's field of view.

[0053] S101, the onboard computer obtains motion information of the moving target from the image data, and predicts a first motion trajectory of the moving target within a first preset time period based on the current motion trajectory and current motion state in the motion information, and executes step S103.

[0054] In some embodiments, the motion information includes: the current actual position, current speed, current motion trajectory, current motion state, etc. of the motion target. The current actual position of the motion target refers to the three-dimensional coordinates (X coordinates) of the motion target in the world three-dimensional coordinate system at the current time t1. t1 ,Y t1 ,Z t1 ).

[0055] In some embodiments, image data methods such as deep learning can be used to process the image data or video data captured by the shooting component to obtain the first relative position of the moving target in the image, that is, to establish a two-dimensional coordinate system with the center point of the captured image as the coordinate origin, and the two-dimensional coordinates (x t ,y t1 ), and then convert the two-dimensional coordinates of the moving object in the image into three-dimensional coordinates in the world coordinate system, that is, the actual position of the moving object is obtained. Specifically, the corresponding three-dimensional coordinates can be obtained using existing coordinate system conversion methods.

[0056] In some embodiments, the current speed (preferably, average speed) of the moving target can be calculated based on the current three-dimensional coordinates and historical three-dimensional coordinates of the moving target (i.e., the three-dimensional coordinates obtained by performing the above-mentioned image processing on multiple images captured over a period of time, i.e., the historical actual position), as well as the time it took for the moving target to travel from the historical actual position to the current actual position; and the current motion trajectory can be obtained based on each historical three-dimensional coordinate. For example, the three-dimensional coordinates corresponding to the two-dimensional coordinates of the moving target in multiple images continuously calculated over a period of time (i.e., the period before the current time t1) are obtained and connected to obtain the current motion trajectory of the moving target.

[0057] In other embodiments, image data methods such as deep learning are used to process the image data or video data captured by the shooting component to obtain the two-dimensional coordinates (x t ,y t1 ), that is, the first relative position, and then calculate the second relative position and relative speed of the moving target relative to the UAV based on the first relative position (that is, the absolute value of the difference between the current speed V1 of the rescue team and the flight speed V2 of the UAV), and then deduce the current coordinate position, current speed and current motion trajectory of the moving target.

[0058] Specifically, since the shooting component is fixed on the drone (for example, the pod is fixed on the drone), the positional relationship between the shooting component and the drone is known; and the two-dimensional coordinates of the moving target in the captured image corresponding to the current moment t1 (i.e., the first relative position) are known, and the second relative position of the moving target relative to the drone can be calculated based on this: 1) According to the coordinates of the moving target in the captured image and the shooting angle of the shooting component (for example, the angle of the pod), the azimuth angle θ of the moving target relative to the drone can be calculated; 2) According to the pixel size of the moving target in the captured image and the actual size of the moving target, the relative distance L between the moving target and the drone is calculated, and then the second relative position is calculated based on the azimuth angle θ and the relative distance L.

[0059] The current motion trajectory refers to the trajectory of the moving target up to the current time t1. Specifically, since the first relative position of the moving target in the image at time t1 can be calculated based on the single image corresponding to the current time t1, the first relative positions of the moving target in multiple images over a period of time (i.e., a period of time before the current time t1) can be continuously calculated, thereby obtaining the current motion trajectory of the moving target based on the coordinate positions of the moving target in these multiple images.

[0060] Of course, in other embodiments, the onboard computer may also predict the first motion trajectory of the moving target within the first preset time period T1 based on the motion information of the moving target. Specifically, the first preset time period T1 is usually 0.5-1s, preferably 1s.

[0061] In some embodiments, based on the current motion trajectory and current motion state (including speed and motion direction) of the moving target, the current speed of the moving target in the current image can be decomposed into two speed components, horizontal and vertical, so that the motion trajectory in the future period can be predicted based on the maximum gradient change direction of the speed component of the moving target, for example, the motion trajectory of the target in the next second can be estimated.

[0062] S102, the onboard computer controls the UAV to continue flying along the current path of the moving target for a second preset time, and after the second preset time has passed, it is determined again whether the moving target is within the shooting field of view of the shooting component. If so, execute step S101, otherwise, execute step S104.

[0063] In order to track moving targets such as rescue teams while taking into account the surrounding environment and road conditions, drones usually have a certain flight altitude. Therefore, when the moving target is moving along the predetermined path, it may enter a tunnel or dense jungle and be blocked by obstructions. At this time, it is inconvenient or impossible for the drone to follow closely, which will cause the rescue team to disappear from the shooting field of view for a short time, but after a short time, it will reappear in the shooting field of view of the shooting component. Therefore, when it is determined that the rescue team is no longer in the shooting field of view, it will continue to track according to the established path and the established flight speed. After a second preset time (for example, 3-5 minutes), when the rescue team appears in the shooting field of view again, it will continue to track along the path. Of course, if the rescue team has not appeared in the shooting field of view, it means that the rescue team may have encountered difficulties and has chosen an alternative rescue path.

[0064] S103: The onboard computer adjusts the shooting posture of the shooting component according to the first motion trajectory so that the moving target is within the shooting field of view of the shooting component.

[0065] S104 , the onboard computer controls the drone to adjust the flight direction and the shooting posture of the shooting component according to the pre-stored alternative path, so that the moving target is within the shooting field of view of the shooting component.

[0066] In some embodiments, a plurality of alternative action paths are pre-stored in the onboard computer. Specifically, in order to reduce the computational complexity and power consumption of the onboard computer, the priorities of the plurality of alternative action paths pre-stored in the onboard computer can be directly marked in advance. For example, when the rescue team encounters difficulties during the process and cannot continue along the current path, it is necessary to switch to the highest priority alternative action path (such as the alternative rescue path). Therefore, when the second preset time has passed and the onboard computer again determines that the moving target is still not within the shooting field of view of the shooting component, the highest priority alternative action path is directly used as the current tracking path, thereby adjusting the flight heading and the shooting posture of the shooting component.

[0067] The priority of each alternative route is obtained by comprehensively evaluating various estimated indicators such as the estimated time required for each route to the destination, the difficulty of transporting relief supplies (for example, whether it is accessible to traffic), and safety.

[0068] Of course, in other embodiments, due to too many emergencies during the rescue process, the moving target may not select a travel path according to the order of priority, but directly select the nearest alternative path. Accordingly, step S104 specifically includes the following steps:

[0069] Obtain the actual historical position of the moving target when the moving target last appeared in the shooting field of view of the shooting component; for example, the actual position of the moving target at the last moment;

[0070] The nearest alternative path is determined from multiple alternative paths according to the historical actual position; for example, the alternative path that appears in a circular area with a preset radius with the historical actual position as the origin (i.e., the alternative path closest to the origin, see Figure 3 Due to the harsh geographical environment of mountainous areas and canyons, there are relatively few alternative paths. Therefore, the pre-screened alternative paths will not be too close to each other. Therefore, there will usually not be two alternative paths (i.e. alternative travel paths) within the circle at the same time.

[0071] The nearest candidate path is used as the current tracking path to adjust the flight heading and the shooting posture of the shooting component so that the moving target is within the shooting field of view of the shooting component.

[0072] As we all know, the straight line distance is the shortest, e.g. Figure 3 In the algorithm, since the distance between the origin and point B on the alternative path is the shortest, that is, point B is the closest, point B is usually used as the target when the rescue team switches routes. Accordingly, point B is also used as the tracking starting point of the drone on the alternative path, and the drone needs to fly to point B first to wait for the rescue team.

[0073] In this embodiment, by presetting the priority of the alternative paths, or presetting the radius, and setting point B as the target when the rescue team switches routes, this is done, on the one hand, to avoid the onboard computer from performing a large number of complex calculations (for example, the onboard computer combining a deep learning algorithm to select the best alternative path from multiple alternative paths), and to avoid the onboard computer from making decisions from a single perspective (for example, the perspective of distance), which may lead to one-sided decisions (for example, ignoring factors such as rescue team members and equipment conditions, as well as surrounding environmental conditions) and randomness (which also means unreliability), thereby increasing the possibility of losing the tracking target. On the other hand, it also enables the drone to quickly know the target or path of the missing rescue team, thereby improving the efficiency of continuous tracking to a certain extent. In other words, a balance is achieved between the drone's intelligence (which affects the power consumption of the onboard computer), the efficiency of the drone's continuous tracking, and the flexibility of the rescue team (such as the rescue team's flexible selection of the target when switching routes).

[0074] Furthermore, in order to improve the tracking accuracy, in other embodiments, after adjusting the shooting posture of the shooting component in step S103, the following steps may be further included:

[0075] S105 , correcting the shooting posture of the shooting component according to the image data captured in real time by the shooting component after the shooting posture is adjusted.

[0076] In some embodiments, step S105 specifically includes the following steps: the onboard computer calculates the current actual position of the moving target based on the image data captured in real time by the shooting component; the onboard computer calculates the deviation between the actual position and the predicted position in the first motion trajectory; the onboard computer readjusts the shooting posture of the shooting component based on the deviation.

[0077] Furthermore, since the pod has a fixed attitude angle range, when it moves to the maximum attitude angle, the pod will no longer be able to track the moving target, and the moving target will exceed the pod image range. Therefore, it is necessary to control the movement of the drone to achieve the purpose of continuous tracking. Therefore, in other embodiments, the method further includes the steps of:

[0078] S106: The onboard computer predicts a second motion trajectory of the moving target within a second preset time period in the future based on the motion information and the latest shooting posture of the shooting component.

[0079] In some embodiments, the second preset time period is 0.5-1s.

[0080] S107 , the onboard computer adjusts the flight path according to the second motion trajectory to ensure continuous tracking.

[0081] In other embodiments, in order to further improve the tracking accuracy, the method further includes step: S108, the onboard computer corrects the flight route of the drone in real time based on the image data captured by the shooting component in real time.

[0082] Furthermore, in some other embodiments, in order to further improve the tracking accuracy, the method further includes the step of: an onboard computer correcting the flight path of the drone in real time based on the drone's flight data, the image data captured in real time by the camera component, and the latest shooting posture of the camera component. Specifically, the step of correcting the flight path of the drone includes the following steps: the onboard computer obtains the latest shooting posture of the camera component; the onboard computer determines whether the posture angle in the shooting posture is the maximum posture angle; if the posture angle is the maximum posture angle, the onboard computer calculates the current actual position of the moving target based on the image data captured in real time by the camera component; the onboard computer calculates the deviation between the actual position and the predicted position in the second motion trajectory; and the onboard computer readjusts the flight path of the drone based on the deviation.

[0083] The following is an example of a drone equipped with a camera through a pod. Figure 2 , the target tracking method of the present invention is described in detail.

[0084] S001, the onboard computer obtains the image data captured by the camera from the pod carried by the UAV in real time, and performs target recognition and capture on the obtained image data to obtain the motion information of the moving target to be tracked.

[0085] S002 , predicting the first motion trajectory of the moving target in the next second based on the motion information obtained in step S001 .

[0086] S003: The onboard computer adjusts the attitude angle of the pod according to the first motion trajectory obtained in step S002, so that the moving target is within the shooting field of view of the pod.

[0087] For example, when the moving target moves upward on the Y axis with the center point as the origin in the image, the pitch angle of the pod is modified, and the pod is raised to ensure that the moving target is near the center of the image; or, the left and right heading angles of the pod are modified to ensure that the moving target is near the center of the image.

[0088] S004, the onboard computer obtains the image data captured by the pod in real time after the shooting attitude is adjusted, and corrects the attitude angle of the pod in real time according to the image data.

[0089] In some embodiments, due to prediction errors on the one hand, and the fact that the motion state of the moving target itself may change on the other hand, such as a sudden change in motion direction, or a sudden acceleration or deceleration of motion speed, the actual motion trajectory of the moving target may deviate from the predicted motion trajectory. Therefore, after adjusting the shooting posture of the shooting component, it is necessary to correct the shooting posture based on the image data captured by the shooting component in real time. For example, after adjusting the shooting posture of the shooting component, the current real-time position of the moving target is immediately calculated based on the image data currently captured by the shooting component, and then the deviation between the real-time position and the estimated position of the moving target in the predicted first motion trajectory is calculated, and the shooting posture of the shooting component is readjusted based on the deviation.

[0090] S005: The onboard computer predicts the second motion trajectory of the moving target in the next second based on the motion information and the latest shooting posture of the pod.

[0091] S006, the onboard computer adjusts the flight route according to the second motion trajectory to ensure continuous tracking.

[0092] S007, the onboard computer obtains the image data taken by the pod in real time, and corrects the flight route of the UAV in real time based on the image data.

[0093] S008, the onboard computer obtains the flight data of the UAV, and makes real-time corrections to the flight route of the UAV based on the flight data and the image data taken in real time by the pod.

[0094] In order to evaluate the performance of the target tracking method of the present invention, the continuous tracking of a rescue vehicle by a drone is taken as an example, and the method is compared with the existing target tracking method in terms of continuous tracking success rate, accuracy and speed.

[0095] The overlap score (OS) and center location error (CLE) of the OPE (One Pass Evaluation) method are used to evaluate the success rate and accuracy of the target tracking algorithm. Specifically, the algorithm is initialized using the ground truth box in the first frame. The predicted boxes in subsequent frames are completely derived from the output of the target tracking algorithm. The differences between the labeled and predicted boxes are then compared, and accuracy and success rate curves can be plotted to evaluate the performance of the tracker.

[0096] See also Figure 4 , first calculate the label box And the prediction box output by the algorithm The Intersection of Union (IoU) is used as the OS of each frame. The formula is as follows: (1),

[0097] in, The OS represents the number of pixels in the region calculated. Success is determined by comparing each frame's OS with a pre-set threshold. The ratio of total successful frames to all frames represents the success rate at that threshold. When plotting the success rate curve, the horizontal axis represents the set threshold (ranging from 0 to 1), and the vertical axis represents the success rate at that threshold. The area under the curve is calculated during evaluation; a larger area indicates stronger algorithm performance.

[0098] For accuracy, see Figure 4 , CLE should be calculated in each frame, and CLE is determined by the center point of the label box The algorithm predicts the center point of the box The distance between is defined as follows: (2), where Represents the Euclidean distance between two points. By calculating the CLE for each video frame, the accuracy metric is expressed as the ratio of the number of frames with a distance error less than a threshold to the total number of frames. Similar to the success rate graph, the horizontal axis represents the pixel distance threshold (ranging from 0 to 50), and the vertical axis represents the accuracy at the corresponding threshold. Typically, the threshold is set to 20 pixels, resulting in distance precision (DP) when CLE = 20 pixels.

[0099] The target tracking method of this embodiment is used as the experimental group and the other nine existing target tracking methods (such as LightTrack, SiamAPN, SiamAPN++, SiamRPN_mobileV2, SiamRPN_ex, SiamFC++, SiamBAN, and SiamCAR) are used as the control group to evaluate the accuracy and tracking success rate. Figure 5 and Figure 6 , and the evaluation results of the three parameters are as follows Figure 7 .Depend on Figure 5 、 Figure 7 and Figure 7 It can be seen that the target tracking method of this embodiment has the highest success rate and one of the best accuracy, that is, it can best perform continuous tracking of the target.

[0100] Based on the above target tracking method, the present invention also provides a target tracking device, which includes:

[0101] A data acquisition module is used to acquire image data captured by the shooting component in real time;

[0102] a first data processing module, configured to process the image data acquired by the data acquisition module to determine whether the moving target to be tracked is currently within the shooting field of view of the shooting component;

[0103] a second data processing module, configured to, when the first data processing module determines that the moving target is currently within the shooting field of view of the shooting component, obtain motion information of the moving target from the image data, and predict a first motion trajectory of the moving target within a first preset time period based on a current motion trajectory and a current motion state in the motion information;

[0104] a third data processing module, configured to control the drone to continue flying along the current path of the moving target for a second preset time when the first data processing module determines that the moving target is not currently within the shooting field of view of the shooting component;

[0105] a fourth data processing module, configured to adjust the shooting posture of the shooting component according to the first motion trajectory predicted by the second data processing module, so that the moving target is within the shooting field of view of the shooting component;

[0106] The fifth data processing module is used to determine whether the moving target appears again in the shooting field of view of the shooting component after a second preset time has passed, and when it is determined that the moving target is not in the shooting field of view of the shooting component, control the drone to adjust the flight heading and the shooting posture of the shooting component according to the pre-stored highest priority alternative path so that the moving target is within the shooting field of view of the shooting component.

[0107] In some embodiments, the first data processing module is specifically used to construct a two-dimensional coordinate system with the center of the image data as the origin to obtain the first relative position of the moving target in the image data; calculate the second relative position and relative speed of the moving target relative to the drone based on the first relative position, the current actual position of the drone and the current flight speed; calculate the current coordinate position, current speed and current motion trajectory of the moving target based on the second relative position and the relative speed; or, to construct a two-dimensional coordinate system with the center of the image data as the origin to obtain the first relative position of the moving target in the image data; calculate the current coordinate position of the moving target based on the first relative position; calculate the current speed and current motion trajectory of the moving target based on the current coordinate position and historical coordinate position of the moving target.

[0108] In other embodiments, the target tracking device also includes a sixth data processing module for correcting the flight route of the drone in real time based on the flight data of the drone, the image data captured by the shooting component in real time, and the latest shooting posture of the shooting component.

[0109] In other embodiments, when the drone fails to wait for a moving target to appear (at point B) on the alternative path (for example, the onboard computer determines that the drone has waited at point B for 15-30 minutes without the moving target appearing), or receives a temporary other mission instruction (for example, to conduct reconnaissance at a specific destination), or when the onboard computer determines that the remaining battery level of the drone, as detected by the power detection module, has reached a preset threshold, the onboard computer sends a control instruction to the drone's flight controller to direct the drone to a designated landing point and wait, thereby preventing the drone from being lost. Because complex environments are prone to signal interference or loss, staged navigation is required for the drone's landing process to the designated landing site.

[0110] In some embodiments, the integrated guided landing device is pre-installed on the drone, see Figure 8 , which includes an onboard computer 11, an inertial navigation module 12, a satellite positioning module 13, a radio guidance module 14, a visual guidance module 15, a first ground beacon 16 for transmitting radio signals, and a second ground beacon 17 for transmitting optical signals, wherein the onboard computer 11 is electrically connected to the flight controller of the UAV, for sending corresponding control instructions to the flight controller, and receiving flight data fed back by the flight controller, such as flight altitude; the inertial navigation module 12 is electrically connected to the above-mentioned onboard computer 11 and flight controller respectively, for guiding the UAV to land to the destination through the flight controller; the satellite positioning module 13 is electrically connected to the above-mentioned onboard computer 11 and flight controller, for real-time positioning of the UAV; the radio guidance module 14 is electrically connected to the above-mentioned onboard computer 11, for receiving the radio signal transmitted by the first ground beacon 16; the visual guidance module 15 is electrically connected to the above-mentioned onboard computer 11, for receiving the optical signal transmitted by the second ground beacon 17.

[0111] In some embodiments, the onboard computer 11 is used to obtain flight data such as the current flight altitude of the UAV from the flight controller or the satellite positioning module, and determine whether the current flight altitude is less than or equal to a first preset altitude threshold and greater than a second preset altitude threshold. If it is greater than the first preset altitude threshold, a first control instruction is sent to the satellite positioning module 13 to control the satellite positioning module 13 to feedback real-time positioning data (for example, the real-time coordinates of the UAV), and to perform real-time corrections to the landing route guided by the inertial navigation module 12 based on the real-time positioning data, that is, the satellite positioning module assists the inertial navigation module in guiding the UAV to land at the destination.

[0112] Specifically, the onboard computer 11 generates a second control instruction indicating a real-time correction of the landing route based on the real-time positioning data, and sends it to the flight controller, which then corrects the flight route / flight trajectory of the UAV in real time based on the second control instruction.

[0113] In some embodiments, when it is determined that the current flight altitude is less than or equal to a first preset altitude threshold and greater than a second preset altitude threshold, the onboard computer 11 is also used to start the radio guidance module 14, and periodically obtain the radio beacon signal received by the radio guidance module 14 and transmitted by the first ground beacon 16, and periodically correct the heading of the UAV according to the radio beacon signal; specifically, the onboard computer 11 determines whether the UAV deviates from the landing heading based on the radio signal. If so, it generates and sends a third control instruction to the flight controller to control the UAV to stop landing, and correct the heading of the UAV before landing.

[0114] Specifically, the radio beacon, that is, the relative position of the first ground signal and the UAV can be calculated based on the strength of the radio beacon signal received by the radio guidance module.

[0115] A three-dimensional coordinate system is constructed with the drone as the coordinate origin. Through radio guidance, the radio beacon, that is, the azimuth of the first ground beacon relative to the origin (that is, the drone), can be obtained. Mobile flight is carried out according to this azimuth. If the control cycle is one second, the drone will compare the azimuths measured before and after every second of flight to see if they are consistent. If there is an angle error, the flight route will be corrected.

[0116] In some embodiments, when it is determined that the current flight altitude is less than or equal to the second preset altitude threshold, the onboard computer 11 sends a fourth control instruction to the visual guidance module 15 to start the visual navigation module 15 to assist the inertial navigation module in guiding the UAV to land to the destination; specifically, the onboard computer 11 obtains the optical signal emitted by the second ground beacon received by the visual navigation module 15, and performs data processing based on the optical information to obtain the fifth control instruction, and sends it to the flight controller to control the UAV to land to the destination.

[0117] In some embodiments, the beacon location only needs to know the distance and azimuth from the landing point, as well as the relative position, and does not need to be set near the landing point.

[0118] In some embodiments, the visual guidance module can guide the UAV flight by terrain matching, but the landing accuracy is limited and needs to be integrated with a satellite positioning module for navigation, or integrated with an inertial navigation module for navigation.

[0119] In some embodiments, the onboard computer is a lightweight onboard computer with low performance.

[0120] See also Figure 9 Based on the guided landing device, the guided landing method of the UAV includes the following steps:

[0121] S201, the onboard computer obtains the current flight altitude of the UAV in real time, and determines whether the current flight altitude is less than or equal to the first preset altitude threshold and greater than the second preset altitude threshold. If it is greater than the first preset altitude threshold, execute step S202; if it is less than or equal to the first preset altitude threshold and greater than the second preset altitude threshold, execute step S203; if it is less than or equal to the second preset altitude threshold, execute step S204.

[0122] In some embodiments, the onboard computer obtains the current flight altitude of the drone from the drone's flight controller or satellite positioning module.

[0123] In some embodiments, the first preset altitude threshold and the second preset altitude threshold are pre-set according to the current geographical environment of the landing point.

[0124] Of course, in other embodiments, the propagation limit distance of the radio signal emitted by the first ground beacon is used as the first preset height threshold, and the propagation limit distance of the optical beacon signal emitted by the second ground beacon is used as the second preset height threshold.

[0125] For example, when the flight altitude exceeds the first preset altitude threshold, the radio guidance module on the drone will not receive the radio beacon signal transmitted by the first ground signal. If the flight altitude is less than or equal to the first preset altitude threshold, the radio guidance module on the drone will receive the radio beacon signal transmitted by the first ground signal. Of course, during the descent of the drone, the first reception of the radio beacon signal transmitted by the first ground beacon may be directly used as the basis. That is, if the first reception of the radio beacon signal transmitted by the first ground signal is received, step S203 is executed.

[0126] For example, when the flight altitude exceeds the second preset altitude threshold, the visual guidance module on the drone will not receive the optical beacon signal transmitted by the second ground signal. If the flight altitude is less than or equal to the second preset altitude threshold, the visual guidance module on the drone will receive the optical beacon signal transmitted by the second ground signal. Of course, during the descent of the drone, the first receipt of the optical beacon signal transmitted by the second ground beacon may be directly used as the criterion. That is, if the first receipt of the radio beacon signal transmitted by the second ground signal is performed, step S204 is executed.

[0127] Once the drone enters the landing phase, the onboard computer begins to obtain the drone's flight altitude in real time and compares the current flight altitude with two preset altitude thresholds. Only when the flight altitude is greater than the preset altitude thresholds will the inertial navigation module be activated to guide the drone to land. Before the drone enters the landing phase, the drone can fly under any guidance method in the existing technology.

[0128] S202: The onboard computer obtains real-time positioning data from the satellite positioning module, and corrects the landing route of the UAV in real time according to the positioning data, and executes step S201.

[0129] In some embodiments, if the current flight altitude of the drone is greater than a first preset altitude threshold, it indicates that the drone is currently still in a high-altitude stage. At this stage, the satellite positioning signal is very strong because it is not affected by the geographical environment of the landing point. Therefore, based on the real-time positioning data of the satellite positioning module, for example, the real-time coordinates of the drone, the flight route / landing route of the drone under the guidance of the inertial navigation module is corrected in real time.

[0130] In some embodiments, the onboard computer obtains real-time positioning data, such as the coordinates of the drone, and generates corresponding navigation commands based on the real-time positioning data and sends them to the flight controller of the drone. The flight controller executes the navigation commands and corrects the landing route in real time.

[0131] S203, the onboard computer periodically obtains the radio beacon signal transmitted by the first ground beacon received by the radio guidance module, and corrects the landing heading of the UAV according to the radio beacon signal, executing step S201.

[0132] In some embodiments, if the drone's current flight altitude is less than or equal to a first preset altitude threshold but greater than a second preset altitude threshold, the drone is currently in mid-air. During this phase, the drone enters the communication range of a first ground beacon, meaning the radio guidance module can receive a radio beacon signal transmitted by the first ground beacon. Therefore, the onboard computer activates the radio guidance module and periodically acquires the radio signal received by the radio guidance module. Based on the radio beacon signal, the onboard computer determines whether the drone has deviated from its course. If so, a fifth control instruction is generated and sent to the flight controller to stop the drone from landing and correct its course. If not, no action is taken, allowing the drone to continue landing under the guidance of the inertial navigation module. When the onboard computer determines that the drone has landed at the second preset altitude threshold, a corresponding control instruction is generated to activate the visual guidance module to assist the inertial navigation module in guiding the drone to land. In other words, while the satellite positioning module performs real-time corrections, the radio guidance module periodically performs corrections, thereby further improving the accuracy and reliability of guidance.

[0133] In this embodiment, radio direction finding technology is used to determine whether the drone has deviated from its course. Specifically, the relative position of the radio beacon, that is, the second ground beacon and the drone can be determined based on the strength of the received radio signal, so that whether the drone has deviated from its course can be determined based on the relative position.

[0134] S204: The onboard computer activates the visual guidance module to receive the optical beacon signal emitted by the second ground beacon, and guides the UAV to land at the destination according to the optical beacon signal.

[0135] In some embodiments, if the drone's flight altitude is less than a second preset altitude, indicating that the drone is currently at a low altitude, due to geographical factors, both satellite and radio signals may be subject to ground interference, resulting in data loss or anomalies, which may render the satellite positioning module and the radio guidance module inoperable. Therefore, the onboard computer transmits a control instruction to activate the visual guidance module, thereby activating the visual guidance module to receive an optical beacon signal from a second ground beacon (of course, the satellite positioning module and the radio guidance module are correspondingly disabled). The visual guidance module then acquires the optical beacon signal and processes the optical beacon signal to generate a third control instruction, which is then transmitted to the drone's flight controller to control the drone's landing. Specifically, when the drone descends to a certain altitude, the visual guidance module assists the inertial navigation module in guiding the drone to land.

[0136] The guided landing method of this embodiment does not require pre-setting of parking spaces and cooperative light arrays, and different modules are used to assist the inertial navigation module in different stages of landing. For example, in the low-altitude stage, only the visual guidance module is used to assist the inertial navigation model for guidance (to avoid the influence of interference signals or no signals); in the medium-altitude stage, the satellite positioning module and the radio guidance module are used to perform real-time and periodic corrections on the inertial navigation, respectively; in the high-altitude stage, the satellite positioning module is used to perform real-time corrections on the inertial navigation module.

[0137] Of course, in other embodiments, if satellite images or high-altitude images of the landing point are pre-stored, when the visual guidance module is started, the image taken by the visual guidance module can be compared with the pre-stored satellite images or high-altitude images, and the inertial navigation module can be corrected according to the comparison results.

[0138] In other embodiments, when it is determined in step S201 that the current flight altitude is less than or equal to a first preset altitude threshold and greater than a second preset altitude threshold (of course, at this time the second preset altitude threshold is not the communication distance limit of the second ground beacon, but is set according to the physical environment of the current landing point, and when it is higher than the second preset altitude threshold, the visual guidance module can also receive the optical beacon signal), the radio guidance module is started while the visual navigation module is also started to receive the optical beacon signal, so as to correct the landing route of the UAV under the guidance of the inertial navigation module according to the optical beacon signal.

[0139] Accordingly, when the UAV descends to the second preset altitude threshold, the onboard computer directly controls to shut down the radio guidance module and the satellite positioning module, and only retains the visual guidance module to assist the inertial navigation module.

[0140] During the guided landing process, different modules are used in stages for guidance or auxiliary guidance. For example, when the drone is in the high-altitude landing phase, the satellite positioning module's positioning data is used to correct the drone's flight path in real time, as satellite signals are not subject to interference. When the drone is in the mid-altitude landing phase, it can receive radio signals. Therefore, the satellite positioning module provides real-time corrections while the radio signals received by the radio guidance module periodically correct the drone's landing course. This not only avoids errors that would occur when using only the inertial navigation module to a certain extent, but also ensures that the drone does not deviate from the established landing path through real-time and periodic course corrections. When the drone is in the low-altitude phase of the landing process, that is, when its flight altitude drops to a second preset altitude threshold, the satellite positioning module's signal is very weak or even absent due to the geographical environment, and the radio guidance module is also susceptible to interference or misleading. Therefore, to avoid interference or misleading that may cause the drone to deviate from its course, the satellite positioning module and the radio guidance module are disabled, and the most intuitive visual guidance module is directly used to assist the inertial navigation model for guidance, improving the accuracy and reliability of landing in complex environments. And because the visual guidance module is used for auxiliary guidance when entering the low-altitude stage, and the low-altitude stage is usually only a few hundred meters high, even if there is sudden fog or heavy rain that interferes with the line of sight after entering the low-altitude stage, it is still possible to land under the guidance of the inertial navigation module.

[0141] Typically, in disaster relief areas with harsh environments, drones need to carry supplies, etc. Therefore, the drone's own weight must not be too heavy. If a large amount of image data is collected and processed, or even the image data is combined with other data for processing, this will inevitably require the drone to be equipped with a high-performance onboard computer. A high-performance onboard computer will inevitably have more computing units than an ordinary onboard computer, which means that its own weight is heavier than that of an ordinary onboard computer, thereby limiting the weight of supplies that the drone can carry.

[0142] Furthermore, there is no need to precisely set up parking spaces, cooperative light arrays and other guidance facilities at the landing point in advance, thus avoiding the waste of manpower and financial resources to build parking spaces and other guidance devices during the post-disaster rescue stage, and also avoiding safety accidents caused by sudden aftershocks or landslides during the construction process. In addition, different guidance modules are used to assist the inertial navigation module in guidance at different stages of landing, thus avoiding interference or signal loss at low altitude or near the landing point, which makes it impossible to perform multi-mode data fusion calculations and thus unable to continue guidance, thereby ensuring the stability of guided landing in complex environments.

[0143] By rationally allocating various modules at various stages of drone landing (for example, when the flight altitude is greater than a first preset altitude threshold, the satellite positioning module and the inertial navigation module are used for guidance; when the flight altitude is less than the first preset altitude threshold but greater than a second preset altitude threshold, the satellite positioning module and the radio module are used to respectively perform real-time and periodic corrections on the landing route guided by the inertial navigation module; and when the flight altitude is less than or equal to the second preset altitude threshold, the visual guidance module is directly used to assist the inertial navigation module for guidance), the overall energy consumption of the drone is reduced, thereby enabling the drone to have a longer cruising time, and thus being able to perform more tasks or different types of tasks at one time, thereby improving work efficiency. For example, search and rescue missions can be carried out after the delivery of supplies, thereby improving rescue efficiency.

[0144] By using the visual guidance module to assist the inertial navigation module in guiding the drone when the drone is close to the destination, that is, when the flight altitude is less than or equal to the second preset altitude threshold, the drone can be prevented from landing accidentally due to radio interference.

[0145] The target tracking method of the embodiment of the present disclosure can be applied to a variety of electronic devices. For example, the electronic device can be: a mobile phone, a tablet computer (Tablet Personal Computer), a digital camera, a personal digital assistant (PDA), a navigation device, a mobile Internet device (Mobile Internet Device, MID), a wearable device (Wearable Device), and other devices capable of object editing. In addition, the object editing scheme of the embodiment of the present disclosure can be implemented not only as a function of the input method, but also as a function of the operating system of the electronic device.

[0146] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it can appear in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiments of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD) or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0147] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. For example, in some embodiments, various examples of the present disclosure (e.g., methods, apparatus, or devices) may be implemented in part or in whole on a computer-readable medium. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, apparatus, systems, techniques, or methods described herein may be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0148] The present disclosure also provides at least one computer program product stored on a non-transient computer-readable storage medium. The computer program product includes computer-executable instructions, such as computer-executable instructions included in a program module executed in a device on a physical or virtual processor of a target to perform the example method described above. Generally speaking, program modules may include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data structures. In various embodiments, the functionality of the program modules may be merged or split between the described program modules. The computer-executable instructions for the program modules may be executed in a local or distributed device. In a distributed device, the program modules may be located in both local and remote storage media.

[0149] The program code for realizing the method of the present disclosure can be written in one or more programming languages.These computer program codes can be provided to the processor of general-purpose computer, special-purpose computer or other programmable data processing device, so that program code, when being executed by computer or other programmable data processing device, causes the function / operation specified in flow chart and / or block diagram to be implemented.Program code can be fully on computer, partly on computer, as independent software package, partly on computer and partly on remote computer or fully on remote computer or server and execute.In the context of the present disclosure, computer program code or related data can be carried by any appropriate carrier, so that equipment, device or processor can perform various processes and operations described above.The example of carrier includes signal, computer readable medium, etc.

[0150] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0151] Those skilled in the art will appreciate that to implement all or part of the process steps in the above-described method embodiments, the process steps can be executed by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium. When executed, the program can include the process steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination thereof. More detailed examples of machine-readable storage media include an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0152] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0153] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a computer terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0154] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A target tracking method based on a drone pod, characterized in that: The target is a moving target in a complex environment, and the method comprises the steps of: S0, the onboard computer processes the image data captured in real time by the shooting component to determine whether the moving target to be tracked is currently within the shooting field of view of the shooting component. If so, execute step S1; otherwise, execute step S2; S1, an onboard computer obtains motion information of the moving target from the image data, and predicts a first motion trajectory of the moving target within a first preset time period based on the current motion trajectory and current motion state in the motion information, and executes step S3; S2, the onboard computer controls the drone to continue flying along the current path of the moving target for a second preset time, and when it is determined again after the second preset time that the moving target is no longer within the shooting field of view of the shooting component, step S4 is executed; S3, the onboard computer adjusts the shooting posture of the shooting component according to the first motion trajectory so that the moving target is within the shooting field of view of the shooting component; Specifically, the step of predicting a first motion trajectory within a first preset time period based on the current motion trajectory and the current motion state includes: dividing the velocity of the motion target in the image data into two velocity components, horizontal and vertical, based on the current motion trajectory and the current motion state of the motion target; and determining the motion trajectory based on the direction of maximum gradient change of the velocity component; S4, the onboard computer controls the drone to adjust the flight direction and the shooting posture of the shooting component according to the nearest alternative path among the multiple pre-stored alternative paths, so that the moving target is within the shooting field of view of the shooting component; The steps of determining the nearest alternative path specifically include: Obtaining the actual historical position of the moving target when the moving target last appeared in the shooting field of view of the shooting component; The nearest candidate path is determined from a plurality of candidate paths according to the historical actual position; specifically, the candidate path that appears within a circular area with a preset radius with the historical actual position as the origin is used as the nearest candidate path.

2. The method according to claim 1, characterized in that The step S1 specifically includes the following steps: The onboard computer constructs a two-dimensional coordinate system with the center of the image data as the origin to obtain a first relative position of the moving target in the image data; The onboard computer calculates a second relative position and relative speed of the moving target relative to the drone based on the first relative position, the current actual position of the drone, and the current flight speed; The current coordinate position, current speed and current motion trajectory of the moving target are calculated according to the second relative position and the relative speed.

3. The method according to claim 1, characterized in that The step S1 specifically includes the following steps: The onboard computer constructs a two-dimensional coordinate system with the center of the image data as the origin to obtain a first relative position of the moving target in the image data; The onboard computer calculates the current coordinate position of the moving target according to the first relative position; The onboard computer calculates the current speed and current motion trajectory of the moving target based on the current coordinate position and the historical coordinate position of the moving target.

4. The method according to claim 2, characterized in that The step of calculating the current coordinate position of the moving target specifically comprises the steps of: Obtaining a current shooting angle of the shooting component, and obtaining an azimuth angle of the drone based on the shooting angle and the first relative position; Obtaining a pixel size of the moving target in the image data, and calculating a relative distance between the moving target and the UAV based on the pixel size; Calculating the current coordinate position of the moving target according to the azimuth angle and the relative distance; and / or, The step of calculating the current speed of the moving target specifically comprises the steps of: Calculating the current speed of the moving target according to the flight speed and the relative speed; and / or, The step of calculating the current motion trajectory of the moving target specifically comprises the steps of: The relative motion trajectory of the moving target is calculated according to the first relative position of the moving target in each image data captured within the second preset time period.

5. The method according to claim 1, characterized in that Also includes the steps: The onboard computer corrects the shooting posture according to the image data captured in real time by the shooting component; and / or, The onboard computer predicts a second motion trajectory of the moving target within a second preset time period based on the motion information and the latest shooting posture of the shooting component; The onboard computer adjusts the flight path according to the second motion trajectory to ensure continuous tracking.

6. The method according to claim 5, characterized in that Also includes the steps: The onboard computer corrects the flight route of the UAV in real time according to the flight data of the UAV, the image data captured by the capturing component in real time, and the latest capturing posture of the capturing component.

7. The method according to claim 6, characterized in that The step of correcting the shooting posture of the shooting component specifically includes the steps of: The onboard computer calculates the current actual position of the moving target based on the image data captured in real time by the capturing component; An onboard computer calculates a deviation between the actual position and the predicted position in the first motion trajectory; The onboard computer readjusts the shooting posture of the shooting component according to the deviation.

8. The method according to claim 6, characterized in that The step of adjusting the flight route of the drone specifically comprises the following steps: The onboard computer obtains the latest shooting posture of the shooting component; The onboard computer determines whether the attitude angle in the shooting attitude is the maximum attitude angle; If the attitude angle is the maximum attitude angle, the onboard computer calculates the current actual position of the moving target based on the image data captured in real time by the capturing component; An onboard computer calculates a deviation between the actual position and a predicted position in the second motion trajectory; The onboard computer readjusts the flight path of the UAV according to the deviation.

9. A target tracking device, characterized in that: include: A data acquisition module is used to acquire image data captured by the shooting component in real time; a first data processing module, configured to process the image data acquired by the data acquisition module to determine whether the moving target to be tracked is currently within the shooting field of view of the shooting component; a second data processing module, configured to, when the first data processing module determines that the moving target is currently within the shooting field of view of the shooting component, obtain motion information of the moving target from the image data, and predict a first motion trajectory of the moving target within a first preset time period based on a current motion trajectory and a current motion state in the motion information; a third data processing module, configured to control the drone to continue flying along the current path of the moving target for a second preset time when the first data processing module determines that the moving target is not currently within the shooting field of view of the shooting component; a fourth data processing module, configured to adjust the shooting posture of the shooting component according to the first motion trajectory predicted by the second data processing module, so that the moving target is within the shooting field of view of the shooting component; Specifically, the step of predicting a first motion trajectory within a first preset time period based on the current motion trajectory and the current motion state includes: dividing the velocity of the motion target in the image data into two velocity components, horizontal and vertical, based on the current motion trajectory and the current motion state of the motion target; and determining the motion trajectory based on the direction of maximum gradient change of the velocity component; a fifth data processing module, configured to determine whether the moving target appears again within the photographing field of view of the photographing component after a second preset time has elapsed, and, if it is determined that the moving target is no longer within the photographing field of view of the photographing component, to control the drone to adjust its flight heading and the photographing posture of the photographing component according to a nearest alternative path among a plurality of pre-stored alternative paths, so as to bring the moving target within the photographing field of view of the photographing component; Among them, the step of determining the nearest alternative path specifically includes: obtaining the historical actual position of the moving target when the moving target last appeared in the shooting field of view of the shooting component; determining the nearest alternative path from multiple alternative paths based on the historical actual position; specifically, the alternative path that appears within a circular area with a preset radius with the historical actual position as the origin is used as the nearest alternative path.

Images