A UAV tracking method, device, optoelectronic integrated equipment and medium

Abstract

The embodiments of the present application provide a method, device, optoelectronic integrated device and medium for tracking a drone. In the method, a current frame image and a historical frame image of a target drone are obtained, and a motion trajectory prediction is performed based on the position information of the target drone in the historical frame image to determine the predicted position information of the target drone in the current frame image. Subsequently, based on the position information of each object to be identified in the current frame image and the predicted position information, the target drone is screened from multiple objects to be identified to determine the position information of the target drone in the current frame image; based on the position information of the target drone in the current frame image, the device tracking parameters are determined; based on the device tracking parameters, the shooting device is controlled to track and shoot the target drone in real time. This achieves accurate tracking of the target drone, effectively improving the tracking accuracy and reliability of the drone.

Description

Technical Field

[0001] The present application relates to the field of drone tracking technology, and in particular to a drone tracking method, device, optoelectronic integrated equipment and medium. Background Art

[0002] In traditional drone tracking solutions, radar systems are usually used to track drones. This method has high equipment configuration costs, and due to the small size of small drones and their corresponding small reflection area, radar cannot accurately detect them. At the same time, the flight trajectory and reflection characteristics of small drones are similar to those of birds, resulting in a high false alarm rate for radar detection. Traditional drone tracking based on radar systems has the problem of poor accuracy.

[0003] The most advanced drone tracking method currently available mainly tracks drones by cracking the communication signals between the drone and the control terminal. Although this method can effectively track the movement path of the drone, when there is no communication interaction between the drone and the control terminal, the radio cracking will become ineffective and the drone cannot be tracked based on the communication signals between the two, so its reliability is poor.

[0004] Therefore, how to solve the problem of poor tracking accuracy and reliability of drones in related technologies has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] Based on the above problems, in order to improve the tracking accuracy and reliability of drones, the embodiments of the present application provide a drone tracking method, device, optoelectronic integrated equipment and medium.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for tracking a drone, comprising:

[0008] Acquire a current frame image and a historical frame image of a target UAV; the current frame image includes a plurality of objects to be identified; and the historical frame image includes the target UAV;

[0009] Performing motion trajectory prediction based on the position information of the target UAV in the historical frame image to determine the predicted position information of the target UAV in the current frame image;

[0010] According to the position information of each of the objects to be identified in the current frame image and the predicted position information, the target drone is screened from the multiple objects to be identified to determine the position information of the target drone in the current frame image;

[0011] Determining device tracking parameters according to the position information of the target UAV in the current frame image;

[0012] Based on the device tracking parameters, the shooting device is controlled to perform real-time tracking and shooting of the target UAV.

[0013] In a possible implementation, screening the target drone from a plurality of objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information to determine the position information of the target drone in the current frame image includes:

[0014] Based on the Hungarian algorithm, a target object to be identified that matches the predicted position information is selected from the position information of each object to be identified;

[0015] The target object to be identified is determined to be the target drone, and the position information of the target object to be identified is determined to be the position information of the target drone in the current frame image.

[0016] In a possible implementation, in the current frame image, the target drone is surrounded by a target detection frame; the position information of the target drone in the current frame image includes: size information of the target detection frame; the device tracking parameters include: shooting magnification;

[0017] The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes:

[0018] Based on the size information of the target detection frame, determining the proportion of the target detection frame in the current frame image;

[0019] The shooting magnification of the shooting device is calculated according to the proportion of the target detection frame in the current frame image.

[0020] In a possible implementation, the position information of the target drone in the current frame image includes: coordinate information of the target detection frame; the device tracking parameters include: rotation direction;

[0021] The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes:

[0022] Determining, according to the coordinate information of the target detection frame, the quadrant information of the target detection frame in the current frame image;

[0023] The rotation direction of the photographing device is calculated based on the quadrant information of the target detection frame.

[0024] In a possible implementation, the device tracking parameters include: a turning speed; the current frame image includes a plurality of concentric ellipses of different levels; and the ellipse sizes of the concentric ellipses of different levels are different;

[0025] The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes:

[0026] Determining the target concentric ellipse level corresponding to the target detection frame according to the coordinate information of the target detection frame;

[0027] Based on the target concentric ellipse level, the steering speed of the photographing device is calculated.

[0028] In a possible implementation, the objects to be identified include: historical objects and non-historical objects; the historical objects are objects that have appeared in the historical frame images; the non-historical objects are objects that have only appeared in the current frame images; the objects to be identified have their own corresponding tracking identifiers.

[0029] In a possible implementation, the performing motion trajectory prediction based on the position information of the target UAV in the historical frame image to determine the predicted position information of the target UAV in the current frame image includes:

[0030] Determine the position information of the target UAV in the historical frame image according to the tracking identifier of the target UAV;

[0031] Based on the Kalman filter algorithm and the position information of the target UAV in the historical frame image, motion trajectory prediction is performed to determine the predicted position information of the target UAV in the current frame image.

[0032] In a second aspect, an embodiment of the present application provides a drone tracking device, comprising:

[0033] An image acquisition module, used to acquire a current frame image and a historical frame image of a target UAV; the current frame image includes a plurality of objects to be identified; the historical frame image includes the target UAV;

[0034] A position prediction module, used to predict the motion trajectory of the target UAV based on the position information of the target UAV in the historical frame image, so as to determine the predicted position information of the target UAV in the current frame image;

[0035] a target drone determination module, configured to screen the target drone from the plurality of objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information, so as to determine the position information of the target drone in the current frame image;

[0036] A tracking parameter determination module, used to determine the device tracking parameters according to the position information of the target UAV in the current frame image;

[0037] The device control module is used to control the shooting device to perform real-time tracking and shooting of the target UAV based on the device tracking parameters.

[0038] In the third aspect, an embodiment of the present application provides an optoelectronic integrated device, including an optoelectronic device and a gimbal device; the optoelectronic device is used to photograph a target drone, and the gimbal device is used to control the rotation direction, steering speed and shooting magnification of the optoelectronic device to implement any possible drone tracking method in the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any possible drone tracking method in the first aspect.

[0040] Compared with the prior art, the present application has the following beneficial effects: the embodiment of the present application provides a method, device, optoelectronic integrated device and medium for tracking a drone. In the method, the current frame image and the historical frame image of the target drone are first obtained. On this basis, the predicted position information of the target drone in the current frame image is determined by predicting the motion trajectory of the position information of the target drone in the historical frame image. Subsequently, the object to be identified whose position information matches the predicted position information best is selected from multiple objects to be identified, and it is determined as the target drone, so as to determine the position information of the target drone in the current frame image, thereby accurately determining the position of the target drone in the current frame image, even if the target drone has no communication information interaction with the remote control terminal, or when there are interference objects such as birds and airplanes around, the target drone can be tracked in real time through the motion trajectory prediction mechanism and the corresponding optoelectronic device, thereby effectively improving the tracking reliability and accuracy of the target drone. Correspondingly, finally, according to the position information of the target drone in the current frame image, the device tracking parameters can be determined. The target UAV is tracked and photographed with specific device tracking parameters, so that various shooting parameters of the shooting equipment can be adjusted according to the specific position of the target UAV in each frame of the image, further improving the tracking accuracy of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 A schematic diagram of a flow chart of a drone tracking method provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a method for predicting drone location information provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a flow chart of a method for determining a shooting magnification of a shooting device provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a flow chart of a method for calculating a rotation direction provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a flow chart of a method for calculating a turning speed provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a multi-level concentric ellipse provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of a drone tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following is a further detailed description of this application in combination with specific embodiments and with reference to the accompanying drawings. It should be noted that the embodiments described in the embodiments of this application are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0050] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "include" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0051] In traditional drone tracking solutions, radar systems are usually used to track drones. This method has high equipment configuration costs, and due to the small size of small drones and their corresponding small reflection area, radar cannot accurately detect them. At the same time, the flight trajectory and reflection characteristics of small drones are similar to those of birds, resulting in a high false alarm rate for radar detection. Traditional drone tracking based on radar systems has the problem of poor accuracy.

[0052] The most advanced drone tracking method currently available mainly tracks drones by cracking the communication signals between the drone and the control terminal. Although this method can effectively track the movement path of the drone, when there is no communication interaction between the drone and the control terminal, the radio cracking will become ineffective and the drone cannot be tracked based on the communication signals between the two, so its reliability is poor.

[0053] In order to solve the above problems, the embodiment of the present application provides a method, device, optoelectronic integrated device and medium for tracking a drone. In the method, the current frame image and the historical frame image of the target drone are first obtained. On this basis, the predicted position information of the target drone in the current frame image is determined by predicting the motion trajectory of the position information of the target drone in the historical frame image. Subsequently, the object to be identified whose position information matches the predicted position information best is selected from multiple objects to be identified, and it is determined as the target drone, so as to determine the position information of the target drone in the current frame image, thereby accurately determining the position of the target drone in the current frame image. Even if the target drone has no communication information interaction with the remote control terminal, or there are interference objects such as birds and airplanes around, the target drone can be tracked in real time through the motion trajectory prediction mechanism and the corresponding optoelectronic device, thereby effectively improving the tracking reliability and accuracy of the target drone. Correspondingly, finally, according to the position information of the target drone in the current frame image, the device tracking parameters can be determined. The target UAV is tracked and photographed with specific device tracking parameters, so that various shooting parameters of the shooting equipment can be adjusted according to the specific position of the target UAV in each frame of the image, further improving the tracking accuracy of the UAV.

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Next, a drone tracking method provided by the embodiment of the present application will be introduced in conjunction with the specific embodiment drawings. Figure 1 , which is a flow chart of a drone tracking method provided by an embodiment of the present application, and specifically includes the following steps:

[0056] S101: Acquire a current frame image and a historical frame image of a target UAV; the current frame image includes a plurality of objects to be identified; and the historical frame image includes the target UAV.

[0057] In the current frame image captured for the target drone, there are multiple objects to be identified, including historical objects and non-historical objects. Historical objects are used to represent objects that have appeared in historical frame images, and non-historical objects represent objects that are newly added in the current frame image. In order to accurately obtain the position information of the target drone in the current frame image, it is necessary to make a judgment based on the movement of the target drone in the historical frame image.

[0058] The historical frame images of the target UAV include the image from the first identification of the target UAV to all images before the current frame, that is, all images from the initial frame image of the target UAV to the current frame image. Through the historical frame images of the target UAV, the motion trajectory of the target UAV can be obtained for subsequent accurate tracking.

[0059] In the actual shooting scene of the target drone, the captured image often does not only include the target drone. Sometimes, there may be other tracking interference objects around the target drone, such as flying birds, buildings, etc. When these tracking interference objects are close to the target drone, they may affect the identification and marking of the target drone.

[0060] Therefore, when the initial frame image of the target drone is captured, the embodiment of the present application assigns corresponding detection frames and tracking identifiers to all the identified objects in the initial frame image. For example, if the initial frame image includes flying birds, target drones, and buildings, tracking representations 1, 2, and 3 can be assigned to them in turn. In this way, combined with the detection frame on each identified object, even if the target drone may overlap with other tracking interferences, the location of the drone can be quickly confirmed through the detection frames and tracking identifiers on the target drone and other tracking interferences, thereby improving the tracking accuracy of the target drone.

[0061] In particular, in a possible implementation, the recognition of the target drone in the historical frame image and the detection and recognition of the target drone can be determined based on a specific image detection algorithm, such as YOLO, SSD, FasterRCNN, etc. Such image detection algorithms can output the detection frame of the target drone while detecting and locating the target drone, which is not described in detail in this embodiment.

[0062] On the other hand, when in a drone tracking scenario at night, the identification and detection of the target drone can be further completed by thermal imaging. The drone is identified by thermal imaging technology to carry out tracking and detection of the target drone. The drone tracking method disclosed in the embodiment of the present application is not affected by day and night.

[0063] S102: Predicting a motion trajectory based on the position information of the target UAV in the historical frame image to determine predicted position information of the target UAV in the current frame image.

[0064] As can be seen from the introduction of step S101, in the historical frame images related to the target drone, the target drone is assigned a separate detection frame and tracking mark. Therefore, by combining the motion trajectory of the target drone detection frame in multiple historical frame images, the motion trajectory of the target drone before the current frame image can be analyzed, and then the motion trajectory can be predicted, thereby predicting the possible position information of the target drone in the current frame image, that is, the predicted position information.

[0065] Specifically, the prediction of the motion trajectory and travel position of the target drone in the embodiment of the present application is intended to be performed through the Kalman filter algorithm. The overall process can be seen in Figure 2 The flowchart of a method for predicting the position information of a drone shown in FIG. 1 specifically includes the following steps:

[0066] S1021: Determine the position information of the target UAV in the historical frame image according to the tracking identifier of the target UAV.

[0067] Based on the unique tracking identifier of the target drone in the historical frame image and the detection frame corresponding to the target drone, the coordinates of the center point of the target drone in the image can be further determined as the location information of the target drone in the historical frame image. When there are multiple historical frame images of the target drone, the motion trajectory of the target drone can be further drawn based on the detection frames of the target drone in different images. The motion trajectory can also be used as the location information of the target drone in the historical frame image to provide a data basis for the subsequent prediction of the location of the target drone in the current frame image.

[0068] S1022: Predicting a motion trajectory based on a Kalman filter algorithm and the position information of the target UAV in the historical frame image, and determining the predicted position information of the target UAV in the current frame image.

[0069] Furthermore, based on the position information of the target UAV in the historical frame image, the predicted position of the target UAV in the current frame image is calculated by the Kalman filter algorithm. The Kalman filter algorithm uses the state transfer matrix to model the motion pattern of the UAV, which enables the algorithm to predict its position in the next frame based on the state of the previous frame. Even if the speed and direction of the UAV change, the algorithm can adapt well. In addition, although the standard Kalman filter is suitable for linear devices, the extended Kalman filter (EKF) and the unscented Kalman filter (UKF) can handle the motion prediction of nonlinear devices, further improving the flexibility of the prediction.

[0070] In particular, in a possible implementation, in addition to predicting the position information of the target UAV in the current frame image, the position information of the historical object in the current frame image can be further predicted. Since each predicted position information will be attached with a corresponding detection frame and tracking mark, through the position prediction of other tracking interference objects in the historical object, it is possible to more accurately eliminate the tracking interference of the tracking interference objects on the target UAV, thereby improving the tracking accuracy of the target UAV.

[0071] The above is a detailed introduction to predicting the target drone's position information through the Kalman filter algorithm. Figure 1 Method introduction:

[0072] S103: Filtering the target UAV from the multiple objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information, so as to determine the position information of the target UAV in the current frame image.

[0073] After determining the predicted position information of the target UAV in the current frame image, the position information of each object to be identified in the current frame image is matched with the predicted position information of the target UAV, so as to screen the position information of multiple objects to be identified that best matches the predicted position information of the target UAV, and determine the corresponding object to be identified as the target UAV, so as to achieve real-time tracking of the target UAV.

[0074] In the current frame image, each object to be identified is assigned a detection frame, and the coordinate information represented by each detection frame corresponds to the location information of the object to be identified. By screening and matching the predicted coordinate information of the target drone with the actual coordinate information of the object to be identified, the coordinate information closest to the predicted coordinate information can be determined, thereby identifying the target drone.

[0075] In one possible implementation, in order to prevent the influence of tracking interference objects, the predicted position information of the historical objects can also be matched and screened with the objects to be identified, thereby eliminating the interference of redundant objects and improving the tracking accuracy of the target UAV.

[0076] Specifically, the process of determining the target drone from multiple objects to be identified needs to be based on the Hungarian algorithm to complete the screening of the objects to be identified. Based on the predicted position information of the target drone and the actual position information of each object to be identified, the IOU (Intersection over Union) of the predicted position information and other position information can be calculated to form a cost matrix. Assuming that we have m predicted positions and n positions of objects to be identified, the cost matrix is ​​an m*n matrix, in which each element represents the cost between a predicted position and a position of an object to be identified.

[0077] After initializing the Hungarian algorithm, the cost matrix is ​​reduced by rows and columns so that at least one element in each row and column of the matrix is ​​0. Then mark the 0 elements in the cost matrix and try to find a minimum set of rows and columns that cover all zero elements. If the minimum set of rows and columns that cover zero elements cannot cover all zero elements, find the uncovered minimum element, add the minimum element to the uncovered rows, and subtract the minimum element from the uncovered columns. Repeat the reduction and marking process until a minimum set of rows and columns that can cover all zero elements is found. Finally, extract an optimal match from the final cost matrix, so that the position information that best matches the predicted position information can be screened, and then the object to be identified corresponding to the position information is determined as the target drone, thereby determining the position information of the target drone in the current frame image.

[0078] S104: Determine device tracking parameters according to the position information of the target UAV in the current frame image;

[0079] S105: Based on the device tracking parameters, control the shooting device to perform real-time tracking and shooting of the target UAV.

[0080] Finally, after determining the position information of the target drone in the current frame image, the device tracking parameters are determined according to the specific position information of the target drone. The device tracking parameters are used to control the rotation direction, steering speed and shooting magnification of the shooting device. If any of these three parameters are adjusted incorrectly, it may lead to the inability to accurately track the target drone. Next, the calculation process of the rotation direction, steering speed and shooting magnification will be introduced in combination with the specific embodiment drawings.

[0081] First, the calculation method of shooting magnification is introduced, see Figure 3, which is a flow chart of a method for determining a shooting magnification of a shooting device provided in an embodiment of the present application, and specifically includes the following steps:

[0082] S1041: Determine, based on the size information of the target detection frame, the proportion of the target detection frame in the current frame image;

[0083] S1042: Calculate the shooting magnification of the shooting device according to the proportion of the target detection frame in the current frame image.

[0084] As we can see from the previous article, whether it is a historical frame image or a current frame image, a corresponding detection frame will be assigned to the object in the image. The size and dimensions of these detection frames are determined by the size of the object in the image. In the drone tracking scenario, the distance between the drone and the camera is not constant. When the distance between the drone and the camera is close, an excessive shooting magnification may also cause the camera to fail to accurately lock the target drone.

[0085] Therefore, the detection frame corresponding to the target drone is used as the target detection frame. When determining the shooting magnification of the shooting device, it is necessary to determine the image ratio occupied by the target detection frame in the current frame image according to the specific size information of the target detection frame, and use this to determine the shooting magnification of the shooting device. When the target detection frame occupies a large proportion in the image, the calculated shooting magnification needs to be adjusted to a smaller proportion to ensure that the target drone always occupies a moderate proportion in the image, which is convenient for the optoelectronic device to control the rotation of the device for tracking, and vice versa.

[0086] The size information of the target detection frame specifically refers to the width and height of the target detection frame. The upper left corner coordinates of the target detection frame in the current frame image are set to (x1, y1), and the lower right corner coordinates are set to (x2, y2). The width of the target detection frame = x2-x1, and the height = y2-y1. In this way, the size information of the target detection frame can be determined.

[0087] The formula for calculating the shooting magnification is as follows:

[0088]

[0089] In the formula, x represents the target shooting magnification to be calculated, Indicates the current shooting magnification of the optoelectronic device. It indicates the target occupancy ratio that the detection frame of the target drone occupies in advance. Indicates the actual proportion of the detection box occupied by the target in the current frame image.

[0090] In a possible implementation, since the proportion of the target drone's detection frame in the captured image is dynamically fluctuating, in order to adjust the shooting ratio when the drone is in a more intense motion state, an occupation ratio threshold or occupation ratio interval can be pre-set. If the occupation ratio of the drone's target detection frame in the current frame image is lower than the set threshold or interval, the shooting ratio is dynamically increased and adjusted to make its occupation ratio reach the set threshold or interval, ensuring that the shooting device can accurately capture the target drone. Similarly, if the occupation threshold is higher than the set threshold or interval, the shooting ratio is dynamically reduced to prevent the shooting ratio from being too large and causing the shooting device to be unable to respond to steering immediately when tracking the drone.

[0091] Next, the calculation method of the rotation direction will be introduced. Figure 4 , which is a flow chart of a method for calculating a rotation direction provided in an embodiment of the present application, and specifically includes the following steps:

[0092] S1043: determining quadrant information of the target detection frame in the current frame image according to the coordinate information of the target detection frame;

[0093] S1044: Calculate the rotation direction of the shooting device based on the quadrant information of the target detection frame.

[0094] Similar to the premise of calculating the shooting magnification, the calculation of the rotation direction also needs to be completed based on the target detection frame corresponding to the target drone. By obtaining the coordinate information of the target detection frame in the current frame image, its quadrant information in the current frame image can be determined.

[0095] Assuming that the initial direction of the shooting device is the center of the image, the rotation direction can be calculated by the relative position of the center of the target detection frame and the center of the image. The rotation direction can be divided into horizontal and vertical directions. When the target detection frame is in the first quadrant, the horizontal rotation direction is positive and the vertical rotation direction is positive; when it is in the second quadrant, the horizontal rotation direction is negative and the vertical rotation direction is positive; when it is in the third quadrant, the horizontal rotation direction is negative and the vertical rotation direction is negative; when it is in the fourth quadrant, the horizontal rotation direction is positive and the vertical rotation direction is negative.

[0096] For example, taking the current frame image resolution of 1920*1080 as an example, the center point coordinates of the target detection frame are set to (400,500). Since its resolution is 1920*1080, the center point of the image is (960,540). The center point of the image is compared horizontally and vertically with the center point of the target detection frame, that is, 400<960, 500<540. It can be determined that the target detection frame is in the second quadrant, and its horizontal rotation direction is negative and its vertical rotation direction is positive.

[0097] Next, the calculation method of steering speed will be introduced. Figure 5 , which is a flow chart of a method for calculating a steering speed provided in an embodiment of the present application, and specifically includes the following steps:

[0098] S1045: Determine, according to the coordinate information of the target detection frame, the target concentric ellipse level corresponding to the target detection frame;

[0099] S1046: Calculate the turning speed of the photographing device based on the target concentric ellipse level.

[0100] In order to determine the steering speed of the camera device through the coordinate information of the target detection frame, when the camera device captures an image, it will automatically attach a preset multi-level concentric ellipse to the current frame image captured. For details, see Figure 6 A schematic diagram of a multi-level concentric ellipse is shown. As shown in the figure, the levels of the concentric ellipse gradually increase from the center of the circle to the outside of the circle, and each level of the concentric ellipse has its own corresponding preset turning speed.

[0101] As mentioned above, the direction of rotation is divided into horizontal and vertical directions. Accordingly, the control of the steering speed needs to be adjusted based on the direction of rotation, and the steering speed also needs to be synchronously divided into the steering speed in the vertical direction and the steering speed in the horizontal direction. Figure 6 In the process of determining the steering speed based on multi-level concentric ellipses, for the coordinate point of the target UAV on the captured image, it is necessary to establish the projection of the coordinate point on the X-axis and Y-axis respectively, and then determine the vertical steering speed and the horizontal steering speed respectively according to the concentric ellipse level matched by its projection, so as to ensure the control accuracy of the steering speed.

[0102] Therefore, by using the coordinate information of the target detection frame and judging the levels of the concentric ellipses corresponding to the projection points of the target detection frame on the X-axis and Y-axis, the preset turning speeds specifically corresponding to the concentric ellipses can be determined respectively, thereby determining the turning speed of the shooting device.

[0103] The embodiment of the present application provides a method, device, optoelectronic integrated device and medium for tracking a drone. In the method, the current frame image and the historical frame image of the target drone are first obtained. On this basis, the predicted position information of the target drone in the current frame image is determined by predicting the motion trajectory of the position information of the target drone in the historical frame image. Subsequently, the object to be identified whose position information matches the predicted position information best is selected from multiple objects to be identified, and it is determined as the target drone, so as to determine the position information of the target drone in the current frame image, thereby accurately determining the position of the target drone in the current frame image. Even if the target drone has no communication information interaction with the remote control terminal, or there are interference objects such as birds and airplanes around, the target drone can be tracked in real time through the motion trajectory prediction mechanism and the corresponding optoelectronic device, thereby effectively improving the tracking reliability and accuracy of the target drone. Correspondingly, the device tracking parameters can be determined according to the position information of the target drone in the current frame image. The target UAV is tracked and photographed with specific device tracking parameters, so that various shooting parameters of the shooting equipment can be adjusted according to the specific position of the target UAV in each frame of the image, further improving the tracking accuracy of the UAV.

[0104] A drone tracking device provided in an embodiment of the present application is introduced below. The drone tracking device described below and the drone tracking method described above can refer to each other.

[0105] See also Figure 7 , which is a schematic diagram of the structure of a drone tracking device provided in an embodiment of the present application, specifically including the following modules:

[0106] The image acquisition module 100 is used to acquire a current frame image and a historical frame image of a target UAV; the current frame image includes a plurality of objects to be identified; and the historical frame image includes the target UAV;

[0107] A position prediction module 200 is used to predict the motion trajectory of the target UAV based on the position information of the target UAV in the historical frame image to determine the predicted position information of the target UAV in the current frame image;

[0108] The target drone determining module 300 is used to select the target drone from the plurality of objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information, so as to determine the position information of the target drone in the current frame image;

[0109] A tracking parameter determination module 400, configured to determine device tracking parameters according to the position information of the target UAV in the current frame image;

[0110] The device control module 500 is used to control the shooting device to perform real-time tracking and shooting of the target UAV based on the device tracking parameters.

[0111] On the other hand, an embodiment of the present application also provides an optoelectronic integrated device, including an optoelectronic device and a gimbal device; the optoelectronic device is used to photograph a target drone, and the gimbal device is used to control the rotation direction, steering speed and shooting magnification of the optoelectronic device to implement the drone tracking method of any of the above embodiments.

[0112] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, an embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the drone tracking method described in any of the above embodiments.

[0113] The computer-readable media of the embodiments of the present application include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0114] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the drone tracking method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0115] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for methods, devices, optoelectronic integrated equipment and media, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The methods, devices, optoelectronic integrated equipment and media described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0116] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A drone tracking method, characterized in that: include: Acquire a current frame image and a historical frame image of a target UAV; the current frame image includes a plurality of objects to be identified; and the historical frame image includes the target UAV; Performing motion trajectory prediction based on the position information of the target UAV in the historical frame image to determine the predicted position information of the target UAV in the current frame image; According to the position information of each of the objects to be identified in the current frame image and the predicted position information, the target drone is screened from the multiple objects to be identified to determine the position information of the target drone in the current frame image; Determining device tracking parameters according to the position information of the target UAV in the current frame image; Based on the device tracking parameters, controlling the shooting device to perform real-time tracking and shooting of the target UAV; The step of screening the target UAV from a plurality of objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information to determine the position information of the target UAV in the current frame image includes: Based on the Hungarian algorithm, a target object to be identified that matches the predicted position information is selected from the position information of each object to be identified; Determine the target object to be identified as the target drone, and determine the position information of the target object to be identified as the position information of the target drone in the current frame image; The objects to be identified include: historical objects and non-historical objects; the historical objects are objects that have appeared in the historical frame images; the non-historical objects are objects that have only appeared in the current frame images; the objects to be identified have respective corresponding tracking identifiers; in the current frame images, the target drone is surrounded by a target detection frame; The position information of the target drone in the current frame image includes: the coordinate information of the target detection frame; the device tracking parameters include: the rotation direction; the rotation direction includes: the vertical direction and the horizontal direction; The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes: Determining, according to the coordinate information of the target detection frame, the quadrant information of the target detection frame in the current frame image; Calculating the rotation direction of the photographing device based on the quadrant information of the target detection frame; The device tracking parameters include: turning speed; the current frame image includes a plurality of concentric ellipses of different levels; the preset turning speeds corresponding to the concentric ellipses of different levels are different; The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes: Determining, according to the coordinate information of the target detection frame, the target concentric ellipse levels corresponding to the target detection frame in the vertical direction and the horizontal direction respectively; Based on the target concentric ellipse levels corresponding to the target detection frame in the vertical direction and the horizontal direction, respectively, the turning speed of the shooting device in the vertical direction and the horizontal direction are calculated respectively.

2. The method according to claim 1, characterized in that: The position information of the target drone in the current frame image includes: the size information of the target detection frame; the device tracking parameters include: the shooting magnification; The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes: Based on the size information of the target detection frame, determining the proportion of the target detection frame in the current frame image; The shooting magnification of the shooting device is calculated according to the proportion of the target detection frame in the current frame image.

3. The method according to claim 1, characterized in that: The performing motion trajectory prediction based on the position information of the target UAV in the historical frame image to determine the predicted position information of the target UAV in the current frame image includes: Determine the position information of the target UAV in the historical frame image according to the tracking identifier of the target UAV; Based on the Kalman filter algorithm and the position information of the target UAV in the historical frame image, motion trajectory prediction is performed to determine the predicted position information of the target UAV in the current frame image.

4. A drone tracking device, characterized in that: include: An image acquisition module, used to acquire a current frame image and a historical frame image of a target UAV; the current frame image includes a plurality of objects to be identified; the historical frame image includes the target UAV; A position prediction module, used to predict the motion trajectory of the target UAV based on the position information of the target UAV in the historical frame image, so as to determine the predicted position information of the target UAV in the current frame image; a target drone determination module, configured to screen the target drone from a plurality of objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information, so as to determine the position information of the target drone in the current frame image; A tracking parameter determination module, used to determine the device tracking parameters according to the position information of the target UAV in the current frame image; A device control module, used to control a shooting device to perform real-time tracking and shooting of the target UAV based on the device tracking parameters; The step of screening the target UAV from a plurality of objects to be identified according to the position information of each object to be identified in the current frame image and the predicted position information to determine the position information of the target UAV in the current frame image includes: Based on the Hungarian algorithm, a target object to be identified that matches the predicted position information is selected from the position information of each object to be identified; Determine the target object to be identified as the target drone, and determine the position information of the target object to be identified as the position information of the target drone in the current frame image; The objects to be identified include: historical objects and non-historical objects; the historical objects are objects that have appeared in the historical frame images; the non-historical objects are objects that have only appeared in the current frame images; the objects to be identified have their own corresponding tracking identifiers; In the current frame image, the target UAV is surrounded by a target detection frame; The position information of the target drone in the current frame image includes: the coordinate information of the target detection frame; the device tracking parameters include: the rotation direction; the rotation direction includes: the vertical direction and the horizontal direction; The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes: Determining, according to the coordinate information of the target detection frame, the quadrant information of the target detection frame in the current frame image; Calculating the rotation direction of the photographing device based on the quadrant information of the target detection frame; The device tracking parameters include: turning speed; the current frame image includes a plurality of concentric ellipses of different levels; the preset turning speeds corresponding to the concentric ellipses of different levels are different; The determining of the device tracking parameters according to the position information of the target UAV in the current frame image includes: Determining, according to the coordinate information of the target detection frame, the target concentric ellipse levels corresponding to the target detection frame in the vertical direction and the horizontal direction respectively; Based on the target concentric ellipse levels corresponding to the target detection frame in the vertical direction and the horizontal direction, respectively, the turning speed of the shooting device in the vertical direction and the horizontal direction are calculated respectively.

5. An optoelectronic integrated device, characterized in that: It comprises an optoelectronic device and a gimbal device; the optoelectronic device is used to photograph a target drone, and the gimbal device is used to control the rotation direction, steering speed and shooting magnification of the optoelectronic device to implement the drone tracking method as described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the drone tracking method described in any one of claims 1 to 3 is implemented.

Images