Space target positioning method based on aerial real-time image

Abstract

The application provides a space target positioning method based on aerial real-time images, which utilizes a visual pod with laser ranging function to obtain the distance between a target object and a UAV and a real-time image, simultaneously obtains real-time parameters of the UAV, and adopts a plane positioning method to calculate the height and longitude and latitude of the target object based on the real-time parameters of the UAV, the distance between the target object and the UAV and the real-time image. The application can realize the three-dimensional coordinate positioning of the target through one image, does not need to construct a three-dimensional map, and has a simple positioning process, so that the daily aerial photography task of the UAV can be easily met.

Description

Technical Field

[0001] This invention relates to the field of UAV vision technology, and in particular to a spatial target localization method based on real-time aerial images. Background Technology

[0002] With the development of small drone technology and 5G communication in recent years, civilian small drones have been widely used, which has also put forward higher requirements for the functions of drones. Existing drone remote control technology, image transmission technology, and data transmission technology have been developed to a relatively mature level. Target detection and recognition technology has emerged one after another with the development of computer vision technology. At the same time, the accuracy and efficiency issues that are of great concern in target detection have also been well resolved.

[0003] However, the real-time positioning of targets after detection and recognition in drone aerial images still needs improvement. At present, image-based target 3D positioning methods are mainly used in point cloud image acquisition and 3D map construction. These algorithms require powerful computing power and pre-built maps, making them difficult to apply to real-time drone reconnaissance images where time efficiency is critical. This results in complex implementation and low real-time performance. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a spatial target localization method based on real-time aerial images, which solves the problems of complex implementation and low real-time performance in real-time reconnaissance images from UAVs where time efficiency is critical.

[0005] This invention provides a spatial target localization method based on real-time aerial images, comprising the following steps:

[0006] S1. The UAV acquires real-time images based on a vision pod with laser ranging capabilities, and obtains the distance P2 between the target and the UAV based on the vision pod.

[0007] S2. Obtain the real-time parameters of the UAV, which include at least the flight altitude H, the pitch angle alfa of the visual pod, the yaw angle belta of the visual pod, the lateral field of view Wangle of the visual pod, the longitudinal field of view Hangle, the image center coordinates (uc, vc), and the focal length f.

[0008] S3. Select the pixel coordinates (u,v) of the target object and the vertical projection pixel coordinates (u1,v1) of the target object in the real-time image screen respectively;

[0009] S4. Calculate the height difference Hp between the target and the visual pod of the UAV using the distance P2 and the vertical deflection angle of the target relative to the UAV, and obtain the height Ht of the target based on the flight altitude of the UAV and the height difference.

[0010] S5. Using a planar positioning method, the latitude and longitude of the target object are calculated by combining the height difference Hp, the pixel coordinates (u,v) of the target object, and the real-time parameters.

[0011] Optionally, the vertical deflection angle of the target relative to the UAV includes:

[0012] The vertical deflection angle of the UAV is calculated based on the pitch angle alfa of the visual pod, the pixel coordinates (u, v) of the target object, the image center coordinates (uc, vc), the size of each pixel in the image, and the focal length f. The formula for calculating the vertical deflection angle is as follows:

[0013] gama=-alfa+actan((v-vc)*Yp / f)

[0014] Where Yp is the width of each pixel in the image.

[0015] Optionally, the size of each pixel in the image includes:

[0016] The pixel dimensions include pixel length Xp and pixel width Yp. The pixel length Xp is calculated by the focal length f and the lateral field of view Wangle of the visual pod. The formula for calculating the pixel length Xp is as follows:

[0017] Xp = 2 * f * tan(Wangle / 2) / number of pixel columns

[0018] The pixel width Yp is calculated using the focal length f and the vertical field of view Hangle. The formula for calculating the pixel width Yp is as follows:

[0019] Yp=2*f*tan(Hangle / 2) / number of pixel rows

[0020] Optionally, the pixel width Yp includes:

[0021] The pixel width Yp is adjusted by the vertical resolution adjustment parameter Ys to form a new pixel width Yp, wherein the vertical resolution adjustment parameter Ys is 0.835.

[0022] Optionally, calculating the height difference Hp between the target and the visual pod of the UAV using the distance P2 and the vertical deflection angle of the target relative to the UAV includes:

[0023] Determine whether the vertical projection pixel coordinates (u1, v1) of the target object and the image center coordinates (uc, vc) are the same point. If they are the same point, the formula for calculating the height difference Hp is:

[0024] Hp = P2 * sin(gama)

[0025] If not, then the formula for calculating the height difference Hp is:

[0026] Hp = P2 * cos(alfa) * tan(gama)

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] This invention is based on a vision pod with laser ranging capabilities. It can acquire and measure the distance between the target and the drone, as well as real-time images. It also acquires the drone's real-time parameters and uses a planar positioning method to calculate the target's height and latitude and longitude based on the drone's real-time parameters, the distance between the target and the drone, and the real-time images. This invention can achieve three-dimensional coordinate positioning of the target in a single image. The positioning process is simple and can easily meet the daily aerial photography tasks of drones. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 The diagram shown is a flowchart of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Functional units with the same reference numerals in the examples of this invention have the same and similar structures and functions.

[0033] See Figure 1This invention provides a spatial target localization method based on real-time aerial images, comprising the following steps:

[0034] S1. The UAV acquires real-time images based on a vision pod with laser ranging capabilities, and obtains the distance P2 between the target and the UAV based on the vision pod.

[0035] S2. Obtain the real-time parameters of the UAV, which include at least the flight altitude H, the pitch angle alfa of the visual pod, the yaw angle belta of the visual pod, the lateral field of view Wangle of the visual pod, the longitudinal field of view Hangle, the image center coordinates (uc, vc), and the focal length f.

[0036] S3. Select the pixel coordinates (u,v) of the target object and the vertical projection pixel coordinates (u1,v1) of the target object in the real-time image screen respectively;

[0037] S4. Calculate the height difference Hp between the target and the visual pod of the UAV using the distance P2 and the vertical deflection angle of the target relative to the UAV, and obtain the height Ht of the target based on the flight altitude of the UAV and the height difference.

[0038] S5. Using a planar positioning method, the latitude and longitude of the target object are calculated by combining the height difference Hp, the pixel coordinates (u,v) of the target object, and the real-time parameters.

[0039] In this embodiment, a vision pod with laser ranging capability is used to acquire the distance between the target and the drone, as well as real-time images. Simultaneously, real-time parameters of the drone are acquired. A planar positioning method is used to calculate the altitude and latitude / longitude of the target based on the drone's real-time parameters, the distance between the target and the drone, and the real-time images. This invention can achieve three-dimensional coordinate positioning of the target in a single image, without the need to construct a three-dimensional map. The positioning process is simple and can easily meet the daily aerial photography tasks of drones.

[0040] Optionally, the vertical deflection angle of the target relative to the UAV includes:

[0041] The vertical deflection angle of the UAV is calculated based on the pitch angle alfa of the visual pod, the pixel coordinates (u, v) of the target object, the coordinates of the image center (uc, vc), the size of each pixel in the image, and the focal length f. The formula for calculating the vertical deflection angle is as follows:

[0042] gama=-alfa+actan((v-vc)*Yp / f)

[0043] Where Yp is the width of each pixel in the image.

[0044] Optionally, the size of each pixel in the image includes:

[0045] The pixel dimensions include pixel length Xp and pixel width Yp. The pixel length Xp is calculated by the focal length f and the lateral field of view Wangle of the visual pod. The formula for calculating the pixel length Xp is as follows:

[0046] Xp = 2 * f * tan(Wangle / 2) / number of pixel columns

[0047] The pixel width Yp is calculated using the focal length f and the vertical field of view Hangle. The formula for calculating the pixel width Yp is as follows:

[0048] Yp=2*f*tan(Hangle / 2) / number of pixel rows

[0049] In this embodiment, the real-time image resolution acquired by the vision pod is 1920x1080, so the number of pixel columns is 1920 and the number of pixel rows is 1080.

[0050] Optionally, the pixel width Yp includes:

[0051] The pixel width Yp is adjusted by the vertical resolution adjustment parameter Ys to form a new pixel width Yp, wherein the vertical resolution adjustment parameter Ys is 0.835.

[0052] In this embodiment, for the aforementioned real-time image with a resolution of 1920x1080, the width of the pixels is adjusted. The new pixel width Yp is obtained by multiplying the pixel width Yp by the vertical resolution adjustment parameter Ys, so as to replace the previous pixel width Yp, that is, Yp(new) = Yp(old) * Ys. The vertical resolution adjustment parameter Ys is a coefficient obtained from test experimental data, which is beneficial to improving positioning accuracy.

[0053] Optionally, calculating the height difference Hp between the target and the visual pod of the UAV using the distance P2 and the vertical deflection angle of the target relative to the UAV includes:

[0054] Determine whether the vertical projection pixel coordinates (u1, v1) of the target object and the image center coordinates (uc, vc) are the same point. If they are the same point, the formula for calculating the height difference Hp is:

[0055] Hp = P2 * sin(gama)

[0056] If not, then the formula for calculating the height difference Hp is:

[0057] Hp = P2 * cos(alfa) * tan(gama)

[0058] In this embodiment, the real-time image fed back by the vision pod can be fed back to the display unit in the background. The controller in the background can click on the real-time image on the display unit to determine the pixel coordinates (u,v) of the target object and the pixel coordinates (u1,v1) of the target object's projection in the real-time image. The vertical projection pixel coordinates (u1,v1) of the target object are usually the center coordinates (uc,vc) of the image. If the vertical projection pixel coordinates (u1,v1) of the target object are not the center point, the vertical projection pixel coordinates (u1,v1) of the target object can be used as the center point for calculation. The deviation between the pixel coordinates (u,v) of the target object and the pixel coordinates of the center can be replaced by the deviation between the pixel coordinates (u1,v1) of the target object and the vertical projection pixel coordinates (u1,v1).

[0059] If the vertical projection pixel coordinates (u1, v1) of the target object and the image center coordinates (uc, vc) are the same point, then the formula for calculating the height difference Hp is:

[0060] Hp = P2 * sin(gama)

[0061] If not, for example, if the vertical projection pixel coordinates (u1, v1) of the target object are on the building surface, meaning the vertical projection pixel coordinates (u1, v1) of the target object and the image center coordinates (uc, vc) are on the same wall but not at the same point, then the formula for calculating the height difference Hp is:

[0062] Hp = P2 * cos(alfa) * tan(gama)

[0063] In fact, the two formulas are the same. If the ranging point and the target point are the same, then gama = alfa; otherwise, they are different. Then, based on the real-time altitude H of the UAV, the altitude Ht of the target object can be calculated.

[0064] Ht = H - Hp

[0065] The calculation process for the latitude and longitude of the target object is as follows:

[0066] Calculate the longitudinal deflection angle Vgama of the target object relative to its center point, i.e.:

[0067] Vgama = actan((v-vc)*Yp / f)

[0068] Calculate the component Y of the distance between the target and the UAV along the longitudinal axis of the pod, i.e.:

[0069] Y = H / tan(gama)

[0070] Calculate the focal length Px of the pixels in the same row as the target object in the image, that is:

[0071] Px = f / cos(vgama)

[0072] Calculate the horizontal deflection angle Delta of the target object, i.e.:

[0073] Delta = actan((u-uc)*Xp / Px)

[0074] Calculate the component X of the distance between the target and the UAV along the horizontal axis of the pod, i.e.:

[0075] X = Y * tan(Delta)

[0076] Calculate the distance P between the target object and the UAV's projection on the ground (i.e., the projection of the target object), that is:

[0077] P = sqrt(X*X + Y*Y)

[0078] Calculate the heading angle (coureAngle) of the target deviating from the direction of the visual pod, i.e.:

[0079] coureAngle=tan(X / Y)

[0080] Using the latitude and longitude (lon,lat), Earth radius r, UAV heading angle airplaneCoa, visual pod yaw angle beltta, and the distance P between the target and the UAV's ground projection, the heading angle between the UAV and the target, bring = airplaneCoa + beltta + coureAngle, can be used to calculate the latitude and longitude (lon2,lat2) of the target. The calculation formula is as follows:

[0081] lat2=acsin(sin(lat)*cos(distance / r)+cos(lat)*sin(distance / r)*cos(bring))

[0082] lon2=(lon+actan2(sin(bring)*sin(distance / r)*cos(lat),cos(distance / r)-sin(lat)*sin(lat2)))

[0083] Spatial positioning of the target object is achieved based on its latitude and longitude (lon2, lat2) and its height Ht.

[0084] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0085] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A spatial target localization method based on real-time aerial images, characterized in that, Includes the following steps: S1. The UAV acquires real-time images based on a vision pod with laser ranging capabilities, and obtains the distance P2 between the target and the UAV based on the vision pod. S2. Obtain the real-time parameters of the UAV, which include at least the flight altitude H, the pitch angle alfa of the visual pod, the yaw angle belta of the visual pod, the lateral field of view Wangle of the visual pod, the longitudinal field of view Hangle, the image center coordinates (uc, vc), and the focal length f. S3. Select the target pixel coordinates (u,v) and the vertical projection pixel coordinates (u1,v1) of the target in the real-time image. S4. Calculate the height difference Hp between the target and the UAV's visual pod using the distance P2 and the vertical deflection angle of the target relative to the UAV. Obtain the target's height Ht based on the UAV's flight altitude and the height difference. Calculate the UAV's vertical deflection angle based on the visual pod's pitch angle alfa, the target's pixel coordinates (u, v), the image center coordinates (uc, vc), the size of each pixel in the image, and the focal length f. The formula for calculating the vertical deflection angle is: gama=-alfa+actan((v-vc)*Yp / f) Where Yp is the width of each pixel in the image frame; S5. Using a planar positioning method, the latitude and longitude of the target object are calculated by combining the height difference Hp, the pixel coordinates (u,v) of the target object, and the real-time parameters.

2. The spatial target localization method based on real-time aerial images according to claim 1, characterized in that, The size of each pixel in the image includes: The pixel dimensions include pixel length Xp and pixel width Yp. The pixel length Xp is calculated by the focal length f and the lateral field of view Wangle of the visual pod. The formula for calculating the pixel length Xp is as follows: Xp = 2 * f * tan(Wangle / 2) / number of pixel columns The pixel width Yp is calculated using the focal length f and the vertical field of view Hangle. The formula for calculating the pixel width Yp is as follows: Yp=2*f*tan(Hangle / 2) / number of pixel rows.

3. The spatial target localization method based on real-time aerial images according to claim 2, characterized in that, The pixel width Yp includes: The pixel width Yp is adjusted by the vertical resolution adjustment parameter Ys to form a new pixel width Yp, wherein the vertical resolution adjustment parameter Ys is 0.

835.

4. The spatial target localization method based on real-time aerial images according to claim 3, characterized in that, The calculation of the height difference Hp between the target and the visual pod of the UAV using the distance P2 and the vertical deflection angle of the target relative to the UAV includes: Determine whether the vertical projection pixel coordinates (u1, v1) of the target object and the image center coordinates (uc, vc) are the same point. If they are the same point, the formula for calculating the height difference Hp is: Hp = P2 * sin(gama) If not, then the formula for calculating the height difference Hp is: Hp = P2 * cos(alfa) * tan(gama).

Images