Ground target measurement method based on AR and UAV monocular vision and its application

Abstract

The present invention belongs to the field of drone reconnaissance, and relates to a ground target measurement method based on AR and drone monocular vision and its application. The method comprises the following steps: the drone flies to a height of S1 meters from the ground target and hovers to obtain a first drone picture; the ground target is drawn in the first drone picture by using AR plotting technology, and the first length pixel and the first width pixel of the ground target in the first drone picture are measured; the drone is controlled to fly to a height of S2 meters from the ground target and hovers to obtain a second drone picture; the ground target is drawn in the second drone picture by using AR plotting technology, and the second length pixel and the second width pixel of the ground target in the second drone picture are measured; the geometric size of the ground target is calculated according to the measured data. The present invention can effectively improve the operability and measurement accuracy of the original monocular ranging method, and can perceive, quickly calculate, and accurately obtain the size of the ground reconnaissance target in advance.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned aerial vehicle (UAV) reconnaissance, and more specifically, relates to a ground target measurement method based on AR and unmanned aerial vehicle (UAV) monocular vision and its application. Background Art

[0002] A drone is a powered, controllable, and multi-task unmanned aircraft. In recent years, drones have been increasingly used in civilian applications, such as public safety, emergency search and rescue, agriculture and forestry, environmental protection, transportation, communications, meteorology, and aerial photography. The drone's gimbal camera uses monocular vision measurement technology from an aerial perspective to accurately measure the size of ground targets ahead, which can assist reconnaissance personnel in early warning and command.

[0003] In the prior art, CN107479059A discloses a device and method for measuring the distance between overhead lines and vegetation based on a drone, wherein a flight control system and a sensor detection integrated system are provided on the drone body, wherein the sensor detection integrated system is connected to a three-axis gimbal stabilization system, an imaging device, a two-dimensional 360° laser scanner, a large-capacity information storage unit, and an airborne second communication module. CN109978948A discloses a distance measurement method based on vision, which uses a mobile intelligent terminal to take a photo of the real scene space at N different viewing angles, and performs three-dimensional stereo reconstruction to obtain a three-dimensional stereo model of the real scene space, and then measures the distance between any two points in the real scene space. The above-mentioned visual measurement based on drones is limited to naked eye recognition of visible light photos, or a certain image processing is performed through binocular stereo images to realize the analysis of spatial distance. At the same time, the use of drones for distance measurement in existing patents usually requires more integrated drone airborne sensors and higher hardware costs. There are problems such as high hardware cost, complex processing process, and poor real-time performance, which cannot be applied to the real-time measurement and early warning of ground targets in combat environments. Summary of the invention

[0004] In view of the defects of the prior art, the purpose of the present invention is to provide a ground target measurement method based on AR and UAV monocular vision and its application, aiming to solve the problems of high hardware cost, complex processing process and poor real-time performance of the existing UAV vision measurement method.

[0005] To achieve the above object, according to one aspect of the present invention, a ground target measurement method based on AR and drone monocular vision is provided, the method comprising the following steps:

[0006] The S1 drone flies to a height of S1 meters from the ground target and hovers to obtain the first drone image;

[0007] S2 uses AR mapping technology to draw a ground target in the first drone picture, and measures a first length and a first width of the ground target in the first drone picture;

[0008] S3 controls the drone to fly to a height of S2 meters above the ground target and hovers to obtain the second drone image;

[0009] S4 uses AR mapping technology to draw the ground target in the second UAV picture, and measures the second length and second width of the ground target in the second UAV picture;

[0010] S5 calculates the geometric size of the ground target based on the data measured in step S2 and step S4.

[0011] As a further preferred embodiment, in step S1 and step S3, the pitch angle of the gimbal camera of the drone is maintained at 90 degrees.

[0012] As a further preferred embodiment, in step S1 and step S3, it is ensured that the ground target is located in the middle of the drone screen.

[0013] As a further preferred embodiment, in step S5, the calculation formula of the ground target geometric size is:

[0014]

[0015]

[0016] Where l is the actual length of the ground target, r is the focal length of the gimbal camera, m is the long frame of the gimbal camera, n is the short frame of the gimbal camera, a1 is the first length pixel of the ground target, a2 is the second length pixel of the ground target, M is the length pixel of the drone screen, w is the actual width of the ground target, b1 is the first width pixel of the ground target, b2 is the second width pixel of the ground target, and N is the width pixel of the drone screen.

[0017] More preferably, the relationship between the height S1 and the height S2 is 1 / 2 to 2 / 1.

[0018] As a further preference, in step S2 and step S4, the starting point of the length measurement is the left edge of the ground target, and the end point of the length measurement is the right edge of the ground target; the starting point of the width measurement is the upper edge of the ground target, and the end point of the width measurement is the lower edge of the ground target.

[0019] According to another aspect of the present invention, there is provided an application of the above-mentioned ground target measurement method based on AR and monocular vision of unmanned aerial vehicles in unmanned aerial vehicles.

[0020] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art:

[0021] 1. The present invention proposes to measure the geometric size of ground targets by combining UAV monocular vision with AR technology, and calculate the actual size by using the size of ground targets in UAV images at different heights, which can effectively improve the operability and measurement accuracy of the original monocular ranging method, and can perceive, quickly calculate, and accurately obtain the size of ground reconnaissance targets in advance;

[0022] 2. At the same time, the present invention ensures the accuracy of AR measurement line drawing by limiting the angle, position and height of the drone, thereby further improving the accuracy of ground target size measurement;

[0023] 3. In addition, the present invention also provides an application of a ground target measurement method based on AR and UAV monocular vision in a UAV, which can be adapted to the combat environment of a small UAV and improve the reliability of UAV reconnaissance. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of a ground target measurement method based on AR and UAV monocular vision provided by an embodiment of the present invention;

[0025] Figure 2 is a schematic diagram of a drone provided by an embodiment of the present invention at a height of S1 meters;

[0026] Figure 3 is a schematic diagram of a drone provided by an embodiment of the present invention at a height of S2 meters;

[0027] Throughout the drawings, the same reference numerals are used to denote the same elements or structures, wherein:

[0028] 1-UAV, 2-Camera frame, 3-Ground target. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0030] like Figures 1 to 3 As shown, the present invention proposes a ground target measurement method based on AR and unmanned aerial vehicle monocular vision, which includes the following steps:

[0031] S1 commands UAV 1 to fly to a height of S1 meters from the ground target and hover through the UAV ground station, switches the UAV state to enter the target measurement mode, and the ground station displays the UAV reconnaissance video in real time, adjusts the pitch angle of the gimbal camera to 90 degrees and ensures that the ground target 3 is located in the middle vertical direction of the UAV screen, and then obtains the first UAV screen. The focal length of the gimbal camera is r mm, and the camera frame 2 of the gimbal camera is m mm × n mm;

[0032] S2 uses the AR plotting technology to draw the ground target 3 in the first UAV screen through the touch screen of the ground station, and measures that the pixel size of the length measurement line of the ground target 3 in the first UAV screen is a1, and the pixel size of the width measurement line is b1;

[0033] S3 controls the drone to hover at a height of S2 meters from the ground target, keeps the pitch angle of the gimbal camera at 90 degrees and ensures that the ground target 3 is located in the middle vertical direction of the drone screen, and then obtains the second drone screen;

[0034] S4 uses the AR plotting technology to draw the ground target in the second UAV screen through the touch screen of the ground station, and measures that the pixel size of the length measurement line of the ground target 3 in the second UAV screen is a2, and the pixel size of the width measurement line is b2;

[0035] S5 calculates the geometric dimensions of the ground target 3 based on the data measured in steps S2 and S4.

[0036] Further, in step S5, the calculation process of the geometric size of the ground target 3 is:

[0037] l=l1×[1-l1 / (l1+l2)]+l2×[1-l2 / (l1+l2)]

[0038] w=w1×[1-w1 / (w1+w2)]+w2×[1-w2 / (w1+w2)]

[0039] in,

[0040]

[0041] Substituting the above formula into

[0042]

[0043]

[0044] Wherein, l is the actual length of the ground target, l1 is the first length of the ground target, l2 is the second length of the ground target, w is the actual width of the ground target, w1 is the first width of the ground target, w2 is the second width of the ground target, L1 is the first real length of the UAV screen, M is the length pixel of the UAV screen, a1 is the first length pixel of the ground target, L2 is the second real length of the UAV screen, a2 is the second length pixel of the ground target, W1 is the first real width of the UAV screen, W2 is the second real width of the UAV screen, r is the focal length of the gimbal camera, m is the long frame of the gimbal camera, and n is the short frame of the gimbal camera.

[0045] Furthermore, the relationship between the height S1 and the height S2 is 1 / 2 to 2 / 1, so as to avoid inaccurate measurement caused by large size differences of ground targets in the gimbal camera, and to facilitate drawing ground targets in the drone screen using AR mapping technology.

[0046] Furthermore, in step S2 and step S4, the starting point of the length measurement is the left edge of the ground target 3, and the end point of the length measurement is the right edge of the ground target; the starting point of the width measurement is the upper edge of the ground target 3, and the end point of the width measurement is the lower edge of the ground target.

[0047] The ground target measurement method based on AR and UAV monocular vision provided by the present invention adopts AR plotting technology to draw the ground target in the UAV picture. The actual length and width of the ground target can be obtained according to the frame of the gimbal camera, the pixels of the ground target in the UAV picture and the pixels of the UAV picture. It does not need to use too much supporting hardware, and the processing process is simple, so it has a wide range of applicability.

[0048] According to another aspect of the present invention, the application of the above-mentioned ground target measurement method based on AR and UAV monocular vision in a UAV is provided, which is suitable for small UAV combat environment, improves the reliability of UAV reconnaissance, and can perceive in advance, quickly calculate, and accurately obtain the size of ground reconnaissance targets.

[0049] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A ground target measurement method based on AR and UAV monocular vision, characterized in that: The method comprises the following steps: The S1 drone flies to a height of S1 meters from the ground target and hovers to obtain the first drone image; S2 uses AR mapping technology to draw a ground target in the first UAV picture, and measures a first length pixel and a first width pixel of the ground target in the first UAV picture; S3 controls the drone to fly to a height of S2 meters above the ground target and hovers to obtain the second drone image; S4 uses AR mapping technology to draw the ground target in the second UAV picture, and measures the second length pixel and the second width pixel of the ground target in the second UAV picture; S5 calculates the ground target geometric size using the following formula based on the data measured in step S2 and step S4: Where l is the actual length of the ground target, r is the focal length of the gimbal camera, m is the long frame of the gimbal camera, n is the short frame of the gimbal camera, a1 is the first length pixel of the ground target, a2 is the second length pixel of the ground target, M is the length pixel of the drone screen, w is the actual width of the ground target, b1 is the first width pixel of the ground target, b2 is the second width pixel of the ground target, and N is the width pixel of the drone screen; In step S1 and step S3, the pitch angle of the drone's gimbal camera is maintained at 90 degrees to ensure that the ground target is located in the middle of the drone's screen, and the relationship between the height S1 and the height S2 is 1 / 2 to 2 / 1.

2. The ground target measurement method based on AR and unmanned aerial vehicle monocular vision according to claim 1, characterized in that: The starting point of the length measurement is the left edge of the ground target, and the end point of the length measurement is the right edge of the ground target; the starting point of the width measurement is the upper edge of the ground target, and the end point of the width measurement is the lower edge of the ground target.

3. Application of the ground target measurement method based on AR and unmanned aerial vehicle monocular vision as described in claim 1 or 2 in unmanned aerial vehicles.

Images