Hovering unmanned aerial vehicle coordinate monitoring system and method based on image recognition and laser ranging

Abstract

The invention discloses a hovering unmanned aerial vehicle coordinate monitoring system and method based on image recognition and laser ranging, and solves the problems that the actual coordinates of a mooring unmanned aerial vehicle are difficult to accurately obtain in real time and the mooring unmanned aerial vehicle deviates from a target position due to long-term accumulation of errors in the prior art. The system comprises a laser ranging unit and a command control unit which are arranged on a to-be-monitored hovering unmanned aerial vehicle, and a horizontal deviation monitoring camera and a numerical control unit which are arranged on the ground, the laser ranging unit is in communication connection with the numerical control unit and is used for measuring the height of the to-be-monitored hovering unmanned aerial vehicle relative to the ground and sending the height to the numerical control unit; the horizontal deviation monitoring camera is arranged at a projection point of the to-be-monitored hovering unmanned aerial vehicle on the ground, is connected with the numerical control unit and is used for measuring pixel offset data of the to-be-monitored hovering unmanned aerial vehicle and sending the pixel offset data to the numerical control unit; the command control unit is connected with the numerical control unit and used for calculating the actual coordinates of the to-be-monitored hovering unmanned aerial vehicle and correcting the position of the to-be-monitored hovering unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to a hovering UAV coordinate monitoring system and method, and in particular to a hovering UAV coordinate monitoring system and method based on image recognition and laser ranging. Background Art

[0002] In the field of UAVs, multi-rotor UAVs have advantages such as simple structure and strong load-carrying capacity, and are widely used in scenarios such as agricultural and forestry detection, emergency rescue, and photography. However, due to the limitation of battery capacity, its flight time is greatly restricted, and the battery often needs to be replaced after flying for dozens of minutes. In some occasions of hovering or small-range free flight, such as emergency lighting, relay communication, fixed-point video recording, and high-altitude monitoring, the UAV often hovers at a fixed point after taking off, or freely flies within a small range. In such scenarios where large-scale maneuvers are not required, the tethered (hovering) UAV based on the tethered mode plays a great role. With continuous power supply from the ground tethered box, it can work continuously for dozens of hours or even more than a day.

[0003] Although tethered UAVs have been widely used in various scenarios in recent years and play an extremely important role in emergency rescue and other work, factors restricting their use still exist. The stability of the attitude of the tethered UAV during operation is an important factor to ensure its effectiveness, and it is particularly important to stabilize its position at high altitude. Currently, the determination of the coordinates of the tethered UAV mainly relies on the inertial navigation system and the GPS satellite navigation system. The cumulative error caused by long-term use of the inertial navigation, the interference and loss of GPS signals, etc. are all factors affecting the UAV's acquisition of its accurate coordinates. When the UAV cannot accurately obtain its real-time position or obtains incorrect position information, it is extremely easy to cause the UAV to deviate from the target position. In this case, it is difficult for the UAV to successfully complete the preset task, and in severe cases, it is extremely easy to break the tether cable until it makes an emergency landing or even is damaged. Therefore, in the field of tethered UAV operation, how to accurately obtain the position information of the UAV under interference conditions is an important challenge for the development of UAVs. Summary of the Invention

[0004] In order to solve the technical problems that it is difficult to obtain the actual coordinates of the tethered UAV in real time and accurately in the prior art, the long-term accumulation of errors causes the tethered UAV to deviate from the target position, making it difficult to successfully complete the preset task, and in severe cases, the tether cable is broken until it makes an emergency landing or even is damaged, the present invention provides a hovering UAV coordinate monitoring system and method based on image recognition and laser ranging.

[0005] The inventive concept of the present invention:

[0006] Image recognition technology has always been a relatively simple technology for determining the position of a target. On the premise of accurately calibrating the internal and external parameters of the camera, it can accurately give the position of the target in the image in the actual geographical space. The present invention considers applying this technology to the position-assisted determination of a hovering unmanned aerial vehicle (UAV), and uses it to determine the offset of the coordinates of the hovering UAV under the condition of satellite navigation signal rejection for correction. However, for a single camera, the image information obtained by shooting is not sufficient. Since it is impossible to determine the depth of field of the target, it is difficult to determine the accurate position of the hovering UAV, and only the coordinate position can be judged to be on a certain determined straight line. Although the binocular technology can accurately determine the position of the target, aerial calibration will bring more difficulties at this time.

[0007] Since the laser has advantages such as good monochromaticity and strong directivity, it can achieve non-contact long-distance measurement and is also applicable under night conditions. Considering the advantages of the laser ranging system, the present invention mounts it on a hovering UAV. By real-time monitoring the height of the hovering UAV relative to the ground, and on the premise of a given flight height of the hovering UAV, the above-mentioned problem that it is difficult for a single camera to determine the accurate position of the hovering UAV can be accurately solved through image recognition technology. In this way, the purpose of real-time calculation of the azimuth of the hovering UAV under the condition of no GPS can be achieved through image recognition and laser ranging technologies.

[0008] In order to achieve the above object and complete the above inventive concept, the present invention adopts the following technical solutions:

[0009] A hovering UAV coordinate monitoring system based on image recognition and laser ranging, which is characterized in that: it includes a laser ranging unit and a command and control unit arranged on the hovering UAV to be monitored, a horizontal deviation monitoring camera and a numerical control unit arranged on the ground;

[0010] The laser ranging unit is communicatively connected to the numerical control unit, and is used for measuring the height of the hovering UAV to be monitored relative to the ground and sending it to the numerical control unit;

[0011] The horizontal deviation monitoring camera is arranged at the projection point of the hovering UAV to be monitored on the ground and is connected to the numerical control unit, and is used for measuring the pixel offset data of the hovering UAV to be monitored and sending it to the numerical control unit;

[0012] The command and control unit is connected to the numerical control unit, and is used for calculating the actual coordinates of the hovering UAV to be monitored according to the height and pixel offset data, and correcting the position of the hovering UAV to be monitored based on the actual coordinates.

[0013] Furthermore, it further includes a remote control vehicle;

[0014] The horizontal deviation monitoring camera is mounted on the remote control vehicle.

[0015] Further, the numerical control unit is wirelessly communication-connected to the command and control unit.

[0016] Further, the laser ranging unit is wirelessly communication-connected to the numerical control unit.

[0017] Further, the horizontal deviation monitoring camera is wirelessly connected to the numerical control unit.

[0018] Further, the numerical control unit is arranged at the ground station.

[0019] A hovering UAV coordinate monitoring method based on image recognition and laser ranging, which adopts the above-mentioned hovering UAV coordinate monitoring system based on image recognition and laser ranging, is characterized in that it includes the following steps:

[0020] Step 1: Preset the target hovering position coordinates (0, 0, H0) of the hovering UAV to be monitored through the numerical control unit and send them to the command and control unit, where H0 is the height between the target hovering position and the ground;

[0021] Step 2: Command and control the hovering UAV to be monitored to fly to the target hovering position through the command and control unit;

[0022] Step 3: Set the horizontal deviation monitoring camera at the projection point of the hovering UAV to be monitored on the ground, so that the hovering UAV to be monitored captured by the horizontal deviation monitoring camera is located at the center of the image, and record its pixel coordinates as (x0, y0), and send them to the numerical control unit;

[0023] Step 4: Obtain the real-time image of the hovering UAV to be monitored in real time through the horizontal deviation monitoring camera, and obtain the pixel coordinates (x i , y i ) of the hovering UAV to be monitored in its real-time image, send them to the numerical control unit, and at the same time obtain the real-time height H i of the hovering UAV to be monitored corresponding to the real-time image through the laser ranging unit, and send it to the numerical control unit;

[0024] Step 5: The numerical control unit calculates the actual coordinates (X i , Y i ), Z i of the hovering UAV to be monitored based on the coordinates (0, 0, H0), (x0, y0), (x i , y i ) and the real-time height H i :

[0025] X i = (x i - x0) · H i · d x / f

[0026] Yi = (y i - y0) · H i · d y / f

[0027] Z i = H i

[0028] Wherein, d x and d y respectively represent the unit lengths of a pixel in the x and y directions, and f is the focal length of the horizontal offset monitoring camera;

[0029] Step 6: The numerical control unit calculates the offset of the hovering UAV to be monitored according to the actual coordinates (X i , Y i , Z i ) and the target hovering position coordinates (0, 0, H0);

[0030] Step 7: The numerical control unit compares the offset with a preset offset range;

[0031] If it is within the preset offset range, return to Step 4;

[0032] If it exceeds the preset offset range, send the offset to the command and control unit and execute Step 8;

[0033] Step 8: The command and control unit corrects the position of the hovering UAV to be monitored according to the received offset and returns to Step 4 until the hovering UAV finishes flying, completing the coordinate monitoring of the hovering UAV based on image recognition and laser ranging.

[0034] Furthermore, Step 3 is specifically:

[0035] The horizontal deviation monitoring camera is set at the projection point of the hovering UAV to be monitored on the ground through a remote control vehicle, so that the hovering UAV to be monitored captured by the horizontal deviation monitoring camera is located at the center of the image, and its pixel coordinates are recorded as (x0, y0) and sent to the command and control unit.

[0036] Furthermore, Steps 1, 3, and 4 all obtain corresponding coordinates based on a rectangular coordinate system.

[0037] Advantages of the present invention:

[0038] 1. The hovering UAV coordinate monitoring system and method based on image recognition and laser ranging provided by the present invention measure the height between the hovering UAV and the ground in real time through the laser ranging unit, and measure the offset of the hovering UAV on the horizontal plane in real time through the horizontal deviation monitoring camera. The numerical control unit calculates the relative position deviation between the hovering UAV and the preset target hovering position accordingly, so that the flight control system (command and control unit) can correct its position in real time, and can correct the actual coordinates of the hovering UAV in real time and accurately. The monitoring system has a simple structure and strong mobility.

[0039] 2. The hovering UAV coordinate monitoring system and method based on image recognition and laser ranging provided by the present invention realizes the positioning of the hovering UAV through the optical ranging and image recognition method, and can work under the conditions of GPS signal interference or even satellite navigation signal rejection, which provides a guarantee for the long-term safe and stable operation of the hovering UAV.

[0040] 3. The hovering UAV coordinate monitoring system and method based on image recognition and laser ranging provided by the present invention can monitor the offset of the hovering UAV in three-dimensional directions in real time and correct it, with simple calculation and strong operability.

[0041] 4. The hovering UAV coordinate monitoring system and method based on image recognition and laser ranging provided by the present invention is not only applicable to the hovering UAV working in the tethered mode, but also applicable to the hovering UAV hovering in the non-tethered mode. Description of the Drawings

[0042] Figure 1 is a schematic structural diagram of an embodiment of a hovering UAV coordinate monitoring system based on image recognition and laser ranging of the present invention;

[0043] Figure 2 is a flowchart of an embodiment of a hovering UAV coordinate monitoring method based on image recognition and laser ranging of the present invention.

[0044] Reference Numerals in the Drawings:

[0045] 1 - Hovering UAV to be monitored, 2 - Laser ranging unit, 3 - Horizontal deviation monitoring camera, 4 - Numerical control unit, 5 - Payload, 6 - Remote control vehicle. Detailed Embodiments

[0046] Next, the technical solutions of the present invention will be described clearly and completely in conjunction with the drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0047] A hovering UAV coordinate monitoring system based on image recognition and laser ranging provided by an embodiment of the present invention is as follows Figure 1 As shown in the figure, the hovering UAV coordinate monitoring system includes a laser ranging unit 2 and a command and control unit provided on the hovering UAV 1 to be monitored, a horizontal deviation monitoring camera 3, a remote control vehicle 6 and a numerical control unit 4 provided on the ground.

[0048] The hovering UAV 1 to be monitored is equipped with a payload 5.

[0049] The laser ranging unit 2 is wirelessly connected to the numerical control unit 4 and is used to measure the height of the hovering UAV 1 to be monitored relative to the ground and send it to the numerical control unit 4;

[0050] The horizontal deviation monitoring camera 3 is arranged at the projection point of the hovering UAV 1 to be monitored on the ground and is wirelessly connected to the numerical control unit 4. It is used to measure the pixel offset data of the hovering UAV 1 to be monitored and send it to the numerical control unit 4; the horizontal deviation monitoring camera 3 is carried on the remote control vehicle 6.

[0051] The numerical control unit 4 is wirelessly connected to the command and control unit and is used to calculate the actual coordinates of the hovering UAV 1 to be monitored according to the height and pixel offset data, and correct the position of the hovering UAV 1 to be monitored based on the actual coordinates to ensure that the hovering UAV 1 to be monitored stably works at the target hovering position. The numerical control unit 4 is arranged at the ground station.

[0052] Relying on this hovering UAV coordinate monitoring system can effectively solve the positioning problem during hovering operation under the condition of satellite navigation signal rejection, and provides an important support for the monitoring and position correction of the coordinate offset of the hovering UAV. It is not only applicable to UAVs working in the tethered mode, but also applicable to UAVs hovering in the untethered mode.

[0053] In this embodiment, the corresponding coordinates are obtained based on the rectangular coordinate system.

[0054] As Figure 2 shown, when the above-mentioned hovering UAV coordinate monitoring system based on image recognition and laser ranging is used for monitoring, it specifically includes the following steps:

[0055] Step 1: First, perform pre-flight preparations, that is, before takeoff, check and test the laser ranging unit of the hovering UAV 1 to be monitored and the ground station, the ground station, the horizontal deviation monitoring camera 3, and the remote control vehicle 6 to ensure that each component meets the takeoff conditions; then preset the target hovering position coordinates (0, 0, H0) of the hovering UAV 1 to be monitored through the numerical control unit 4 and send them to the command and control unit, where H0 is the height between the target hovering position and the ground;

[0056] Step 2: Command and control the hovering UAV 1 to be monitored to fly to the target hovering position through the command and control unit;

[0057] Under normal circumstances, when the satellite navigation signal is denied or even in an environment without satellite navigation signals at all, the hovering drone 1 to be monitored will deviate from the target hovering position, and this deviation is very likely to accumulate over time and needs to be continuously monitored and corrected.

[0058] Step 3: Use the remote control vehicle 6 to set the horizontal deviation monitoring camera 3 at the projection point of the hovering drone 1 to be monitored on the ground, so that the hovering drone 1 captured by the horizontal deviation monitoring camera 3 is located at the center of the image, and record its pixel coordinates as (x0, y0), and send them to the numerical control unit 4.

[0059] Step 4: Use the horizontal deviation monitoring camera 3 to obtain the real-time image of the hovering drone 1 to be monitored in real time, and obtain the pixel coordinates (x i , y i ) of the hovering drone 1 in its real-time image, and send them to the numerical control unit 4. At the same time, use the laser ranging unit 2 to obtain the real-time height H i of the hovering drone 1 corresponding to the real-time image, and send it to the numerical control unit 4;

[0060] Step 5: The numerical control unit 4 calculates the actual coordinates (X i , Y i ), Z i i of the hovering drone 1 to be monitored based on the coordinates (0, 0, H0), (x0, y0), (x i , y i ) and the real-time height H i :

[0061] X i = (x i - x0) · H i · d x / f

[0062] Y i = (y i - y0) · H i · d y / f

[0063] Z i = H i

[0064] Among them, d x , d y respectively represent the unit lengths of a pixel in the x and y directions, and f is the focal length of the horizontal deviation monitoring camera;

[0065] Step 6: The numerical control unit 4 determines according to the actual coordinates (X i , Y i , Z i) and the target hovering position coordinates (0, 0, H0) to calculate the offset (ΔX i , ΔY i , ΔZ i ) of the hovering UAV 1 to be monitored;

[0066] ΔX i = X i ;

[0067] ΔY i = Y i ;

[0068] ΔZ i = Z i - H0;

[0069] Step 7, the numerical control unit 4 compares the offset with the preset offset range;

[0070] If it is within the preset offset range, return to Step 4;

[0071] If it exceeds the preset offset range, send the offset to the command and control unit and execute Step 8;

[0072] Step 8, the command and control unit corrects the position of the hovering UAV 1 to be monitored according to the received offset so that it is within the preset offset range (±0.5 m), and returns to Step 4 until the hovering UAV finishes flying, completing the coordinate monitoring of the hovering UAV based on image recognition and laser ranging.

[0073] The system and method of this embodiment can effectively solve the problem of coordinate offset of the hovering UAV during long-term operation under GPS interference or even under complete satellite navigation signal rejection conditions, providing positioning accuracy guarantee for the long-term safe and reliable operation of the hovering UAV.

[0074] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A hovering UAV coordinate monitoring system based on image recognition and laser ranging, characterized in that: It includes a laser ranging unit (2) and a command and control unit provided on the hovering drone (1) to be monitored, a horizontal deviation monitoring camera (3) and a numerical control unit (4) provided on the ground; The laser ranging unit (2) is communicatively connected to the numerical control unit (4), and is used to measure the height of the hovering drone (1) to be monitored relative to the ground and send it to the numerical control unit (4); The horizontal deviation monitoring camera (3) is arranged at the projection point of the hovering drone (1) to be monitored on the ground and is connected to the numerical control unit (4), and is used to measure the pixel offset data of the hovering drone (1) to be monitored and send it to the numerical control unit (4); The command and control unit is connected to the numerical control unit (4), and is used to calculate the actual coordinates of the hovering drone (1) to be monitored according to the height and pixel offset data, and correct the position of the hovering drone (1) to be monitored based on the actual coordinates.

2. The hovering drone coordinate monitoring system based on image recognition and laser ranging according to claim 1, wherein: It further includes a remote control vehicle (6); The horizontal deviation monitoring camera (3) is carried on the remote control vehicle (6).

3. The hovering drone coordinate monitoring system based on image recognition and laser ranging according to claim 2, wherein: The numerical control unit (4) is wirelessly communicatively connected to the command and control unit.

4. The hovering drone coordinate monitoring system based on image recognition and laser ranging according to claim 3, wherein: The laser ranging unit (2) is wirelessly communicatively connected to the numerical control unit (4).

5. The hovering drone coordinate monitoring system based on image recognition and laser ranging according to claim 4, wherein: The horizontal deviation monitoring camera (3) is wirelessly connected to the numerical control unit (4).

6. The hovering drone coordinate monitoring system based on image recognition and laser ranging according to claim 5, wherein: The numerical control unit (4) is arranged at the ground station.

7. A hovering UAV coordinate monitoring method based on image recognition and laser ranging, which uses the hovering UAV coordinate monitoring system according to any one of claims 1-6, characterized in that, It includes the following steps: Step 1: Preset the target hovering position coordinates (0, 0, H0) of the hovering drone (1) to be monitored through the numerical control unit (4) and send them to the command and control unit, where H0 is the height between the target hovering position and the ground; Step 2: Command and control the hovering drone (1) to be monitored to fly to the target hovering position through the command and control unit; Step 3: Set the horizontal deviation monitoring camera (3) at the projection point of the hovering drone (1) to be monitored on the ground, so that the hovering drone (1) captured by the horizontal deviation monitoring camera (3) is located at the center of the image, and record its pixel coordinates as (x0, y0), and send them to the numerical control unit (4); Step 4: Use the horizontal deviation monitoring camera (3) to obtain the real-time image of the hovering drone (1) to be monitored in real time, and obtain the pixel coordinates (x i , y i ) of the hovering drone (1) to be monitored in its real-time image, and send them to the numerical control unit (4). At the same time, use the laser ranging unit (2) to obtain the real-time height H i of the hovering drone (1) corresponding to the real-time image, and send it to the numerical control unit (4); Step 5: The numerical control unit (4) calculates the actual coordinates (X i , Y i ) of the hovering UAV (1) to be monitored based on the coordinates (0, 0, H0), (x0, y0), (x i , y i ) and the real-time height H i : i ) X i = (x i - x0) · H i · d x / f Y i =(y i - y0)·H i ·d y / f Z i = H i where d x and d y represent the unit lengths of a pixel in the x and y directions respectively, and f is the focal length of the horizontal offset monitoring camera; Step 6. The numerical control unit (4) calculates the offset of the hovering UAV (1) to be monitored according to the actual coordinates (X i , Y i , Z i ) and the target hovering position coordinates (0, 0, H0); Step 7: The numerical control unit (4) compares the offset with the preset offset range; If it is within the preset offset range, return to Step 4; If it exceeds the preset offset range, send the offset to the command and control unit and execute Step 8; Step 8: The command and control unit corrects the position of the hovering drone (1) to be monitored according to the received offset and returns to Step 4 until the hovering drone flight ends, completing the hovering drone coordinate monitoring based on image recognition and laser ranging.

8. The hovering UAV coordinate monitoring method based on image recognition and laser ranging according to claim 7, characterized in that, Step 3 specifically is as follows: The horizontal deviation monitoring camera (3) is set at the projection point of the hovering UAV (1) to be monitored on the ground by means of the remote control vehicle (6), so that the hovering UAV (1) captured by the horizontal deviation monitoring camera (3) is located at the center of the image, and its pixel coordinates are recorded as (x0, y0) and sent to the command and control unit.

9. The hovering UAV coordinate monitoring method based on image recognition and laser ranging according to claim 8, characterized in that: Steps 1, 3, and 4 all obtain corresponding coordinates based on a rectangular coordinate system.

Images