A UAV panoramic image system, a UAV, and a method for generating a UAV panoramic image

Abstract

The present invention discloses a panoramic imaging system for unmanned aerial vehicles, an unmanned aerial vehicle, and a method for generating panoramic imaging of unmanned aerial vehicles. The panoramic imaging system for unmanned aerial vehicles comprises: a panoramic acquisition device, wherein the panoramic acquisition device comprises a camera for acquiring images of the surrounding environment of the unmanned aerial vehicle; an image processing module, wherein the image processing module is connected to the panoramic acquisition device and is used to generate a panoramic image of the surrounding environment of the unmanned aerial vehicle according to the images acquired by the camera; and an image display device, wherein the image display device is connected to the image processing module and is used to display a panoramic image of the surrounding environment of the unmanned aerial vehicle, thereby solving the technical problem in the related art that the unmanned aerial vehicle cannot observe the flight environment around itself.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to an unmanned aerial vehicle panoramic imaging system, an unmanned aerial vehicle, and a method for generating a panoramic image of an unmanned aerial vehicle. Background Art

[0002] At present, when a drone is flying, it can only observe the images captured by the onboard camera, but cannot check the flying environment around the drone. In this way, it is impossible to detect obstacles in the flying environment around the drone in time only by relying on the onboard camera.

[0003] Therefore, with respect to the technical problem that the above-mentioned UAV cannot observe the flying environment around itself, there is no effective solution in the relevant technology. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a UAV panoramic imaging system, a UAV and a method for generating UAV panoramic images, so as to solve the technical problem in the related art that the UAV cannot observe the flight environment around itself.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: According to one aspect of the present invention, a drone panoramic imaging system is provided, comprising:

[0006] A panoramic acquisition device, the panoramic acquisition device comprising a camera for acquiring an image of the surrounding environment of the drone;

[0007] An image processing module, which is connected to the panoramic acquisition device and is used to generate a panoramic image of the environment around the drone based on the image acquired by the camera;

[0008] An image display device is connected to the image processing module and is used to display a panoramic image of the environment surrounding the drone.

[0009] Optionally, the cameras include a first camera, a second camera, a third camera and a fourth camera arranged around the drone body in the horizontal direction, and a fifth camera and a sixth camera arranged above and below the drone body, respectively.

[0010] Optionally, the first camera and the second camera are respectively located on both sides of the head of the drone body; the third camera and the fourth camera are respectively located on both sides of the tail of the drone body.

[0011] Optionally, the camera is a wide-angle camera.

[0012] According to another aspect of the present invention, a drone is provided, comprising the above-mentioned drone panoramic imaging system.

[0013] According to another aspect of the present invention, a method for generating a panoramic image of an unmanned aerial vehicle is provided, using the above-mentioned panoramic image system of the unmanned aerial vehicle, and the generating method comprises:

[0014] Collect images of the drone’s surroundings;

[0015] generating a panoramic image of the environment surrounding the drone based on the image;

[0016] The panoramic image is displayed.

[0017] Optionally, collecting images of the environment surrounding the drone specifically includes:

[0018] Four images are collected through the first camera, the second camera, the third camera and the fourth camera around the drone body in the horizontal direction; two images are collected through the fifth camera and the sixth camera above and below the drone body;

[0019] Generating a panoramic image of the environment around the drone based on the image specifically includes:

[0020] By stitching and fusing the images, a panoramic image of the environment around the drone is obtained.

[0021] Optionally, the background image of the panoramic image is spherical; the center position of the sphere is the actual position of the drone.

[0022] Optionally, the method further includes:

[0023] According to the coordinates of the marking point of the image taken by the camera and the yaw angle, pitch angle and roll angle of the drone, the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image are calculated; wherein the marking point is the center point of the image taken by the camera.

[0024] Optionally, calculating the actual yaw angle, actual pitch angle, and actual roll angle of the panoramic image according to the coordinates of the mark points of the image captured by the camera and the yaw angle, pitch angle, and roll angle of the drone specifically includes:

[0025] According to the coordinates of the marking points on both sides of the head or tail of the drone, the difference in yaw angle between the sphere and the drone is calculated;

[0026] According to the coordinates of the marker points on the same side of the front and back of the drone, the difference in pitch angle between the sphere and the drone is calculated;

[0027] According to the coordinates of the marking points on both sides of the head or tail of the drone, the difference between the spherical shape and the drone’s roll angle is calculated;

[0028] According to the yaw angle, pitch angle, roll angle of the drone, and the yaw angle difference, pitch angle difference, and roll angle difference, the actual yaw angle, actual pitch angle, and actual roll angle of the panoramic image are calculated.

[0029] The present invention provides a UAV panoramic imaging system, a UAV and a method for generating a panoramic image of a UAV. An image acquisition device is provided for the UAV to acquire images of the flight environment around the UAV, and then the images are processed to generate a panoramic image of the environment around the UAV, so as to solve the technical problem in the related art that the UAV cannot observe the flight environment around itself. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the structure of a panoramic imaging system for unmanned aerial vehicles provided by an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of a top view structure of a drone provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of the main structure of a drone provided by an embodiment of the present invention;

[0033] Figure 4 A flowchart of a method for generating a panoramic image of an unmanned aerial vehicle provided by an embodiment of the present invention;

[0034] Figure 5 A flowchart of image processing steps in a method for generating panoramic images of an unmanned aerial vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0036] In the related art, during the flight of the drone, the user can only observe the images captured by the onboard camera, and cannot view the flight environment around the drone. Currently, there is no better solution.

[0037] Based on the above problems, the present invention proposes a UAV panoramic image system, a UAV and a method for generating a UAV panoramic image to solve the above technical problems in the related art.

[0038] Combination Figure 1-3An unmanned aerial vehicle provided by an embodiment of the present invention includes an unmanned aerial vehicle body 7, and the unmanned aerial vehicle body is equipped with an unmanned aerial vehicle panoramic imaging system. As an example, the unmanned aerial vehicle panoramic imaging system includes a panoramic acquisition device, an image processing module and an image display device, wherein the panoramic acquisition device includes a camera for acquiring images of the surrounding environment of the unmanned aerial vehicle, and the image processing module is connected to the panoramic acquisition device, and is used to generate a panoramic image of the surrounding environment of the unmanned aerial vehicle according to the images acquired by the camera. The image display device is connected to the image processing module, and is used to display a panoramic image of the surrounding environment of the unmanned aerial vehicle. The image display device can adopt various types of displays, which are installed in the unmanned aerial vehicle control console to display the panoramic bird's-eye view. In some embodiments, it can also be used to realize the dynamic linkage between the unmanned aerial vehicle posture and the panoramic view.

[0039] As an example, the panoramic acquisition device includes six cameras, namely, a first camera 1, a second camera 2, a third camera 3 and a fourth camera 4, which are arranged around the drone body in the horizontal direction, and a fifth camera 5 and a sixth camera 6, which are arranged above and below the drone body, respectively. Since the camera acquisition range includes not only the surroundings of the horizontal range, but also the top and bottom of the drone, after splicing and fusing the images collected by the six cameras, a 720° panoramic image of the flight environment around the drone can be obtained.

[0040] Specifically, the first camera 1 and the second camera 2 are respectively located on both sides of the head of the drone body, and are used to collect images of the front end environment of the drone body. The third camera 3 and the fourth camera 4 are respectively located on both sides of the tail of the drone body, and are used to collect images of the rear end environment of the drone body.

[0041] In the embodiment of the present invention, each camera may adopt a wide-angle camera to capture images of a sufficiently large range, so that a more complete and accurate panoramic image can be formed after the images are stitched together.

[0042] Based on the above-mentioned drone panoramic imaging system, the embodiment of the present invention also provides a corresponding drone panoramic imaging generation method, combined with Figure 4 , the method comprising:

[0043] Step S101, collecting images of the surrounding environment of the drone;

[0044] Step S103, generating a panoramic image of the surrounding environment of the drone according to the image;

[0045] Step S105: displaying the panoramic image.

[0046] As an example, step S101 specifically includes:

[0047] Four images are collected by the first camera 1, the second camera 2, the third camera 3 and the fourth camera 4 around the drone body in the horizontal direction; two images are collected by the fifth camera 5 and the sixth camera 6 above and below the drone body.

[0048] Furthermore, step S103 specifically includes:

[0049] By stitching and fusing the images, a panoramic image of the environment around the drone is obtained.

[0050] Combination Figure 5 In step S103, the six images taken by each camera can be stitched together, the center point of the image can be marked as the relative position of the drone, and then the images can be stitched together, that is, the feature points of the overlapping areas can be matched for stitching and fusion, and finally a panoramic image can be obtained.

[0051] Optionally, the background image of the panoramic image is spherical; the center position of the sphere is the actual position of the drone; and the method further includes:

[0052] Step S107, according to the coordinates of the marking point of the image taken by the camera and the yaw angle, pitch angle and roll angle of the drone, the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image are calculated; wherein the marking point is the center point of the image taken by the camera. After obtaining the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image, the image display device can be used to rotate the sphere according to the actual yaw angle, actual pitch angle and actual roll angle with the center of the spherical background model of the panoramic image as the origin, so as to realize the dynamic linkage between the drone posture and the panoramic view.

[0053] Optionally, step S107 specifically includes:

[0054] Step S1071, calculating the yaw angle difference between the sphere and the drone according to the coordinates of the marking points on both sides of the head or the tail of the drone;

[0055] Step S1072, calculating the pitch angle difference between the sphere and the drone according to the coordinates of the marking points on the same side of the front and rear of the drone;

[0056] Step S1073, calculating the roll angle difference between the sphere and the drone according to the coordinates of the marking points on both sides of the drone's head or tail;

[0057] Step S1074, calculating the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image according to the yaw angle, pitch angle and roll angle of the drone, as well as the yaw angle difference, pitch angle difference and roll angle difference.

[0058] For example, suppose the yaw angle of the drone is Yaw, the pitch angle is Pitch, and the roll angle is Roll. The center point of the image taken by the first camera is marker point 1, and its spherical coordinates are (x1, y1, z1). The center point of the image taken by the second camera is marker point 2 (x2, y2, z2), and so on for 6 marker points.

[0059] According to the coordinates of the marker points on both sides of the drone's head or tail, the yaw angle difference between the sphere and the drone is calculated as _yaw. Taking marker point 1 and marker point 2 as an example (marker point 3 and marker point 4 can also be used), the atan2 function can be called to calculate the yaw angle difference between the sphere and the drone, that is:

[0060] _yaw=atan2(y2-y1,x2-x1) / 2

[0061] According to the coordinates of the marker points on the same side of the front and back of the drone, the pitch angle difference between the sphere and the drone is calculated.

[0062] Taking marker 1 and marker 4 as an example (marker 2 and marker 3 can also be used), the atan2 function can be called to calculate the difference in pitch angle between the sphere and the drone, that is:

[0063] _pitch=atan2(z4-z1,sqrt((x4-x1)^2+(y4-y1)^2))

[0064] According to the coordinates of the marker points on both sides of the drone's head or tail, calculate the roll angle difference between the sphere and the drone _roll. Taking marker point 1 and marker point 2 as an example (marker point 3 and marker point 4 can also be used), you can call the atan2 function to calculate the roll angle difference between the sphere and the drone, that is:

[0065] _roll=atan2(y2-y1,z2-z1).

[0066] In this way, the actual yaw angle of the panoramic model is Yaw+_yaw, the pitch angle is Pitch+_pitch, and the roll angle is Roll+_roll. According to the calculation results of the angles, the drone attitude and the panoramic model can be linked.

[0067] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0068] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A drone panoramic imaging system, characterized in that: include: A panoramic acquisition device, the panoramic acquisition device comprising a camera for acquiring an image of the surrounding environment of the drone; An image processing module, which is connected to the panoramic acquisition device and is used to generate a panoramic image of the environment around the drone based on the image acquired by the camera; An image display device, which is connected to the image processing module and is used to display a panoramic image of the surrounding environment of the drone; The collecting of images of the environment surrounding the drone specifically includes: Four images are collected through the first camera, the second camera, the third camera and the fourth camera around the drone body in the horizontal direction; two images are collected through the fifth camera and the sixth camera above and below the drone body; Generating a panoramic image of the environment around the drone based on the image specifically includes: The images are stitched and fused to obtain a panoramic image of the environment surrounding the drone; The background image of the panoramic image is spherical; the center position of the sphere is the actual position of the drone; Also includes: According to the coordinates of the marking point of the image taken by the camera and the yaw angle, pitch angle and roll angle of the drone, the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image are calculated; wherein the marking point is the center point of the image taken by the camera; The calculating the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image according to the coordinates of the mark points of the image captured by the camera and the yaw angle, pitch angle and roll angle of the drone specifically includes: According to the coordinates of the marking points on both sides of the head or tail of the drone, the difference in yaw angle between the sphere and the drone is calculated; According to the coordinates of the marker points on the same side of the front and back of the drone, the difference in pitch angle between the sphere and the drone is calculated; According to the coordinates of the marking points on both sides of the head or tail of the drone, the difference between the spherical shape and the drone’s roll angle is calculated; Calculate the actual yaw angle, actual pitch angle, and actual roll angle of the panoramic image according to the yaw angle, pitch angle, and roll angle of the drone, as well as the yaw angle difference, pitch angle difference, and roll angle difference; Taking the center of the spherical background model of the panoramic image as the origin, the sphere is rotated at corresponding angles according to the actual yaw angle, actual pitch angle and actual roll angle.

2. The drone panoramic imaging system according to claim 1, characterized in that: The camera is a wide-angle camera.

3. A drone, characterized in that: The invention comprises the unmanned aerial vehicle panoramic imaging system as described in any one of claims 1-2.

4. A method for generating a panoramic image of an unmanned aerial vehicle, characterized in that: Using the drone panoramic imaging system described in any one of claims 1-2, the generation method comprises: Collect images of the drone’s surroundings; generating a panoramic image of the environment surrounding the drone based on the image; Displaying the panoramic image; The collecting of images of the environment surrounding the drone specifically includes: Four images are collected through the first camera, the second camera, the third camera and the fourth camera around the drone body in the horizontal direction; two images are collected through the fifth camera and the sixth camera above and below the drone body; Generating a panoramic image of the environment around the drone based on the image specifically includes: The images are stitched and fused to obtain a panoramic image of the environment surrounding the drone; The background image of the panoramic image is spherical; the center position of the sphere is the actual position of the drone; The method further comprises: According to the coordinates of the marking point of the image taken by the camera and the yaw angle, pitch angle and roll angle of the drone, the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image are calculated; wherein the marking point is the center point of the image taken by the camera; The calculating the actual yaw angle, actual pitch angle and actual roll angle of the panoramic image according to the coordinates of the mark points of the image captured by the camera and the yaw angle, pitch angle and roll angle of the drone specifically includes: According to the coordinates of the marking points on both sides of the head or tail of the drone, the difference in yaw angle between the sphere and the drone is calculated; According to the coordinates of the marker points on the same side of the front and back of the drone, the difference in pitch angle between the sphere and the drone is calculated; According to the coordinates of the marking points on both sides of the head or tail of the drone, the difference between the spherical shape and the drone’s roll angle is calculated; Calculate the actual yaw angle, actual pitch angle, and actual roll angle of the panoramic image according to the yaw angle, pitch angle, and roll angle of the drone, as well as the yaw angle difference, pitch angle difference, and roll angle difference; Taking the center of the spherical background model of the panoramic image as the origin, the sphere is rotated at corresponding angles according to the actual yaw angle, actual pitch angle and actual roll angle.

Images