Unmanned aerial vehicle based scene three-dimensional reconstruction method, device and equipment

Abstract

The application discloses a scene three-dimensional reconstruction method, device and equipment based on a UAV, comprising: when the UAV is controlled to be above the center position of a scene in a to-be-photographed area, setting the surrounding flight track parameters of the UAV according to the average height value obtained in advance; setting the gimbal angle according to the surrounding flight track parameters; calculating all the aerial photograph points of the UAV according to the preset aerial photograph point quantity and the surrounding flight radius; when it is judged that the current photographing time obtained in advance is in the preset daytime period, controlling the UAV to perform surrounding flight according to the preset flight speed and the surrounding flight track parameters, and photographing the scene in the to-be-photographed area at each aerial photograph point according to the gimbal angle; obtaining a plurality of to-be-reconstructed images photographed by the UAV, so as to perform three-dimensional reconstruction on the scene in the to-be-photographed area. The embodiment of the application can guarantee the comprehensiveness and authenticity of the reconstructed disaster scene information, and provides strong support for disaster relief command decision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle, and in particular to a scene three-dimensional reconstruction method, device and equipment based on unmanned aerial vehicle. BACKGROUND

[0002] At present, a large number of ground two-dimensional images can be obtained by unmanned aerial vehicle, and a three-dimensional model of a disaster scene can be restored by three-dimensional reconstruction of the photographed disaster scene, which can provide data support for disaster relief command decision-making. However, the traditional scene three-dimensional reconstruction method based on unmanned aerial vehicle generally only controls the unmanned aerial vehicle to take pictures above the disaster scene, and does not consider the influence of the flight height and the angle of the camera lens turning downward from the horizontal line on the photographed scene content. When the angle is too small, two situations occur: if the flight height is too high, a large area of sky image will be photographed, resulting in too few image feature points, which cannot be used for three-dimensional reconstruction of the disaster scene; if the flight height is too low, the top of the building cannot be photographed, resulting in incomplete information of the three-dimensional reconstructed disaster scene; when the angle is too large, the reconstructed disaster scene information has a large height error, which affects the authenticity of the disaster scene information. SUMMARY

[0003] The present application provides a scene three-dimensional reconstruction method, device and equipment based on unmanned aerial vehicle, which can photograph comprehensive building and height information, realize three-dimensional reconstruction of the disaster scene, ensure the completeness and authenticity of the reconstructed disaster scene information, and provide strong support for disaster relief command decision-making.

[0004] To achieve the above-mentioned purpose, the present application provides a scene three-dimensional reconstruction method based on unmanned aerial vehicle, comprising:

[0005] obtaining the average height value of the buildings in the scene of the area to be photographed;

[0006] setting the orbit flight track parameters of the unmanned aerial vehicle according to the average height value; wherein the orbit flight track parameters include orbit flight height and orbit flight radius;

[0007] setting the gimbal angle of the onboard camera of the unmanned aerial vehicle according to the orbit flight track parameters;

[0008] calculating all the aerial photograph points of the unmanned aerial vehicle according to the preset number of aerial photograph points and the orbit flight radius;

[0009] When it is judged that the pre-acquired current shooting time is in the preset daytime time period, the unmanned aerial vehicle is controlled to perform a surrounding flight above the scene to be shot according to a preset flight speed and the surrounding flight track parameter with the preset position as the center, and the scene to be shot is shot at each aerial shot point according to the gimbal angle.

[0010] The plurality of images to be reconstructed shot by the unmanned aerial vehicle are obtained for three-dimensional reconstruction of the scene to be shot.

[0011] As an improvement of the above-mentioned solution, the setting of the surrounding flight track parameter of the unmanned aerial vehicle according to the average height value comprises:

[0012] The surrounding flight height of the unmanned aerial vehicle is set according to a preset ratio range of the surrounding flight height to the average height value and the average height value, wherein the ratio range of the surrounding flight height to the average height value is greater than or equal to 2 and less than or equal to 3.

[0013] The surrounding flight radius of the unmanned aerial vehicle is set according to a preset ratio range of the surrounding flight height to the surrounding flight radius and the surrounding flight height, wherein the ratio range of the surrounding flight height to the surrounding flight radius is greater than or equal to and less than or equal to .

[0014] As an improvement of the above-mentioned solution, the setting of the surrounding flight radius of the unmanned aerial vehicle according to the preset ratio range of the surrounding flight height to the surrounding flight radius and the surrounding flight height comprises:

[0015] The average floor-to-floor distance value of the building of the scene to be shot is obtained.

[0016] The size relationship between the surrounding flight height and the surrounding flight radius is determined according to the average floor-to-floor distance value.

[0017] The surrounding flight radius of the unmanned aerial vehicle is set according to the preset ratio range of the surrounding flight height to the surrounding flight radius and the size relationship.

[0018] As an improvement of the above-mentioned solution, the setting of the gimbal angle of the onboard camera of the unmanned aerial vehicle according to the surrounding flight track parameter comprises:

[0019] The result obtained by substituting the surrounding flight height and the surrounding flight radius into a preset trigonometric function relationship is set as the gimbal angle.

[0020] The trigonometric function relationship is as follows:

[0021]

[0022] wherein H is the circling flight height, and R is the circling flight radius.

[0023] As an improvement of the above solution, the all aerial photograph points of the UAV are calculated according to the preset aerial photograph point quantity and the circling flight radius, comprising:

[0024] The circling flight mileage of the UAV is calculated according to the circling flight radius;

[0025] The circling flight mileage is divided by the preset aerial photograph point quantity to obtain the all aerial photograph points of the UAV.

[0026] As an improvement of the above solution, the several to-be-reconstructed images photographed by the UAV are obtained for three-dimensional reconstruction of the to-be-photographed region scene, comprising at least one of the following:

[0027] The several to-be-reconstructed images photographed by the UAV are obtained, and the to-be-photographed region scene is three-dimensionally reconstructed at a local end;

[0028] The several to-be-reconstructed images photographed by the UAV are obtained, and the several to-be-reconstructed images are sent to a background end or a cloud end, and the to-be-photographed region scene is three-dimensionally reconstructed at the background end or the cloud end.

[0029] As an improvement of the above solution, the to-be-photographed region scene is three-dimensionally reconstructed by the following steps:

[0030] Feature points are extracted from each to-be-reconstructed image, and similarity matching is performed on adjacent to-be-reconstructed images according to the extracted feature points to obtain several matching pairs;

[0031] Feature points are matched in all the matching pairs by a feature matching algorithm, and erroneous feature points are filtered therefrom;

[0032] According to the filtered matching pairs, a global motion recovery structure algorithm is used to obtain a camera pose of each to-be-reconstructed image, and each to-be-reconstructed image is simultaneously subjected to a distortion removal process to obtain each distortion-removed image;

[0033] Each depth map of each distortion-removed image is generated, all the depth maps are fused according to each camera pose to obtain a three-dimensional point cloud of the to-be-photographed region scene;

[0034] The three-dimensional point cloud is subjected to a triangular partitioning process to obtain a coarse grid of the to-be-photographed region scene, and the coarse grid is refined by using image gradient differences of adjacent depth maps to obtain a fine grid of the to-be-photographed region scene;

[0035] Texture mapping is performed on the fine grid to obtain a three-dimensional model of the scene of the to-be-photographed region.

[0036] As an improvement of the above scheme, the scene three-dimensional reconstruction method based on the unmanned aerial vehicle further comprises:

[0037] When it is judged that the current photographing time is in the preset night time period, the unmanned aerial vehicle is controlled to perform a surrounding flight above the scene of the to-be-photographed region with a preset position as the center, at a preset flight speed and according to the surrounding flight track parameter, and to perform photographing on the scene of the to-be-photographed region at each of the aerial photographing points according to a preset night light sensitivity and the gimbal angle within a preset time length of hovering.

[0038] To achieve the above object, the embodiment of the present application further provides a scene three-dimensional reconstruction device based on an unmanned aerial vehicle, comprising:

[0039] an average height value acquisition module configured to acquire an average height value of a building of a scene of a to-be-photographed region;

[0040] a surrounding flight track parameter setting module configured to set a surrounding flight track parameter of the unmanned aerial vehicle according to the average height value, wherein the surrounding flight track parameter comprises a surrounding flight height and a surrounding flight radius;

[0041] a gimbal angle setting module configured to set a gimbal angle of an on-board camera of the unmanned aerial vehicle according to the surrounding flight track parameter;

[0042] an aerial photographing point acquisition module configured to calculate all aerial photographing points of the unmanned aerial vehicle according to a preset number of aerial photographing points and the surrounding flight radius;

[0043] a first to-be-photographed region photographing module configured to, when it is judged that a previously acquired current photographing time is in a preset day time period, control the unmanned aerial vehicle to perform a surrounding flight above the scene of the to-be-photographed region with a preset position as the center, at a preset flight speed and according to the surrounding flight track parameter, and to perform photographing on the scene of the to-be-photographed region at each of the aerial photographing points according to the gimbal angle;

[0044] a three-dimensional reconstruction module configured to obtain a plurality of to-be-reconstructed images photographed by the unmanned aerial vehicle, for three-dimensional reconstruction of the scene of the to-be-photographed region.

[0045] To achieve the above object, the embodiment of the present application further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the scene three-dimensional reconstruction method based on the unmanned aerial vehicle as described above.

[0046] Compared with the prior art, the unmanned aerial vehicle based scene three-dimensional reconstruction method, device and equipment provided by the embodiment of the present application can ensure that the unmanned aerial vehicle captures comprehensive and real building information, guarantee the comprehensiveness and authenticity of the reconstructed disaster scene information, reduce the flight route and flight time through the surrounding flight shooting of the scene to be shot, thereby reducing the modeling time, and quickly reconstructing the scene of the shooting area in three dimensions to provide strong support for disaster relief command decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of the unmanned aerial vehicle based scene three-dimensional reconstruction method provided by the embodiment of the present application;

[0048] Figure 2 is a schematic diagram of the relationship between the surrounding flight height, the surrounding flight radius and the gimbal angle provided by the embodiment of the present application.

[0049] Figure 3 is a structural block diagram of the unmanned aerial vehicle based scene three-dimensional reconstruction device provided by the embodiment of the present application;

[0050] Figure 4 is a structural block diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0052] Referring to Figure 1 , Figure 1 is a flowchart of the unmanned aerial vehicle based scene three-dimensional reconstruction method provided by the embodiment of the present application, and the unmanned aerial vehicle based scene three-dimensional reconstruction method comprises:

[0053] S1, acquiring an average height value of a building of a scene to be shot;

[0054] S2, setting a surrounding flight trajectory parameter of the unmanned aerial vehicle according to the average height value; wherein the surrounding flight trajectory parameter comprises a surrounding flight height and a surrounding flight radius;

[0055] S3, setting a gimbal angle of an onboard camera of the unmanned aerial vehicle according to the surrounding flight trajectory parameter;

[0056] S4, calculating all the aerial photograph points of the UAV according to the preset aerial photograph point quantity and the surrounding flight radius;

[0057] S5, when judging that the current shooting time obtained in advance is in the preset daytime time period, controlling the UAV to perform surrounding flight above the scene to be shot in the preset position as the center, according to the preset flight speed and the surrounding flight track parameter, and shooting the scene to be shot at each aerial photograph point according to the holder angle;

[0058] S6, obtaining several images to be reconstructed shot by the UAV, for three-dimensional reconstruction of the scene to be shot.

[0059] It can be understood that in step S1, the average height value can be obtained by consulting the relevant information of the buildings in the scene to be shot, or the UAV can be controlled to fly above the center position of the scene to be shot, at this time the current flight height of the UAV needs to be greater than the height of the highest building in the scene to be shot, and the average height value is obtained by using the image of the scene to be shot shot by the UAV on-board camera. Preferably, the current flight height is the sum of the height value of the highest building in the scene to be shot and a preset initial height value, and the initial height value is 100 meters.

[0060] Specifically, step S2 of setting the surrounding flight track parameter of the UAV according to the average height value comprises:

[0061] setting the surrounding flight height of the UAV according to the ratio range of the preset surrounding flight height and the average height value and the average height value; wherein the ratio range of the surrounding flight height and the average height value is greater than or equal to 2 and less than or equal to 3;

[0062] It can be understood that generally, the surrounding flight height is set to 3 times the average height value, and when the height difference of the buildings is relatively large, the surrounding flight height is set to 2 times the average height value. For example, when the average height value of the buildings is less than 20 meters and the scene is open, the surrounding flight height is set to 60 meters, which can be appropriately lowered; when the average height value of the buildings is 35 meters, the surrounding flight height is set to 100 meters, which can be appropriately lowered; when the average height value of the buildings is 80 and there is a low floor, the surrounding flight height is set to 160 meters.

[0063] setting the surrounding flight radius of the UAV according to the ratio range of the preset surrounding flight height and the surrounding flight radius and the surrounding flight height; wherein the ratio range of the surrounding flight height and the surrounding flight radius is greater than or equal to and less than or equal to .

[0064] Specifically, the setting of the circling flight radius of the unmanned aerial vehicle according to the preset ratio range of the circling flight height to the circling flight radius and the circling flight height comprises:

[0065] obtaining an average floor-to-floor distance value of the buildings in the scene to be photographed;

[0066] determining a size relationship between the circling flight height and the circling flight radius according to the average floor-to-floor distance value;

[0067] setting the circling flight radius of the unmanned aerial vehicle according to the preset ratio range of the circling flight height to the circling flight radius and the size relationship.

[0068] It can be understood that the average floor-to-floor distance value can be obtained by consulting relevant materials of the buildings in the scene to be photographed, or the unmanned aerial vehicle can be controlled to fly above the center position of the scene to be photographed, and the average floor-to-floor distance value can be obtained by using the image of the scene to be photographed captured by the onboard camera of the unmanned aerial vehicle.

[0069] When the average floor-to-floor distance of the buildings is large, i.e., the average floor-to-floor distance value is greater than a preset distance, the size relationship is that the circling flight radius is greater than the circling flight height; when the average floor-to-floor distance value is equal to the distance, the size relationship is that the circling flight radius is equal to the circling flight height; when the average floor-to-floor distance of the buildings is narrow, i.e., the average floor-to-floor distance value is less than the distance, the size relationship is that the circling flight radius is less than the circling flight height; after the size relationship between the circling flight radius and the circling flight height is determined, the circling flight radius is set within a ratio range of the circling flight height to the circling flight radius, the ratio range is greater than or equal to and less than or equal to Moreover, circling flight is performed under the circling flight height and the circling flight radius provided in the embodiment of the present application, which can ensure the safety of the unmanned aerial vehicle and prevent the unmanned aerial vehicle from crashing.

[0070] Specifically, the setting of the gimbal angle of the onboard camera of the unmanned aerial vehicle according to the circling flight trajectory parameter in step S3 comprises:

[0071] the result obtained by substituting the circling flight height and the circling flight radius into a preset trigonometric function relationship is set as the gimbal angle ;

[0072] The trigonometric function relationship is:

[0073]

[0074] In the formula, H is the circling flight height, and R is the circling flight radius.

[0075] In this embodiment of the invention, - indicates that the camera lens of the drone rotates downwards from the horizontal line, and the gimbal angle in this embodiment of the invention... The angle at which a drone's camera lens rotates downwards from the horizontal line is the angle formed by the straight line passing through the center of the lens and the horizontal plane.

[0076] It is understandable that the orbital flight altitude, orbital flight radius, and gimbal angle have some relationship. Figure 2 The relationship shown indicates that the gimbal angle and orbital flight altitude affect the acquisition of scene information in the area to be photographed. This embodiment of the invention utilizes... Setting the relationship between the orbital flight altitude, orbital flight radius, and gimbal angle allows the scene to be reconstructed to appear within the effective acquisition range of the camera as much as possible, resulting in a more comprehensive and realistic 3D model of the area to be photographed.

[0077] When the ratio of the orbital flight altitude to the orbital flight radius is greater than or equal to and less than or equal to At this time, the absolute value of the gimbal angle is greater than or equal to 30° and less than or equal to 60°. Within this range, more comprehensive and realistic scene information can be captured, ensuring the reconstruction effect. More specifically, when the average building spacing is greater than the preset spacing distance and the orbital flight radius is greater than the orbital flight altitude, the absolute value of the gimbal angle is greater than or equal to 30° and less than 45°; when the average building spacing is equal to the preset spacing distance and the orbital flight radius is equal to the orbital flight altitude, the absolute value of the gimbal angle is equal to 45°; when the average building spacing is less than the preset spacing distance and the orbital flight radius is less than the orbital flight altitude, the absolute value of the gimbal angle is greater than 45° and less than or equal to 60°.

[0078] Specifically, step S4, which calculates all the aerial photography points of the UAV based on the preset number of aerial photography points and the orbital flight radius, includes:

[0079] The orbital flight distance of the UAV is calculated based on the orbital flight radius.

[0080] Divide the orbital flight distance by the preset number of aerial photography points to obtain all the aerial photography points of the drone.

[0081] Preferably, the number of aerial photograph points is 16. In general, a photograph is taken at every 16th point of the circumnavigation flight route. The 16 photographs can make the reconstructed scene meet the needs of the emergency field. The number of images used for three-dimensional reconstruction is small, which can quickly construct the three-dimensional model of the scene of the to-be-photographed region and improve the three-dimensional reconstruction efficiency. In the embodiment of the present application, the circumnavigation flight distance of the unmanned aerial vehicle is divided into 16 equal parts to obtain 16 aerial photograph points. A photograph is taken at each aerial photograph point. At this time, the photographing frequency can be set. The aerial photograph points are divided by the preset flight speed to obtain the photographing frequency. In fact, the scene of the to-be-photographed region can also be reconstructed by taking a photograph at every 6th point of the circumnavigation flight route. At a flight height of 100 meters, 12 photographs or even 10 photographs can reconstruct a scene that meets the needs of the emergency field.

[0082] It can be understood that in step S5, the position is preferably the center position of the scene of the to-be-photographed region, and the flight speed is preferably the maximum flight speed of 10 m / s. When the current photographing time is in the preset daytime period, the unmanned aerial vehicle is controlled to fly around the scene of the to-be-photographed region with the center position of the scene of the to-be-photographed region as the center, at a flight speed of 10 m / s and according to the circumnavigation flight trajectory parameters, and to take photographs of the scene of the to-be-photographed region at each aerial photograph point according to the gimbal angle. The daytime sensitivity during photographing is automatically set by the onboard camera, which is not limited here. In the embodiment of the present application, the to-be-reconstructed image can be obtained by circumnavigating the to-be-photographed region once, which shortens the flight route, reduces the flight time, reduces the modeling time, and improves the efficiency.

[0083] Specifically, the step S6 of obtaining the plurality of to-be-reconstructed images taken by the unmanned aerial vehicle for three-dimensional reconstruction of the scene of the to-be-photographed region comprises at least one of the following:

[0084] obtaining the plurality of to-be-reconstructed images taken by the unmanned aerial vehicle, and performing three-dimensional reconstruction of the scene of the to-be-photographed region locally;

[0085] obtaining the plurality of to-be-reconstructed images taken by the unmanned aerial vehicle, sending the plurality of to-be-reconstructed images to a background end or a cloud end, and performing three-dimensional reconstruction of the scene of the to-be-photographed region at the background end or the cloud end.

[0086] It can be understood that the plurality of to-be-reconstructed images taken by the unmanned aerial vehicle obtained locally can be directly three-dimensionally reconstructed locally, or can be sent to the cloud end or the background end for three-dimensional reconstruction.

[0087] Specifically, the scene of the to-be-photographed region is three-dimensionally reconstructed by the following steps:

[0088] extract feature points from each of the to-be-reconstructed images, and perform similarity matching on adjacent to-be-reconstructed images according to the extracted feature points to obtain a plurality of matching pairs;

[0089] perform feature point matching on all the matching pairs through a feature matching algorithm, and filter out erroneous feature points therefrom;

[0090] obtain camera poses of each of the to-be-reconstructed images through a global structure from motion algorithm according to the filtered matching pairs, and simultaneously perform distortion removal on each of the to-be-reconstructed images to obtain each distortion-removed image;

[0091] generate each depth map of each of the distortion-removed images, and fuse all the depth maps according to each of the camera poses to obtain a three-dimensional point cloud of the to-be-photographed region scene;

[0092] perform triangulation processing on the three-dimensional point cloud to obtain a coarse mesh of the to-be-photographed region scene, and perform refinement on the coarse mesh using image gradient differences of adjacent depth maps to obtain a fine mesh of the to-be-photographed region scene;

[0093] perform texture mapping on the fine mesh to obtain a three-dimensional model of the to-be-photographed region scene.

[0094] It can be understood that three-dimensional reconstruction is performed through the following steps at a local end, a background end, or a cloud end:

[0095] extract feature points from each of the to-be-reconstructed images, and perform similarity matching on adjacent to-be-reconstructed images according to the extracted feature points as sparse feature information to obtain a plurality of matching pairs;

[0096] perform feature point matching through a feature matching algorithm to obtain relative positions and poses of the matching pairs, and simultaneously filter out erroneous feature points in the matching pairs;

[0097] perform global position and pose recovery of images through a global structure from motion algorithm (GSfM) to obtain camera poses of each of the to-be-reconstructed images; and simultaneously perform distortion removal on each of the to-be-reconstructed images by optimizing intrinsic parameters of each of the to-be-reconstructed images to obtain each best distortion-removed image.

[0098] calculate each depth map of each of the distortion-removed images using a multi-view depth map calculation method, and fuse the depth maps to obtain a three-dimensional point cloud of the to-be-photographed region scene;

[0099] Delaunay network and classification are performed on the three-dimensional point cloud to obtain a rough grid, and the gradient difference of adjacent depth maps is used to refine the rough grid to optimize the grid smoothness and detail expression, and a fine grid is obtained.

[0100] The fine grid is textured by a multi-view texture mapping method to obtain the final textured grid model data, i.e., the three-dimensional model of the scene to be photographed.

[0101] In an optional embodiment, the scene three-dimensional reconstruction method based on the unmanned aerial vehicle further comprises:

[0102] When it is judged that the current photographing time is within the preset night time period, the unmanned aerial vehicle is controlled to fly around according to the preset flight speed and the surrounding flight trajectory parameter, and to hover at each aerial photographing point for a preset time length to photograph the scene to be photographed according to the preset night sensitivity and the gimbal angle.

[0103] Preferably, the night sensitivity is 1600, the time length is 5 seconds, the position is the center position of the scene to be photographed, and the flight speed is 10 m / s.

[0104] In the embodiment, when the current photographing time is within the preset night time period, the night sensitivity is set to 1600, the unmanned aerial vehicle is controlled to fly around above the scene to be photographed with the center position of the scene to be photographed as the center and at a flight speed of 10 m / s and the surrounding flight trajectory parameter, and to hover at each aerial photographing point for 5 seconds to photograph.

[0105] The scene three-dimensional reconstruction method based on the unmanned aerial vehicle provided in the embodiment can set the surrounding flight trajectory parameter of the unmanned aerial vehicle by the average height value of the building, and set the gimbal angle of the onboard camera of the unmanned aerial vehicle by the surrounding flight trajectory parameter, so that the unmanned aerial vehicle can photograph comprehensive and real building information, and the comprehensiveness and authenticity of the reconstructed disaster scene information are ensured, the photographing of the scene to be photographed by surrounding flight reduces the flight route and shortens the flight time, thereby reducing the modeling time, and the scene to be photographed can be quickly three-dimensionally reconstructed to provide strong support for disaster rescue command decision-making in a timely manner.

[0106] Referring to Figure 2 , Figure 2 is a structural block diagram of a scene three-dimensional reconstruction device 10 based on an unmanned aerial vehicle provided in the embodiment, the scene three-dimensional reconstruction device 10 based on the unmanned aerial vehicle comprises:

[0107] The average height value acquisition module 11 is configured to acquire the average height value of the building of the scene to be photographed.

[0108] The orbit flight trajectory parameter setting module 12 is configured to set orbit flight trajectory parameters of the UAV according to the average height value, wherein the orbit flight trajectory parameters include an orbit flight height and an orbit flight radius.

[0109] The gimbal angle setting module 13 is configured to set a gimbal angle of an onboard camera of the UAV according to the orbit flight trajectory parameters.

[0110] The aerial photography point acquisition module 14 is configured to calculate all aerial photography points of the UAV according to a preset number of aerial photography points and the orbit flight radius.

[0111] The first to-be-photographed region scene photographing module 15 is configured to, when it is judged that the pre-acquired current photographing time is within a preset daytime period, control the UAV to perform orbit flight above the to-be-photographed region scene with a preset position as the center, at a preset flight speed and according to the orbit flight trajectory parameters, and photograph the to-be-photographed region scene at each aerial photography point according to the gimbal angle.

[0112] The three-dimensional reconstruction module 16 is configured to obtain a plurality of to-be-reconstructed images photographed by the UAV, for three-dimensional reconstruction of the to-be-photographed region scene.

[0113] Preferably, the setting of the orbit flight trajectory parameters of the UAV according to the average height value comprises:

[0114] setting the orbit flight height of the UAV according to a preset ratio range of the orbit flight height to the average height value and the average height value, wherein the ratio range of the orbit flight height to the average height value is greater than or equal to 2 and less than or equal to 3.

[0115] setting the orbit flight radius of the UAV according to a preset ratio range of the orbit flight height to the orbit flight radius and the orbit flight height, wherein the ratio range of the orbit flight height to the orbit flight radius is greater than or equal to and less than or equal to .

[0116] Preferably, the setting of the orbit flight radius of the UAV according to the preset ratio range of the orbit flight height to the orbit flight radius and the orbit flight height comprises:

[0117] acquiring an average floor-to-floor distance value of buildings of the to-be-photographed region scene;

[0118] determining a size relationship between the orbit flight height and the orbit flight radius according to the average floor-to-floor distance value.

[0119] According to the preset ratio range of the surrounding flight height and the surrounding flight radius and the size relationship, the surrounding flight radius of the unmanned aerial vehicle is set.

[0120] Preferably, the gimbal angle of the onboard camera of the unmanned aerial vehicle according to the surrounding flight trajectory parameter is set by:

[0121] The result obtained by substituting the surrounding flight height and the surrounding flight radius into a preset trigonometric function relationship is set as the gimbal angle.

[0122] The trigonometric function relationship is:

[0123]

[0124] In the formula, H is the surrounding flight height, and R is the surrounding flight radius.

[0125] Preferably, all the aerial photography points of the unmanned aerial vehicle are calculated according to the preset number of aerial photography points and the surrounding flight radius, including:

[0126] The surrounding flight mileage of the unmanned aerial vehicle is calculated according to the surrounding flight radius.

[0127] The surrounding flight mileage is divided by the preset number of aerial photography points to obtain all the aerial photography points of the unmanned aerial vehicle.

[0128] Preferably, the plurality of to-be-reconstructed images photographed by the unmanned aerial vehicle are obtained for three-dimensional reconstruction of the to-be-photographed region scene, including at least one of the following:

[0129] The plurality of to-be-reconstructed images photographed by the unmanned aerial vehicle are obtained, and the three-dimensional reconstruction of the to-be-photographed region scene is performed at a local end.

[0130] The plurality of to-be-reconstructed images photographed by the unmanned aerial vehicle are obtained, and the plurality of to-be-reconstructed images are sent to a background end or a cloud end, and the three-dimensional reconstruction of the to-be-photographed region scene is performed at the background end or the cloud end.

[0131] Preferably, the three-dimensional reconstruction of the to-be-photographed region scene is performed by:

[0132] Feature points are extracted from each to-be-reconstructed image, and similarity matching is performed on adjacent to-be-reconstructed images according to the extracted feature points to obtain a plurality of matching pairs.

[0133] Feature points are matched in all the matching pairs by a feature matching algorithm, and incorrect feature points are filtered out.

[0134] According to the filtered matching pairs, camera poses of each of the to-be-reconstructed images are obtained by using a global motion structure-from-motion algorithm, and each of the to-be-reconstructed images is simultaneously subjected to a de-distortion process to obtain each de-distorted image;

[0135] Each depth map of each of the de-distorted images is generated, and all the depth maps are fused according to each of the camera poses to obtain a three-dimensional point cloud of the to-be-photographed region scene;

[0136] The three-dimensional point cloud is subjected to a triangulation process to obtain a coarse mesh of the to-be-photographed region scene, and the coarse mesh is subjected to refinement by using image gradient differences of adjacent depth maps to obtain a fine mesh of the to-be-photographed region scene;

[0137] The fine mesh is subjected to a texture mapping process to obtain a three-dimensional model of the to-be-photographed region scene.

[0138] Preferably, the scene three-dimensional reconstruction device based on a UAV further comprises:

[0139] The second to-be-photographed region scene photographing module is configured to, when it is judged that the current photographing time is within a preset night time period, control the UAV to perform a circular flight above the to-be-photographed region scene with a preset position as a center, at a preset flight speed and according to the circular flight track parameters, and to perform photographing of the to-be-photographed region scene at each of the aerial photographing points within a preset time length, at a preset night light sensitivity and according to the gimbal angle.

[0140] It is worth noting that the working processes of the various modules in the scene three-dimensional reconstruction device 10 based on a UAV according to the embodiments of the present application can refer to the working processes of the scene three-dimensional reconstruction method based on a UAV described in the above embodiments, and will not be described herein again.

[0141] The scene three-dimensional reconstruction device 10 based on a UAV provided by the embodiments of the present application sets the circular flight track parameters of the UAV by using the average height value of the building, and sets the gimbal angle of the on-board camera of the UAV by using the circular flight track parameters, so that the UAV can photograph comprehensive and real building information, and the comprehensiveness and reality of the reconstructed disaster site scene information are ensured. The photographing of the to-be-photographed region scene by the circular flight reduces the flight route and shortens the flight time, thereby reducing the modeling time, and the to-be-photographed region scene can be quickly three-dimensionally reconstructed, and strong support can be provided for disaster relief command decision-making in a timely manner.

[0142] Referring to Figure 4 , Figure 4is a structural block diagram of an electronic device 20 provided by an embodiment of the present application, and the electronic device 20 includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. The processor 21 implements the steps in the above-mentioned embodiment of the method for three-dimensional reconstruction of a scene based on a UAV when executing the computer program. Alternatively, the processor 21 implements the functions of each module / unit in the above-mentioned embodiments of the device when executing the computer program.

[0143] For example, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device 20.

[0144] The electronic device 20 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device 20 can include, but is not limited to, the processor 21 and the memory 22. Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 20, and does not limit the electronic device 20, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the electronic device 20 can also include an input / output device, a network access device, a bus, and the like.

[0145] The processor 21 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor 21 is a control center of the electronic device 20, and connects each part of the entire electronic device 20 through various interfaces and lines.

[0146] The memory 22 can be used to store the computer programs and / or modules, and the processor 21 realizes various functions of the electronic device 20 by running or executing the computer programs and / or modules stored in the memory 22, and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 22 can include a high-speed random access memory, and can also include a nonvolatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0147] The modules / units integrated in the electronic device 20 are stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor 21 executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0148] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separate, and the units shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0149] The above is the preferred embodiment of the present application. It should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements are also considered within the scope of protection of the present application.

Claims

1. A method for three-dimensional reconstruction of a scene based on a UAV, characterized in that, The method comprises the following steps: obtaining an average height value of buildings in a scene of a region to be photographed; setting a circling flight track parameter of the unmanned aerial vehicle according to the average height value, wherein the circling flight track parameter comprises a circling flight height and a circling flight radius; setting a gimbal angle of an on-board camera of the unmanned aerial vehicle according to the circling flight track parameter; calculating all aerial photograph points of the unmanned aerial vehicle according to a preset number of aerial photograph points and the circling flight radius; when it is judged that a current photographing time obtained in advance is within a preset daytime period, controlling the unmanned aerial vehicle to perform circling flight above the scene of the region to be photographed with a preset position as the center, at a preset flight speed and according to the circling flight track parameter, and to perform photographing of the scene of the region to be photographed at each aerial photograph point according to the gimbal angle; obtaining a plurality of images to be reconstructed photographed by the unmanned aerial vehicle, for three-dimensional reconstruction of the scene of the region to be photographed; wherein the setting of the circling flight track parameter of the unmanned aerial vehicle according to the average height value comprises: setting the circling flight height of the unmanned aerial vehicle according to a preset ratio range of the circling flight height to the average height value and the average height value, wherein the ratio range of the circling flight height to the average height value is greater than or equal to 2 and less than or equal to 3; According to the preset ratio range of the surrounding flight height and the surrounding flight radius and the surrounding flight height, the surrounding flight radius of the unmanned aerial vehicle is set; wherein the ratio range of the surrounding flight height and the surrounding flight radius is greater than or equal to and less than or equal to the setting of the gimbal angle of the on-board camera of the unmanned aerial vehicle according to the circling flight track parameter comprises: setting the gimbal angle as a result calculated by substituting the circling flight height and the circling flight radius into a preset trigonometric function relationship; wherein the trigonometric function relationship is: wherein H is the circling flight height and R is the circling flight radius. 2.The UAV-based scene 3D reconstruction method of claim 1, wherein, the setting of the circling flight radius of the unmanned aerial vehicle according to the preset ratio range of the circling flight height to the circling flight radius and the circling flight height comprises: obtaining an average floor spacing value of buildings in the scene of the region to be photographed; determining a size relationship between the circling flight height and the circling flight radius according to the average floor spacing value; setting the circling flight radius of the unmanned aerial vehicle according to the preset ratio range of the circling flight height to the circling flight radius and the size relationship. 3.The UAV-based scene 3D reconstruction method of claim 1, wherein, the calculation of all aerial photograph points of the unmanned aerial vehicle according to the preset number of aerial photograph points and the circling flight radius comprises: calculating a circling flight mileage of the unmanned aerial vehicle according to the circling flight radius; dividing the circling flight mileage by the preset number of aerial photograph points to obtain all aerial photograph points of the unmanned aerial vehicle. 4.The UAV-based scene 3D reconstruction method of claim 1, wherein, the obtaining of a plurality of images to be reconstructed photographed by the unmanned aerial vehicle for three-dimensional reconstruction of the scene of the region to be photographed comprises at least one of the following: obtaining a plurality of images to be reconstructed photographed by the unmanned aerial vehicle, and performing three-dimensional reconstruction of the scene of the region to be photographed at a local end; obtaining a plurality of images to be reconstructed photographed by the unmanned aerial vehicle, sending the plurality of images to be reconstructed to a background end or a cloud end, and performing three-dimensional reconstruction of the scene of the region to be photographed at the background end or the cloud end. 5.The UAV-based scene 3D reconstruction method of claim 1, wherein, three-dimensional reconstruction of the scene of the region to be photographed is performed through the following steps: Feature point extraction is performed on each of the to-be-reconstructed images, similarity matching is performed on adjacent to-be-reconstructed images according to the extracted feature points, and a plurality of matching pairs are obtained; Feature point matching is performed on all the matching pairs through a feature matching algorithm, and incorrect feature points are filtered out; Camera poses of each of the to-be-reconstructed images are obtained through a global motion recovery structure algorithm according to the filtered matching pairs, and each of the to-be-reconstructed images is simultaneously subjected to a distortion removal process, and each distortion-removed image is obtained; Each depth map of each of the distortion-removed images is generated, all the depth maps are fused according to each of the camera poses, and a three-dimensional point cloud of the to-be-photographed region scene is obtained; Triangular partitioning is performed on the three-dimensional point cloud, a coarse mesh of the to-be-photographed region scene is obtained, and the coarse mesh is refined using image gradient differences of adjacent depth maps, and a fine mesh of the to-be-photographed region scene is obtained; Texture mapping is performed on the fine mesh, and a three-dimensional model of the to-be-photographed region scene is obtained. 6.The UAV-based scene 3D reconstruction method of claim 1, wherein, The scene three-dimensional reconstruction method based on the unmanned aerial vehicle further includes: When it is determined that the current photographing time is within a preset night time period, the unmanned aerial vehicle is controlled to perform circular flight above the to-be-photographed region scene with a preset position as the center, at a preset flight speed and according to the preset circular flight track parameters, and to perform photographing of the to-be-photographed region scene at each of the aerial photographing points within a preset time length, at a preset night light sensitivity and according to the gimbal angle. 7.A device for three-dimensional reconstruction of a scene based on a UAV, characterized in that, It includes: An average height value acquisition module is configured to acquire an average height value of a building in a to-be-photographed region scene; A circular flight track parameter setting module is configured to set a circular flight track parameter of the unmanned aerial vehicle according to the average height value, wherein the circular flight track parameter includes a circular flight height and a circular flight radius; A gimbal angle setting module is configured to set a gimbal angle of an on-board camera of the unmanned aerial vehicle according to the circular flight track parameter; An aerial photographing point acquisition module is configured to calculate all aerial photographing points of the unmanned aerial vehicle according to a preset number of aerial photographing points and the circular flight radius; A first to-be-photographed region scene photographing module is configured to, when it is determined that a previously acquired current photographing time is within a preset day time period, control the unmanned aerial vehicle to perform circular flight above the to-be-photographed region scene with a preset position as the center, at a preset flight speed and according to the preset circular flight track parameters, and to perform photographing of the to-be-photographed region scene at each of the aerial photographing points according to the gimbal angle; A three-dimensional reconstruction module is configured to obtain a plurality of to-be-reconstructed images photographed by the unmanned aerial vehicle, for three-dimensional reconstruction of the to-be-photographed region scene; The setting of the circular flight track parameter of the unmanned aerial vehicle according to the average height value includes: The circular flight height of the unmanned aerial vehicle is set according to a preset ratio range of the circular flight height to the average height value and the average height value, wherein the ratio range of the circular flight height to the average height value is greater than or equal to 2 and less than or equal to 3. According to the preset ratio range of the surrounding flight height and the surrounding flight radius and the surrounding flight height, the surrounding flight radius of the unmanned aerial vehicle is set; wherein the ratio range of the surrounding flight height and the surrounding flight radius is greater than or equal to and less than or equal to The gimbal angle of the on-board camera of the unmanned aerial vehicle is set according to the circling flight trajectory parameter, and the setting includes: The result obtained by substituting the circling flight height and the circling flight radius into a preset trigonometric function relationship is set as the gimbal angle; The trigonometric function relationship is: In the formula, H is the circling flight height, and R is the circling flight radius.

8. An electronic device, comprising: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the method for three-dimensional reconstruction of a scene based on an unmanned aerial vehicle according to any one of claims 1 to 6 when executing the computer program.

Images