Aerial photography plan creation device and method

Abstract

Provided are an aerial photography plan creation device and method with which it is possible to easily create an aerial photography plan for favorable aerial photography of the appearance of a target photography object and that requires few aerial photography images. This aerial photography plan creation device (20) comprises: a memory (210) that stores an aerial photography plan including a flight path of a flying object set when performing aerial photography of a photography region, and a plurality of photography points on the flight path; and a processor (200). The process (200) performs an aerial photography simulation on the photography region in accordance with the aerial photography plan stored in the memory (210), generates a simulation image corresponding to a real aerial photography image with the aerial photography simulation, calculates, with respect to a target photography object within the photography region on the basis of the simulation image, proportion information indicating a proportion of a photographable region or a proportion of a non-photographable region relative to the entire photography region of the target photography object, and evaluates an aerial photography plan set on the basis of the proportion information.

Description

Aerial photography planning device and method

[0001] The present invention relates to an aerial photography plan creation device and method, and more particularly to a technique for creating an aerial photography plan that takes a small number of aerial images and captures good aerial photography of the exterior of a subject.

[0002] In order to grasp the extent of damage when a disaster occurs, in addition to the traditional method of photographing using aircraft, in recent years drones have also been used to photograph.

[0003] When photographing houses (structures) using aircraft, the flight altitude is relatively high (around 1,000 m), so it is not possible to grasp the detailed damage status of individual houses from the aerial images. On the other hand, when photographing using drones, the altitude is low (150 m or less), so it is possible to confirm the detailed status of individual houses from the aerial images.

[0004] However, in order to confirm all the houses in a shooting area, it is common to shoot with a large amount of overlap, which increases the number of aerial images and makes confirmation time-consuming. For example, a technology is becoming popular in which a large number of images are shot with a large amount of overlap, a method called SfM (Structure from Motion) is used to generate a 3D model of the object being shot, and the image of the object being shot is then mapped onto the surface of the 3D model. However, SfM requires long processing times, and it takes several tens of minutes or more to process images shot in a shooting area of ​​500 m x 500 m.

[0005] Conventionally, an information processing device has been proposed that can suppress a decrease in the efficiency of photographing by an aircraft and obtain a large amount of information on the front of the terrain by the aircraft (Patent Document 1).

[0006] The information processing device described in Patent Document 1 generates a flight path for an aircraft to fly by acquiring topographical information of the flight range in which the aircraft will fly, generating a flight path including shooting positions in three-dimensional space for photographing the terrain of the flight range based on the topographical information of the flight range, and deriving a shooting angle for each shooting position on the flight path based on the topographical information of the flight range and the flight path.

[0007] In particular, the information processing device described in Patent Document 1 derives the camera's shooting angle so that, when an undulating ground surface is photographed from the air using an aircraft equipped with a camera, the ground surface can be photographed from the air as directly as possible (from the front), thereby enabling aerial photography of the front of the terrain even if the ground surface is a mountain slope, and enabling the acquisition of a large amount of information about the front of the terrain.

[0008] Furthermore, Patent Document 2 describes a flight plan creation system that creates a flight plan for an autonomously flying drone.

[0009] The flight planning system described in Patent Document 2 accepts specifications for a photography area (designated area) identified by the latitudes and longitudes of multiple vertices, the latitude and longitude of a flight start point in the photography area, the field of view and altitude above ground of a camera used for photography, and photography overlap and side lap rates, and determines a flight line in the photography area consisting of multiple straight flight lines according to the various specified conditions. Then, when it is determined that the elevation difference between passing points at a predetermined distance on the determined straight flight line is equal to or greater than a predetermined value, the altitude of the passing point at the current travel point is corrected, thereby automatically optimizing the flight line and passing points in the photography area if the photography area includes undulating terrain.

[0010] JP 2020-043543 A JP 2021-196299 A

[0011] The information processing devices and flight plan creation systems described in Patent Documents 1 and 2 are both designed to create aerial photography plans suitable for photographing an uneven ground surface (photography area), but are not designed to create aerial photography plans that take good aerial photographs of the appearance of the object being photographed within the photography area.

[0012] In particular, the information processing device described in Patent Document 1 derives the shooting angle of a camera that takes aerial photographs facing the ground surface, so there is a problem in that only the roof of a house on flat ground directly below the camera can be photographed.

[0013] Furthermore, the flight plan creation system described in Patent Document 2 accepts specifications for the camera's angle of view and altitude above ground, as well as specifications for the overlap rate and side lap rate of photography, to optimize the flight line and passing points in the photography area, so the flight plan (aerial photography plan) including the flight line and passing points is determined based on the conditions specified by the user. The aerial photography plan determined in this manner does not optimize the number of aerial images, nor does it take into consideration taking good aerial photographs of the appearance of the subject in the photography area.

[0014] The present invention has been made in consideration of the above circumstances, and aims to provide an aerial photography plan creation device and method that can easily create an aerial photography plan that takes a small number of aerial images and captures good aerial photography of the appearance of the object being photographed.

[0015] In order to achieve the above-mentioned object, the invention of a first aspect is an aerial photography plan creation device that includes a memory that stores an aerial photography plan that includes a flight path of an aircraft and multiple photography points on the flight path that are set when photographing a photography area from the air, and a processor, wherein the processor performs an aerial photography simulation of the photography area in accordance with the aerial photography plan stored in the memory, generates a simulation image that corresponds to the actual aerial image from the aerial photography simulation, calculates percentage information for the photography subject within the photography area based on the simulation image, which indicates the proportion of the photographable area or the proportion of the unphotographable area of ​​the photography subject to the total photography area, and evaluates the aerial photography plan that has been set based on the percentage information.

[0016] According to the first aspect of the present invention, an aerial photography simulation of a photography area is performed in accordance with a pre-set aerial photography plan, information (ratio information) indicating whether the subject to be photographed is being photographed appropriately is calculated, and the set aerial photography plan is evaluated based on the calculated ratio information, so that the aerial photography plan can be evaluated in advance without actually photographing the photography area from the air.

[0017] In the aerial photography planning device according to the second aspect of the present invention, in the first aspect, it is preferable that the processor acquires appearance information regarding the appearance of the object to be photographed within the photography area, and generates a simulation image based on three-dimensional information about the photography point, camera information including the focal length and attitude of the camera mounted on the aircraft, and the appearance information of the object to be photographed.

[0018] In other words, if the three-dimensional information of the photographing point on the flight path, the camera information of the camera mounted on the aircraft, and the appearance information of the object to be photographed are known, it is possible to generate a simulation image that corresponds to the actual aerial image that would be obtained if the object were photographed from a camera located at that photographing point.

[0019] In the aerial photography planning device of the third aspect of the present invention, in the second aspect, the object to be photographed is a house, and it is preferable that the processor obtains the latitude, longitude, and altitude that identify the polygon of the house from the map database as appearance information.

[0020] In the aerial photography planning device according to a fourth aspect of the present invention, in the first aspect, it is preferable that the object to be photographed is a house, and the processor calculates the proportion information of all houses within the photography area.

[0021] In the aerial photography planning device of the fifth aspect of the present invention, in the fourth aspect, when the ratio information is an appearance rate indicating the ratio of the area that can be photographed to the total photography area of ​​a house, the processor evaluates that the aerial photography plan that has been set is appropriate if Na / N is equal to or greater than a second threshold, where N is the number of houses in the photography area and Na is the number of houses in the photography area that have an appearance rate calculated to be equal to or greater than a first threshold.

[0022] In the aerial photography planning device of the sixth aspect of the present invention, in the fifth aspect, the processor evaluates as optimal the aerial photography plan in which Na / N is greater than or equal to a second threshold and Na / N is the largest among the allowable number of photographs.

[0023] In the aerial photography planning device of the seventh aspect of the present invention, in the fourth aspect, the processor generates multiple simulation images corresponding to multiple shooting points in accordance with the aerial photography plan, calculates proportion information for all houses in the multiple simulation images, and when multiple proportion information for the same house is calculated based on the multiple simulation images, it is preferable to adopt the proportion information of the multiple proportion information with the largest photographable area as the proportion information for the same house.

[0024] If the same house is included in multiple simulation images, multiple ratio information will be calculated for the same house. In this case, the ratio information that maximizes the photographable area among the multiple ratio information is used as the ratio information for that house, allowing for more appropriate evaluation of the aerial photography plan.

[0025] In the aerial photography planning device according to the eighth aspect of the present invention, in any of the fourth to seventh aspects, when the processor calculates the proportion information of a specific house among the houses that are the subject of photography, it is preferable that the processor calculates as proportion information the proportion between the area of ​​the specific house that appears in the simulation image, or the area of ​​the specific house that is obscured by the house in front and does not appear in the simulation image, and the area of ​​the specific house that appears in the simulation image when the specific house is not obscured by the house in front, or assuming that it is not obscured.

[0026] In the aerial photography planning device according to a ninth aspect of the present invention, in the eighth aspect, it is preferable that the area of ​​the house is an area of ​​the exterior walls excluding the roof.

[0027] This is because the exterior wall area is easy to capture as a simulation image and is also likely to be blocked by a house in the foreground.

[0028] In the aerial photography plan creation device of the 10th aspect of the present invention, in any of the first to seventh aspects, it is preferable that the processor selects an aerial photography plan that is evaluated as optimal or appropriate from the multiple aerial photography plans based on the evaluation.

[0029] In the aerial photography plan creation device of an eleventh aspect of the present invention, in any of the first to seventh aspects, a plurality of aerial photography plans are stored in the memory, and the processor automatically selects a specified aerial photography plan from the plurality of aerial photography plans, or accepts manual input of an aerial photography plan selection and sets the aerial photography plan, and if it evaluates that the set aerial photography plan is inappropriate, it is preferable to select and reset an aerial photography plan from the plurality of aerial photography plans that will take a greater number of images than the set aerial photography plan, or an aerial photography plan that will increase the proportion of the area that can be photographed of the subject.

[0030] In the aerial photography plan creation device according to a twelfth aspect of the present invention, in any of the first to seventh aspects, the processor accepts a manually created aerial photography plan including a flight path of the aircraft and multiple photography points along the flight path and stores it in memory, and if it evaluates that the aerial photography plan stored in memory is inappropriate, it automatically revises it to an aerial photography plan with more photography points than the aerial photography plan stored in memory, or an aerial photography plan with a higher proportion of the area that can be photographed of the subject, and updates the aerial photography plan stored in memory to the automatically revised aerial photography plan.

[0031] The invention of the thirteenth aspect is an aerial photography plan creation method executed by an aerial photography plan creation device having a memory that stores an aerial photography plan including a flight path of an aircraft and multiple photography points on the flight path that are set when photographing a photography area from the air, and a processor, in which the processor executes each of the steps including the steps of: performing an aerial photography simulation of the photography area in accordance with the aerial photography plan stored in the memory and generating a simulation image that corresponds to an actual aerial image through the aerial photography simulation; calculating, based on the simulation image, percentage information for a subject within the photography area that indicates the proportion of the photographable area or the proportion of the unphotographable area of ​​the subject to the total photography area of ​​the subject; and evaluating the set aerial photography plan based on the percentage information.

[0032] In the aerial photography planning method according to the fourteenth aspect of the present invention, in the thirteenth aspect, the step of generating a simulation image preferably acquires appearance information regarding the appearance of the object to be photographed within the photography area, and generates the simulation image based on three-dimensional information about the photography point, camera information including the focal length and attitude of the camera mounted on the aircraft, and the appearance information of the object to be photographed.

[0033] In the aerial photography planning method according to a fifteenth aspect of the present invention, in the thirteenth aspect, it is preferable that the object to be photographed is a house, and the step of calculating the proportion information calculates the proportion information of all houses within the photography area.

[0034] In the aerial photography plan creation method according to the 16th aspect of the present invention, in the 15th aspect, when the ratio information is an appearance rate indicating the ratio of the area that can be photographed to the total photography area of ​​the object to be photographed, the step of calculating the ratio information calculates the appearance rate of all houses within the photography area, and the step of evaluating the aerial photography plan evaluates the set aerial photography plan as appropriate if Na / N is equal to or greater than a second threshold, where N is the number of houses within the photography area and Na is the number of houses within the photography area that have an appearance rate calculated to be equal to or greater than a first threshold.

[0035] In the aerial photography planning method according to the 17th aspect of the present invention, in the 15th aspect, the step of calculating the proportion information calculates the proportion information of all houses in multiple simulation images corresponding to multiple shooting points in accordance with the aerial photography plan, and when multiple proportion information is calculated for the same house based on the multiple simulation images, it is preferable to adopt the proportion information of the multiple proportion information with the largest photographable area as the proportion information of the same house.

[0036] In the aerial photography planning method according to the 18th aspect of the present invention, in any of the 15th to 17th aspects, when calculating the proportion information of a specific house among the houses that are the subject of photography, it is preferable that the step of calculating the proportion information calculates as proportion information the ratio between the area of ​​the specific house that appears in the simulation image, or the area of ​​the specific house that does not appear in the simulation image because the specific house is obstructed by the house in front, and the area of ​​the specific house that appears in the simulation image when the specific house is not obstructed by the house in front, or if it is assumed that the specific house is not obstructed.

[0037] In the aerial photography planning method according to a nineteenth aspect of the present invention, in the eighteenth aspect, it is preferable that the area of ​​the house is an area of ​​the exterior walls excluding the roof.

[0038] An aerial photography plan creation method according to a 20th aspect of the present invention, in any of the 13th to 17th aspects, preferably comprises the steps of: storing a plurality of aerial photography plans in the memory; and the processor automatically selecting a specified aerial photography plan from the plurality of aerial photography plans; or setting the aerial photography plan by accepting manual input of an aerial photography plan selection; and, if the step of evaluating the aerial photography plan evaluates that the set aerial photography plan is inappropriate, resetting from the plurality of aerial photography plans an aerial photography plan that takes a greater number of images than the set aerial photography plan, or an aerial photography plan that has a higher proportion of the area that can be photographed of the subject; and the processor repeatedly executing the steps of generating simulation images according to the reset aerial photography plan, calculating proportion information, and evaluating the aerial photography plan.

[0039] An aerial photography plan creation method according to a 21st aspect of the present invention, in any of the 13th to 17th aspects, preferably includes a step in which a processor accepts a manually created aerial photography plan including a flight path of the aircraft and a plurality of photography points along the flight path and stores the plan in memory; if the processor evaluates that the aerial photography plan stored in memory is inappropriate, the processor automatically modifies the aerial photography plan to one with more photography points than the aerial photography plan stored in memory, or to an aerial photography plan with a higher proportion of the area that can be photographed of the subject, and updates the aerial photography plan stored in memory to the automatically modified aerial photography plan; and the processor repeatedly executes the steps of generating simulation images according to the updated aerial photography plan, calculating proportion information, and evaluating the aerial photography plan.

[0040] According to the present invention, it is possible to easily create an aerial photography plan that takes a small number of aerial images and captures good aerial images of the external appearance of the object to be photographed.

[0041] FIG. 1 is a schematic diagram showing an example of the configuration of a system including an aerial photography planning device according to the present invention. FIG. 2 is a block diagram showing an embodiment of the hardware configuration of the aerial photography planning device shown in FIG. 1. FIG. 3 is a functional block diagram showing a first embodiment of the aerial photography planning device according to the present invention. FIG. 4 is a diagram showing an example of a photography plan prepared in advance. FIG. 5 is a diagram showing the relationship between an aerial image obtained by a camera taking an aerial photograph from a certain photography point and a map showing the ground surface captured by the aerial image. FIG. 6 is a diagram showing a map including a house indicated by a rectangular frame. FIG. 7 is a diagram showing exterior information of a house linked to a house ID. FIG. 8 is a diagram used to explain the position and attitude of a camera. FIG. 9 is a diagram showing an example of the relationship between a three-dimensional spatial coordinate system with three axes corresponding to the three-dimensional coordinates (x', y', z') obtained by the coordinate transformation of Equation 1 and the image coordinate system defined by the camera's image sensor. FIG. 10 is a diagram showing an example of a simulation image corresponding to an aerial image. FIG. 11 is a diagram showing a first embodiment of a method for calculating the appearance rate of a house. Fig. 12 is a diagram showing a second embodiment of a method for calculating the appearance rate of a house. Fig. 13 is a diagram showing the process of storing and updating the appearance rate in association with a house ID. Fig. 14 is a functional block diagram showing a second embodiment of an aerial photography planning device according to the present invention. Fig. 15 is a flowchart showing an embodiment of an aerial photography planning method according to the present invention.

[0042] Hereinafter, preferred embodiments of an aerial photography planning system and method according to the present invention will be described with reference to the accompanying drawings.

[0043] [System Including Aerial Photography Planning Apparatus] FIG. 1 is a schematic diagram showing an example of the configuration of a system including an aerial photography planning apparatus according to the present invention.

[0044] 1 includes a drone 12, which is an aerial vehicle for aerial photography, a remote controller 16, and an aerial photography planning device 20. The drone 12, the remote controller 16, and the aerial photography planning device 20 are connected via a network 22. The aerial photography planning device 20 can also connect via the network 22 to a map database of the Geospatial Information Authority of Japan and an OpenStreetMap map database (not shown).

[0045] The drone 12 flies according to the aerial photography plan created by the aerial photography plan creation device 20, and automatically takes aerial photographs of the photography area at photography points along the flight path using the camera 14 mounted on the drone 12. The aerial photography plan is transmitted from the aerial photography plan creation device 20 to the drone 12 via the network 22 and a remote controller 16 that remotely controls the drone 12, or it can be transmitted directly from the aerial photography plan creation device 20 to the drone 12 via the network 22, or it can be stored in advance in a memory (not shown) within the drone 12.

[0046] The aerial photography plan includes the flight path of the flying object (drone 12) set when photographing the photography area from the air and multiple photography points along the flight path, and may also include camera information for the camera 14 that will take the aerial photography. The camera information includes the focal length and attitude of the camera 14 when taking the aerial photography. Details of the aerial photography plan and camera information will be described later.

[0047] When a disaster occurs, the drone 12 takes aerial photographs of the area where the disaster occurred (photography area) to investigate the damage to the object within the area. In this example, the object is a house. The house is not limited to a house where people live, but also includes a factory, a store, a warehouse, and other buildings.

[0048] The camera 14 is mounted on the drone 12 via a gimbal head 13. The camera 14 or drone 12 has a GPS (Global Positioning System) receiver, a barometric pressure sensor, a direction sensor, a gyro sensor, etc., and acquires the position (latitude, longitude, altitude) of the camera 14 during aerial photography, sets a position on the flight route of the aerial photography plan as a target position, and automatically flies so as to pass through the photography points on the flight route in sequence.

[0049] When the drone 12 reaches a photography point on its flight path, it stops in the air (hovers) and outputs an aerial photography command to the camera 14. When the camera 14 receives the aerial photography command from the drone 12, it takes an aerial photograph of the photography area.

[0050] Images captured using the camera 14 (hereinafter referred to as "aerial images IM") can be stored in an internal storage built into the camera 14 and / or in a storage device such as a memory card removably attached to the camera 14. The aerial images IM can be transferred to the remote controller 16 via wireless communication or to the aerial photography planning device 20. It is also preferable that information about the shooting point during aerial photography be recorded in the header section of the image file in which the aerial images IM are recorded.

[0051] The aerial photography planning device 20 is configured using a computer. The computer applied to the aerial photography planning device 20 may be a server, a personal computer, or a workstation.

[0052] The aerial photography planning device 20 can perform data communication with the drone 12, the remote controller 16, an external map database, etc. via a network 22. The network 22 may be a local area network or a wide area network.

[0053] The aerial photography planning device 20 can acquire aerial images IM from the drone 12 or the camera 14 via the network 22, or via the remote controller 16. The aerial photography planning device 20 can also acquire aerial images IM from the memory card of the camera 14, etc., without going through the network 22. This is because it is conceivable that the network in some areas may be down due to a disaster.

[0054] <Overview of the present invention> The operation of the above-described system 10 is, for example, the operation when the drone 12 takes aerial photographs in accordance with an aerial photography plan for a photography area created by the aerial photography plan creation device 20 in order to investigate the damage to houses in an area where a disaster has occurred (photography area), but the main function of the aerial photography plan creation device 20 is to evaluate the aerial photography plan that has been created and set in advance.

[0055] (1) Setting an aerial photography plan: The flight path to be flown and the photography points (latitude, longitude, altitude) to be used for photography are set for a photography area (e.g., 500 m x 500 m). In this case, the focal length and attitude of the camera 14 (azimuth angle and depression angle indicating the photography direction) are assumed to be determined in advance.

[0056] The size of the shooting area may be determined based on the specifications of the drone 12 (maximum flight time (battery dependent), maximum flight speed, etc.).

[0057] The aerial photography plan is set so that the same house in the photography area is photographed multiple times in aerial images corresponding to the number of photography points, i.e., the house in the photography area is photographed from multiple different photography points.

[0058] A plurality of aerial photography plans with different flight routes and photography points are prepared in advance, and the prepared aerial photography plans are stored in the memory 210 (FIG. 2) in the aerial photography plan creation device 20.

[0059] (2) Evaluation of the aerial photography plan The aerial photography plan creation device 20 performs an aerial photography simulation according to the aerial photography plan set in (1) above, and generates simulation images that correspond to the actual aerial images taken from the aerial at each shooting point through the aerial photography simulation. The generation of simulation images will be described in detail later.

[0060] Based on the simulation image, the aerial photography planning device 20 calculates percentage information for a house within the photography area, indicating the proportion of the photographable area (appearance rate) to the total photography area of ​​the house, or the proportion of the area that cannot be photographed (occlusion rate).

[0061] Here, the appearance rate of a particular house is the ratio of the area that can actually be photographed to the total photographable area of ​​the particular house that can be photographed if the house in front of the particular house does not obstruct the photographing (if the particular house is not blocked by the house in front) as seen from the camera. Therefore, the appearance rate of the particular house is 1 if the house in front of the particular house does not obstruct the photographing, and is 0 if the house cannot be photographed at all due to the obstructing house.

[0062] On the other hand, the occlusion rate of a particular house is the ratio of the area blocked by the house in front (area that cannot be photographed) to the total photographable area of ​​the particular house that could be photographed if the house in front of the particular house did not obstruct the photographing, as seen from the camera. Therefore, the occlusion rate of a particular house is 0 if the house in front of the particular house does not obstruct the photographing, and is 1 if the photographing is completely impossible due to the house obstructing the photographing.

[0063] In this example, the appearance rate is used as ratio information indicating the appearance rate or occlusion rate of the houses in the shooting area, but since the appearance rate = 1 - occlusion rate, the occlusion rate may be used instead of the appearance rate.

[0064] The aerial photography planning device 20 generates multiple simulation images corresponding to all of the photography points within the photography area through aerial photography simulation at all of the photography points, and calculates the appearance rate of the house in each simulation image. In this case, since the aerial photography plan is set so that the same house is photographed multiple times, multiple appearance rates (multiple ratio information) are calculated for one house, and the maximum appearance rate among the multiple appearance rates is adopted as the appearance rate of that one house.

[0065] The aerial photography planning device 20 evaluates the aerial photography plan set in (1) above based on the appearance rates of all the houses within the photography area calculated in this way.

[0066] For example, the evaluation of an aerial photography plan can be performed by dividing the number of houses Na by the number of houses N, where N is the number of houses in the photography area whose appearance rate is calculated to be equal to or higher than the first threshold, and quantifying the value (Na / N) as the OK house ratio.

[0067] (3) Calculation of the optimal aerial photography plan The aerial photography plan creation device 20 performs the evaluation (2) above on multiple aerial photography plans stored in the memory 210, and can calculate the aerial photography plan with the highest evaluation as the optimal aerial photography plan.

[0068] The aerial photography plan creation device 20 can also calculate an aerial photography plan that satisfies a preset allowable number of aerial images and maximizes the acceptable house ratio as the optimal aerial photography plan. Furthermore, it can calculate an aerial photography plan that maximizes the acceptable house ratio by exceeding a second threshold (acceptable value) as the appropriate aerial photography plan.

[0069] [Hardware Configuration of Aerial Photography Planning Apparatus] FIG. 2 is a block diagram showing an embodiment of the hardware configuration of the aerial photography planning apparatus shown in FIG.

[0070] The aerial photography planning device 20 shown in FIG. 2 includes a processor 200 , a memory 210 , a map database 220 , a display device 230 , an input / output interface 240 , and an operation unit 250 .

[0071] The processor 200 is composed of a CPU (Central Processing Unit) and other components, and controls all the components of the aerial photography plan creation device 20, and also performs the simulation image generation process for generating the simulation images described above, the appearance rate calculation process, and the aerial photography plan evaluation process. Details of the various processes performed by the processor 200 will be described later.

[0072] The memory 210 includes flash memory, read-only memory (ROM), random access memory (RAM), a hard disk drive, etc. The flash memory, ROM, or hard disk drive is a non-volatile memory that stores various programs including an operating system. The RAM functions as a working area for processing by the processor 200 and temporarily stores programs stored in the flash memory, etc. Note that the processor 200 may have a portion of the memory 210 (RAM) built in.

[0073] In addition, the memory 210 stores aerial photography plans prepared in advance, and when aerial images IM are acquired according to the aerial photography plan, it functions as an image storage unit that stores the aerial images IM, and can save and manage the aerial images IM.

[0074] The map database 220 is a part that manages map information of the photographing area, and in this example, in addition to the map, it also manages appearance information relating to the appearance of the photographing object (house) on the map.

[0075] Details of the appearance information relating to the exterior appearance of houses managed by the map database 220 will be described later. The map database 220 may be configured inside the aerial photography planning device 20, or may be an external map database accessible via the network 22, such as a map database of the Geospatial Information Authority of Japan that manages basic map information, or a map database of OpenStreetMap.

[0076] The display device 230 displays the aerial photography plan to be evaluated, the evaluation results of the aerial photography results, simulation images, etc., in response to instructions from the processor 200, and is also used as part of a GUI (Graphical User Interface) when accepting various types of information from the operation unit 250.

[0077] The display device 230 may be included in the aerial photography planning device 20, or may be provided externally, separate from the aerial photography planning device 20, as shown in FIG.

[0078] The input / output interface 240 includes a connection unit connectable to an external device, a communication unit connectable to a network, etc. As the connection unit connectable to an external device, a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI) (HDMI is a registered trademark), etc. can be applied.

[0079] The operation unit 250 includes a pointing device such as a mouse, a keyboard, and the like, and functions as part of a GUI that accepts input of various information and instructions by user operation.

[0080] [Aerial Photography Planning Apparatus] <First Embodiment of Aerial Photography Planning Apparatus> FIG. 3 is a functional block diagram showing a first embodiment of an aerial photography planning apparatus according to the present invention.

[0081] FIG. 3 is a functional block diagram mainly showing the functions of a processor 200-1 corresponding to the processor 200 of the aerial photography planning device 20 shown in FIG.

[0082] In FIG. 3, the processor 200-1 functions as a simulation image generation unit 201, an appearance rate calculation unit 202, a storage processing unit 203, an aerial photography plan evaluation unit 204, and an aerial photography plan setting unit 205.

[0083] In this example, a plurality of aerial photography plans are stored in advance in the memory 210. The plurality of aerial photography plans may each have a different flight path and / or photography points for the photography area.

[0084] FIG. 4 is a diagram showing an example of an imaging plan prepared in advance.

[0085] In Figure 4, the shooting area is a 500m x 500m square area indicated by the dotted line, and the flight path is a grid-like flight path indicated by the thick arrow, which is a flight path that is flown in a single stroke from start to end. The distance between adjacent flight paths on the grid-like flight path (flight interval) is 125m. Note that, although a grid-like flight path is used in this example, the present invention is not limited to this.

[0086] In this example, the multiple photography points on the flight path are intersections where the lattice-shaped flight path intersects. There are 25 of these intersections, but because all of the intersections are passed over twice, the total number of photography points is 50. The spacing between photography points is 125 m, the same as the flight spacing, but it may be narrower than 125 m. However, it is preferable to include the intersections of the lattice-shaped flight path as photography points.

[0087] Furthermore, the grid-like flight path is aligned with the east-west and north-south directions. In this case, by identifying the starting position (latitude, longitude) of the flight path, a 500 m x 500 m shooting area can be identified. Furthermore, the flight path altitude can be set to a constant altitude of, for example, 150 m or less. This also allows the flight path and the three-dimensional position (latitude, longitude, altitude) of the shooting point to be identified.

[0088] In urban areas divided into grids, it is preferable to set the shooting area to match the area and the flight path to follow the area. In this case, by specifying the positions (latitude and longitude) of the four corners of the shooting area, it is possible to specify a shooting area of ​​500 m x 500 m.

[0089] It is preferable that the multiple aerial photography plans pre-stored in memory 210 are all set based on the camera information of the camera 14 mounted on the drone 12 so that the same house within the photography area is photographed multiple times.

[0090] The camera information is the focal length and attitude (azimuth angle and depression angle indicating the shooting direction) of the camera 14, which are assumed to be set in advance. The focal length of the camera 14 is a fixed focus that is set in advance, and the attitude of the camera 14 relative to the drone 12 is also fixed. For example, the shooting direction of the camera 14 has a depression angle that shoots diagonally downward, and is the same as the flight direction of the drone 12. Note that if the camera information of the camera 14 is changed, the aerial photography plan must also be changed in accordance with the changed camera information. This is because the shooting range of the aerial image captured changes depending on the focal length of the camera 14.

[0091] 3, the aerial photography plan setting unit 205 selects a pre-set aerial photography plan from multiple aerial photography plans stored in memory 210, reads the selected aerial photography plan from memory 210, and sends it to the simulation image generation unit 201. In this case, the aerial photography plan setting unit 205 can select and set the aerial photography plan with the smallest number of photography points (number of aerial images) from the multiple aerial photography plans.

[0092] The aerial photography plan setting unit 205 may also accept manual selection input of an aerial photography plan from the operation unit 250 and initially set an aerial photography plan desired by the user.

[0093] When the simulation image generating unit 201 acquires the aerial photography plan to be evaluated from the memory 210, it performs an aerial photography simulation of the photography area in accordance with the aerial photography plan, and generates a simulation image corresponding to the actual aerial image through the aerial photography simulation.

[0094] When generating a simulation image, the simulation image generation unit 201 acquires appearance information regarding the appearance of the house within the shooting area managed by the map database 220, and generates a simulation image corresponding to each aerial image obtained when aerial photography is performed by the camera 14 from each shooting point based on the three-dimensional information (latitude, longitude, altitude) of each shooting point included in the aerial photography plan, camera information including the focal length and attitude of the camera 14 mounted on the drone 12, and the exterior information of the house acquired from the map database 220.

[0095] FIG. 5 is a diagram showing the relationship between an aerial image IM obtained by taking an aerial photograph with the camera 14 from a certain photographing point, and a map MP showing the ground surface photographed by this aerial image IM.

[0096] When evaluating and creating an aerial photography plan, an aerial photograph of an aerial image IM is not actually taken, but a simulation image corresponding to the aerial photograph IM is generated. However, for the sake of convenience, FIG. 5 shows the aerial photograph IM.

[0097] The simulation image generation unit 201 can predict the area (city block) of the aerial image IM obtained when photographing from the air from the shooting point based on three-dimensional information of the shooting point and camera information including the focal length and attitude of the camera 14, and obtains a map MP of the area corresponding to the predicted aerial image IM from the map database 220.

[0098] In FIG. 5, black circles on the map MP indicate the vertices of polygons on the ground perimeter of houses, and have three-dimensional information indicating latitude, longitude, and altitude.

[0099] The simulation image generating unit 201 identifies the positions of the vertices of the polygons indicated by black circles on the map MP on the simulation image corresponding to the aerial image IM.

[0100] In map MP, each house is assigned a house ID (Identification) as identification information to identify the house, and the latitude, longitude, and altitude of the vertices of the polygon surrounding the ground perimeter of the house are recorded linked to the house ID.

[0101] <<About the camera matrix>> The problem of finding the correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates can be solved by finding the camera matrix as a transformation matrix for perspective projection transformation using the following formula based on the camera model: Image coordinates (u, v) = camera matrix * three-dimensional coordinates (x, y, z)

[0102] The camera matrix can be expressed as the product of an internal parameter matrix and an external parameter matrix. The external parameter matrix is ​​a matrix that converts three-dimensional coordinates (world coordinates) into camera coordinates. The external parameter matrix is ​​a matrix determined by the shooting point (camera position) and attitude during aerial photography, and includes translation parameters and rotation parameters.

[0103] The internal parameter matrix is ​​a matrix that converts camera coordinates into image coordinates, and is determined by the specifications of the camera 14, such as the focal length of the camera, the sensor size and aberration (distortion) of the image sensor, etc.

[0104] By converting from the three-dimensional coordinates (x, y, z) to the camera coordinates using the extrinsic parameter matrix, and then converting from the camera coordinates to the image coordinates (u, v) using the intrinsic parameter matrix, the three-dimensional coordinates (x, y, z) can be associated (converted) with the image coordinates (u, v).

[0105] The intrinsic parameter matrix can be specified in advance, whereas the extrinsic parameter matrix depends on the position and orientation of the camera and therefore needs to be set for each simulation image corresponding to the aerial image IM.

[0106] <<Explanation of Perspective Projection Transformation Using Camera Matrix>> The map database 220 stores a map MP and appearance information relating to the appearance of the object (house) to be photographed included in the map MP.

[0107] That is, in the map MP, each house is assigned a house ID (Identification) as identification information for identifying the house, and three-dimensional information of multiple specific points on the ground perimeter of the house (vertices of a polygon representing the ground perimeter of the house) is recorded as appearance information related to the house's appearance, linked to the house ID. The three-dimensional information of the vertices of the polygon representing the ground perimeter of the house is information indicating latitude, longitude, and altitude. In addition, height information of the house may also be recorded as appearance information related to the house's appearance.

[0108] The simulation image generating unit 201 acquires, from the map database 220, three-dimensional information on the vertices of polygons that indicate the ground perimeters of houses included in the map MP.

[0109] FIG. 6 is a diagram showing a map MP including a house H indicated by a rectangular frame, and FIG. 7 is a diagram showing exterior information of a house linked to a house ID.

[0110] The polygon representing the ground perimeter of the house H shown in FIG. 6 is a quadrilateral.

[0111] In FIG. 7, the house ID of house H is ID0001, and associated with this ID0001, latitude, longitude, and altitude are recorded as three-dimensional information of the four vertices of a polygon indicating the ground perimeter of house H.

[0112] Here, we will explain in detail the calculation method for converting the three-dimensional coordinates (x, y, z) of the vertices of the polygon representing the ground perimeter of the house included in the map MP into coordinates (image coordinates (u, v)) when projected onto the image sensor of the camera 14.

[0113] In the three-dimensional coordinates (x, y, z) of the vertices of the polygon representing the ground perimeter of the house, x and y are latitude and longitude converted into UTM coordinates, which are an orthogonal coordinate system, and z is altitude.

[0114] As shown in FIG. 7, the latitude, longitude, and altitude of the vertices of the polygon that indicates the ground perimeter of the house can be obtained from the map database 220 based on the house ID.

[0115] Furthermore, if height information of the house (building) is recorded in the map database 220, the height information can be used to calculate the three-dimensional position of the lowest point of the roof on the image.If height information is not available, the altitude of the lowest point of the roof can be calculated depending on the type of house (single-story, two-story, building (number of floors of the building), etc.), for example, in the case of a two-story building, the height of the lowest point of the roof can be assumed to be 6 m.

[0116] The three-dimensional position of the camera 14 during aerial photography is defined as (xc, yc, zc), where xc and yc are the latitude and longitude of the photography point converted into UTM coordinates, and zc is the altitude.

[0117] Furthermore, the attitude (shooting direction) of the camera 14 during aerial photography is specified by the azimuth angle θh, tilt angle θt, and roll angle θr. The azimuth angle θh is the angle from north, with north as the reference point. The tilt angle θt is the angle of the camera toward the ground (depression angle). The roll angle θr is the inclination from the horizontal. In this example, the attitude of the camera 14 is fixed relative to the gimbal head 13, and the shooting direction of the camera 14 during aerial photography is determined by the flight direction of the drone 12 (east-west direction, north-south direction).

[0118] FIG. 8 is a diagram used to explain the position and orientation of the camera.

[0119] In the UTM coordinate system, the x-axis is defined as east and the y-axis is defined as north. In Fig. 8, the position of the camera 14 is defined as Pc(xc, yc, zc). Arrow A indicates the shooting direction specified by the attitude of the camera 14.

[0120] The equation for converting the coordinates (x, y, z) of the vertices of the polygon on the ground perimeter of the house into the origin of the projection center (i.e., the origin which is the shooting point (camera position) when taking an aerial photograph) is expressed as the following [Equation 1].

[0121]

[0122] Furthermore, the rotation matrices Mh, Mt, and Mr are defined by the following [Equation 2], [Equation 3], and [Equation 4].

[0123]

[0124]

[0125]

[0126] In equations (2) to (4), the azimuth angle θh, tilt angle θt, and roll angle θr, which indicate the shooting direction of the camera 14, are known.

[0127] The coordinates (x', y', z') of the vertices of the polygon with the projection center as the origin are converted into camera coordinates using the following equation.

[0128]

[0129] The origin of the camera coordinate system is the center of projection, the X axis is the horizontal direction of the image sensor, the Y axis is the vertical direction of the image sensor, and the Z axis is the depth direction.

[0130] FIG. 9 is a diagram illustrating an example of the relationship between a three-dimensional spatial coordinate system having three axes corresponding to the three-dimensional coordinates (x', y', z') obtained by the coordinate transformation of Equation 1 and the image coordinate system of the image sensor of camera 14.

[0131] The coordinates (X, Y, Z) (unit: meters) of the vertices of the polygon obtained by the above formula (5) are converted into camera coordinates (unit: pixels) by the following formula.

[0132]

[0133] In Equation 6, f is the focal length of the camera 14, and p is the pixel pitch. The pixel pitch p is the distance between pixels of the image sensor and is usually the same in both the vertical and horizontal directions. Uc and Vc are the image center coordinates (in pixels).

[0134] The simulation image generation unit 201 shown in Figure 3 acquires the latitude, longitude, and altitude (Figure 7) of the four vertices of the polygon on the ground perimeter of a house H shown in Figure 6, and can associate the coordinates (x, y, z) of the four vertices of the polygon on the ground perimeter of house H with the image coordinates (u, v) of the camera coordinates (perform perspective projection transformation) by performing calculations from [Equation 1] to [Equation 6] based on three-dimensional information of the shooting point and camera information including the focal length and attitude of the camera.

[0135] The simulation image generating unit 201 generates a simulation image including a polygon of the ground perimeter of the house H by connecting the four vertices of the polygon that has been perspectively projected onto the image coordinates (u, v).

[0136] FIG. 10 is a diagram showing an example of a simulation image corresponding to the aerial image IM.

[0137] 10 shows the polygons (polygons PG indicated by thick frames) of the ground perimeter of each house superimposed on the aerial image, but the simulation image does not include the aerial image because there are no aerial images actually taken from the air in accordance with the aerial photography plan.

[0138] Furthermore, in FIG. 10, polygons PG of the ground perimeter of each house are shown, but since each house has a height, polygons showing the exterior of each house are not shown.

[0139] Next, a polygon representing the exterior of a house H will be described.

[0140] Of the latitudes, longitudes, and altitudes of the four vertices of the polygon that indicates the ground perimeter of house H shown in Figure 7, the latitude, longitude, and altitude of the vertices of the polygon that indicates the perimeter of the lowest point of the roof of house H can be obtained by changing only the building height. The building height of house H can be obtained from the map database 220 based on the house ID of house H. If the map database 220 does not contain building height information, the building height can be uniformly set to, for example, 6 m. Since the map database of the Geospatial Information Authority of Japan does not contain building height information, when the map database is used as the map database 220, all building heights are treated as 6 m.

[0141] The simulation image generation unit 201 also performs perspective projection transformation of the vertices of the polygon representing the perimeter of the lowest point of the roof of the house H into image coordinates, in the same way as the vertices of the polygon representing the perimeter of the ground surface of the house H, to generate a simulation image.

[0142] 11A shows a polygon representing the ground perimeter of a house H on a perspective projection converted simulation image, and a polygon representing the perimeter of the lowest point of the roof of the house H. The polygon representing the perimeter of the lowest point of the roof of the house H is higher than the polygon representing the ground perimeter of the house H by the height of the building.

[0143] The simulation image generation unit 201 generates a perimeter polygon showing the appearance of the house H by connecting the six outermost points of the eight points of the polygon shown in Fig. 11(A) (Fig. 11(B)). Then, for all houses in an aerial image IM obtained by aerial photography at a certain shooting point and camera orientation, the simulation image generation unit 201 performs perspective projection transformation of each vertex of the perimeter polygon showing the appearance of the house into the image coordinates of the camera 14, and generates a simulation image by connecting the outermost points of each house.

[0144] Returning to FIG. 3, the appearance rate calculation unit 202 calculates the appearance rate of each house based on the simulation image generated by the simulation image generation unit 201.

[0145] FIG. 11 is a diagram showing a first embodiment of a method for calculating the appearance rate of a specific house H. In FIG.

[0146] To calculate the appearance rate of the house H, a polygon representing the outer periphery of the house H is extracted from the simulation image (FIG. 11B), and the area of ​​the polygon is filled in white in the black image to obtain a first black-and-white image (FIG. 11C). The number of white pixels (n1) within the filled-in polygon in the first black-and-white image is counted.

[0147] Next, in the first black-and-white image shown in Fig. 11(C), the polygon on the periphery showing the exterior of the house in front of house H as seen from camera 14 is filled in with black, and a second black-and-white image is obtained (Fig. 11(D)). The number of white pixels (n2) within the polygon filled in with white in the second black-and-white image is counted.

[0148] The black area in the difference between the first black-and-white image shown in Fig. 11(C) and the second black-and-white image shown in Fig. 11(D) is an area where the house H is not photographed due to the obstruction of the foreground house. Note that if the foreground house does not obstruct photography, the first black-and-white image and the second black-and-white image will match.

[0149] The appearance rate calculation unit 202 calculates the ratio (n1 / n2) of the number of white pixels (n2) in the second black-and-white image to the number of white pixels (n1) in the first black-and-white image as the appearance rate of the house H.

[0150] The appearance rate calculation unit 202 calculates the appearance rates of all the houses in the simulation image, and outputs the appearance rates to the storage processing unit 203 in association with the house IDs.

[0151] FIG. 12 is a diagram showing a second embodiment of a method for calculating the appearance rate of a specific house H. In FIG.

[0152] The second embodiment of the method for calculating the appearance rate shown in Figure 12 differs from the first embodiment of the method for calculating the appearance rate shown in Figure 11 in that the appearance rate is calculated by focusing only on the exterior wall area of ​​the house.

[0153] Of the eight vertices of the polygon representing the ground perimeter of house H shown in Figure 12(A) and the polygon representing the roof perimeter of house H, six vertices corresponding to the area of ​​the exterior walls of house H are selected, and a polygon representing the area outside house H (the area of ​​the exterior walls excluding the roof) is extracted by connecting the six vertices (Figure 12(B)).

[0154] Next, a first black-and-white image is obtained in which the polygon area representing the exterior wall area in the black image is filled in white (Figure 12 (C)), and the number of white pixels (n1) in the polygon filled in white in the first black-and-white image is counted.

[0155] Next, in the first black-and-white image shown in Figure 12(C), the outer polygon representing the exterior of the house in front of house H as seen from camera 14 is filled in with black, and a second black-and-white image is obtained (Figure 12(D)). Note that the outer polygon representing the exterior of the foreground house is not a polygon representing the outer wall area. Then, the number of white pixels (n2) within the filled-in polygon in the second black-and-white image is counted.

[0156] The appearance rate calculation unit 202 calculates the appearance rate of the house H as the ratio (n1 / n2) of the number of white pixels (n2) in the second black-and-white image shown in FIG. 12(D) to the number of white pixels (n1) in the first black-and-white image shown in FIG. 12(C), in the same manner as the method of calculating the appearance rate of the first embodiment shown in FIG. 11 .

[0157] The appearance rate calculated in this way is an index showing the visibility of the exterior walls of houses in the aerial image.

[0158] The appearance rate calculation unit 202 of this example calculates the appearance rate of each house by applying the appearance rate calculation method of the second embodiment shown in FIG.

[0159] The simulation image generator 201 shown in Figure 3 generates simulation images corresponding to aerial images IM obtained when aerial photography is performed at each photography point and camera posture according to the aerial photography plan. That is, the simulation image generator 201 generates the same number of simulation images as the number of photography points set in the aerial photography plan.

[0160] The appearance rate calculation unit 202 calculates the appearance rates of all the houses in each simulation image for each simulation image, the number of which is the same as the number of shooting points in the aerial photography plan.

[0161] The memory processing unit 203 is a part that stores appearance rates in association with house IDs, and when multiple appearance rates are calculated for the same house, the largest appearance rate among the multiple appearance rates is adopted (stored) as the appearance rate for the same house.

[0162] FIG. 13 is a diagram showing a process of storing and updating the appearance rate in association with the house ID.

[0163] The appearance rate of each house in a simulation image SI1 is calculated by the appearance rate calculation unit 202 as described above, and the storage processing unit 203 records the calculated appearance rate of each house by linking it to the house ID (ID_00001, ID_00002, ...) of each house included in the map MP.

[0164] When the appearance rate of each house in the simulation image SI2 is calculated based on the simulation image SI2 taken at a different shooting point, the storage processing unit 203 stores the appearance rate of each house calculated in the same manner as above, linked to the house ID of each house included in the map MP.

[0165] Here, if simulation image SI1 and simulation image SI2 partially overlap, appearance rates are calculated for the same house in the overlapping portion. If, among the appearance rates calculated based on simulation image SI2, an appearance rate for the same house calculated based on simulation image SI1 has already been stored and the appearance rate for the same house exceeds the already stored appearance rate, the storage processing unit 203 updates the stored appearance rate. In the example shown in FIG. 13 , the appearance rates for house IDs ID_00001 and ID_00002 have been updated.

[0166] The storage processing unit 203 stores and updates the appearance rates calculated based on each simulation image for all shooting points, and when multiple appearance rates are calculated for the same house, it stores the maximum appearance rate among the multiple appearance rates as the appearance rate for the same house.

[0167] Returning to Figure 3, once the aerial photography plan evaluation unit 204 has calculated the appearance rates of all houses within the photography area (in this example, 500m x 500m), it evaluates the aerial photography plan currently being evaluated based on the appearance rates of all houses.

[0168] An example of the evaluation of the aerial photography plan by the aerial photography plan evaluation unit 204 will be described below.

[0169] The aerial photography plan evaluation unit 204 counts the number of houses in the photography area as N and the number of houses in the photography area whose appearance rate is calculated to be equal to or greater than the first threshold as Na. Here, the first threshold can be set to, for example, 0.8.

[0170] The aerial photography plan evaluation unit 204 calculates the ratio (Na / N) of the number of houses Na whose appearance rate is calculated to be equal to or higher than the first threshold to the number of houses N in the photography area, and sets Na / N as the OK house ratio.

[0171] The aerial photography plan evaluation unit 204 can use the OK house ratio as an evaluation value for the aerial photography plan. Furthermore, if the OK house ratio is equal to or greater than a threshold (second threshold), the aerial photography plan evaluation unit 204 determines that the aerial photography plan to be evaluated is appropriate, and if it is less than the second threshold, determines that the aerial photography plan to be evaluated is inappropriate, and can output these determination results. Furthermore, the determination results are not limited to a two-level evaluation of appropriate / inappropriate, and a five-level evaluation is also possible.

[0172] Furthermore, an aerial photography plan with a large number of images taken will have a higher acceptable house ratio (Na / N) than an aerial photography plan with a small number of images taken. On the other hand, an aerial photography plan with a large number of images taken will have the problem of longer times required for the actual aerial photography and the processing of the aerial images. Therefore, it is preferable to create an aerial photography plan with a small number of images taken and with an acceptable house ratio (Na / N) that satisfies the reference value (second threshold).

[0173] To create an optimal aerial photography plan in accordance with the above conditions, the aerial photography plan setting unit 205 sets the aerial photography plan as follows.

[0174] First, when the aerial photography plan evaluation unit 204 determines that the aerial photography plan currently being evaluated is inappropriate, the aerial photography plan setting unit 205 selects an aerial photography plan that has a higher OK house ratio (Na / N) than the aerial photography plan currently being evaluated from the multiple aerial photography plans stored in memory 210. For example, an aerial photography plan with more photography points than the number of photography points in the aerial photography plan currently being evaluated is selected.

[0175] The selected aerial photography plan is output to the simulation image generation unit 201. The simulation image generation unit 201, appearance rate calculation unit 202, storage processing unit 203, and aerial photography plan evaluation unit 204 each generate a simulation image, calculate an appearance rate based on the generated simulation image, store the calculated appearance rate, and evaluate the aerial photography plan based on the stored appearance rate, in the same way as for the initially set aerial photography plan.

[0176] The aerial photography plan evaluation unit 204 can then adopt the aerial photography plan when the OK house ratio (Na / N) meets the reference value, and output the aerial photography plan together with the evaluation results.

[0177] The aerial photography plan evaluation unit 204 determines as optimal the aerial photography plan that maximizes the acceptable house ratio (Na / N) among the acceptable number of photographs, and can also determine as optimal the aerial photography plan that minimizes the number of photographs while the acceptable house ratio (Na / N) satisfies the acceptable standard value. Furthermore, among the aerial photography plans that maximize the acceptable house ratio (Na / N) of 1 or a value close to 1, the aerial photography plan that minimizes the number of photographs can be determined as optimal.

[0178] Furthermore, the aerial photography plan evaluation unit 204 may evaluate the aerial photography plan using a representative value (e.g., average value) of the appearance rate of all houses within the photography area, without using the OK house ratio (Na / N).

[0179] In addition, the aerial photography plan evaluation unit 204 accepts input of the specifications of the drone 12 (maximum flight time (battery dependent), maximum flight speed, etc.), and if the specifications of the drone 12 are restrictive, the length of the flight path can also be used as one of the evaluation criteria.

[0180] In this way, an aerial photography plan can be evaluated without actually using drone 12 to take aerial photographs in accordance with the plan, and an aerial photography plan can be easily created that requires a small number of aerial images and takes good aerial photographs of the exterior of the object to be photographed (a house).

[0181] <Second Embodiment of Aerial Photography Planning Apparatus> FIG. 14 is a functional block diagram showing a second embodiment of an aerial photography planning apparatus according to the present invention.

[0182] Figure 14 is a functional block diagram mainly showing the functions of the processor 200-2 corresponding to the processor 200 of the aerial photography planning device 20 shown in Figure 2. In Figure 14, parts that are common to the first embodiment of the aerial photography planning device shown in Figure 3 are given the same reference numerals, and detailed explanations thereof will be omitted.

[0183] The processor 200-2 of the aerial photography plan creation device 20 of the second embodiment shown in FIG. 14 differs in that it has an aerial photography plan setting update unit 206 instead of the aerial photography plan setting unit 205 shown in FIG.

[0184] The aerial photography plan setting update unit 206 accepts a manual aerial photography plan from the operation unit 250 and stores the accepted aerial photography plan as the first aerial photography plan to be evaluated in the memory 210. The aerial photography plan setting update unit 206 may also store a default aerial photography plan or the previously created aerial photography plan in the memory 210 as the first aerial photography plan to be evaluated.

[0185] The aerial photography plan to be evaluated stored in the memory 210 is output to the simulation image generation unit 201 .

[0186] The simulation image generation unit 201, appearance rate calculation unit 202, storage processing unit 203, and aerial photography plan evaluation unit 204 generate a simulation image, calculate an appearance rate based on the generated simulation image, store the calculated appearance rate, and evaluate the aerial photography plan based on the stored appearance rate, in the same manner as in the first embodiment, based on the aerial photography plan to be evaluated stored in memory 210.

[0187] If the evaluation result of the aerial photography plan to be evaluated by the aerial photography plan evaluation unit 204 is inappropriate, the aerial photography plan setting update unit 206 automatically corrects the aerial photography plan to be currently evaluated so that the evaluation result becomes appropriate, and rewrites the aerial photography plan stored in memory 210 with the automatically corrected aerial photography plan.

[0188] For example, if the OK house ratio (Na / N) is below a reference value, the aerial photography plan setting update unit 206 increases the number of photography points from the current photography plan, or makes modifications to shorten the flight interval and increase the number of photography points, and rewrites the aerial photography plan stored in memory 210 with the modified aerial photography plan.

[0189] Conversely, if the OK house ratio (Na / N) is significantly higher than the reference value, the aerial photography plan setting update unit 206 will reduce the number of photography points from the current photography plan, or make modifications such as increasing the flight interval and reducing the number of photography points, and rewrite the aerial photography plan stored in memory 210 with the modified aerial photography plan. This allows optimization so that the photography time and number of aerial images are minimized.

[0190] [Embodiment of Aerial Photography Planning Method] FIG. 15 is a flowchart showing an embodiment of an aerial photography planning method according to the present invention.

[0191] The aerial photography planning method shown in FIG. 15 is a method performed by the aerial photography planning device of the first embodiment shown in FIG.

[0192] In Figure 15, when processor 200-1 first sets an aerial photography plan to be evaluated, it automatically selects a specified aerial photography plan from multiple aerial photography plans stored in memory 210, or accepts manual input of a selected aerial photography plan and sets the aerial photography plan (step S10).

[0193] Next, a parameter i indicating a photographing point Pi in the aerial photography plan is set to 1 (step S12). The total number of photographing points is M.

[0194] The processor 200-1 generates a simulation image SIi of the photographing point Pi (step S14). The simulation image SIi can be generated based on three-dimensional information (latitude, longitude, and altitude) of the photographing point Pi, camera information including the focal length and attitude of the camera 14 mounted on the drone 12, and exterior information of each house acquired from the map database 220 (the latitude, longitude, and altitude of the vertices of the polygon on the ground perimeter of the house).

[0195] Next, the processor 200-1 calculates the appearance ratio of each of all the houses included in the simulation image SIi based on the simulation image SIi (step S16). In this example, the appearance ratio of the exterior wall of each house is used as the appearance ratio of the house.

[0196] The processor 200-1 stores the appearance rate of each house calculated in step S16 in association with the house ID (step S18).

[0197] It is determined whether the parameter i is M (step S20), and if the parameter i is not M, the parameter i is incremented by 1 (step S22), and the process returns to step S14.

[0198] Then, the process from step S14 to step S20 is repeated for the next photographing point Pi. Note that in step S18, if the appearance rate of the same house ID is stored and the appearance rate exceeds the stored appearance rate, the stored appearance rate is updated.

[0199] In step S20, when the parameter i is determined to be M (when processing from step S14 to step S18 is completed for all shooting points), the processor 200-1 evaluates the aerial photography plan for the current evaluation target set in step S10 based on the appearance rates of all houses within the shooting area.

[0200] In this example, the processor 200-1 counts the number of houses in the image capture area as N, and the number of houses in the image capture area whose appearance rate is calculated to be equal to or greater than the first threshold as Na. Here, the first threshold can be set to, for example, 0.8.

[0201] The processor 200-1 then calculates the ratio (Na / N) of the number of houses Na whose appearance rate is calculated to be equal to or greater than the first threshold to the number of houses N in the photography area, and sets Na / N as the OK house ratio. The processor 200-1 determines whether the OK house ratio (Na / N) is equal to or greater than the second threshold (step S24), and if it determines that the OK house ratio (Na / N) is less than the second threshold (if "No"), it evaluates the aerial photography plan to be evaluated as inappropriate, and returns to step S10.

[0202] The processor 200-1 resets an aerial photography plan from among the multiple aerial photography plans stored in the memory 210 that has a higher OK house ratio (Na / N) than the aerial photography plan evaluated as inappropriate, and repeats the processing from step S12 to step S24.

[0203] On the other hand, if it is determined in step S24 that the OK house ratio (Na / N) is equal to or greater than the second threshold (if "Yes"), the aerial photography plan to be evaluated is evaluated as appropriate, and this process is terminated.

[0204] [Other] The aerial photography plan setting update unit 206 may automatically revise the aerial photography plan to be evaluated by instructing the aerial photography plan to further improve the appearance rate of the houses in the area of ​​interest. Furthermore, the aerial photography plan may be evaluated based on criteria such as "how many exterior walls are visible" and "the minimum appearance rate of the visible exterior walls," in addition to the appearance rate of the exterior walls of each house.

[0205] Furthermore, in this embodiment, the hardware structure of a processing unit that executes various processes, such as a CPU (Central Processing Unit), is the following various processors: The various processors include a CPU, which is a general-purpose processor that executes software (programs) to function as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0206] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Multiple processing units may also be configured with a single processor. Examples of multiple processing units configured with a single processor include: a first configuration, as typified by client or server computers, in which a single processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units; and a second configuration, as typified by system-on-chip (SoC), in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip. In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0207] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0208] Furthermore, the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications are possible without departing from the spirit of the present invention.

[0209] 10...System 12...Drone 13...Gimbal head 14...Camera 16...Remote controller 20...Aerial photography plan creation device 22...Network 200...Processor 200-1...Processor 200-2...Processor 201...Simulation image generation unit 202...Appearance rate calculation unit 203...Storage processing unit 204...Aerial photography plan evaluation unit 205...Aerial photography plan setting unit 206...Aerial photography plan setting update unit 210...Memory 220...Map database 230...Display device 240...Input / output interface 250...Operation unit H...House IM...Aerial image MP...Map S10 to S24...Steps SI1, SI2, SIi...Simulation image

Claims

1. An aerial photography plan creation device comprising: a memory for storing an aerial photography plan including a flight path of an aircraft and a plurality of photography points on the flight path that are set when photographing a photography area from the air; and a processor, wherein the processor: performs an aerial photography simulation of the photography area in accordance with the aerial photography plan stored in the memory; generates a simulation image that corresponds to an actual aerial photograph image from the aerial photography simulation; calculates, based on the simulation image, percentage information for a subject within the photography area that indicates the proportion of the photographable area or the proportion of the unphotographable area of ​​the subject to the total photography area; and evaluates the set aerial photography plan based on the percentage information.

2. The aerial photography planning device of claim 1, wherein the processor acquires appearance information regarding the appearance of the object to be photographed within the photography area, and generates the simulation image based on three-dimensional information of the photography point, camera information including the focal length and attitude of the camera mounted on the aircraft, and appearance information of the object to be photographed.

3. The aerial photography planning device according to claim 2, wherein the object to be photographed is a house, and the processor acquires the latitude, longitude, and altitude that identify the polygon of the house from a map database as the appearance information.

4. The aerial photography planning device according to claim 1, wherein the object to be photographed is a house, and the processor calculates the proportion information of all houses within the photographing area.

5. The aerial photography plan creation device of claim 4, wherein when the ratio information is an appearance ratio indicating the ratio of the photographable area to the total photographable area of ​​the house, the processor evaluates that the set aerial photography plan is appropriate if Na / N is equal to or greater than a second threshold, where N is the number of houses in the photographable area and Na is the number of houses in the photographable area whose appearance ratio is calculated to be equal to or greater than a first threshold.

6. The aerial photography plan creation device according to claim 5, wherein the processor evaluates as optimal an aerial photography plan in which Na / N is equal to or greater than a second threshold and in which Na / N is maximized among the allowable number of photographs.

7. The aerial photography plan creation device of claim 4, wherein the processor generates a plurality of simulation images corresponding to the plurality of photography points in accordance with the aerial photography plan, calculates proportion information for all houses in the plurality of simulation images, and, when a plurality of proportion information for the same house is calculated based on the plurality of simulation images, adopts the proportion information of the plurality of proportion information with the largest photographable area as the proportion information for the same house.

8. An aerial photography planning device as described in any one of claims 4 to 7, wherein when the processor calculates the proportion information of a specific house among the houses that are the subject of photography, the processor calculates as the proportion information the area of ​​the specific house that appears in the simulation image, or the area of ​​the specific house that is obscured by a house in front and does not appear in the simulation image, and the area of ​​the specific house that appears in the simulation image when the specific house is not obscured by the house in front or if it is assumed that the specific house is not obscured.

9. The aerial photography planning device according to claim 8, wherein the area of ​​the house is an area of ​​the exterior walls excluding the roof.

10. The aerial photography plan creation device according to any one of claims 1 to 7, wherein the processor selects an aerial photography plan that is evaluated as optimal or appropriate from among a plurality of aerial photography plans based on the evaluation.

11. An aerial photography plan creation device as described in any one of claims 1 to 7, wherein the memory stores a plurality of aerial photography plans, and the processor automatically selects a specified aerial photography plan from the plurality of aerial photography plans or accepts manual input of an aerial photography plan selection and sets the aerial photography plan, and if the set aerial photography plan is evaluated as inappropriate, selects and resets from the plurality of aerial photography plans an aerial photography plan that takes a greater number of images than the set aerial photography plan or an aerial photography plan that has a higher proportion of the area that can be photographed of the subject.

12. An aerial photography plan creation device as described in any one of claims 1 to 7, wherein the processor accepts an aerial photography plan including a manually created flight path of the aircraft and multiple photography points along the flight path and stores it in the memory, and if it evaluates that the aerial photography plan stored in the memory is inappropriate, it automatically modifies it to an aerial photography plan with more photography points than the aerial photography plan stored in the memory or an aerial photography plan with a higher proportion of the area that can be photographed of the subject, and updates the aerial photography plan stored in the memory to the automatically modified aerial photography plan.

13. An aerial photography plan creation method executed by an aerial photography plan creation device equipped with a memory that stores an aerial photography plan including a flight path of an aircraft and multiple photography points on the flight path that are set when photographing a photography area from the air, and a processor, wherein the processor executes each of the following steps: a step of simulating aerial photography of the photography area in accordance with the aerial photography plan stored in the memory, and generating a simulation image that corresponds to an actual aerial photograph image through the aerial photography simulation; a step in which the processor calculates, based on the simulation image, percentage information for a subject within the photography area that indicates the proportion of the photographable area or the proportion of the unphotographable area of ​​the subject to the total photography area of ​​the subject; and a step of evaluating the set aerial photography plan based on the percentage information.

14. The aerial photography planning method described in claim 13, wherein the step of generating the simulation image acquires appearance information regarding the appearance of the object to be photographed within the photography area, and generates the simulation image based on three-dimensional information of the photography point, camera information including the focal length and attitude of the camera mounted on the aircraft, and appearance information of the object to be photographed.

15. The aerial photography planning method according to claim 13, wherein the object to be photographed is a house, and the step of calculating the proportion information calculates the proportion information of all houses within the photographing area.

16. The method for creating an aerial photography plan described in claim 15, wherein, when the ratio information is an appearance rate indicating the ratio of the area that can be photographed to the total photography area of ​​the subject, the step of calculating the ratio information calculates the appearance rate of all houses in the photography area, and the step of evaluating the aerial photography plan evaluates the set aerial photography plan as appropriate if Na / N is equal to or greater than a second threshold, where N is the number of houses in the photography area and Na is the number of houses in the photography area that have an appearance rate equal to or greater than a first threshold.

17. The aerial photography plan creation method described in claim 15, wherein the step of calculating the proportion information calculates proportion information for all houses in multiple simulation images corresponding to the multiple photography points in accordance with the aerial photography plan, and when multiple proportion information for the same house is calculated based on the multiple simulation images, the proportion information with the largest photographable area among the multiple proportion information is adopted as the proportion information for the same house.

18. A method for creating an aerial photography plan according to any one of claims 15 to 17, wherein the step of calculating the proportion information, when calculating the proportion information of a specific house among the houses that are the subject of photography, calculates as the proportion information the ratio between the area of ​​the specific house that appears in the simulation image, or the area of ​​the specific house that is obscured by a house in front and does not appear in the simulation image, and the area of ​​the specific house that appears in the simulation image when the specific house is not obscured by the house in front, or if it is assumed that the specific house is not obscured.

19. The aerial photography planning method according to claim 18, wherein the area of ​​the house is an area of ​​the exterior walls excluding the roof.

20. The method for creating an aerial photography plan according to any one of claims 13 to 17, wherein the memory stores a plurality of aerial photography plans, and the processor includes a step of automatically selecting a specified aerial photography plan from the plurality of aerial photography plans or accepting manual input of an aerial photography plan selection and setting the aerial photography plan, and when the step of evaluating the aerial photography plan evaluates the set aerial photography plan as inappropriate, the step of setting the aerial photography plan resets from the plurality of aerial photography plans to an aerial photography plan that will take more images than the set aerial photography plan or an aerial photography plan that will have a higher proportion of the area that can be photographed of the subject, and the processor repeatedly executes the steps of generating the simulation image according to the reset aerial photography plan, calculating the proportion information, and evaluating the aerial photography plan.

21. A method for creating an aerial photography plan as claimed in any one of claims 13 to 17, comprising the steps of: receiving an aerial photography plan including a manually created flight path of the aircraft and a plurality of photography points along the flight path, and storing the plan in the memory; and if the processor evaluates that the aerial photography plan stored in the memory is inappropriate, automatically revising the aerial photography plan to one with more photography points than the aerial photography plan stored in the memory, or to one with a higher proportion of the photographable area of ​​the subject, and updating the aerial photography plan stored in the memory to the automatically revised aerial photography plan; and the processor repeatedly executes the steps of generating the simulation image according to the updated aerial photography plan, calculating the proportion information, and evaluating the aerial photography plan.

Images