Image processing device, image processing method, and program

Abstract

The present invention provides an image processing device, an image processing method, and a program with which it is possible for a user such as a camera operator to easily grasp the image-capturing state of a setting area. The present invention comprises one or more processors, and one or more memories in which programs to be executed by the one or more processors are stored. The processor acquires a setting area by executing a program instruction, acquires an image group in which the setting area is captured using a camera, acquires a map including the setting area, calculates meta information of each image of the image group, calculates the image-capturing state of the setting area on the basis of the meta information of each image, and outputs visualization information in which the calculated result is superimposed on the setting area of the map.

Description

Image processing device, image processing method and program

[0001] The present invention relates to an image processing device, an image processing method, and a program, and more particularly to a technique for visualizing the photographing situation of an area.

[0002] When a disaster occurs, local governments will fly multiple drones simultaneously to take aerial photographs of the local government area and use the aerial images to determine the extent of damage to homes.

[0003] Patent Document 1 discloses a technology for determining the extent of damage to houses by aligning a captured image with map data and comparing and cutting out the areas of individual houses shown in the captured image with the map data.

[0004] International Publication No. 2023 / 047799

[0005] When aerial images are taken of a target area, if there are unphotographed areas, it is difficult to grasp the extent of damage in those areas, so it is preferable that there are as few unphotographed areas as possible. Also, when aerial images have a low ground resolution, it is difficult to grasp the extent of damage in the target area, so it is preferable that they have a high ground resolution. Furthermore, in order to grasp the extent of damage to houses, it is preferable that the aerial images are taken at an appropriate camera angle.

[0006] However, it has been difficult for users such as photographers to understand the shooting conditions of such aerial images.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an image processing device, an image processing method, and a program that allow users such as photographers to easily understand the shooting conditions of a set area.

[0008] In order to achieve the above object, an image processing device according to a first aspect of the present disclosure is an image processing device that includes one or more processors and one or more memories that store programs to be executed by the one or more processors, and the processor executes instructions of the program to acquire a set area, acquire a group of images of the set area taken using a camera, acquire a map including the set area, calculate meta information for each image in the group of images, calculate the shooting conditions of the set area based on the meta information for each image, and output visualized information in which the calculated results are superimposed on the set area on the map.

[0009] According to the first aspect, a user such as a photographer can easily grasp the shooting conditions of the set area.

[0010] In an image processing device according to a second aspect of the present disclosure, in the image processing device according to the first aspect, it is preferable that the processor acquires a plurality of set areas, acquires a group of images in which each set area of ​​the plurality of set areas is photographed, acquires a map including the plurality of set areas, calculates the photographing conditions of each set area based on meta information of each image, and superimposes the calculated results on each set area of ​​the map.

[0011] In the image processing device according to the third aspect of the present disclosure, in the image processing device according to the first or second aspect, it is preferable that the processor superimposes at least one of color, shading, hatching, and contour lines according to the calculated result on the set area of ​​the map.

[0012] In an image processing device according to a fourth aspect of the present disclosure, in an image processing device according to any one of the first to third aspects, it is preferable that the meta information includes latitude and longitude information of the four corners of each image, and the shooting situation is the number of images.

[0013] In an image processing device according to a fifth aspect of the present disclosure, in an image processing device according to any one of the first to fourth aspects, it is preferable that the meta information includes position information and attitude information of the camera at the time of shooting, and the shooting situation is the angular distribution of the camera.

[0014] An image processing device according to a sixth aspect of the present disclosure is an image processing device according to any one of the first to fifth aspects, wherein the map includes information about houses, the meta information includes position information and attitude information of the camera at the time of shooting, the processor calculates the shooting conditions of the set area based on the focal length and sensor size of the camera, and the shooting conditions are preferably a distribution of ground resolution for each house included in the image.

[0015] In an image processing device according to a seventh aspect of the present disclosure, in an image processing device according to any of the first to sixth aspects, it is preferable that the meta information includes latitude and longitude information for the four corners of each image, and the shooting situation is an area coverage rate, which is the ratio between the area of ​​the set area and the area photographed.

[0016] An image processing device according to an eighth aspect of the present disclosure is an image processing device according to any one of the first to seventh aspects, wherein the map includes information about houses, the meta information includes latitude and longitude information for the four corners of each image, and the shooting situation is preferably a house coverage rate, which is the ratio between the number of houses in a set area and the number of houses photographed.

[0017] An image processing device according to a ninth aspect of the present disclosure is preferably the image processing device according to any one of the first to eighth aspects, and outputs text information of the result of calculation for a set area of ​​the map.

[0018] An image processing device according to a tenth aspect of the present disclosure is an image processing device according to any one of the first to ninth aspects, wherein the group of images is preferably a group of images taken by a camera mounted on an aircraft.

[0019] In order to achieve the above object, an image processing method according to an eleventh aspect of the present disclosure is an image processing method executed by one or more processors, in which the processor acquires a set area, acquires a group of images of the set area photographed using a camera, acquires a map including the set area, calculates meta information for each image in the group of images, calculates the shooting conditions of the set area based on the meta information for each image, and outputs visualized information in which the calculated results are superimposed on the set area on the map.

[0020] According to the eleventh aspect, a user such as a photographer can easily grasp the shooting conditions of the set area.

[0021] In order to achieve the above object, a program according to a twelfth aspect of the present disclosure is a program that causes a computer to execute the image processing method according to the eleventh aspect. A non-transitory computer-readable recording medium, such as a CD-ROM (Compact Disk-Read Only Memory), that stores the program according to the twelfth aspect is also included in the present disclosure.

[0022] According to the present invention, a user such as a photographer can easily grasp the shooting conditions of a set area.

[0023] FIG. 1 is a schematic diagram showing an example of the configuration of a photographed image processing system. FIG. 2 is a block diagram showing an example of the electrical configuration of a drone equipped with a camera. FIG. 3 is a block diagram showing an example of the hardware configuration of an image processing device. FIG. 4 is a block diagram showing an example of the functional configuration of an image processing device. FIG. 5 is a flowchart showing each step of an image processing method. FIG. 6 is a diagram showing an example of a setting area. FIG. 7 is a diagram showing an example of a calculation result of a meta information calculation unit. FIG. 8 is a diagram showing an example of visualized information. FIG. 9 is a diagram showing another example of visualized information. FIG. 10 is a diagram showing another example of visualized information. FIG. 11 is a diagram showing another example of visualized information. FIG. 12 is a diagram showing another example of visualized information.

[0024] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In this specification, the same components are designated by the same reference numerals, and redundant explanations will be omitted where appropriate.

[0025] 1 is a schematic diagram showing an example configuration of a photography and image processing system 10. The photography and image processing system 10 includes a drone 12 for aerial photography, a camera 14 mounted on the drone 12, a remote controller 16, and an image processing device 20. The drone 12 is an unmanned aerial vehicle that is remotely controlled by a photographer taking aerial photographs using the remote controller 16, and is an example of an air vehicle. The drone 12 may have an autopilot function that flies according to a program.

[0026] The camera 14 is mounted on the drone 12 via a gimbal head 13. The camera 14 includes an optical system, an image sensor, and a signal processing circuit (not shown). The optical system includes one or more lenses such as a focus lens. The image sensor may be, for example, a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor.

[0027] The camera 14 generates digital image data of the photographed subject by processing signals obtained from the image sensor using a signal processing circuit. The digital image data generated by the camera 14 can be considered an "image." Images captured using the camera 14 are stored in an internal storage device built into the drone 12 and / or a storage device such as a memory card removably attached to the drone 12. In addition, images captured using the camera 14 may be transferred to the remote controller 16 or the image processing device 20 using wireless communication.

[0028] The remote controller 16 is a transmitter that controls the operation of the camera 14 and the drone 12 via wireless communication. The wireless communication may be in the form of a wireless local area network (LAN). The wireless communication may be in the form of a communication using radio waves in the 2.4 GHz band or 5.7 GHz band. The wireless communication may be in the form of a mobile communication network. The communication format for the control signal for operating the drone 12 and the communication format for transferring images captured using the camera 14 may be different or the same.

[0029] The remote controller 16 includes a display 16A, left and right sticks (not shown) for controlling the flight operations of the drone 12, a lever (not shown) for operating the gimbal head 13, a shooting button (not shown) for instructing the camera 14 to take a photograph, and a shooting mode button (not shown) for switching between video shooting and still image shooting.

[0030] The display 16A may be a touch panel display. Various operations on the drone 12, the gimbal head 13, and the camera 14 may be performed by touching the touch panel display. Touch operations include a tap operation, a double tap operation, a flick operation, a swipe operation, a drag operation, a pinch-in operation, and a pinch-out operation.

[0031] The live video captured by the camera 14 is displayed on the display 16A of the remote controller 16. The remote controller 16 also grasps the status of the drone 12, such as its flight position and flight speed, in real time based on data from various sensors provided on the drone 12. Flight information indicating the status of the drone may be displayed on the display 16A.

[0032] The photographed image processing system 10 photographs a plurality of still images (photographed images) from the air using a camera 14 and processes the photographed images in an image processing device 20 .

[0033] The image processing device 20 automatically sets GCPs (Ground Control Points) required for generating a highly accurate 3D model and an orthoimage from a group of aerial images in which adjacent images have overlapping regions. A GCP is a point on the ground whose latitude, longitude, and altitude are known and which is a characteristic topographical point visible on the image. The image processing device 20 may generate a highly accurate 3D model and an orthoimage from a group of aerial images.

[0034] The image processing device 20 is configured by a computer. The computer applied to the image processing device 20 may be a server, a personal computer, or a workstation.

[0035] The image processing device 20 performs data communication with the remote controller 16 via a network 22. The network 22 may be a local area network or a wide area network. The image processing device 20 acquires various information from the drone 12 and the camera 14. The image processing device 20 also acquires map data of the imaging target range from a geographic information system (not shown) via the network 22. The image processing device 20 may acquire the map data in advance before capturing an image with the camera 14, or may acquire the map data after capturing an image with the camera 14.

[0036] 2 is a block diagram showing an example of the electrical configuration of a drone 12 equipped with a camera 14. The drone 12 includes a GPS (Global Positioning System) receiver 30, an air pressure sensor 32, a direction sensor 34, a gyro sensor 36, a motor 38, a processor 40, a storage device 42, a communication interface 44, a battery (not shown), and a battery charging terminal.

[0037] The GPS receiver 30 acquires the latitude and longitude of the position of the drone 12. The barometric pressure sensor 32 detects the barometric pressure at the position of the drone 12. The drone 12 acquires the altitude of the position of the drone 12 based on the barometric pressure detected using the barometric pressure sensor 32. The term "acquire" includes the concept of generating information by data processing such as calculation. The latitude, longitude, and altitude of the drone 12 constitute position information including latitude information, longitude information, and altitude information of the camera 14.

[0038] The orientation sensor 34 may be, for example, a geomagnetic sensor. The drone 12 uses the orientation sensor 34 to detect the azimuth angle at which the lens of the camera 14 is facing.

[0039] The gyro sensor 36 detects a roll angle representing the angle of rotation about the roll axis, a pitch angle representing the angle of rotation about the pitch axis, and a yaw angle representing the angle of rotation about the yaw axis. The drone 12 acquires attitude information of the camera 14 based on the rotation angles acquired using the gyro sensor 36. Note that some or all of the sensors, such as the GPS receiver 30, the air pressure sensor 32, the orientation sensor 34, and the gyro sensor 36, may be located on the camera 14 side.

[0040] The motor 38 is a power source that rotates rotors (not shown). The drone 12 includes a plurality of motors 38 that drive a plurality of rotors.

[0041] The storage device 42 may be a memory, an internal storage, an external storage device, or a combination thereof. The processor 40 serves as a flight controller and performs various calculations necessary for flight control of the drone 12 based on sensor data obtained from various sensors.

[0042] The communication interface 44 is a communication unit that performs wireless communication with the remote controller 16, etc. The communication interface 44 may include a communication terminal that supports wired communication.

[0043] 3 is a block diagram showing an example of the hardware configuration of the image processing device 20. The image processing device 20 includes one or more processors 202, one or more computer-readable media 204, a communication interface 206, an input / output interface 208, and a bus 210.

[0044] The hardware structure of the processor 202 is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various functional units, a GPU (Graphics Processing Unit), which is a processor specialized for image processing, a PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor having a circuit configuration designed specifically for executing specific processing such as an ASIC (Application Specific Integrated Circuit).

[0045] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Multiple functional units may also be configured with a single processor. Examples of multiple functional units configured with a single processor include: a first configuration, as typified by a client or server computer, in which a single processor is configured with a combination of one or more CPUs and software, and this processor operates as multiple functional units; and a second configuration, as typified by a SoC (System on Chip), in which a processor is used to realize the functions of an entire system including multiple functional units on a single IC (Integrated Circuit) chip. In this way, the various functional units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0046] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0047] Processor 202 is connected to computer-readable medium 204 , communication interface 206 , and input / output interface 208 via bus 210 .

[0048] The computer-readable medium 204 stores instructions to be executed by the processor 202. The computer-readable medium 204 includes a memory serving as a main storage device and a storage serving as an auxiliary storage device. The computer-readable medium 204 may be, for example, a semiconductor memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these. The computer-readable medium 204 stores various programs, including an image processing program, and data.

[0049] The communication interface 206 controls communications over the network 22 .

[0050] The input / output interface 208 is connected to an input device 214 and a display device 216 and controls input and output to and from the image processing device 20 .

[0051] The input device 214 is configured by, for example, a keyboard, a mouse, a multi-touch panel, other pointing devices, a voice input device, or an appropriate combination of these.

[0052] The display device 216 is configured by, for example, a liquid crystal display, an organic electro-luminescence (OEL) display, a projector, or an appropriate combination of these.

[0053] The image processing device 20 may include an input device 214 and a display device 216 .

[0054] 4 is a block diagram showing an example of the functional configuration of the image processing device 20. The image processing device 20 includes a set area acquisition unit 100, a captured image acquisition unit 102, a map information acquisition unit 104, a meta information calculation unit 106, a shooting situation calculation unit 108, and a visualization information output unit 110. Each function of the image processing device 20 is realized by the processor 202 executing a program stored in a computer-readable medium 204.

[0055] The set area acquisition unit 100 acquires a set area for visualizing the shooting situation. For example, a user divides an aerial photography area, which is a fixed shooting area, into multiple areas for which the user wishes to visualize the shooting situation. It is preferable that the multiple areas are divided so that there are no gaps between adjacent areas and no overlaps between adjacent areas. Each of the multiple areas may be a different administrative district. The set area acquisition unit 100 acquires each of the divided areas as a multiple set area.

[0056] The captured image acquisition unit 102 acquires a plurality of images (one example of an "image group") of the aerial photography area captured using the camera 14 from the camera 14. The captured image acquisition unit 102 acquires a plurality of images, each of which is a portion of the aerial photography area, and each of which has an overlapping area with adjacent images.

[0057] The map information acquisition unit 104 acquires map information including the image capture areas of the multiple images acquired by the captured image acquisition unit 102, i.e., map information including the aerial capture area. The map information acquisition unit 104 may acquire the map information from the computer-readable medium 204, may acquire the map information from the input device 214, or may acquire the map information via the network 22.

[0058] The meta information calculation unit 106 calculates meta information for each of the multiple images acquired by the captured image acquisition unit 102. The meta information for each image includes, for example, latitude and longitude information including the latitude and longitude of the four corners of the capture area of ​​each image, and the ground resolution of the houses included in each image.

[0059] The meta information calculation unit 106 may acquire sensor data and camera information recorded as exchange image file format (EXIF) data from each image. The sensor data includes the latitude, longitude, altitude, roll angle, pitch angle, and yaw angle of the camera 14 at the time of capture. The camera information includes the sensor size and focal length of the camera 14.

[0060] The meta information calculation unit 106 may match the sensor data and camera information of each image with the map information acquired by the map information acquisition unit 104, and may obtain accurate position information and accurate attitude information of the camera 14 at the time of shooting. The meta information calculation unit 106 may apply geocoding technology to the captured image, and may project road lines on the map onto the captured image with an accuracy of approximately 3 to 5 meters.

[0061] The meta information calculation unit 106 may acquire house information from the map information acquired by the map information acquisition unit 104 .

[0062] The shooting condition calculation unit 108 calculates the shooting conditions of the set area based on the meta information of each image calculated by the meta information calculation unit 106. The shooting conditions of the set area include the number of shots, camera angle distribution, ground resolution distribution, shooting coverage rate, and shooting overlap rate.

[0063] The number of images taken is the number of images taken of the set area. The number of images taken may be the total number of images taken of the set area, or the number of images per unit area. The unit area may be one square kilometer.

[0064] The camera angle is the angle of downward tilt of the optical axis of the lens of the camera 14 relative to the horizontal direction. The camera angle is determined from the attitude information of the camera 14 or the pitch angle of the sensor data. The camera angle distribution is the distribution of the frequency of occurrence of the camera angle for each image. The camera angle distribution is expressed, for example, as a histogram indicating the number of images that belong to each predetermined interval of camera angle values. Furthermore, the camera angle distribution may be the number of images captured directly below and the number of images captured obliquely, or the ratio between these.

[0065] The ground resolution is the size in real space of the houses included in the image, i.e., the size of each pixel of the house captured in the image. The ground resolution is calculated from the sensor size of the camera 14, the actual distances to the four corners of the capture range, and information on the location of the house obtained from map information. The ground resolution distribution is the distribution of the frequency of occurrence of the ground resolution for each house. The ground resolution distribution is expressed, for example, as a histogram showing the number of houses that belong to each of the predetermined intervals of the ground resolution value.

[0066] The photographing coverage rate includes the area coverage rate and the house coverage rate. The area coverage rate is the ratio between the area of ​​the set area and the area of ​​the photographed area of ​​the set area. The area of ​​the set area is determined from the map information of the set area. The area of ​​the photographed area is determined from the latitude and longitude information of the four corners of each image. The house coverage rate is the ratio between the number of houses in the set area and the number of photographed houses. The number of houses in the set area is determined from the house information in the map information. The number of photographed houses is determined from the house information in the map information and the latitude and longitude information of the four corners of each image.

[0067] The shooting overlap rate is the ratio of the area of ​​an image captured in an image that is also captured in other images. The shooting overlap rate is calculated by averaging the ratio of the area of ​​each image's shooting area to the area of ​​the overlapping area of ​​that image.

[0068] The visualization information output unit 110 generates and outputs visualization information in which the calculated results of the shooting conditions are superimposed on a set area of ​​a map. The visualization information output unit 110 may generate and output visualization information in which the calculated results of the shooting conditions are superimposed on a set area of ​​a map according to a predetermined legend. The predetermined legend may fill the set area with at least one of a color, a shade, and a hatching according to the calculated results of the shooting conditions. Filling includes providing transparency so that map information such as roads and houses can be recognized even after filling. The predetermined legend may superimpose a contour line according to the calculated results of the shooting conditions. The contour line may outline the set area with at least one of a color, a shade, and a line type according to the calculated results of the shooting conditions.

[0069] The visualization information output from the visualization information output unit 110 is displayed, for example, on a display device 216. The visualization information may be stored in a computer-readable medium 204, or may be transmitted to a device connected to the network 22 via the communication interface 206.

[0070] The visualization information output from the visualization information output unit 110 allows a user such as a photographer to easily understand the shooting conditions in the set area.

[0071] [Image Processing Method] Figure 5 is a flowchart showing the steps of the image processing method. The image processing method allows a user, such as a photographer, to easily understand the shooting conditions of a set area for multiple aerial images taken using the drone 12. The image processing method is realized by the processor 202 executing an image processing program stored in the computer-readable medium 204. The image processing program may be provided by a computer-readable non-transitory storage medium or via the network 22.

[0072] In step S1, the user sets an area for grasping the shooting situation. The image processing device 20 acquires the area set by the user as a set area.

[0073] For example, a user of a local government taking aerial photographs may divide the local government area, which is the aerial photographing area, into multiple areas in units of districts, towns, blocks, or towns. When the user divides the local government area into multiple areas, the set area acquisition unit 100 acquires the multiple areas as each set area. The user may set an area by inputting a selection instruction using the input device 214 on the map displayed on the display device 216.

[0074] 6 is a diagram showing an example of a set area. Here, the user divides the local government area into multiple areas, "A Town 1-chome," "A Town 2-chome," and "A Town 3-chome." As shown in FIG. 6, "A Town 1-chome" is set as set area AS1, "A Town 2-chome" is set as set area AS2, and "A Town 3-chome" is set as set area AS3.

[0075] In step S2, the image processing device 20 acquires a plurality of images.

[0076] Here, the captured image acquisition unit 102 acquires a plurality of still images that are taken with overlapping portions of the municipality area during a single flight of the drone 12, each of which captures a portion of the municipality area.

[0077] In step S3, the image processing device 20 calculates meta information for each of the multiple images acquired in step S2.

[0078] Here, the map information acquisition unit 104 acquires map information including the local government area, which is the aerial photography area. The meta information calculation unit 106 acquires house information from the map information acquired by the map information acquisition unit 104.

[0079] The meta information calculation unit 106 also acquires exif data from each of the multiple images acquired in step S2. Furthermore, the meta information calculation unit 106 acquires sensor data and camera information from the exif data.

[0080] Next, the meta information calculation unit 106 matches the sensor data and camera information of each image with the map information to determine the accurate position information and accurate orientation information of the camera 14 at the time of shooting. Next, the meta information calculation unit 106 calculates the latitude and longitude information of the four corners of the shooting area of ​​each image based on the determined position information and orientation information of the camera 14. Furthermore, the meta information calculation unit 106 calculates the ground resolution of each house in the image based on the house information and the position information and orientation information of the camera 14.

[0081] Fig. 7 is a diagram showing an example of the calculation results of the meta information calculation unit 106. F7A shown in Fig. 7 is a table showing the camera position information, camera attitude information, and latitude and longitude information of the four corners of the shooting area at the time of shooting for each image. In F7A, latitude and longitude information is calculated as camera position information, and roll angle, pitch angle, and yaw angle are calculated as camera attitude information for each of the images "0001.jpg", "0002.jpg", ...

[0082] F7B in Fig. 7 is a diagram showing an image of the four corners of the shooting area of ​​the image "0001.jpg." In F7B, the latitude and longitude information of the four corners of the shooting area of ​​the image "0001.jpg," (lat1, lon1), (lat2, lon2), (lat3, lon3), and (lat4, lon4), are plotted at the corresponding positions on the map, and the shooting area is indicated by surrounding the plotted points with straight lines.

[0083] F7C in FIG. 7 is a table showing the ground resolution for each image of each house. Here, the house IDs "Bld001," "Bld002," ... are linked to the image and ground resolution that each house has. For example, it shows that the house with house ID "Bld001" is shown with a ground resolution of "5 cm" in image "0001.jpg" and with a ground resolution of "15 cm" in image "0002.jpg."

[0084] Returning to the description of FIG. 5, in step S4, the image processing device 20 tally up the meta information calculated in step S3 for each set area acquired in step S1.

[0085] Here, the shooting situation calculation unit 108 calculates the number of shots, camera angle distribution, ground resolution distribution, and shooting coverage rate for each of the plurality of set areas acquired in step S1.

[0086] Finally, in step S5, the image processing device 20 generates visualized information in which the meta information for each set area collected in step S4 is superimposed on the region of the set area on the map, and outputs the visualized information to the display device 216. The display device 216 displays the visualized information acquired from the image processing device 20.

[0087] The visualized information output in step S5 allows a user such as a photographer to easily understand the shooting conditions in the set area.

[0088] [Display of Visualized Information] Figure 8 is a diagram showing an example of visualized information displayed on the display device 216 in step S5. In visualized information IV1 shown in Figure 8, set areas AS1, AS2, and AS3 are hatched according to the meta information, and a predetermined hatching legend is superimposed on the region of each set area on the map. The meta information includes at least one of the number of images taken, the camera angle distribution, the ground resolution distribution, the image capture coverage rate, and the image capture overlap rate.

[0089] The visualized information IV1 allows the user to understand the differences in the shooting conditions for each set area.

[0090] 8 is displayed on the display device 216, when the user selects one of the setting areas AS1, AS2, and AS3 using the input device 214, a pop-up display of the shooting status of the selected setting area is displayed. The selection operation is, for example, a click operation after moving the mouse cursor to the desired setting area.

[0091] FIG. 9 is a diagram illustrating an example of visualized information displayed on the display device 216 after a setting area is selected. F9A in FIG. 9 illustrates visualized information IV2 when setting area AS2 in visualized information IV1 is selected. In visualized information IV2, a pop-up display PU1 is displayed as text information for setting area AS2. The pop-up display PU1 has a speech bubble shape and includes a rectangular portion indicating information about the setting area and a pointed portion indicating which setting area it is. The pop-up display PU1 includes the "name of the area," "number of images," "camera angle distribution," "ground resolution distribution," and "photography coverage rate." In this example, the "name of the area" is "A-cho 2-chome," the "number of images" is "A," and the "photography coverage rate" is "B%."

[0092] Here, an example has been described in which visualized information IV2 is displayed as a pop-up on visualized information IV1, but the visualized information may be displayed by transitioning from the display screen of visualized information IV1 to the display screen of visualized information IV2.

[0093] 9 shows visualized information IV3 when the setting area AS3 of visualized information IV1 is selected. In visualized information IV3, a pop-up display PU2 is displayed as text information for the setting area AS3. The pop-up display PU2 includes the "name of area," "number of shots," "camera angle distribution," "ground resolution distribution," and "photography coverage rate." In this example, the "name of area" is "A-cho 3-chome," the "number of shots" is "C," and the "photography coverage rate" is "D%."

[0094] In visualization information IV2 and visualization information IV3, values ​​are not displayed for the "camera angle distribution" and "ground resolution distribution," but they may be displayed. For example, for the "camera angle distribution," the "number of images taken directly below" and the "number of images taken at an angle" may be displayed. Furthermore, for the "ground resolution distribution," the "average ground resolution value of all houses" may be displayed.

[0095] The visualized information IV2 and the visualized information IV3 allow the user to understand the shooting conditions of the set area about which the user wants to know more details.

[0096] 9 is displayed on the display device 216, when the user selects one of the items "camera angle distribution" and "ground resolution distribution" using the input device 214, the display device 216 transitions to displaying the visualized information of the selected item. The visualized information of the selected item is, for example, a graph representation of the selected item.

[0097] FIG. 10 is a diagram illustrating an example of visualized information displayed on the display device 216 after an item selection operation. F10A in FIG. 10 illustrates visualized information IV4 when "Camera Angle Distribution" is selected in the pop-up display PU1. The visualized information IV4 includes a histogram showing the distribution of camera angles of images captured in the set area AS2. In the histogram, the horizontal axis represents "camera angle (unit: degrees)" and the vertical axis represents "number of images (unit: images)." Here, images captured at a camera angle of 25 to 34 degrees are classified as "30 degrees," and images captured at a camera angle of 35 to 44 degrees are classified as "40 degrees." Similarly, images captured at a camera angle of 75 to 84 degrees are classified as "80 degrees," and images captured at a camera angle of 85 to 94 degrees are classified as "90 degrees." Furthermore, with regard to "camera angle," "90 degrees" and "80 degrees" are classified as "directly below," and "40 degrees" and "30 degrees" are classified as "diagonal."

[0098] Visualized information IV4 allows the user to grasp the camera angle distribution of the set area of ​​which the user wants to know more details.

[0099] When visualization information IV4 shown in F10A is displayed on the display device 216, if the user selects one of the "camera angles" in the histogram using the input device 214, visualization information for the selected camera angle is displayed. The visualization information for the selected camera angle is, for example, an area captured at the selected camera angle.

[0100] 10 shows visualized information IV5 obtained when "40 degrees" is selected from the camera angles in visualized information IV4. The visualized information IV5 includes a map MP1 of the set area AS2, and superimposed on the map MP1 are 12 areas AP captured using 12 images captured at camera angles ranging from 35 degrees to 44 degrees.

[0101] Visualized information IV5 allows the user to understand the area of ​​the set area that was photographed at the selected camera angle.

[0102] FIG. 11 shows another example of visualized information displayed on the display device 216 after an item selection operation. F11A in FIG. 11 shows visualized information IV6 when "Ground Resolution Distribution" is selected in the pop-up display PU1. Visualized information IV6 includes a histogram showing the distribution of ground resolutions of houses captured in images of the set area AS2. The histogram's horizontal axis represents "ground resolution (unit: centimeters / pixel)" and the vertical axis represents "number of houses (unit: houses)." Here, each house is classified into a ground resolution range of 4 to 30 centimeters / pixel in increments of 1 centimeter / pixel. For example, a house with a ground resolution of 2.5 to 3.4 centimeters / pixel would be classified as "3 centimeters / pixel."

[0103] Visualization information IV6 allows the user to grasp the ground resolution distribution of the set area for which the user wants to know more details.

[0104] When visualization information IV6 shown in F11A is displayed on the display device 216, if the user selects one of the "Ground Resolutions" in the histogram using the input device 214, visualization information of the selected ground resolution is displayed. The visualization information of the ground resolution is, for example, the positions of houses photographed at the selected ground resolution.

[0105] 11 shows visualization information IV7 obtained when "3 centimeters / pixel" is selected from the "Ground Resolution" options in visualization information IV6. Visualization information IV7 includes map MP1 of the set area AS2, and the positions of houses BL with a ground resolution of 3 centimeters / pixel are superimposed on map MP1.

[0106] The visualization information IV7 allows the user to understand which houses in the set area have been photographed at the selected ground resolution.

[0107] FIG. 12 is a diagram showing another example of visualized information displayed on the display device 216. F12A in FIG. 12 shows visualized information IV8. The visualized information IV8 includes a map MP2 that includes an aerial photographed area, and an area AF captured in a photograph is superimposed on the map MP2. The area AF is an area captured in multiple images. The area AF is calculated from latitude and longitude information of the four corners of the captured area in each image. The number of images captured in each area AF may be superimposed on the area AF.

[0108] The visualization information IV8 allows the user to understand the range of the photographed area.

[0109] When the visualized information IV8 shown in F12A is displayed on the display device 216, if the user selects any of the areas AF using the input device 214, the display device 216 will transition to displaying the shooting status of the selected area AF.

[0110] 12 shows visualization information IV9 when one of the areas AF in visualization information IV8 is selected. Visualization information IV9 shows the shooting status for the selected area AF. Visualization information IV9 includes the "number of shots," "camera angle," "ground resolution," and "shooting overlap rate." Here, the "number of shots" is "E (F per square kilometer)," the "camera angle" is "G shots directly below / H shots obliquely," the "ground resolution" is "average I centimeters," and the "shooting overlap rate" is J%.

[0111] When the user selects one of the items "Number of Shots," "Camera Angle," "Ground Resolution," and "Shooting Overlap Rate" using the input device 214 while visualization information IV9 shown in F12B is displayed on the display device 216, the display device 216 may transition to displaying visualization information of the selected item. The visualization information of the selected item may be, for example, a graph representation of the selected item.

[0112] [Others] The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations of the respective embodiments can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention.

[0113] DESCRIPTION OF SYMBOLS 10...Photographed image processing system 12...Drone 13...Gimbal head 14...Camera 16...Remote controller 16A...Display 20...Image processing device 22...Network 30...GPS receiver 32...Atmospheric pressure sensor 34...Orientation sensor 36...Gyro sensor 38...Motor 40...Processor 42...Storage device 44...Communication interface 100...Setting area acquisition unit 102...Photographed image acquisition unit 104...Map information acquisition unit 106...Meta information calculation unit 108...Photographing situation calculation unit 110...Visualization information output unit 202...Processor 204...Computer readable medium 206...Communication interface 208...Input / output interface 210...Bus 214...Input device 216...Display device AF...Area AP...Area AS1...Setting area AS2...Setting area AS3...Setting area BL...House ID...House IV1...Visualized information IV2...Visualized information IV3...Visualized information IV4...Visualized information IV5...Visualized information IV6...Visualized information IV7...Visualized information IV8...Visualized information IV9...Visualized information MP1...Map MP2...Map PU1...Pop-up display PU2...Pop-up display S1 to S5...Steps of image processing method

Claims

1. An image processing device comprising: one or more processors; and one or more memories in which programs to be executed by the one or more processors are stored, wherein the processor executes instructions of the programs to obtain a set area, obtain a group of images of the set area taken using a camera, obtain a map including the set area, calculate meta information for each image in the group of images, calculate the shooting conditions of the set area based on the meta information of each image, and output visualized information in which the calculated results are superimposed on the set area of ​​the map.

2. The image processing device according to claim 1, wherein the processor acquires a plurality of set areas, acquires a group of images in which each of the plurality of set areas has been photographed, acquires a map including the plurality of set areas, calculates the photographing conditions of each of the set areas based on meta information of each of the images, and superimposes the calculated results on each of the set areas of the map.

3. The image processing device according to claim 1, wherein the processor superimposes at least one of a color, a shade, a hatching, and a contour line according to the calculated result on the set area of ​​the map.

4. The image processing device according to claim 1, wherein the meta information includes latitude and longitude information of the four corners of each of the images, and the shooting conditions are the number of the images.

5. The image processing device according to claim 1, wherein the meta information includes position information and attitude information of the camera at the time of shooting, and the shooting conditions are an angle distribution of the camera.

6. The image processing device of claim 1, wherein the map includes information on houses, the meta information includes position information and attitude information of the camera at the time of shooting, the processor calculates the shooting conditions of the set area based on the focal length and sensor size of the camera, and the shooting conditions are a distribution of ground resolution for each house included in the image.

7. The image processing device according to claim 1, wherein the meta information includes latitude and longitude information of the four corners of each of the images, and the shooting conditions are an area coverage rate which is the ratio between the area of ​​the set area and the area photographed.

8. The image processing device according to claim 1, wherein the map includes information on houses, the meta information includes latitude and longitude information for the four corners of each of the images, and the shooting conditions are a house coverage rate which is the ratio between the number of houses within the set area and the number of houses photographed.

9. The image processing device according to claim 1, further comprising: an image processing device which outputs text information of the calculation result for the set area of ​​the map.

10. An image processing device according to any one of claims 1 to 9, wherein the group of images is a group of images taken by the camera mounted on an aircraft.

11. An image processing method executed by one or more processors, wherein the processors acquire a set area, acquire a group of images of the set area taken using a camera, acquire a map including the set area, calculate meta information for each image in the group of images, calculate the shooting conditions of the set area based on the meta information of each image, and output visualization information in which the calculated results are superimposed on the set area of ​​the map.

12. A program for causing a computer to execute the image processing method according to claim 11.

13. A non-transitory computer-readable recording medium having the program according to claim 12 recorded thereon.

Images