Information processing device, information processing method, program, and information processing system

Abstract

Provided are an information processing device, an information processing method, a program, and an information processing system which can assist in specifying an un-photographed house when photographing an investigation range. The information processing device, which comprises a processor and memory that stores a program to be executed by the processor: acquires position and orientation information that indicates the position and orientation of a camera mounted on a flying body; acquires camera configuration information including an optical configuration of the camera; calculates a photographing range of the camera on the basis of the position and orientation information and the camera configuration information; acquires an investigation range map that indicates the investigation range; acquires house-to-be-investigated information including the position of a house to be investigated from the investigation range map; determines whether the house to be investigated is included in the photographing range; specifies the un-photographed house which has never been determined to be included in the photographing range; and overlaps information about the un-photographed house with the investigation range map.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and an information processing system. [Background technology]

[0002] Conventional flight photography systems display images obtained from a camera mounted on an aircraft on a monitor installed on the ground, and the photographer takes the photograph while checking the images displayed on the monitor.

[0003] Patent Document 1 describes a mobile station video transmission system that includes a mobile station equipped with a camera and a receiving station that receives images captured by the camera and transmitted from the mobile station. The system described in this document generates a map image that indicates the position and capture range of the mobile station, etc., using the captured images and accompanying data of the captured images transmitted from the mobile station, and displays the generated map image on a monitor.

[0004] The imaging range of the mobile station is generated by calculating the coordinates of the four corners of the imaging range from the position data of the helicopter that constitutes the mobile station, the attitude angle data of the helicopter, the pointing direction data of the camera, and the zoom ratio of the camera.

[0005] Patent Document 2 describes a structure deterioration inspection support system that generates a three-dimensional model of a structure from a group of images obtained by photographing the structure from multiple viewpoints and displays the three-dimensional model of the structure.

[0006] The system described in the document displays an image selected by the user from multiple images on an individual screen, clearly showing areas of the structure that have not yet been photographed, allowing the user to recognize areas of the structure that have not been inspected for deterioration.

[0007] Patent Document 3 describes a disaster situation assessment system that assesses the situation in disaster-stricken areas based on images taken from the air by aircraft such as helicopters. In the system described in this document, when a disaster occurs, a marker is attached to damaged structures, and relevant information related to the structures is displayed. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Publication No. 11-187363 [Patent Document 2] Japanese Patent Publication No. 2020-1586 [Patent Document 3] International Publication No. 2013 / 051300 Summary of the Invention [Problem to be solved by the invention]

[0009] However, in residential damage assessment surveys conducted by local governments, at least one image of the target house is required. Any houses that are not included in the images taken using the camera mounted on the aircraft are photographed by an on-site surveyor. Therefore, missing images taken using the aircraft can cause delays in the issuance of disaster certificates.

[0010] Conventional aerial photography systems have difficulty determining whether at least one image has been captured for each house under investigation. Furthermore, when photography is performed using a camera mounted on an aircraft, it is difficult to determine which houses have not been photographed from the images displayed on the monitor. Furthermore, it is difficult to guide the aircraft toward unphotographed houses.

[0011] In the system described in Patent Document 1, the photographed area captured by the camera is displayed on a map, and the location of the disaster area can be identified from the map, but it is difficult to identify houses in the disaster area that have not been photographed from the map.

[0012] The system described in Patent Document 2 can identify parts of a structure that have not been inspected for deterioration, but Patent Document 2 does not describe how to identify structures that have not been photographed.

[0013] The system described in Patent Document 3 can grasp the characteristics of damaged structures, but it is difficult to identify unphotographed structures within the photography area.

[0014] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, information processing method, program, and information processing system that can assist in identifying houses that have not been photographed when photographing a survey area. [Means for solving the problem]

[0015] The information processing device of the present disclosure is an information processing device that includes a processor and a memory in which a program to be executed by the processor is stored, and the processor executes instructions of the program to acquire position and attitude information representing the position and attitude of a camera mounted on an aircraft, acquire camera configuration information including the optical configuration of the camera, calculate the camera's shooting range based on the position and attitude information and the camera configuration information, acquire a survey range map representing the survey range, acquire survey target house information including the position of the survey target house on the survey range map, determine whether the survey target house is included in the shooting range, identify unphotographed houses that do not have a history of being determined to be included in the shooting range, and superimpose information about the unphotographed houses on the survey range map.

[0016] According to the information processing device of the present disclosure, unphotographed house images corresponding to unphotographed houses that have not been determined to be included in the camera's shooting range are superimposed on the survey area map. This makes it possible to identify unphotographed houses.

[0017] The survey area map may apply a map of the survey area and an aerial photograph of the survey area as the geographical conditions of the survey area.

[0018] Information Processing Devicemay include a display device, and the processor may transmit a display signal to the display device, the display signal representing the search area map with the unphotographed house image superimposed thereon. In this aspect, the display device may display the search area map with the unphotographed house image superimposed thereon.

[0019] In another aspect of the information processing device, a processor superimposes an unphotographed house image corresponding to an unphotographed house on a survey area map.

[0020] According to this aspect, the visibility of unphotographed houses can be improved.

[0021] In another aspect of the information processing device, a processor superimposes information about a photographed house that has a history of determining that the house to be investigated is included in the photographing range on the investigation range map.

[0022] According to this aspect, it is possible to identify the house that has been photographed.

[0023] In another aspect of the information processing device, a processor superimposes a photographed house image corresponding to a photographed house on a survey area map.

[0024] According to this aspect, the visibility of the photographed house can be improved.

[0025] In an information processing device according to another aspect, a processor superimposes camera position information indicating the position of the camera on the survey area map onto the survey area map.

[0026] According to this aspect, the visibility of the camera position can be improved.

[0027] In an information processing device according to another aspect, a processor superimposes first navigation information indicating a direction in which an unphotographed house is located on a search range map.

[0028] According to this aspect, house It is possible to identify the direction in which

[0029] In this aspect, a first navigation image representing the first navigation information may be applied as the first navigation information.

[0030] In an information processing device according to another aspect, a processor superimposes first navigation information including information indicating the number of unphotographed houses in a direction indicated by the first navigation information onto a survey range map.

[0031] According to this aspect, it is possible to identify the number of unphotographed houses in a direction in which the unphotographed houses exist.

[0032] In another aspect of the information processing device, the processor superimposes second navigation information indicating the distance and direction from the photographed range to the unphotographed house of interest on the survey range map.

[0033] According to this aspect, the direction of the unphotographed house of interest and the distance to the unphotographed house of interest can be identified.

[0034] In this embodiment, the second navigation information may be a second navigation image representing the second navigation information, and the second navigation image may include text information.

[0035] In an information processing device according to another aspect, a processor superimposes third navigation information including a wide-area map obtained by reducing the entire search range map onto the search range map.

[0036] According to this aspect, the location of the survey map within the entire survey area can be identified.

[0037] In this aspect, a third navigation image representing the third navigation information may be used as the third navigation information.

[0038] In an information processing device according to another aspect, a processor acquires, as the position and orientation information, the position of the camera, the altitude of the camera, the direction of the optical axis of the camera, and the depression angle of the optical axis.

[0039] According to this aspect, the angle of view of the camera can be specified.

[0040] In another aspect of the information processing device, the processor determines whether or not a house that has not yet been photographed falls within the photographing range when the photographing range moves in accordance with the movement of the aircraft.

[0041] According to this aspect, photographing can be performed multiple times to obtain photographed images of the entire survey area.

[0042] In another aspect of the information processing device, the processor stores a history of when the house to be investigated is determined to be included in the image capturing range.

[0043] According to this aspect, it is possible to identify houses that have not been photographed based on the determination history of houses to be surveyed.

[0044] The information processing method disclosed herein is an information processing method in which a computer acquires position and attitude information representing the position and attitude of a camera mounted on an aircraft, acquires camera configuration information including the optical configuration of the camera, calculates the camera's shooting range based on the position and attitude information and the camera configuration information, acquires a survey range map representing the survey range, acquires survey target house information including the position of the survey target house on the survey range map, determines whether the survey target house is included in the shooting range, identifies unphotographed houses that do not have a history of being determined to be included in the shooting range, and superimposes information about the unphotographed houses on the survey range map.

[0045] According to the information processing method of the present disclosure, it is possible to obtain the same effects as those of the information processing device of the present disclosure. The constituent elements of the information processing device of other aspects may be applied to the constituent elements of the information processing method of other aspects.

[0046] The program of the present disclosure is a program that enables a computer to perform the following functions: acquire position and attitude information representing the position and attitude of a camera mounted on an aircraft; acquire camera configuration information including the optical configuration of the camera; calculate the camera's shooting range based on the position and attitude information and the camera configuration information; acquire a survey range map representing the survey range; acquire information about the house to be surveyed including the position of the house to be surveyed on the survey range map; determine whether the house to be surveyed is included in the shooting range; identify unphotographed houses that do not have a history of being determined to be included in the shooting range; and superimpose information about unphotographed houses on the survey range map.

[0047] According to the program of the present disclosure, it is possible to obtain the same effects as those of the information processing device of the present disclosure. The components of the information processing device of other aspects may be applied to the components of the program of other aspects.

[0048] The information processing system of the present disclosure is an information processing system comprising an aircraft, a camera mounted on the aircraft, and an information processing device having a processor and a memory for storing a program to be executed by the processor, wherein the processor executes instructions of the program to acquire position and attitude information representing the position and attitude of the camera mounted on the aircraft, acquire camera configuration information including the optical configuration of the camera, calculate the camera's shooting range based on the position and attitude information and the camera configuration information, acquire a survey range map representing the survey range, acquire survey target house information including the position of the survey target house on the survey range map, determine whether the survey target house is included in the shooting range, identify unphotographed houses that do not have a history of being determined to be included in the shooting range, and superimpose information about the unphotographed houses on the survey range map.

[0049] According to the information processing system of the present disclosure, it is possible to obtain the same effects as the information processing device of the present disclosure. The components of the information processing device of other aspects may be applied to the components of the information processing system of other aspects. [Effects of the Invention]

[0050] According to the present invention, the survey area map is overlaid with unphotographed house images corresponding to unphotographed houses that have not been determined to be included in the camera's shooting range. This makes it possible to identify unphotographed houses. [Brief explanation of the drawings]

[0051] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a functional block diagram showing the electrical configuration of the photography drone shown in FIG. [Figure 3] FIG. 3 is a functional block diagram of the controller shown in FIG. [Figure 4] FIG. 4 is a functional block diagram of the terminal device shown in FIG. [Figure 5] FIG. 5 is a block diagram showing an example of the hardware configuration of a terminal device. [Figure 6] FIG. 6 is a flowchart showing the procedure of the information processing method according to the embodiment. [Figure 7] FIG. 7 is a schematic diagram of a method for calculating the angle of view. [Figure 8] FIG. 8 is a schematic diagram of the calculation of the shooting range. [Figure 9] Figure 9 is an explanatory diagram of the survey area map. [Figure 10] FIG. 10 is an explanatory diagram of a survey area map onto which an unphotographed house image is superimposed. [Figure 11] FIG. 11 is an explanatory diagram of the transition of the survey area map. [Figure 12] FIG. 12 is an explanatory diagram of a survey area map according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0052] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In this specification, the same components are designated by the same reference numerals, and redundant explanations will be omitted where appropriate.

[0053] [Configuration example of image processing system according to embodiment] [Overall structure] 1 is a schematic diagram showing an example of the configuration of an information processing system according to an embodiment. The following describes an information processing system that processes various types of information acquired during aerial photography, taking as an example a residential damage certification survey conducted when issuing a disaster damage certificate.

[0054] The information processing system 10 shown in the figure includes an imaging drone 12, a camera 14, a controller 16, a terminal device 18, and a server 20. The imaging drone 12, the camera 14, the terminal device 18, and the server 20 can communicate data with each other via a network 22.

[0055] In Figure 1, the arrow representing the data communication between the photography drone 12 and the network 22 and the arrow representing the data communication between the camera 14 and the network 22 are illustrated using a single arrow.

[0056] The photographing drone 12 and the camera 14 may individually communicate with the network 22, or the data communication between the photographing drone 12 and the network 22 and the data communication between the camera 14 and the network 22 may be shared.

[0057] The photography drone 12 is an unmanned aerial vehicle that is remotely controlled using a controller 16. The photography drone 12 may have an autopilot function that flies according to a specified program. The photography drone 12 described in the embodiment is an example of an air vehicle.

[0058] The camera 14 is supported by a gimbal 13 at the bottom of the main body 12A of the photography drone 12 and is mounted on the photography drone 12. The bottom of the main body 12A of the photography drone 12 refers to the side opposite the rotor 12B when the position where the rotor 12B is attached is the top of the main body 12A of the photography drone 12.

[0059] The gimbal 13 rotatably supports the camera 14. The gimbal 13 may support the camera 14 rotatably in two directions, with each of two mutually orthogonal directions being set as a rotation axis.

[0060] The camera 14 includes an optical system and an image sensor. The optical system includes one or more lenses, such as a focus lens. The image sensor may be a CCD image sensor or a CMOS image sensor. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary Metal-Oxide Semiconductor.

[0061] The optical system and image sensor are not shown in Fig. 1. The photographing lens included in the optical system is shown in Fig. 7 with reference numeral 14B, and the image sensor is shown in Fig. 7 with reference numeral 14A.

[0062] The camera 14 forms an image of the subject light received from the imaging range F on the imaging surface of the imaging element. The imaging element outputs an electrical signal representing the received subject light. The camera 14 generates an image signal of the subject from the electrical signal representing the subject light using a signal processing circuit, and outputs the image signal of the subject. The camera 14 may send an image signal representing a through image to the controller 16. The display 16A of the controller 16 may display the through image.

[0063] The camera 14 photographs the houses in the survey area SA from above the survey area SA. The camera 14 takes at least one photograph and generates one or more still images of the entire survey area SA. The camera 14 may photograph multiple houses in one photograph. Taking a photograph once is synonymous with generating one still image.

[0064] For example, the camera 14 changes the photographing range F in accordance with the movement of the photographing drone 12, photographs the entire range of the survey range SA, and generates multiple still images corresponding to the entire range of the survey range SA.

[0065] The controller 16 transmits wireless control signals to the photographic drone 12 to control the operation of the photographic drone 12. That is, in response to the pilot's operation of the controller 16, control signals indicating the direction and speed of rotation of the rotor 12B are transmitted from the controller 16 to the photographic drone 12. Based on the control signals transmitted from the controller 16, the photographic drone 12 performs movements such as ascending, descending, turning, moving forward, backward, left, and right, and hovering.

[0066] The controller 16 may be in data communication with a terminal device 18 and a server 20 via a network 22 .

[0067] The terminal device 18 functions as an information processing device that supports the operation of the photography drone 12. The terminal device 18 acquires a survey area map that shows geographic information of the survey area SA. The terminal device 18 can superimpose various information, such as information showing the shooting history of the camera 14, onto the survey area map.

[0068] The terminal device 18 includes a display 18A. The terminal device 18 uses the display 18A to display a survey area map showing geographical information of the survey area SA. Details of the survey area map will be described later.

[0069] The terminal device 18 may have the functions of the controller 16. That is, the terminal device 18 may have a function for wireless communication with the photography drone 12 and a function for executing a control program for the photography drone 12.

[0070] A computer is used as the server 20. The computer may be a personal computer or a workstation. The server 20 acquires various types of information transmitted from the filming drone 12, the camera 14, and the terminal device 18.

[0071] The network 22 may employ wireless communication such as the 2.4 GHz band. The network 22 may employ any wired communication type. The network 22 may combine wireless and wired communication types. The network 22 may employ any communication standard. The network 22 may be electrically connected to computers and the like provided at the survey base or the like so as to be able to communicate freely.

[0072] [Example of a drone configuration for filming] Figure 2 is a functional block diagram showing the electrical configuration of the photography drone shown in Figure 1. The photography drone 12 is equipped with a GPS receiver 30, a barometric pressure sensor 32, a direction sensor 34, a gyro sensor 36, and a communication interface 38. Note that GPS is an abbreviation for Global Positioning System.

[0073] The GPS receiver 30 acquires the latitude and longitude of the filming drone 12. The barometric pressure sensor 32 detects the air pressure in the filming drone 12. The filming drone 12 acquires the altitude of the filming drone 12 based on the air pressure detected using the barometric pressure sensor 32. The latitude, longitude, and altitude of the filming drone 12 constitute the position information of the filming drone 12.

[0074] Acquiring altitude information of the photography drone 12 includes calculating the altitude of the photography drone 12 from the air pressure detected using the air pressure sensor 32. In other words, acquiring can include the meaning of generating information, such as calculation, computation, and calculation.

[0075] The orientation sensor 34 detects a predetermined forward direction of the photography drone 12. The photography drone 12 acquires the traveling direction of the photography drone 12 based on the orientation acquired using the orientation sensor 34.

[0076] The gyro sensor 36 detects a roll angle representing a rotation angle about a roll axis, a pitch angle representing a rotation angle about a pitch axis, and a yaw angle representing a rotation angle about a yaw axis. The filming drone 12 acquires attitude information of the filming drone 12 based on the rotation angles acquired using the gyro sensor 36.

[0077] That is, the photography drone 12 may include a processing unit that calculates parameters and the like to be applied to control the photography drone 12 from various information acquired using various sensors. The processing unit may be a combination of a processor and a memory. The processor and memory will be described in detail later.

[0078] The communication interface 38 controls wireless communication of the filming drone 12 via the network 22. The communication interface 38 may include a communication terminal compatible with wired communication.

[0079] The filming drone 12 includes a battery and a charging terminal for the battery. The filming drone 12 may also include a fuel gauge that notifies the remaining battery charge. The fuel gauge may transmit remaining battery charge information to the controller 16 via the communication interface 38. The battery, charging terminal, and fuel gauge are not shown in the drawings.

[0080] [Controller configuration example] Figure 3 is a functional block diagram of the controller shown in Figure 1. The controller 16 includes a user interface 50, an image capture setting unit 52, a flight control unit 54, a sensor data storage unit 56, an image data storage unit 58, an image capture control unit 60, and a communication interface 62.

[0081] The user interface 50 includes operation members such as a joystick and buttons operated by the operator, and a display 16A. The operation members output operation signals corresponding to the operator's operations. The display 16A displays various information related to the control of the filming drone 12.

[0082] The photography setting unit 52 generates photography settings based on the operation of the operator and the plan for the residential damage certification survey, and transmits the photography settings to the camera 14. The camera 14 performs photography based on the photography settings.

[0083] The flight control unit 54 generates flight settings based on the operator's operations and the plan for the residential damage certification survey, and transmits the flight settings to the photographing drone 12. The photographing drone 12 flies based on the flight settings.

[0084] The sensor data storage unit 56 stores the sensor data transmitted from each of the barometric pressure sensor 32, the orientation sensor 34, and the gyro sensor 36 provided in the photography drone 12. The flight control unit 54 controls the flight of the photography drone 12 based on the sensor data stored in the sensor data storage unit 56.

[0085] Image data storage unit 58 stores image data transmitted from camera 14. The image data may be associated with the shooting date and time, shooting conditions set in camera 14, sensor data, etc. Display 16A may display an image represented by the image data stored in image data storage unit 58.

[0086] The photography control unit 60 controls the camera 14 to capture still images based on the photography settings and flight settings. The photography control unit 60 may also control the camera 14 to capture still images in response to an operation by an operator. That is, the photography control unit 60 adjusts the angle of view of the camera 14 at a specified photography position and causes the camera 14 to take a photograph. The photography control unit 60 acquires image data from the camera 14 and stores the acquired image data in the image data storage unit 58.

[0087] The communication interface 62 performs wireless communication with the communication interface 38 provided in the filming drone 12. The communication interface 62 may be electrically connected to the network 22 shown in FIG. 1 so as to be able to freely communicate data with it.

[0088] [Example of terminal device configuration] Fig. 4 is a functional block diagram of the terminal device shown in Fig. 1. The terminal device 18 includes a survey area map acquisition unit 80. The survey area map acquisition unit 80 acquires a map of the survey area or geographical information corresponding to the map from the server 20 shown in Fig. 1.

[0089] The survey area map may be a map of the survey area SA that specifies the addresses of the houses to be surveyed, an aerial photograph of the survey area SA, or a processed map image in which the map components are replaced with schematic images.

[0090] For example, the survey area map may be a map image based on a map of the survey area SA or an aerial photograph of the survey area SA, to which various types of geographical information have been added. The geographical information may include text, photographs, illustrations, diagrams, and symbols.

[0091] Obtaining the search area map may include obtaining geographic information of the search area and generating the search area map based on the geographic information of the search area. Obtaining the search area map may include obtaining a map of the search area and processing the map of the search area.

[0092] A two-dimensional coordinate system is set for the survey range map. A Cartesian coordinate system may be applied to the two-dimensional coordinate system. The origin of the two-dimensional coordinate system may be applied to any position in the survey range map. Any position in the survey range map may be specified using coordinate values.

[0093] The terminal device 18 includes a camera attitude information acquisition unit 82. The camera attitude information acquisition unit 82 acquires sensor information from the photography drone 12 at a specified photography position, and acquires camera attitude information representing the attitude of the camera 14 at the specified photography position based on the acquired sensor information.

[0094] The terminal device 18 includes a camera configuration information acquisition unit 83. The camera configuration information acquisition unit 83 acquires camera configuration information from the camera 14, including the optical configuration of the camera 14, such as the focal length of the photographic lens provided in the camera 14 and the size of the image sensor provided in the camera 14.

[0095] The camera configuration information may be acquired and stored in advance as a constant by the terminal device 18. For example, the terminal device 18 may acquire identification information of the camera 14, and then acquire camera configuration information that matches the identification information from the server 20.

[0096] The terminal device 18 includes a shooting range calculation unit 84. The shooting range calculation unit 84 calculates the shooting range F of the camera 14 on the ground when the shooting drone 12 is located at a specified shooting position, based on the camera attitude information acquired using the camera attitude information acquisition unit 82 and the camera configuration information acquired using the camera configuration information acquisition unit 83.

[0097] The photographing range calculation unit 84 calculates the coordinate values ​​of the photographing range F of the camera 14 in the investigation range map. For example, the photographing range calculation unit 84 calculates the coordinate values ​​of the vertices that form a trapezoid that represents the photographing range F of the camera 14.

[0098] The terminal device 18 includes a survey target house information acquisition unit 86. The survey target house information acquisition unit 86 acquires survey target house information including the address and shape of the survey target house. The survey target house information is applied to identify the survey target house in the survey range map acquired using the survey range map acquisition unit 80.

[0099] The survey target house information acquisition unit 86 calculates the coordinate values ​​of the survey target house in the survey area map. The survey target house information acquisition unit 86 calculates the coordinate values ​​of the vertices that form the polygon representing the survey target house.

[0100] The terminal device 18 is equipped with a photographed house determination unit 88. The photographed house determination unit 88 determines a survey target house that is included in the shooting range F of the camera 14 as a photographed house. The photographed house determination unit 88 stores photographed house information regarding the photographed house. The photographed house determination unit 88 stores the coordinate values ​​of the vertices that make up a polygon representing the photographed house as the photographed house information.

[0101] The photographed house determination unit 88 stores the history of the determination results for all the houses to be investigated. The storage device provided in the terminal device 18 is used to store the history of the determination results.

[0102] The terminal device 18 is equipped with an unphotographed house determination unit 90. The unphotographed house determination unit 90 determines a house that is not included in the shooting range F of the camera 14 as an unphotographed house. The unphotographed house determination unit 90 stores unphotographed house information relating to unphotographed houses. The unphotographed house determination unit 90 stores, as the unphotographed house information, the coordinate values ​​of the vertices that make up a polygon representing an unphotographed house.

[0103] The terminal device 18 includes a survey area map processing unit 92. The survey area map processing unit 92 processes the survey area map, such as superimposing unphotographed house information on the survey area map.

[0104] The survey area map processing unit 92 may superimpose information about the photographed area of ​​the camera 14 at the photographing position where the still image is photographed onto the survey area map. The survey area map processing unit 92 may superimpose information about photographed houses onto the survey area map.

[0105] The survey area map processing unit 92 includes an unphotographed house image generation unit 94. The unphotographed house image generation unit 94 generates an unphotographed house image representing an unphotographed house. The survey area map processing unit 92 may also include a shooting range image generation unit 96. The shooting range image generation unit 96 generates a shooting range image representing the shooting range F of the camera 14. Furthermore, the survey area map processing unit 92 includes a photographed house image generation unit 98. The photographed house image generation unit 98 generates a photographed house image representing a photographed house.

[0106] The display control unit 99 transmits a display signal representing the investigation area map that has been subjected to the above-described processing to the display 18A. For example, the display control unit 99 may transmit a display signal representing the investigation area map on which the photographed house image is superimposed to the display 18A.

[0107] 5 is a block diagram showing an example of the hardware configuration of a terminal device 18. The terminal device 18 includes a processor 100, a computer-readable medium 102 which is a non-transitory tangible entity, a communication interface 104, and an input / output interface 106. The terminal device 18 also includes an input device 110 and a display 18A.

[0108] The processor 100 includes a CPU (Central Processing Unit). The processor 100 may also include a GPU (Graphics Processing Unit). The processor 100 is connected to a computer-readable medium 102, a communication interface 104, and an input / output interface 106 via a bus 108. An input device 110 and a display 18A are connected to the bus 108 via the input / output interface 106.

[0109] The computer-readable medium 102 includes a memory serving as a main storage device and a storage serving as an auxiliary storage device. The computer-readable medium 102 may be a semiconductor memory such as a read-only memory (ROM) and a random access memory (RAM), a hard disk drive, a solid-state drive, or the like. The computer-readable medium 102 may be any combination of multiple devices.

[0110] A hard disk drive may be referred to as an HDD, which is an abbreviation of the English term Hard Disk Drive, and a solid state drive may be referred to as an SSD, which is an abbreviation of the English term Solid State Drive.

[0111] The terminal device 18 is connected to the network 22 shown in FIG. 1 via the communication interface 104, and is connected to the filming drone 12, the server 20, etc. via the network 22 so as to be able to communicate with them.

[0112] The computer-readable medium 102 stores various programs corresponding to various functions of the terminal device 18, such as an image processing program 120, a storage control program 122, a communication control program 124, and a display control program 126.

[0113] The image processing program 120 corresponds to the processing functions of the investigation range map, such as the investigation range map acquisition unit 80 and the investigation range map processing unit 92 shown in Fig. 5. For example, the image processing program 120 performs processing of the investigation range map acquired using the investigation range map acquisition unit 80.

[0114] The image processing program 120 also generates images representing various information to be superimposed on the survey area map, such as information on houses that have not been photographed, and performs processing to superimpose the various information on the survey area map.

[0115] The storage control program 122 controls the storage of various types of data. The storage control program 122 controls the storage of image data representing various types of images generated using the image processing program 120. The storage control program 122 controls the storage of various types of information acquired by the terminal device 18.

[0116] The communication control program 124 controls the communication carried out via the communication interface 104. The display control program 126 controls the display of various information on the display 18A.

[0117] Various programs stored in the computer-readable medium 102 include one or more instructions. The computer-readable medium 102 stores various data, various parameters, and the like.

[0118] In the terminal device 18, the processor 100 executes various programs stored in the computer-readable medium 102, thereby realizing various functions of the terminal device 18. Note that the term "program" is synonymous with the term "software."

[0119] The terminal device 18 performs data communication with an external device via a communication interface 104. The communication interface 104 may be compliant with various standards such as USB (Universal Serial Bus). The communication form of the communication interface 104 may be either wired communication or wireless communication.

[0120] An input device 110 and a display 18A are connected to the terminal device 18 via the input / output interface 106. The input device 110 may be an input device such as a keyboard or a mouse. A touch panel system may be used for the input device 110.

[0121] The display 18A may be a liquid crystal display, an organic EL display, or any combination of multiple devices. The EL in organic EL display is an abbreviation for Electro-Luminescence.

[0122] Examples of the hardware structure of the processor 100 include a CPU, a GPU, a PLD (Programmable Logic Device), and an ASIC (Application Specific Integrated Circuit). A CPU is a general-purpose processor that executes programs and functions as various functional units. A GPU is a processor specialized for image processing.

[0123] A PLD is a processor whose electrical circuit configuration can be changed after the device is manufactured. An example of a PLD is an FPGA (Field Programmable Gate Array). An ASIC is a processor with dedicated electrical circuitry designed specifically to perform a specific task.

[0124] A processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types. Examples of combinations of various processors include a combination of one or more FPGAs and one or more CPUs, and a combination of one or more FPGAs and one or more GPUs. Another example of a combination of various processors is a combination of one or more CPUs and one or more GPUs.

[0125] A single processor may be used to configure multiple functional units. An example of using a single processor to configure multiple functional units is a configuration in which a single processor is configured by applying a combination of one or more CPUs and software, such as an SoC (System On a Chip), which is typified by a computer such as a client or server, and this processor operates as multiple functional units.

[0126] Another example of using one processor to configure multiple functional units is to use a processor that uses one IC chip to realize the functions of an entire system including multiple functional units. Note that IC is an abbreviation for Integrated Circuit.

[0127] In this way, the various functional units are configured as hardware structures using one or more of the various processors described above.More specifically, the hardware structures of the various processors described above are electric circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0128] The hardware configuration of the terminal device 18 shown in FIG. 5 can also be applied to the control device, controller 16, and server 20 of the photographing drone 12 shown in FIG.

[0129] [Procedure of information processing method according to embodiment] 6 is a flowchart showing the procedure of the information processing method according to the embodiment. In the search range map acquisition step S10, the search range map acquisition unit 80 shown in FIG.

[0130] In the survey target house information acquisition step S12, the survey target house information acquisition unit 86 acquires the survey target house information from the server 20. A survey target house information display step may be executed to display the survey target house information acquired in this step.

[0131] In the position and attitude information acquisition step S14, the camera attitude information acquisition unit 82 acquires position information and attitude information of the camera 14. In this step, the camera attitude information acquisition unit 82 acquires sensor information from various sensors equipped in the photography drone 12, and can calculate the position and attitude of the camera 14 based on the acquired sensor information. A camera position and attitude information display step may be executed to display the position information and attitude information of the camera 14 acquired in this step.

[0132] In a camera configuration information acquisition step S16, the camera configuration information acquisition unit 83 acquires pre-stored camera configuration information of the camera 14. In this step, the camera configuration information stored in the controller 16 or the server 20 may be acquired. A camera configuration information display step may be executed to display the camera configuration information acquired in this step.

[0133] In the shooting range calculation step S18, the shooting range calculation unit 84 calculates the shooting range F of the camera 14 using the position information of the camera 14, the attitude information of the camera 14, and the camera configuration information of the camera 14. A shooting range information storage step may be executed to store the shooting range information generated in this step.

[0134] In the photographed house determination process S20, the photographed house determination unit 88 determines the surveyed houses included in the shooting range F of the camera 14 calculated in the shooting range calculation process S18 as photographed houses. A photographed house information storage process may be executed to store the photographed house information generated in this process. In addition, a determination history storage process may be executed to store the history of the determination results in the photographed house determination process S20 for all surveyed houses.

[0135] In the unphotographed house determination step S22, the unphotographed house determination unit 90 determines unphotographed houses among the houses under investigation that have not been determined to be photographed houses in the photographed house determination step S20. An unphotographed house information storage step may be performed to store unphotographed house information generated in the same step. The photographed house determination step S20 and the unphotographed house determination step S22 may be performed in parallel.

[0136] In the survey area map processing step S24, the survey area map processing unit 92 processes the survey area map. In this step, the unphotographed house image generation unit 94 generates an unphotographed house image representing an unphotographed house, and the survey area map processing unit 92 performs processing to superimpose the unphotographed house image on the survey area map.

[0137] In this process, the shooting range image generation unit 96 can generate a shooting range image representing the shooting range F of the camera 14. Also, in this process, the photographed house image generation unit 98 can generate a photographed house image representing the photographed house. The survey range map processing unit 92 can perform processing to superimpose the shooting range image, etc. on the survey range map.

[0138] In the investigation area map display step S26, the display control unit 99 transmits a display signal representing the processed investigation area map to the display 18 A. The display 18 A displays the investigation area map on which the unphotographed house image is superimposed.

[0139] In the shooting range update determination step S28, the terminal device 18 determines whether or not the shooting range F of the camera 14 has been updated. Examples of when the shooting range F of the camera 14 is updated include moving the shooting position and reshooting.

[0140] For example, when the camera attitude information acquisition unit 82 acquires new position information of the camera 14, the terminal device 18 determines that the shooting range F of the camera 14 has been updated. Furthermore, when a user input signal indicating that reshooting will be performed is acquired, the terminal device 18 may determine that reshooting will be performed.

[0141] In this step, if the terminal device 18 determines that the shooting range F of the camera 14 has been updated, the result is a Yes determination. If the determination is Yes, the process proceeds to the position and orientation information acquisition step S14, and the steps from the position and orientation information acquisition step S14 to the shooting range update determination step S28 are executed until the determination is No in the shooting range update determination step S28. If the shooting range F has been updated, the camera configuration information acquisition step S16 may be omitted.

[0142] On the other hand, in the shooting range update determination step S28, if the terminal device 18 determines that the shooting range F of the camera 14 has not been updated, the determination is No. If the determination is No, a specified termination process is executed, and the procedure of the information processing method is terminated.

[0143] [Example of shooting range calculation] Calculation of the imaging range F of the camera 14 shown in FIG. 1 includes calculation of the angle of view of the camera 14 and calculation of the imaging range F of the camera 14 based on the angle of view of the camera 14.

[0144] 7 is a schematic diagram of a method for calculating the angle of view. Let D be the size of image sensor 14A provided in camera 14, f be the focal length of photographing lens 14B provided in camera 14, and L be the photographing distance. The angle of view α when there is no distortion from photographing lens 14B is expressed as α = (D × L) / f. Here, the angle of view α refers to the length of one side of the photographing range F when it is a rectangle corresponding to the planar shape of image sensor 14A. When photographing range F is rectangular, the length of each of the two mutually perpendicular sides is calculated as the angle of view α.

[0145] 8 is a schematic diagram of how the shooting range is calculated. The shooting range of camera 14 on the ground G is the cross section of an imaginary quadrangular pyramid whose apex is the principal point of shooting lens 14B of camera 14 and whose perpendicular line is the optical axis A of shooting lens 14B, intersecting with the ground G.

[0146] The flight altitude H of the photography drone 12 corresponds to the photography distance L shown in Fig. 7. The flight altitude H is calculated by correcting the detection value of the atmospheric pressure sensor 32 shown in Fig. 2 using the relative height. The flight altitude may be stored as a value based on GPS positioning acquired using the GPS receiver 30 and a value based on the detection value of the atmospheric pressure sensor 32.

[0147] For example, the flight altitude H based on GPS positioning may be acquired immediately after the photography drone 12 takes off, and the flight altitude H may be corrected using the detection value of the barometric pressure sensor 32 while the photography drone 12 is flying.

[0148] θ is the depression angle of the camera 14, and is calculated based on the attitude information of the photographing drone 12 and the attitude information of the camera 14. Symbol P1 represents the photographing position of the photographing drone 12. The coordinate value of the photographing position P1 is calculated based on the latitude information and longitude information of the photographing drone 12.

[0149] Point P2 is the point where the optical axis A of the photographing lens 14B intersects with the ground G. Point P3 is one end of the photographing range. Point P4 is the other end of the photographing range.

[0150] The coordinate values ​​of point P3 and point P4 can be calculated by applying the parameters shown in Fig. 8 to geometric calculations. The length β of any one side constituting the shooting range is calculated as the distance between point P3 and point P4.

[0151] The shooting range of the camera 14 on the ground G provided on the filming drone 12 during flight has a trapezoidal shape with a relatively short base on the side closer to the camera 14 and a relatively long base on the side farther from the camera 14. The ground here refers to a surface that serves as a reference for the altitude of the filming drone 12.

[0152] The photographing range of the camera 14 is determined by the secondary image defined in the survey area map. Former For example, the imaging range of the camera 14 can be defined using the coordinate values ​​of each vertex of a trapezoid representing the imaging range.

[0153] The imaging range of the camera 14 may be defined using position information of the imaging drone 12. For example, the latitude and longitude of each vertex of a trapezoidal shape representing the imaging range may be calculated based on the latitude and longitude of the imaging drone 12.

[0154] [Example of survey area map] 9 is an explanatory diagram of the survey area map. The figure shows a survey area map 200 corresponding to a part of the survey area SA, corresponding to the size of the display 18A of the terminal device 18. The survey area map 200 has the photographing range 204 of the camera 14 and the position information 206 of the photographing drone 12 superimposed thereon. The vertices P constituting the photographing range 204 of the camera 14 are 11, vertex P 12 , vertex P 13 and vertex P 14 The coordinate values ​​on the investigation area map 200 are defined for the position information 206 of the photographing drone 12. Similarly, the coordinate values ​​on the investigation area map 200 are defined for the position information 206 of the photographing drone 12. The photographing area 204 corresponds to the photographing area F shown in FIG.

[0155] 9 shows an example in which an image resembling the photography drone 12 is applied as the position information 206 of the photography drone 12, but any image, text information, etc. may be applied as the position information 206 of the photography drone 12. Note that the position information 206 of the photography drone 12 described in the embodiment is an example of camera position information.

[0156] When an operator, such as the pilot of the photographing drone 12, touches any position on the display 18A, the terminal device 18 may enlarge and display the survey area map 200, centered on the position on the survey area map 200 corresponding to the touched position.

[0157] Furthermore, when the operator scrolls an arbitrary position on the display 18A, the terminal device 18 may change the area of ​​the investigation range map 200 to be displayed on the display 18A in accordance with the operator's scrolling.

[0158] That is, when the operator operates the terminal device 18, the terminal device 18 may change the display mode of the survey area map 200 displayed on the display 18A in accordance with the operator's operation. Changes in the display mode of the survey area map 200 may include zooming in, zooming out, shifting, and the like.

[0159] 10 is an explanatory diagram of a survey area map onto which an unphotographed house image is superimposed. The survey area map 220 shown in the figure has a survey target house image superimposed thereon as survey target house information. house The images are classified into unphotographed house images 210 representing houses that have not been photographed and photographed house images 212 representing houses that have been photographed.

[0160] The image of the house to be surveyed is a polygon corresponding to the outer shape of the house to be surveyed. 21 , vertex P 22 , vertex P 23 and vertex P 24 The coordinate values ​​of each vertex in the survey area map 200 are defined for each vertex of the survey target house image. The coordinate values ​​of the vertices constituting the survey target house image are used to determine whether the survey target house is a photographed house or an unphotographed house. In addition, in FIG. 10, the symbol P representing the vertex is attached to each vertex constituting an arbitrary unphotographed house image 210 which is the survey target house image. 21 , P 22 , P 23 , P 24 , P 25 , P 26 and P 27 is attached.

[0161] An unphotographed house is a house under investigation that has not been determined to be within the photographing range 204 of the camera 14. A photographed house is a house under investigation that has been determined to be within the photographing range 204 of the camera 14 at least once.

[0162] Examples of buildings that may be subject to the survey include buildings in each city, town, or village that are listed in the Basic Resident Register. In other words, the buildings subject to the survey are determined by the local government where the residential damage certification survey is conducted.

[0163] Before the shooting range 204 for the first shooting is determined, the survey range map 200 may have unphotographed house images 210 created for all surveyed houses, and the survey range map 200 may have unphotographed house images 210 corresponding to all surveyed houses superimposed thereon.

[0164] In response to the determination of the photographing range 204, a determination is made as to which houses have been photographed and which have not been photographed, and in response to the determination result, an unphotographed house image 210 and a photographed house image 212 are superimposed on the survey range map 220.

[0165] Fig. 11 is an explanatory diagram of the transition of the investigation range map. Fig. 11 shows the transition of the investigation range map when the investigation range is photographed at the first photographing position and the second photographing position, which are different from each other. The investigation range map 230 is the investigation range map at the first photographing position. Inspection standard The first photographing position is generated based on photographing the surrounding area. The position information 206A of the photographing drone 12 is superimposed on the first photographing position.

[0166] 4 determines that an unphotographed house included in the photographing range 204A in the survey range map 230 is a photographed house. The unphotographed house determination unit 90 determines that an unphotographed house, the coordinate values ​​of one or more vertices of which in the unphotographed house image 210 are not included in the photographing range 204A, is an unphotographed house not included in the photographing range 204A.

[0167] 11, all vertices of the unphotographed house image 210A are included in the photographing range 204A, so the unphotographed house corresponding to the unphotographed house image 210A is determined to be a photographed house. For a house determined to be a photographed house, a photographed house image 212 having the same external shape as the unphotographed house image 210A is superimposed in place of the unphotographed house image 210A.

[0168] The survey area map 240 is generated based on the photographing of the survey area at the second photographing position. For example, the second photographing position is the next to be photographed after the first photographing position. Inspection standard The second photographing position is a photographing position where photographing of the surrounding area is performed. but are superimposed.

[0169] In the survey area map 240, a photographed house image 212 is superimposed at the position of an unphotographed house image 210A that has been determined to be a photographed house in the survey area map 230. A display mode is applied to the photographed house image 212 so that it can be visually distinguished from the unphotographed house image 210. As an example of a display mode applied to the photographed house image 212, a mode that makes the color, brightness, pattern, etc. different from the unphotographed house image 210 may be applied.

[0170] 4 determines that an unphotographed house included in the photographing range 204B in the investigation range map 240 is a photographed house. At the position of an unphotographed house determined to be a photographed house, a photographed house image is superimposed in place of the unphotographed house image 210B.

[0171] In this way, the photographing drone 12 is moved and the photographing position within the survey area is changed in sequence, and the survey target house that is included in the photographing range 204 of the camera 14 at least once is superimposed with the photographed house image 212 as a photographed house.

[0172] [Variation of the survey area map] 12 is an explanatory diagram of a survey area map according to a modified example. The survey area map 250 shown in the figure includes navigation information. In the figure, the navigation information includes a first navigation image 252, a second navigation image 254, and a third navigation image 256.

[0173] The first navigation image 252 is an arrow line indicating the direction in which there are unphotographed houses based on the position of the photographing drone 12. The first navigation image 252 also includes text information indicating the number of unphotographed houses. The number of unphotographed houses applied to the first navigation image 252 is the number of unphotographed houses within a range that is a specified distance from the position of the photographing drone 12.

[0174] 12, first navigation images 252 are arranged for each of the north, south, east, and west directions. The survey area map 250 may include a first navigation image 252 facing northeast and a first navigation image 252 facing southwest, etc. The survey area map 250 may include a first navigation image 252 that faces northeast and a first navigation image 252 that faces southwest, etc. The survey area map 250 may include a first navigation image 252 that has zero unphotographed houses that has been deleted.

[0175] The second navigation image 254 includes an arrow indicating the direction of the unphotographed house of interest at the position of the photographing drone 12 and text information indicating the distance from the position of the photographing drone 12 to the unphotographed house of interest.

[0176] The distance from the position of the photographing drone 12 to the unphotographed house of interest may be the distance from the position of the photographing drone 12 to the position on the flight path of the photographing drone 12 that corresponds to the position of the unphotographed house of interest.

[0177] A wide-area map showing the entire survey area SA is applied to the third navigation image 256. In the third navigation image 256, a frame 258 showing the area corresponding to the survey area map 250 is superimposed on the wide-area map.

[0178] The first navigation image 252 described in the embodiment is an example of first navigation information, the second navigation image 254 described in the embodiment is an example of second navigation information, and the third navigation image 256 described in the embodiment is an example of third navigation information.

[0179] [Operational Effects of the Information Processing System and Information Processing Method According to the Embodiment] The information processing system and information processing method according to the embodiment can achieve the following advantageous effects.

[0180] [1] An investigation area map 220 showing geographical information of the investigation area is superimposed with an imaging area 204 on the ground G of the camera 14 provided in the imaging drone 12 and an unphotographed house image 210 showing an unphotographed house. The investigation area map 220 with the unphotographed house image 210 superimposed thereon is displayed on the display 18A of the terminal device 18.

[0181] This allows the operator of the photographing drone 12 to recognize the shooting range of the camera 14 during flight and the houses that have not yet been photographed, and can fly the photographing drone 12 toward the houses that have not yet been photographed.

[0182] [2] The survey area map 220 has photographed house images 212 representing photographed houses superimposed thereon. The survey area map 220 with the photographed house images 212 superimposed thereon is displayed on the display 18A of the terminal device 18. This allows the operator of the photographing drone 12 to distinguish between unphotographed houses and photographed houses.

[0183] [3] The survey area map 220 has the position information 206 of the filming drone 12 superimposed thereon. This allows the operator of the filming drone 12 to understand the positions of the filming drone 12 and the camera 14.

[0184] [4] The survey area map 250 has a first navigation image 252 superimposed thereon, which indicates the direction in which unphotographed houses exist, etc. This allows the operator of the photography drone 12 to ascertain the direction in which unphotographed houses exist.

[0185] [5] The first navigation image 252 is an unphotographed image in the direction indicated by the first navigation image 252. house This includes the number of unfilmed house The camera drone 12 may be guided according to the number of cases.

[0186] [6] A second navigation image 254 is superimposed on the survey area map 250, showing the direction from the camera 14 to the unphotographed house of interest and the distance from the camera 14 to the unphotographed house of interest. This allows the operator of the photography drone 12 to understand the direction and distance from the camera 14 to the unphotographed house of interest.

[0187] [7] Survey Area Map 250 is the survey area map. Inspection standard A third navigation image 256 including a wide-area map showing the entire area is superimposed. Inspection standard The position of the filming drone 12 within the entire area can be determined.

[0188] [8] An unphotographed house in which all vertices constituting the unphotographed house image 210A are included in the photographed range 204A is determined to be a photographed house. This can prevent missing parts from appearing in the photographed image of the photographed house.

[0189] For all houses to be surveyed, if they are determined to be houses that have already been photographed, the history of the determination results is stored, which makes it possible to avoid photographing the same house twice.

[0190] [Modification of the information processing system according to the embodiment] A part of the information processing performed in the terminal device 18 may be performed by other devices of the terminal device 18, such as the controller 16 and the server 20. For example, the calculation of the imaging range of the camera 14 on the ground G may be performed by the controller 16.

[0191] The terminal device 18 can acquire the survey area map 220, the position of the survey target house, and the size of the survey target house from the server 20, generate an unphotographed house image 210 in response to the photographing performed by the camera 14, and superimpose the unphotographed house image 210 on the survey area map 220. The terminal device 18 can also acquire the survey area map 220 in which a two-dimensional coordinate system is set in advance.

[0192] The above-described embodiments of the present invention may be modified, added, or deleted as appropriate within the scope of the spirit of the present invention. The present invention is not limited to the above-described embodiments, and many modifications may be made by a person skilled in the art within the technical concept of the present invention. Furthermore, the embodiments, modifications, and applications may be implemented in appropriate combinations. [Explanation of symbols]

[0193] 10 Information Processing Systems 12 Filming drone 12A main unit 12B rotor blade 13 Gimbal 14 Camera 14A image sensor 14B shooting lens 16 Controllers 18 Terminal Equipment 18A Display 20 servers 22 Network 30 GPS receivers 32 Barometric pressure sensor 34 Orientation sensor 36 Gyro sensor 38 Communication Interface 50 User Interface 52 Shooting Settings 54 Flight control unit 56 Sensor data storage unit 58 Image data storage unit 60 Shooting control unit 62 Communication Interface 80 Survey area map acquisition section 82 Camera posture information acquisition unit 83 Camera configuration information acquisition unit 84 Shooting range calculation unit 86 Survey target house information acquisition department 88 Photographed House Identification Unit 90 Unphotographed House Identification Section 92 Survey Area Map Processing Section 94 Unphotographed house image generation unit 96 Shooting range image generation unit 98 Photographed house image generation unit 99 Display control section 100 processors 102 Computer-readable medium 104 Communication Interface 106 Input / Output Interface 108 Bus 110 Input Device 120 Image Processing Program 122 Memory Control Program 124 Communication Control Program 126 Display Control Program 200 Survey Area Map 204 Shooting range 204A Shooting Range 204B Shooting Range 206 Location information of filming drone 12 206A Location information of filming drone 12 210 Unphotographed house images 210A Unphotographed house image 210B Unphotographed house image 212 house images 220 Survey Area Map 230 Survey Area Map 240 Survey Area Map 250 Survey Area Map 252 First Navigation Image 254 Secondary Navigation Image 256 Third Navigation Image S10~S28 Each step of the information processing method

Claims

1. a processor; a memory for storing a program to be executed by the processor; An information processing device comprising: The processor executes the instructions of the program, acquiring position and orientation information representing the position and orientation of a camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing device that superimposes information about the unphotographed houses on the survey area map in a display format that allows the information to be visually distinguished.

2. The information processing device according to claim 1 , wherein the processor superimposes an unphotographed house image corresponding to the unphotographed house on the survey area map.

3. The information processing device according to claim 2 , wherein a polygon corresponding to a periphery shape of the unphotographed house is applied to the unphotographed house image.

4. A processor; a memory for storing a program to be executed by the processor; An information processing device comprising: The processor executes the instructions of the program, acquiring position and orientation information representing the position and orientation of a camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing device that superimposes, on the survey area map, an unphotographed house image to which a polygon corresponding to the outer periphery shape of the unphotographed house is applied as information about the unphotographed house.

5. An information processing device described in any one of claims 1 to 4, wherein the processor determines that the house being surveyed is an unphotographed house if one or more of the multiple vertices of a polygon corresponding to the outer peripheral shape of the house being surveyed are not included in the shooting range.

6. The information processing device according to claim 1 , wherein the processor superimposes information about a photographed house that has a history of determining that the house to be investigated is included in the photographing range on the investigation range map.

7. The information processing device according to claim 6 , wherein the processor superimposes a photographed house image corresponding to the photographed house on the survey area map.

8. The information processing device described in claim 6 or 7, wherein the processor determines that the house under investigation is a house that has been photographed when all of the multiple vertices of a polygon corresponding to the outer peripheral shape of the house under investigation are included in the photographing range.

9. The information processing device according to claim 1 , wherein the processor superimposes camera position information indicating a position of the camera on the survey area map onto the survey area map.

10. The information processing device according to claim 1 , wherein the processor superimposes first navigation information indicating a direction in which the unphotographed house is located on the survey area map.

11. a processor; a memory for storing a program to be executed by the processor; An information processing device comprising: The processor executes the instructions of the program, acquiring position and orientation information representing the position and orientation of a camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing device that superimposes information about the unphotographed house on the survey area map, The processor superimposes first navigation information indicating a direction in which the unphotographed house is located on the survey range map; an information processing device that superimposes the first navigation information, which includes information indicating the number of houses that have not yet been photographed in the direction indicated by the first navigation information, on the survey area map;

12. The information processing device according to claim 1 , wherein the processor superimposes second navigation information indicating a distance and a direction from the photographing range to the unphotographed house of interest on the survey range map.

13. The information processing device according to claim 1 , wherein the processor superimposes third navigation information, including a wide-area map obtained by reducing the entire search range map, on the search range map.

14. The information processing apparatus according to claim 1 , wherein the processor acquires, as the position and orientation information, a position of the camera, an altitude of the camera, a direction of an optical axis of the camera, and a depression angle of the optical axis.

15. The information processing device according to claim 1 , wherein the processor determines whether the unphotographed house falls within the photography range when the photography range moves in accordance with the movement of the flying object.

16. The information processing device according to claim 1 , wherein the processor stores a history of when the house to be investigated is determined to be included in the image capturing range.

17. The computer acquiring position and orientation information representing the position and orientation of a camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing method in which information about the unphotographed houses is superimposed on the survey area map in a display format that allows visual distinction.

18. The computer acquiring position and orientation information representing the position and orientation of a camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing method for superimposing information about the unphotographed houses on the survey area map, superimposing first navigation information indicating a direction in which the unphotographed house exists on the survey area map; An information processing method for superimposing the first navigation information, which includes information indicating the number of houses not yet photographed in the direction indicated by the first navigation information, on the survey area map.

19. A computer comprising: acquiring position and orientation information representing the position and orientation of a camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing method for superimposing, on the survey area map, an unphotographed house image to which a polygon corresponding to the outer periphery shape of the unphotographed house is applied as information about the unphotographed house.

20. On the computer, A function of acquiring position and attitude information representing the position and attitude of a camera mounted on an aircraft; obtaining camera configuration information including the optical configuration of the camera; a function of calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; The ability to obtain a survey area map showing the survey area; A function of acquiring information about the house to be surveyed, including the location of the house to be surveyed on the survey area map; a function of determining whether the house to be surveyed is included in the photographing range; A function to identify unphotographed houses that have not been determined to be included in the photography range; and A program that realizes the function of superimposing information about the unphotographed houses on the survey area map in a display format that allows visual distinction.

21. On the computer, A function of acquiring position and attitude information representing the position and attitude of a camera mounted on an aircraft; obtaining camera configuration information including the optical configuration of the camera; a function of calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; The ability to obtain a survey area map showing the survey area; A function of acquiring information about the house to be surveyed, including the location of the house to be surveyed on the survey area map; a function of determining whether the house to be surveyed is included in the photographing range; A function to identify unphotographed houses that have not been determined to be included in the photography range; and A program that realizes a function of superimposing information about the unphotographed houses on the survey area map, A function of superimposing first navigation information indicating the direction in which the unphotographed house exists on the survey area map; and A program that realizes a function of superimposing the first navigation information, which includes information indicating the number of unphotographed houses in the direction indicated by the first navigation information, on the survey area map.

22. A computer comprising: A function of acquiring position and attitude information representing the position and attitude of a camera mounted on an aircraft; obtaining camera configuration information including the optical configuration of the camera; a function of calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; The ability to obtain a survey area map showing the survey area; A function of acquiring information about the house to be surveyed, including the location of the house to be surveyed on the survey area map; a function of determining whether the house to be surveyed is included in the photographing range; A function to identify unphotographed houses that have not been determined to be included in the photography range; and A program that realizes the function of superimposing an unphotographed house image onto the survey area map, to which a polygon corresponding to the outer shape of the unphotographed house is applied as information about the unphotographed house.

23. A non-transitory computer-readable recording medium having the program according to any one of claims 20 to 22 recorded thereon.

24. Aircraft and a camera mounted on the aircraft; an information processing device including a processor and a memory in which a program to be executed by the processor is stored; An information processing system comprising: The processor: Executing the instructions of said program, acquiring position and attitude information representing the position and attitude of the camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing system that superimposes information about the unphotographed houses on the survey area map in a display format that allows visual distinction.

25. Aircraft and a camera mounted on the aircraft; an information processing device including a processor and a memory in which a program to be executed by the processor is stored; An information processing system comprising: The processor executes the instructions of the program, acquiring position and attitude information representing the position and attitude of the camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing system that superimposes information about the unphotographed houses on the survey area map, The processor superimposes first navigation information indicating a direction in which the unphotographed house is located on the survey range map; An information processing system that superimposes the first navigation information, which includes information indicating the number of houses that have not yet been photographed in the direction indicated by the first navigation information, onto the survey area map.

26. An aircraft; a camera mounted on the aircraft; an information processing device including a processor and a memory in which a program to be executed by the processor is stored; An information processing system comprising: The processor executes the instructions of the program, acquiring position and attitude information representing the position and attitude of the camera mounted on the aircraft; obtaining camera configuration information including an optical configuration of the camera; calculating a shooting range of the camera based on the position and orientation information and the camera configuration information; Obtain a survey area map showing the survey area; Obtain survey target house information including the location of the survey target house on the survey area map; determining whether the house to be investigated is included in the photographing range; Identifying an unphotographed house that has no history of being determined to be included in the photography range; An information processing system that superimposes, on the survey area map, an unphotographed house image to which a polygon corresponding to the outer periphery shape of the unphotographed house is applied as information about the unphotographed house.

Images