Information processing device, information processing method, and information processing system

Abstract

This information processing device is provided with: a display processing unit that displays, on a display unit, geographical information created on the basis of images captured by a camera provided on a moving body; an associating unit that associates re-imaging information for re-imaging by the camera with the geographic information on the basis of an input from a user; and a movement plan creating unit that creates a movement plan for the moving body for performing the re-imaging by means of the camera on the basis of the re-imaging information.

Description

Information processing device, information processing method, and information processing system

[0001] The present technology relates to an information processing device, an information processing method, and an information processing system.

[0002] Conventional techniques include a technique for creating point clouds or orthoimages representing terrain from multiple image data captured by a camera mounted on a mobile object such as a drone. Furthermore, a technique for re-capturing an object when the image capture using such a mobile object fails has also been proposed (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-191523

[0004] In the technology described in Patent Document 1, if a failure to capture an object is detected, the object is re-photographed from the position where the failure to capture the object was detected. However, in order to create geographic information such as a point cloud with high accuracy, it is important to create an appropriate movement plan for the moving object, which specifies the route and area through which the object will be moved to capture the image again.

[0005] The present technology has been developed in consideration of such problems, and aims to provide an information processing device, an information processing method, and an information processing system that can create a travel plan for re-photographing for creating geographic information.

[0006] In order to solve the above-mentioned problems, the first technology is an information processing device that includes a display processing unit that displays geographic information created based on an image captured by a camera equipped on a moving object on a display unit, a matching unit that matches re-photographing information for re-photographing by the camera with the geographic information based on input from a user, and a movement plan creation unit that creates a movement plan for the moving object for re-photographing by the camera based on the re-photographing information.

[0007] The second technology is an information processing method that displays geographical information created based on images taken by a camera equipped on a moving body on a display unit, associates re-photographing information for re-photographing by the camera with the geographical information based on input from a user, and creates a movement plan for the moving body for re-photographing by the camera based on the re-photographing information.

[0008] The third technology is an information processing system comprising an information processing device that includes a geographic information creation device that creates geographic information based on images captured by a camera equipped on a moving object, a display processing unit that displays the geographic information on a display unit, an association unit that associates re-photographing information for re-photographing by the camera with the geographic information based on input from a user, and a movement plan creation unit that creates a movement plan for the moving object to re-photograph by the camera based on the re-photographing information.

[0009] 1 is a diagram showing the configuration of an information processing system 1000. FIG. 1 is a block diagram showing the configuration of a moving body 100. FIG. 2 is a block diagram showing the configuration of a geographic information creation device 200. FIG. 3 is a block diagram showing the hardware configuration of an information processing device 300. FIG. 4 is a block diagram showing the configuration of a processing block of the information processing device 300. FIG. 5 is a flowchart showing processing in the information processing system 1000. FIG. 6 is a diagram showing geographic information displayed on the display unit 307 in 2D display. FIG. 7 is a diagram showing geographic information displayed on the display unit 307 in 3D display. FIG. 8 is an explanatory diagram of input for associating re-photographing information with geographic information. FIG. 9 is an explanatory diagram of re-photographing information associated with geographic information. FIG. 10 is an explanatory diagram of oblique photography. FIG. 11 is an explanatory diagram of associating re-photographing information with an orthoimage. FIG. 11 is an explanatory diagram of a movement plan. FIG. 12 is an explanatory diagram of the reliability of geographic information. FIG. 13 is an example of a display of importance when the present technology is used in as-built management.

[0010] Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order: <Embodiment> [Configuration of information processing system 1000] [Configuration of mobile body 100] [Configuration of geographic information creation device 200] [Configuration of information processing device 300] [Processing in information processing system 1000] <Modification>

[0011] <Embodiment> [Configuration of Information Processing System 1000] As shown in FIG. 1, the information processing system 1000 includes a mobile object 100, a geographic information creation device 200, and an information processing device 300.

[0012] The mobile object 100 is connected to a geographic information creation device 200. Furthermore, the geographic information creation device 200 is connected to an information processing device 300. Furthermore, the information processing device 300 is connected to the mobile object 100. In this embodiment, the mobile object 100 is an unmanned aerial vehicle flying in the sky, and therefore the geographic information creation device 200 and the information processing device 300 are connected by a wireless connection. The geographic information creation device 200 is connected to the information processing device 300 by a wired connection or a wireless connection.

[0013] Wired connection methods include, for example, HDMI (High-Definition Multimedia Interface) and USB (Universal Serial Bus), while wireless connection methods include, for example, Wi-Fi, wireless LAN (Local Area Network), networks such as 4G (Fourth Generation Mobile Communication System) and 5G (Fifth Generation Mobile Communication System), Bluetooth (registered trademark), NFC (Near Field Communication), Ethernet (registered trademark), etc. Note that the supply of data and information from the mobile object 100 to the geographic information creation device 200, the supply of data and information from the geographic information creation device 200 to the information processing device 300, and the supply of data and information from the information processing device 300 to the mobile object 100 may be performed via a recording medium such as a USB flash memory or an SD memory card.

[0014] The mobile object 100 is an unmanned aerial vehicle such as a drone that flies in the sky. The mobile object 100 is equipped with a camera 105 and can perform automatic flight and automatic photography based on a preset movement route (pre-movement route) or a movement plan created by the information processing device 300. The pre-movement route can be set using existing mobile object control software, etc.

[0015] When performing automatic flight and automatic photography, route information, photography position, photography direction, photography timing, etc. are set in advance, and the mobile body 100 performs flight control and photography control in accordance with the settings. The mobile body 100 flies above the photography range, i.e., the range for creating geographic information (geographic information creation range), and photographs the ground surface with the camera 105 during flight. The mobile body 100 also acquires camera position information, which is the position of the camera 105 at the time of photography, and transmits the image and camera position information to the geographic information creation device 200. In addition to automatic flight, the mobile body 100 may also be capable of being flown manually by a pilot.

[0016] The geographic information creation device 200 creates geographic information of a geographic information creation range based on images captured by the camera 105 of the mobile object 100. The geographic information is a three-dimensional point cloud, an orthoimage, or the like.

[0017] A 3D point cloud is a set of data consisting of multiple points with location and color information. It can represent terrain, objects, etc. as a large number of point sets, and is geographic information that can be used in various fields such as civil engineering, architecture, and manufacturing.

[0018] An orthoimage is an image that has been transformed (orthotransformed) to eliminate misalignment of the image on the image and display the aerial photograph in the correct size and position without tilt, just like a map viewed from directly above. Because the shape and position of the image are correct in an orthoimage, it is possible to accurately measure position, area, distance, etc. on the image, and it is geospatial information that can be used by overlaying it with map data, etc.

[0019] Furthermore, the geographic information creation device 200 estimates the orientation of the camera 105 at the time of image capture based on an image captured by the camera 105 of the moving object 100. The geographic information creation device 200 transmits the geographic information, camera position information, and camera orientation information to the information processing device 300.

[0020] The information processing device 300 displays a GUI (Graphical User Interface) to present the user with geographical information, camera position information, camera attitude information, etc. The information processing device 300 also associates re-photographing information with geographical information based on input from the user, and creates a movement plan based on the re-photographing information.

[0021] The re-photographing information is information associated with geographic information for creating a travel plan. In this technology, the re-photographing information includes three types of information: designated route information, priority area information, and unnecessary area information. The re-photographing information will be described in detail later.

[0022] The movement plan is created by the information processing device 300 based on the re-imaging information, and indicates the movement route of the moving body 100 for re-imaging. The movement route will be described in detail later.

[0023] [Configuration of moving body 100] The configuration of moving body 100 will be described with reference to Fig. 2. Although not shown, moving body 100 has an exterior configuration including a housing, rotors, and rotor support shafts. However, the rotors are not limited to rotors and may be fixed wings.

[0024] The UAV (Unmanned Aerial Vehicle) control unit 101 is composed of a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and controls the entire moving body 100 and each part thereof by executing processes and issuing commands according to programs. The UAV control unit 101 also controls the moving speed, moving direction, turning direction, etc. of the moving body 100 by supplying control signals to the actuators 102 that control the output of the actuators 102.

[0025] The UAV control unit 101 also controls the output of the actuator 102 while comparing the current position of the moving body 100 with the previous movement route, thereby controlling the moving body 100 to move along the previous movement route. The UAV control unit 101 also controls the output of the actuator 102 while comparing the current position of the moving body 100 with the movement plan transmitted from the information processing device 300, thereby controlling the moving body 100 to move along the movement plan.

[0026] The actuator 102 is a drive source for driving the rotor, and is provided at the tip of the rotor support shaft, etc. The actuator 102 operates under the control of the UAV control unit 101.

[0027] The gimbal 103 rotatably supports the camera 105 at the bottom of the moving body 100. The gimbal 103 is a type of support that supports the camera 105 on, for example, two or three axes.

[0028] The gimbal control unit 104 controls the operation of the gimbal 103 that supports the camera 105. The attitude of the camera 105 can be freely adjusted by controlling the rotation of the axis of the gimbal 103 with the gimbal control unit 104. This allows the attitude of the camera 105 to be adjusted in accordance with the movement plan created by the information processing device 300 and photographing to be performed.

[0029] The camera 105 includes an imaging element, a signal processing circuit, etc., and captures RGB (Red, Green, Blue) or monochrome images. The imaging element may be a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc.

[0030] The camera 105 photographs a subject and obtains an image as digital data. The camera 105 may also have a function of performing AI-based processing on the image. Examples of this processing include image recognition processing and image detection processing.

[0031] The camera 105 is mounted, for example, suspended from the bottom surface of the housing of the mobile body 100 via a gimbal 103 in order to capture images of the ground surface within the geographic information creation range. The camera 105 is capable of capturing images by pointing its lens in any direction from 360 degrees horizontally to vertically downward by driving the gimbal 103. Note that the camera 105 may be mounted on the mobile body 100 in any manner as long as it can capture images by pointing its lens in any direction from 360 degrees horizontally to vertically downward.

[0032] The position information acquisition unit 106 acquires the position (e.g., latitude, longitude, and altitude) of the camera 105 using, for example, a Global Navigation Satellite System (GNSS) device or a Global Positioning System (GPS) device. Note that since the position of the camera 105 is the position of the moving body 100, it is also possible to detect the speed of the moving body 100 from changes in the position information and the elapsed time.

[0033] The attitude information acquisition unit 107 is configured by, for example, an IMU (Inertial Measurement Unit), and acquires the attitudes of the moving body 100 and the camera 105 by obtaining three-dimensional angular velocity and acceleration using a two-axis or three-axis acceleration sensor, an angular velocity sensor, a gyro sensor, etc. The attitude information is, for example, a three-axis rotation vector (Yaw, Pitch, Roll) calculated from the acceleration and angular velocity of the moving body 100.

[0034] The sensor control unit 108 controls the acquisition of an image by the camera 105 and the acquisition of camera position information by the position information acquisition unit 106 so that they are performed in synchronization. The sensor control unit 108 also attaches time information, such as a timestamp, to the image and the camera position information to indicate that they are synchronized.

[0035] The communication unit 109 is a communication terminal or communication module for various purposes, which performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, etc. The mobile object 100 transmits images and camera position information to the geographic information creation device 200 via the communication unit 109. Note that the mobile object 100 may transfer this information to the geographic information creation device 200 via an external storage medium such as a USB flash memory or an SD memory card.

[0036] In addition, a recording medium such as a semiconductor memory may be used, and the recording medium may be removed from the mobile body 100 after the mobile body 100 has landed, and connected to a recording medium connection slot (not shown) of the geographic information creation device 200, thereby allowing images and camera position information to be transferred.

[0037] For example, a mobile body 100 such as a drone can fly as desired by controlling the output of actuators 102. When hovering in the air, the attitude information acquisition unit 107 or a separately provided gyro sensor detects tilt, and the output of the actuator 102 on the side where the aircraft is lowered is increased and the output of the actuator 102 on the side where the aircraft is raised is decreased, thereby keeping the aircraft horizontal. Furthermore, when moving forward, the output of the actuator 102 in the direction of travel is decreased and the output of the actuator 102 in the opposite direction is increased, thereby causing the aircraft to assume a forward-leaning attitude and generating propulsion in the direction of travel.

[0038] The moving body 100 may be equipped with a 360-degree camera that can capture images of the surrounding area 360 degrees. Images of clouds and the sun captured by the 360-degree camera can be used for various processing.

[0039] The mobile object 100 may be equipped with an altimeter, a compass, etc. The altimeter may be a barometric altimeter or a radio altimeter that measures the altitude of the mobile object 100. The compass detects the traveling direction of the mobile object 100 using the action of a magnet.

[0040] In this embodiment, the mobile object 100 is equipped with sensors such as a camera 105, a position information acquisition unit 106, and an attitude information acquisition unit 107, but the mobile object 100 may also be configured to be equipped with a standalone sensor device that has these functions.

[0041] [Configuration of Geographic Information Creation Device 200] Next, the configuration of the geographic information creation device 200 will be described with reference to FIG.

[0042] The receiving unit 201 receives the image and camera position information transmitted from the moving body 100 and supplies them to the geographic information creating unit 202 and the attitude estimating unit 203 .

[0043] The geographic information creation unit 202 creates geographic information based on the image supplied from the receiving unit 201. The geographic information creation unit 202 creates a three-dimensional point cloud using, for example, the SfM (Structure From Motion) method. The geographic information creation unit 202 may create an orthoimage instead of or in addition to the three-dimensional point cloud. The type of geographic information the geographic information creation unit 202 creates may be set in advance, or may be selectable by the user. The orthoimage can be created using an existing method such as SfM-MVS (Structure-from-Motion / Multi-View-Stereo).

[0044] The attitude estimation unit 203 estimates the attitude of the camera 105 at the time of shooting based on the image supplied from the receiving unit 201, and outputs camera attitude information. SfM is a method for determining the attitude of the camera 105 from the captured image. Note that the camera attitude information may be acquired by an attitude information acquisition unit 107 included in the moving body 100.

[0045] The transmitting unit 204 transmits the geographic information, camera position information, and camera attitude information to the information processing device 300 via a transmission path such as the Internet or the above-mentioned communication method. The geographic information creation device 200 may transfer the information to the information processing device 300 via an external storage medium such as a USB flash memory or an SD memory card. The geographic information creation device 200 may also transmit the information to a cloud or a server.

[0046] The geographic information creation device 200 is installed as a fixed station within the geographic information creation range. The geographic information creation device 200 may be configured as a standalone device, or may be configured as an electronic device having information processing and communication functions, such as a personal computer, smartphone, or tablet terminal. The geographic information creation device 200 may also be realized by an electronic device having computer functions executing a program. The program may be pre-installed on the electronic device, or may be distributed by download or storage medium, etc., and installed by a user, etc. The geographic information creation device 200 may also be configured as a cloud server.

[0047] [Configuration of Information Processing Device 300] Next, the hardware configuration of the information processing device 300 will be described with reference to FIG.

[0048] The CPU 301 functions as an arithmetic processing unit that performs various processes and controls the entire information processing device 300 and each unit. The CPU 301 executes various processes in accordance with programs stored in the ROM 302 or nonvolatile memory unit 304, or programs loaded from the storage unit 308 to the RAM 303. The nonvolatile memory unit 304 may be, for example, an EEPROM (Electrically Erasable Programmable Read Only Memory). The RAM 303 also stores data and the like required for the CPU 301 to execute various processes as appropriate.

[0049] The CPU 301, ROM 302, RAM 303, and nonvolatile memory unit 304 are interconnected via a bus, to which an input / output interface 305 is also connected.

[0050] The input / output interface 305 is connected to an input unit 306 , a display unit 307 , a storage unit 308 , a communication unit 309 , and a drive 310 .

[0051] The input unit 306 is, for example, various types of controls or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, a remote controller, etc. The input unit 306 detects a user operation, and the CPU 301 interprets a signal corresponding to the input operation.

[0052] The display unit 307 is configured by, for example, a display device such as a liquid crystal display or an organic electroluminescence (EL) display provided in the housing of the computer device, a separate display device connected to the computer device, etc. The display unit 307 is connected to the input / output interface 305 either integrally with it or separately.

[0053] The display unit 307 displays various images, videos, etc. on the display screen based on instructions from the CPU 301. The display unit 307 also displays various operation menus, icons, messages, etc. based on instructions from the CPU 301. The display unit 307 also displays a GUI (Graphical User Interface) for displaying geographic information according to the present technology.

[0054] The storage unit 308 is a large-capacity storage medium such as a hard disk, flash memory, etc. The storage unit 308 stores various applications, data, information, etc.

[0055] The communication unit 309 is a communication terminal or communication module for various purposes that performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, etc. The information processing device 300 receives geographic information, camera position information, and camera attitude information transmitted from the geographic information creation device 200 via the communication unit 309. The information processing device 300 also transmits a movement plan to the mobile object 100 via the communication unit 309. The information processing device 300 may transfer information and data to the geographic information creation device 200 and the mobile object 100 via an external storage medium such as a USB flash memory or an SD memory card.

[0056] A drive 310 is connected to the input / output interface 305 as needed. A removable storage medium 311 is appropriately attached via the drive 310. The removable storage medium 311 includes a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like.

[0057] The drive 310 can read data files such as programs used in various processes from a removable storage medium 311. The read data files are stored in the storage unit 308. Images included in the data files are displayed on the display unit 307. Computer programs read from the removable storage medium 311 are installed in the storage unit 308 as needed.

[0058] In the information processing device 300, for example, software for processing of the present technology can be installed via network communication by the communication unit 309 or a removable storage medium 311. The software may also be stored in advance in the ROM 302, the storage unit 308, etc. Images captured by the camera 105 and processing results of AI image processing may also be received, and the images and processing results may be stored in the storage unit 308 or the removable storage medium 311.

[0059] Next, the configuration of the processing blocks in the information processing device 300 will be described with reference to FIG.

[0060] The information acquisition unit 321 acquires the geographic information, camera position information, and camera attitude information transmitted from the geographic information creation device 200 and received by the communication unit 309, and supplies them to the display processing unit 323. The information acquisition unit 321 also supplies the geographic information to the anomaly detection unit 322 and the association unit 324.

[0061] The anomaly detection unit 322 detects anomalies in the geographic information. If there is a defect in the image captured by the camera 105, an anomaly occurs in the geographic information. Anomalies in the three-dimensional point cloud as geographic information include sparse points, or the absence of points in areas where points should be present. Furthermore, anomalies in the orthoimage as geographic information include image distortion, missing images (exposing the background), and objects hidden by shadows.

[0062] The display processing unit 323 performs processing to display geographic information, camera position information, camera attitude information, the movement path of the moving object 100, and abnormality detection results on the display unit 307.

[0063] The association unit 324 performs processing to associate re-photographing information for creating a movement plan with geographic information based on input from the user.

[0064] The coordinate conversion unit 325 converts the re-photographing information expressed in the site coordinate system into the drone coordinate system. The site coordinate system is a coordinate system in which coordinate axes are set at an arbitrary origin set in a local location such as a construction site. The drone coordinate system is a global coordinate system expressed in latitude and longitude.

[0065] The movement plan creation unit 326 creates a movement plan based on the re-photographing information associated with the geographic information.

[0066] The information processing device 300 is configured as described above. The information processing device 300 is configured by an electronic device having information processing and communication functions, such as a personal computer, a smartphone, or a tablet terminal. The processing blocks and information processing method of the information processing device 300 according to the present technology may be realized by the electronic device executing a program. The program may be pre-installed on the electronic device, or may be distributed by download or storage medium, etc., and installed by a user, etc. Furthermore, the processing unit of the information processing device 300 may be configured in a cloud server.

[0067] A cloud server is not limited to being configured by a single computer device, but may be configured by a system of multiple computer devices. The multiple computer devices are systemized, for example, by a local area network (LAN). Furthermore, multiple computer devices located in remote locations may be systemized by a virtual private network (VPN) using the Internet or the like. The multiple computer devices may include computer devices as a server group (cloud) available through a cloud computing service.

[0068] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to perform the described functions. Processors include transistors and other circuits and are considered circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in memory. In this specification, a circuit, unit, or means is hardware that is programmed to perform or executes the described functions. The hardware may be any hardware disclosed herein or any hardware known to be programmed to perform or execute the described functions. When the hardware is a processor, which is considered a type of circuitry, the circuit, means, or unit is a combination of hardware and software used to configure the hardware and / or processor.

[0069] [Processing in Information Processing System 1000] Next, processing in the information processing system 1000 will be described with reference to FIG.

[0070] First, in step S11, the moving body 100 moves (flies) above the geographic information creation range based on a pre-created preliminary movement route, and takes photographs with the camera 105. While the moving body 100 is moving along the pre-created movement route, the camera 105 repeatedly and continuously takes photographs at predetermined regular intervals.

[0071] Furthermore, the position information acquisition unit 106 acquires the camera position information at the time of image capture in synchronization with the image capture by the camera 105. Then, the mobile object 100 transmits the image and the camera position information to the geographic information creation device 200.

[0072] The moving object 100 can take pictures while changing the moving route in response to instructions from the user via wireless communication while moving. Furthermore, the timing of taking pictures can be added or changed in response to instructions from the user via wireless communication while moving.

[0073] Next, in step S12, the attitude estimation unit 203 of the geographic information creation device 200 estimates the attitude of the camera 105 based on the image captured by the camera 105 and outputs camera attitude information.

[0074] Next, in step S13, the geographic information creation unit 202 of the geographic information creation device 200 creates geographic information based on the image. The geographic information creation device 200 transmits the geographic information, camera position information, and camera attitude information to the information processing device 300.

[0075] Next, in step S14, the anomaly detection unit 322 of the information processing device 300 detects an anomaly in the geographic information and supplies information on the location where the anomaly exists in the geographic information (anomaly location information) as an anomaly detection result to the display processing unit 323. An anomaly in a three-dimensional point cloud as geographic information can be detected, for example, by measuring the distance between each point constituting the point cloud and determining whether the distance is equal to or greater than a predetermined threshold. Furthermore, since the point cloud is created in advance to satisfy a predetermined density, an area where the density of the created point cloud is equal to or less than a predetermined density can be detected as an anomaly.

[0076] Next, in step S15, the geographic information is displayed on the display unit 307 by processing of the display processing unit 323 of the information processing device 300. There are two display formats: 2D display and 3D display. In the case of 2D display, the geographic information displayed on the display unit 307 is as shown in FIG. 7. In the example of FIG. 7, the geographic information is a three-dimensional point cloud. In this embodiment, it is assumed that the ground, roads, embankments, buildings, and trees exist on the ground surface photographed by the camera 105. However, this configuration of the ground surface is merely an example, and the present technology is not limited to such a configuration of the ground surface.

[0077] In addition to the geographic information, the display processing unit 323 displays, on the display unit 307, camera position information at the time of shooting, which was acquired in synchronization with the shooting of the image used to create the geographic information. The camera position information is displayed as a point indicating the position on the geographic information. This allows the user to easily understand the position of the camera 105 at which the image was shot.

[0078] Furthermore, the display processing unit 323 displays camera attitude information on the display unit 307 in addition to the geographic information. The camera attitude information is represented by a rectangular frame indicating the angle of view of the camera 105 and lines extending from points indicating the camera position information to each corner of the frame. This allows the user to easily understand the attitude of the camera 105 at which the image was captured. The angle of view can be identified from the image used to create the geographic information.

[0079] In addition to the geographic information, the display processing unit 323 also displays on the display unit 307 the movement path of the moving object 100 in the sky above the geographic information creation range. The movement path is displayed as a line connecting multiple positions indicated by the camera position information in chronological order (photographing order). To achieve this, for example, it is advisable to associate time information such as a timestamp with the camera position information in the moving object 100 and refer to this information. By displaying this movement path, the user can easily understand the route taken by the moving object 100 to take photographs for creating geographic information.

[0080] Furthermore, the display processing unit 323 displays, in addition to the geographical information, anomaly location information, which is the anomaly detection result, on the display unit 307. This allows the user to understand where the anomaly is in the geographical information, identify locations or areas within the geographical information creation range where re-imaging is required, and perform input for associating the imaging information.

[0081] The user can also find anomalies by looking at the displayed geographical information. Therefore, displaying the anomaly location information is not essential. Displaying the anomaly location information assists the user in inputting information to associate it with the imaging information. The anomaly location information may be highlighted, for example, to make it easier for the user to understand. In addition to the anomaly location information, information indicating the type of the anomaly may also be supplied from the anomaly detection unit 322 to the display processing unit 323 and displayed on the display unit 307.

[0082] In the example of Fig. 7, camera position information, camera attitude information, movement path, and anomaly position information are displayed together with geographic information, but it is not necessary to display all of the information at the same time. The user may be able to select which information to display. Also, the display or non-display of selected information may be switched according to a selection input from the user.

[0083] The geographic information can also be displayed in 3D, and in the case of 3D display, the geographic information etc. displayed on the display unit 307 is as shown in Fig. 8. The geographic information in Fig. 8 is also a three-dimensional point cloud. In the 3D display in Fig. 8, camera position information, camera attitude information, movement path, and abnormality position information are displayed superimposed on the geographic information, as in the 2D display in Fig. 7. The user may be able to select and switch between 2D display and 3D display.

[0084] The user looks at the geographical information displayed on the display unit 307 and inputs information for re-photographing to the input unit 306 to associate the information with the geographical information.

[0085] When the user inputs information while viewing the display of the geographical information and travel route shown in FIG. 7, the input result will be, for example, as shown in FIG.

[0086] Next, in step S16, the association unit 324 of the information processing device 300 performs processing to associate the re-photograph information with the geographic information based on the input result of the user. The re-photograph information associated with the geographic information based on the input result shown in FIG. 9 becomes as shown in FIG.

[0087] 10 , in the present technology, the re-photographing information includes three pieces of information: designated route information, priority area information, and unnecessary area information. The re-photographing information is information that specifies either or both of the route and area along which the moving body 100 moves for re-photographing by the camera 105, and is also information that specifies areas into which the moving body 100 does not need to move.

[0088] The designated route information is information that directly specifies a specific movement route of the mobile object 100. To associate the designated route information with the geographic information, the user inputs a line indicating the designated route on the geographic information displayed on the display unit 307. The tip of the arrow is the end point of the designated route, and the end point on the opposite side of the arrow is the start point of the designated route. The designated route may be input by specifying the start point and the end point. In this case, the straight line connecting the start point and the end point becomes the designated route. Multiple designated route information may be associated with the geographic information. Multiple designated route information may be associated with distant positions, or the designated routes may be connected and associated. Furthermore, the movement route information may be specified as a line other than a straight line, such as a curved line, a wavy line, or a free line.

[0089] The specified route information may include the moving speed of the mobile object 100 when moving along the specified route and the attitude of the camera 105. The moving speed may be input as a specific speed value, or may be input as the length of time required to move along the input specified route. The attitude of the camera 105 may be input as a specific angle value, or the attitude of the camera 105 corresponding to the input angle value may be visually displayed using animation or the like.

[0090] The specified route information can also be used to specify a route for so-called oblique photography, in which the camera 105 is tilted diagonally rather than pointing vertically downward to take photographs while moving diagonally relative to the direction of the round-trip route after moving back and forth multiple times within the geographic information creation range, as shown in Figure 11. Using images obtained by oblique photography to create geographic information can improve the accuracy of the geographic information.

[0091] The priority area information does not specify a specific travel route like the designated route information, but specifies an area (priority area) to be photographed with priority within the geographic information creation range.

[0092] In order to associate the priority area information with the geographic information, the user performs an input to specify the priority area on the geographic information displayed on the display unit 307. The priority area may be input as a rectangular area as shown in Fig. 9, or may be input as an area of ​​other shape such as a circle or a free-form shape, or may be specified by a waypoint.

[0093] In the orthoimage, as shown in FIG. 12, it is preferable to associate the orthoimage with key area information in order to focus on photographing areas where distortion occurs or areas where dark or long shadows exist.

[0094] Furthermore, it is preferable to associate priority area information to focus on capturing images of areas where blind spots or shadows are created depending on the shooting direction, such as tall buildings. The priority area information makes it possible to focus on capturing images of areas where occlusion occurs, where the area behind is hidden by a building or object in the foreground.

[0095] The unnecessary area information is information that specifies an area (unnecessary area) that does not need to be photographed and does not need to be included in the movement plan. Areas that do not need to be photographed include, for example, areas covered with trees such as forests and woods, water surfaces such as oceans, lakes, and rivers, and snow surfaces. However, the unnecessary area is not limited to these, and can be any area that the user considers unnecessary to photograph for creating geographic information.

[0096] In order to associate the unnecessary area information with the geographic information, the user inputs information to specify the unnecessary area on the geographic information displayed on the display unit 307. The unnecessary area may be input as a rectangular area as shown in Fig. 9, or may be input as an area of ​​other shape such as a circle or a free shape, or may be specified by a waypoint.

[0097] It is not necessary to associate all of the designated route information, priority area information, and unnecessary area information with the geographic information. Only one type of information or two types of information may be associated with the geographic information. Also, multiple pieces of information may be associated with the geographic information.

[0098] Next, in step S17, the coordinate conversion unit 325 converts the designated route, key area, and unnecessary area indicated in the re-photographing information associated with the geographic information from the site coordinate system to the drone coordinate system.

[0099] Next, in step S18, the movement plan creation unit 326 creates a movement plan based on the re-photographing information associated with the geographic information. The movement plan is created so that the mobile object 100 moves along the designated route in the designated route information and the priority area in the priority area information, and does not move through the unnecessary area in the unnecessary area information. The movement plan is also created as a route connecting the designated route and the priority area via the shortest distance.

[0100] Regarding the designated route information, the designated route, the moving speed of the moving body 100, and the attitude of the camera 105 are reflected directly in the movement plan.

[0101] Furthermore, for the priority area information, a movement plan can be created using a plurality of movement methods, such as round trip movement, up and down movement, and movement at a reduced speed.

[0102] Reciprocating movement is a method in which the mobile object 100 photographs the area of ​​importance by traveling back and forth horizontally multiple times above the area of ​​importance. This increases the number of images of the area of ​​importance, i.e., the number of images used to create geographic information, thereby increasing the image overlap rate. Increasing the image overlap rate improves the accuracy of the geographic information, making it possible to create geographic information without abnormalities. Note that the image overlap rate can be further increased by narrowing the distance between the reciprocating movement paths.

[0103] Vertical movement is a method of capturing images by pointing the lens of the camera 105 toward a wall and moving up and down relative to the wall in order to create geographic information for a high area, such as the wall of a building. This allows images to be captured of areas that are difficult to capture from the air, allowing geographic information to be created. Note that by narrowing the distance between the round-trip routes taken by the moving body 100 as it moves up and down, the image overlap rate can be increased.

[0104] The object to be photographed by moving up and down may be perpendicular to the ground surface, or may be any object such as a terrain or building that is not perpendicular but has a height relative to the ground surface.

[0105] The slower speed movement is a method of capturing images by slowing down the speed of the moving body 100 moving over the priority area compared to when moving over other areas. This increases the number of times images are captured, increases the overlap rate of the images, and improves the accuracy of the geographic information, making it possible to create geographic information without abnormalities.

[0106] The method for moving the priority area to capture images may be set in advance by the user, or may be automatically determined by an algorithm by the information processing device 300. Furthermore, the method is not limited to one, and a movement plan may be created by combining a plurality of methods. For example, the movement speed may be reduced and images may be captured while moving back and forth.

[0107] The movement start position of the moving body 100 in the movement plan may be the current position of the moving body 100 or any position specified by the user. When the user specifies the movement start position, it is preferable that the user be able to input the movement start position together with inputting the information for re-imaging. If the user does not specify a position, it is preferable that the current position of the moving body 100 or a position set in advance as a default position be set as the movement start position.

[0108] Therefore, for example, when three pieces of information for re-photographing, namely, designated route information, priority area information, and unnecessary area information, are associated with geographic information as shown in FIG. 10, a travel plan is created as shown in FIG.

[0109] In the movement plan shown in Figure 13, the moving body 100 first moves from the movement start position to priority area A, and moves back and forth so as to focus on photographing within priority area A. Next, the moving body 100 moves from priority area A to one end of the designated route, moves along the designated route, and then moves from the other end of the designated route to priority area B. Next, the moving body 100 moves so as to focus on photographing within priority area B. Because priority area B is an area with height, such as a wall, the moving body 100 photographs while moving up and down within priority area B. Then, the movement of the moving body 100 ends.

[0110] The movement plan creation unit 326 creates a movement plan based on the re-photographing information associated with the geographical information. Therefore, if the only information associated with the geographical information as re-photographing information is designated route information, the movement plan creation unit 326 creates a movement plan that reflects only the designated route information. Also, if the only information associated with the geographical information as re-photographing information is key area information, the movement plan creation unit 326 creates a movement plan that reflects only the key area information.

[0111] When multiple pieces of re-photographing information are associated with geographic information, the user may be able to select from the re-photographing information to be reflected in the movement plan. By selecting the re-photographing information to be reflected in the movement plan, the movement plan can be adjusted or changed. Furthermore, multiple different movement plans can be created based on the re-photographing information associated with the geographic information.

[0112] The display processing unit 323 may display a preview of the movement plan created based on the re-capture information on the display unit 307. By viewing the preview, the user can redo input to associate the re-capture information or perform input to add or delete the re-capture information. The preview display may be either 2D display or 3D display, but displaying it in 3D allows the user to easily and intuitively grasp the movement plan, particularly the details of vertical movement.

[0113] If the user does not input the travel speed and camera attitude in the specified route information, the travel speed and camera attitude are automatically set to a predetermined value. Also, a notification prompting the user to input the travel speed and camera attitude may be displayed on the display unit 307.

[0114] When inputting information for association with re-photographing, the user may input conditions such as the desired photographing time and weather, and the movement plan creation unit 326 may create a movement plan that reflects those conditions.

[0115] When a plurality of pieces of re-photographing information are associated with geographic information, the user may be allowed to set a priority for each piece of re-photographing information. When a priority is set for each piece of re-photographing information, the movement plan creation unit 326 creates a movement plan according to the priority.

[0116] For example, two priorities, high priority and low priority, can be set for the re-photographing information. In this case, the high-priority designated route information or key area information is reflected in the travel plan regardless of the remaining battery power of the mobile object 100. The low-priority designated route information or key area information is reflected in the travel plan only when the remaining battery power of the mobile object 100 is equal to or greater than a predetermined amount.

[0117] The movement plan creation unit 326 may create a movement plan based on the remaining battery power of the moving body 100. For example, when the remaining battery power of the moving body 100 is equal to or less than a predetermined threshold, an upper limit is set for the total length of the route in the movement plan, and a movement plan is created within the upper limit of the length of the movement route, with priority given to re-photographing information associated with a position close to the movement start position of the moving body 100. Furthermore, when the remaining battery power is equal to or less than a predetermined threshold, a movement plan may be created by reducing round trips in movement of the priority area.

[0118] The movement plan creation unit 326 may also create a movement plan based on the time required for imaging. For example, a time for imaging, such as 30 minutes, is set in advance, and the movement plan is created by giving priority to re-imaging information set in a position close to the movement start position of the moving body 100 so that the time it takes for the moving body 100 to travel the route in the movement plan is within 30 minutes. Re-imaging information that does not fit within 30 minutes is not included in the movement plan.

[0119] The movement plan may also include the movement start time of the moving object 100 and the time period during which the moving object 100 moves. Furthermore, the movement plan creation unit 326 may create a movement plan based on the weather. For example, on a sunny day, the movement start time and movement time period in the movement plan are determined based on the daylight hours and the position of the sun so as to minimize shadows.

[0120] The movement plan creation unit 326 may also create a movement plan based on wind direction. For example, when the wind is blowing from the south, it is more efficient for the moving body 100 to move from upwind to downwind, i.e., from south to north. Therefore, a movement plan is created according to the wind direction, such as a movement route within the priority area that moves the moving body 100 from south to north.

[0121] The information processing device 300 can acquire weather information from the Internet or the like in order to create a travel plan.

[0122] Returning to the explanation of the flowchart in Figure 6, the movement plan created by the movement plan creation unit 326 is transmitted from the information processing device 300 to the moving body 100. Next, in step S19, the moving body 100 moves above the geographic information creation range based on the movement plan and re-images the image using the camera 105. The UAV control unit 101 of the moving body 100 controls the output of the actuator 102 while comparing the current position of the moving body 100 with the movement plan, thereby controlling the moving body 100 to move in accordance with the movement plan. In addition, the gimbal control unit 104 controls the gimbal 103 based on the movement plan to adjust the attitude of the camera 105 and perform image capture. This allows re-image capture based on the movement plan. The moving body 100 then transmits the images acquired during the re-image capture to the geographic information creation device 200.

[0123] Then, in step S20, the geographic information creation unit 202 of the geographic information creation device 200 recreates geographic information based on the image.

[0124] The flowchart in Figure 6 shows the process of re-photographing based on a movement plan and re-creating geographic information using the images obtained by the re-photographing. However, it is also possible to create a movement plan after that and perform a second re-photographing and re-creation of geographic information. It is possible to create a movement plan, re-photograph, and re-create geographic information any number of times.

[0125] The processing of the present technology is performed as described above. According to the present technology, by associating re-photographing information with geographic information, it is possible to easily create a travel plan for re-photographing for creating geographic information.

[0126] In addition, when entering information to associate re-photographing information with geographic information, the user can check the travel route of past photographs displayed on the geographic information, making it easier and more accurate to enter information.

[0127] If a user were to create a travel plan by themselves while looking at geographic information, the geographic information and the travel plan would have to be displayed on separate screens, and the user would have to create the travel plan by comparing the two. However, with this technology, the user only needs to input information to associate the re-photographing information with the displayed geographic information, making it easy to create a travel plan.

[0128] The information processing device 300 is equipped with an anomaly detection unit 322, and displays the anomaly detection results on the display unit 307 in addition to the geographical information, so that the user can reliably recognize anomalies in the geographical information and create a travel plan to resolve the anomalies.

[0129] This technology is characterized by displaying geographic information and associating it with re-photographing information, so its implementation in other companies' products can be verified by checking the GUI display of those products.

[0130] The number of moving bodies 100 that move based on the movement plan is not limited to one, and may be multiple. That is, movement based on the movement plan and re-imaging may be performed using multiple moving bodies 100. When there are multiple moving bodies 100, the re-imaging information may be divided by type of information to create a movement plan for each moving body 100, or one movement plan may be created and divided for each moving body 100.

[0131] For example, if there are two moving bodies, such as moving body 100A and moving body 100B, the movement plan is divided into two, left and right, with the left side being the movement plan for moving body 100A and the right side being the movement plan for moving body 100B. Dividing into two, left and right, is merely an example, and the movement plan may also be divided into top and bottom, or in any other manner. If three moving bodies 100 have abnormalities, the movement plan may also be divided into three or more parts.

[0132] Furthermore, when there are multiple moving bodies, such as moving body 100A and moving body 100B, a movement plan based on the designated route information is created for moving body 100A, and a movement plan based on the priority area information is created for moving body 100B. Furthermore, multiple moving bodies 100 may be moved based on the same movement plan.

[0133] After photographing the entire geographic information creation range, a movement plan may be created for a portion of the geographic information creation range, and that portion of the geographic information creation range may be photographed again based on the movement plan. Alternatively, after photographing the entire geographic information creation range, a movement plan may be created for a portion of the geographic information creation range, and the entire geographic information creation range may be photographed again based on the movement plan for that portion of the geographic information creation range.

[0134] In a 3D point cloud, the reliability of geographic information decreases around objects with a certain height, such as the walls of buildings. For example, when a blind spot occurs due to a tall object as shown in Figure 14, the overlap rate of multiple images captured by the camera 105 of the moving object 100 decreases, and the reliability of the 3D point cloud decreases.

[0135] Furthermore, as shown in Figure 14, shadows are cast around objects of a certain height, such as building walls. The influence of these shadows can sometimes reduce the reliability of the 3D point cloud. In SfM, a method for creating a 3D point cloud, shadow boundaries are likely to be treated as feature points. Feature points are used to align multiple images, so points that are treated as the same feature point in multiple images must exist at the same coordinates. However, due to the movement path of the moving object 100, a time difference may occur when the same location is photographed at different times. If the time difference between these different times is long, the position of the shadow may have changed. If the shifted shadow boundary is treated as a feature point, the image alignment may attempt to match an incorrect position, adversely affecting the creation of the 3D point cloud. For this reason, the shadow boundary is also considered to have low reliability in the 3D point cloud.

[0136] Therefore, the reliability of the 3D point cloud can be considered low around tall objects such as building walls due to the reduced overlap rate caused by blind spots and the influence of changing shadows.

[0137] This low reliability area may be calculated from the position of the sun or the attitude of the camera 105, and displayed together with the geographic information on the display unit 307. A user viewing this display can input re-photographing information to associate the low reliability area with the re-photographing information. The movement plan creation unit 326 then creates a movement plan based on the re-photographing information, thereby enabling highly accurate geographic information to be created even for areas with low reliability.

[0138] For the same reason, the reliability of the 3D point cloud may be reduced due to the influence of cloud shadows. Therefore, a 360-degree camera capable of capturing images of the surrounding area is mounted on the mobile object 100, and images of the clouds and the sun are captured by the 360-degree camera in addition to images of the ground surface by the camera 105. The position of the cloud shadow is identified from the 360-degree image obtained by the image capture. The boundary of the shadow may then be displayed on the display unit 307 together with geographical information as an area of ​​low reliability.

[0139] When this technology is used for as-built management (managing whether the constructed structure meets the standard criteria intended by the client), the importance of the geographic information may be displayed together with the geographic information on the display unit 307 so that the user can input information to associate with the information for re-photography.

[0140] For example, since the soil ground that will be the actual construction surface needs to be photographed with priority, the display unit 307 displays a high level of importance along with the geographic information. Furthermore, embankments, construction machinery, buildings, etc., are displayed with medium level of importance along with the geographic information. Furthermore, the display unit 307 displays a low level of importance for non-construction surfaces, such as forests, trees, rivers, ponds, puddles, snow, and the periphery of the captured image frame, along with the geographic information. The level of importance can be displayed, for example, by surrounding the range indicating the level of importance in the geographic information with a frame and applying different colors within the frame depending on the level of importance, as shown in FIG. 15 . Furthermore, the thickness, type, and color of the frame lines may be changed depending on the level of importance.

[0141] In addition, areas where the overlap rate of the image on which the geographic information is based is low (due to the effects of photography failure, such as wind or weather conditions) or areas with low point cloud density are areas that should be re-photographed, so the display unit 307 displays the high importance of these areas together with the geographic information.

[0142] By displaying the importance in this way, the user can recognize the importance and appropriately associate the re-photographing information with the location that should be re-photographed.

[0143] The importance can be determined based on the results of topographical recognition of the geographical information creation range, the results of object detection within the shooting range, the overlap rate of the captured images, the density of the point cloud created as geographical information, shooting camera position information (information on whether the area is in the center or periphery), etc. The importance may be determined based on any one of these, or on a combination of any two or more of them.

[0144] The reliability-related processing and importance-related processing may be performed by the anomaly detection unit 322, or the information processing device 300 may be provided with dedicated processing units for each.

[0145] <Modifications> Although the embodiments of the present technology have been specifically described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical ideas of the present technology are possible.

[0146] Different types of geographic information, for example, a 3D point cloud and an orthoimage, may be simultaneously displayed on the display unit 307. In this case, the 3D point cloud and the orthoimage may be displayed superimposed on each other, or the display screen of the display unit 307 may be divided into two display areas, with the 3D point cloud displayed in one display area and the orthoimage displayed in the other display area.

[0147] The latest geographic information and past geographic information may be displayed simultaneously on the display unit 307. For example, the display screen of the display unit 307 may be divided into two display areas, with the latest point cloud displayed in one display area and the past point cloud displayed in the other display area.

[0148] By simultaneously displaying the latest geographic information and past geographic information, the user can view and compare the latest geographic information with the past geographic information, and can input information to associate key area information with areas in the latest geographic information where there are abnormal differences from the past geographic information. Then, by creating a movement plan based on the key area information and taking photographs again, the latest geographic information can be created in which the abnormal differences have been resolved.

[0149] The information processing device 300 may include the geographic information creation unit 202, or the geographic information may be created by a cloud server or the like.

[0150] The geographic information may be information other than 3D point clouds and orthoimages, such as a DEM (Digital Elevation Model). A DEM is a digital elevation model that digitally represents 3D coordinates using planar positions (2D) and elevation values ​​measured using various surveying methods.

[0151] The mobile object 100 may be a manned aircraft, a glider, a helicopter, a balloon, an airship, a rocket, or a device that can move along a rail installed in a high position such as the ceiling of a building, other than an unmanned aircraft. Furthermore, the mobile object 100 is not limited to those that move in the sky, but may be an automobile, a motorcycle, a bicycle, personal mobility, an airplane, a ship, a robot, construction machinery, agricultural machinery, or the like.

[0152] The present technology can also be configured as follows. (1) An information processing device comprising: a display processing unit that displays, on a display unit, geographical information created based on an image captured by a camera provided in a moving object; an association unit that associates re-imaging information for re-imaging with the camera with the geographical information based on input from a user; and a movement plan creation unit that creates a movement plan for the moving object to re-imaging with the camera based on the re-imaging information. (2) The information processing device described in (1) that includes an anomaly detection unit that detects an anomaly in the geographical information. (3) The information processing device described in (2) that the display processing unit displays the detected anomaly together with the geographical information on the display unit. (4) The information processing device described in any of (1) to (3) that the display processing unit displays, on the display unit, the movement path of the moving object when capturing an image with the camera together with the geographical information. (5) The information processing device described in any of (1) to (4) that the display processing unit displays, on the display unit, the position of the camera when capturing an image together with the geographical information. (6) The information processing device according to any one of (1) to (5), wherein the display processing unit displays the attitude of the camera when capturing an image on the display unit together with the geographic information. (7) The information processing device according to any one of (1) to (6), wherein the re-capture information is information specifying one or both of a route and an area along which the moving object will move for re-capturing by the camera. (8) The information processing device according to (7), wherein the re-capturing information is designated route information that specifies a route along which the moving object will move. (9) The information processing device according to (8), wherein the designated route information further specifies a moving speed of the moving object. (10) The information processing device according to (8) or (9), wherein the designated route information further specifies the attitude of the camera. (11) The information processing device according to any one of (7) to (10), wherein the re-capturing information is priority area information that specifies an area to be primarily captured by the camera. (12) The information processing device according to any one of (1) to (11), wherein the re-capturing information is unnecessary area information that specifies an area where capturing by the camera is unnecessary.(13) The information processing device according to any one of (7) to (12), wherein the movement plan creation unit creates the movement plan so as to include either or both of a route and an area specified in the re-photographing information, resulting in the shortest distance. (14) The information processing device according to any one of (1) to (13), wherein the geographic information is a three-dimensional point cloud. (15) The information processing device according to any one of (1) to (14), wherein the geographic information is an orthoimage. (16) The information processing device according to any one of (1) to (15), comprising a geographic information creation unit that creates the geographic information. (17) An information processing method comprising: displaying geographic information created based on an image captured by a camera provided in a moving object on a display unit; associating re-photographing information for re-photographing by the camera with the geographic information based on input from a user; and creating a movement plan for the moving object for re-photographing by the camera based on the re-photographing information. (18) An information processing system comprising: a geographic information creation device that creates geographic information based on an image captured by a camera equipped on a moving body; an information processing device that includes: a display processing unit that displays the geographic information on a display unit; an association unit that associates re-photographing information for re-photographing by the camera with the geographic information based on input from a user; and a movement plan creation unit that creates a movement plan for the moving body to re-photograph by the camera based on the re-photographing information.

[0153] DESCRIPTION OF SYMBOLS 100: Mobile object 105: Camera 202: Geographic information creation unit 300: Information processing device 307: Display unit 322: Abnormality detection unit 323: Display processing unit 324: Correspondence unit 326: Movement plan creation unit 1000: Information processing system

Claims

1. An information processing device comprising: a display processing unit that displays geographic information created based on an image taken by a camera equipped on a moving body on a display unit; an association unit that associates re-photographing information for re-photographing by the camera with the geographic information based on input from a user; and a movement plan creation unit that creates a movement plan for the moving body for re-photographing by the camera based on the re-photographing information.

2. The information processing device according to claim 1, further comprising an anomaly detection unit that detects anomalies in the geographic information.

3. The information processing device according to claim 2, wherein the display processing unit displays the detected abnormality together with the geographic information on the display unit.

4. The information processing device according to claim 1, wherein the display processing unit displays on the display unit the route of the moving object when the image was captured by the camera, together with the geographic information.

5. The information processing device according to claim 1, wherein the display processing unit displays the position of the camera when the image was taken on the display unit together with the geographic information.

6. The information processing device according to claim 1, wherein the display processing unit displays the orientation of the camera when the image was captured on the display unit together with the geographic information.

7. The information processing device according to claim 1, wherein the re-photographing information is information specifying either or both of the route and area along which the moving object will move for re-photographing by the camera.

8. The information processing device according to claim 7, wherein the re-photographing information is designated route information that designates a travel route of the moving object.

9. The information processing device according to claim 8, wherein the specified route information further specifies a moving speed of the mobile object.

10. The information processing device according to claim 8, wherein the specified route information further specifies the attitude of the camera.

11. The information processing device according to claim 7, wherein the re-photographing information is priority area information that specifies an area to be photographed with priority by the camera.

12. The information processing device according to claim 1, wherein the re-photographing information is unnecessary area information that specifies an area that does not need to be photographed by the camera.

13. The information processing device according to claim 7, wherein the movement plan creation unit creates the movement plan so as to include either or both of the route and area specified in the re-photographing information and to result in the shortest distance.

14. The information processing device according to claim 1, wherein the geographic information is a three-dimensional point cloud.

15. The information processing device according to claim 1, wherein the geographic information is an orthoimage.

16. The information processing device according to claim 1, further comprising a geographic information creation unit that creates the geographic information.

17. An information processing method comprising: displaying geographical information created based on an image taken by a camera equipped on a moving object on a display unit; associating re-photographing information for re-photographing by the camera with the geographical information based on input from a user; and creating a movement plan for the moving object to re-photograph using the camera based on the re-photographing information.

18. An information processing system comprising: a geographic information creation device that creates geographic information based on images taken by a camera equipped on a moving object; an information processing device that has: a display processing unit that displays the geographic information on a display unit; an association unit that associates re-photographing information for re-photographing by the camera with the geographic information based on input from a user; and a movement plan creation unit that creates a movement plan for the moving object to re-photograph using the camera based on the re-photographing information.

Images