Information processing device, information processing method, program, and information processing system

Abstract

Provided are an information processing device, an information processing method, a program, and an information processing system, by which formulation of a preferable survey plan for each analysis area in a survey target region is realized. The information processing device acquires map information in which a geographical position and a shape are represented, acquires feature meta-information pertaining to each of one or more features, acquires an aerial image of a survey target region, derives a degree of damage for each of the one or more features on the basis of the map information, the feature meta-information, and the aerial image, statistically processes the degree of damage derived for each of the one or more features included in an analysis area, derives representative information representing the analysis area, and causes the analysis area to be displayed in association with the representative information.

Description

Information processing device, information processing method, program, and information processing system

[0001] The present invention relates to an information processing device, an information processing method, a program, and an information processing system.

[0002] When a disaster such as an earthquake occurs, a survey of the affected area is conducted. Before investigators visit the affected area to conduct the survey, information on the situation of the affected area is collected from aerial images of the affected area.

[0003] In the survey of the damage situation, a survey plan is formulated for each area as determined by the local government, etc., which will determine priorities and assign surveyors. For example, a survey plan is formulated using aerial images used to grasp the status of land features such as houses.

[0004] Patent Document 1 describes a disaster situation assessment system that assesses the damage situation in a disaster-stricken area using images captured by an aircraft. Figures 5 to 7 of the document show two-dimensional maps on which damage information is superimposed. Figure 8 shows a three-dimensional map on which damage information is superimposed.

[0005] Patent Document 2 describes a restoration support method that reduces or automates the on-site survey and plan preparation work required for restoration work after a disaster. The method described in this document displays the scale of restoration work obtained by comparing orthoimages before and after a landslide disaster on an orthoimage after the landslide disaster.

[0006] Patent Document 3 describes a map updating system that compares a map with an aerial photograph image taken after the map was created and detects changes to structures in the aerial photograph image based on the comparison results. The system described in this document assigns attributes to structures, such as remaining, removed, expanded or remodeled, newly constructed, and requiring investigation, and displays map figures of the structures on the map according to the attributes. The document describes display colors and line types of map figures as examples of displaying map figures according to attributes.

[0007] Patent Document 4 describes a population estimation device that estimates population using aerial photograph images. The device described in this document accepts an area for which a user wants to estimate the population, input from a user terminal.

[0008] Patent Document 5 describes a forest information processing system that automatically calculates tree volume in a forest area from forest image information and tree height information. The system described in this document extracts the apex of each tree from image data obtained by photographing an area including a forest from the air, extracts each tree crown, creates information including the sunny part of each tree crown, identifies the tree species of the crown based on the spectral reflectance characteristics of the sunny part of each tree crown, identifies the tree species of the tree apex, and calculates the number of trees included in the entire area of ​​the forest area or an arbitrary range by tree species.

[0009] International Publication No. 2013 / 051300 JP 2006-276306 A JP 2004-198530 A Japanese Patent No. 4914266 A Japanese Patent No. 4858793 A

[0010] However, in disaster damage surveys, priorities are determined and survey personnel are assigned for each area as determined appropriately by local governments, etc., but determining priorities and assigning survey personnel directly from aerial images takes time and requires skill.

[0011] Furthermore, if only information on the damage status of each feature is provided, there will be insufficient information to formulate an investigation plan. For example, in the case of an investigation into damaged houses, the difference between wooden and non-wooden structures, and the difference between determining whether a house is completely destroyed or partially destroyed, requires different levels of expertise and knowledge, making it difficult to formulate an investigation plan, including the allocation of investigators.

[0012] If information aggregated for each area that a survey team is responsible for surveying is not provided, it is difficult to formulate a survey plan, such as assigning survey areas to survey teams, determining the composition of each survey team, and changing survey team members.In other words, even if information on geographical features is already available, it is difficult to formulate a survey plan, such as selecting survey team members.

[0013] The system described in Patent Document 1 displays various information related to the disaster superimposed on a two-dimensional map, allowing the extent of the damage to be grasped to a certain extent, but the information obtained from the system is insufficient to be used in formulating an investigation plan.

[0014] The method described in Patent Document 2 is related to landslides that occur locally, and it is difficult to apply it to the investigation of disaster situations where the investigation area is wide.

[0015] The system described in Patent Document 3 is specialized in analyzing the characteristics of each feature in aerial photograph images, and it is difficult to apply it to investigating the damage situation in areas that include multiple features.

[0016] The device described in Patent Document 4 relates to estimating population, and it is difficult to apply it to surveys of disaster situations, which include surveys of situations at multiple features.

[0017] The system described in Patent Document 5 relates to forest surveys, and it is difficult to apply it to surveys of disaster situations, which involve surveys of the situations of multiple features.

[0018] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, a program, and an information processing system that enable the formulation of a desirable survey plan for each analysis area in a survey area.

[0019] An information processing device according to a first aspect of the present disclosure includes a processor and a memory storing a program to be executed by the processor, and the processor executes the program stored in the memory to acquire map information representing the geographical position and shape of one or more features included in a survey area, acquire feature meta information for each of the one or more features, acquire aerial images of the survey area, derive a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images, statistically process the degree of damage derived for each of the one or more features for an analysis area included in the survey area, derive representative information representing the analysis area, and display the analysis area in association with the representative information.

[0020] According to the information processing device according to the first aspect of the present disclosure, for an analysis area included in a survey area, the damage levels derived for each of one or more features are statistically processed to derive representative information. The representative information for the analysis area is displayed in association with the analysis area. This allows for the formulation of an investigation plan appropriate for the analysis area.

[0021] The damage level is an index that indicates the degree of damage to a feature. Examples of the damage level include complete destruction, partial destruction, partial damage, and normal damage.

[0022] In the information processing device of the second aspect, in the information processing device of the first aspect, a processor extracts multiple analysis area images of the analysis area from an aerial image, aligns the positions and contour shapes of features of the multiple analysis area images with respect to a map including the analysis area, generates an aligned composite image by combining the multiple analysis area images, and associates the aligned composite image with representative information and displays it.

[0023] An information processing device according to a third aspect may be the information processing device of the second aspect, in which the processor statistically processes the degree of damage derived for each feature for each sub-analysis area into which the analysis area is subdivided, generates representative information for each sub-analysis area, and associates and displays an aligned composite image with the representative information for each sub-analysis area.

[0024] In the information processing device of the fourth aspect, in the information processing device of the second or third aspect, the processor may derive the degree of damage for each feature, apply a display mode according to the degree of damage, and display the degree of damage by superimposing it on the aligned composite image.

[0025] An information processing device according to a fifth aspect is an information processing device according to any one of the first to fourth aspects, wherein the processor may determine the analysis area using boundary information representing boundaries included in the map information.

[0026] In an information processing device according to a sixth aspect, in an information processing device according to any one of the first to fifth aspects, the processor may determine an analysis area containing multiple features based on aerial images and map information.

[0027] An information processing device according to a seventh aspect is the information processing device of any one of the first to sixth aspects, wherein the processor may acquire information indicating an analysis area designated by a user.

[0028] In an information processing device according to an eighth aspect, in an information processing device according to any one of the first to seventh aspects, a processor may acquire survey plan information for an analysis area that indicates at least one of the survey priority, the period required for the survey, the number of investigators, the expertise of the investigators, and the proficiency of the investigators, and display the survey plan information and representative information.

[0029] An information processing method according to a ninth aspect of the present disclosure is an information processing method that executes the steps of acquiring map information representing the geographical position and shape of one or more features included in a survey area, acquiring feature meta information for each of the one or more features, acquiring aerial images of the survey area, deriving a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images, statistically processing the degree of damage derived for each of the one or more features for an analysis area included in the survey area, and deriving representative information that represents the analysis area, and associating the analysis area with the representative information and displaying it.

[0030] According to the information processing method according to the ninth aspect of the present disclosure, it is possible to obtain the same effects as those of the information processing device according to the first aspect of the present disclosure.

[0031] In the information processing method according to the ninth aspect, it is possible to appropriately combine the same items as those specified in the second to eighth aspects. In this case, the components that perform the processing or function specified in the information processing device can be understood as components of the information processing method that perform the corresponding processing or function.

[0032] A program according to a tenth aspect of the present disclosure is a program that enables a computer functioning as an information processing device to perform the following functions: acquire map information representing the geographical position and shape of one or more features included in a survey area; acquire feature meta information for each of the one or more features; acquire aerial images of the survey area; derive a degree of damage for each of the one or more features based on the map information, feature meta information, and aerial images; statistically process the degree of damage derived for each of the one or more features for an analysis area included in the survey area and derive representative information that represents the analysis area; and associate the analysis area with the representative information and display it.

[0033] According to the program according to the tenth aspect of the present disclosure, it is possible to obtain the same effects as those of the information processing device according to the first aspect of the present disclosure.

[0034] In the program according to the tenth aspect, it is possible to appropriately combine the same items as those specified in the second to eighth aspects. In this case, the components that perform the processes or functions specified in the information processing device can be understood as the components of the program that perform the corresponding processes or functions.

[0035] An information processing system according to an eleventh aspect of the present disclosure includes an information processing device that is communicatively connected to an external device, the information processing device including a processor and a memory that stores a program to be executed by the processor, and the processor executes the program stored in the memory to acquire map information representing the geographical position and shape of one or more features included in a survey area, acquire feature meta information for each of the one or more features, acquire aerial images of the survey area, derive a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images, statistically process the degree of damage derived for each of the one or more features for an analysis area included in the survey area, derive representative information that represents the analysis area, and display the representative information in association with the analysis area.

[0036] According to the information processing system according to the eleventh aspect of the present disclosure, it is possible to obtain the same effects as those of the information processing device according to the first aspect of the present disclosure.

[0037] In the information processing system according to the eleventh aspect, it is possible to appropriately combine the same items as those specified in the second to eighth aspects. In this case, the components that perform the processes or functions specified in the information processing device can be understood as components of the program that performs the corresponding processes or functions.

[0038] According to the present invention, for an analysis area included in a survey area, the damage levels derived for each of one or more features are statistically processed to derive representative information. The representative information for the analysis area is displayed in association with the analysis area. This allows for the formulation of an investigation plan appropriate for the analysis area.

[0039] FIG. 1 is a schematic diagram showing an example of the configuration of an information processing system according to an embodiment. FIG. 2 is a functional block diagram showing the electrical configuration of an information processing device according to an embodiment. FIG. 3 is a block diagram showing the hardware configuration of the information processing device shown in FIG. 2. FIG. 4 is an explanatory diagram of information processing performed using an information processing system. FIG. 5 is an explanatory diagram of derivation of the degree of damage. FIG. 6 is an explanatory diagram of a specific example of determination of an analysis area and statistical processing of the degree of damage. FIG. 7 is an explanatory diagram showing a specific example of an aligned composite image. FIG. 8 is an explanatory diagram showing an example of a display screen for representative information. FIG. 9 is an explanatory diagram showing an example of display of priority order.

[0040] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification, identical components are designated by the same reference numerals, and duplicate descriptions will be omitted where appropriate. Furthermore, when multiple components are listed in the following embodiments, it can be interpreted as including at least one of the multiple components.

[0041] 1 is a schematic diagram showing an example of the configuration of an information processing system according to an embodiment. In this embodiment, an information processing system 10 is illustrated that ascertains the damage status of houses included in aerial images of a disaster area taken using a camera 14 mounted on a drone 12. In this specification, the term "image" may include the meaning of image data, which is an electrical signal representing an image.

[0042] The information processing system 10 includes a drone 12 for aerial photography, a camera 14 mounted on the drone 12, a remote controller 16, and an information processing device 20. The drone 12 is an unmanned aerial vehicle that is remotely controlled using the remote controller 16. The drone 12 may have an autopilot function that flies according to a program.

[0043] The camera 14 is mounted on the drone 12 via a gimbal head 13. The camera 14 includes an optical system, an image sensor, and a signal processing circuit. Note that the details of the optical system, the image sensor, and the signal processing circuit are not shown in the figure.

[0044] The optical system includes one or more lenses. The image sensor may be a CCD image sensor or a CMOS image sensor, where CCD is an abbreviation for Charge Coupled Device and CMOS is an abbreviation for Complementary Metal-Oxide Semiconductor.

[0045] The camera 14 processes the signal obtained from the image sensor using a signal processing circuit to generate digital image data of the photographed object. The digital image data generated by the camera 14 can be a photographed image.

[0046] The captured images taken and generated using the camera 14 can be stored in an internal storage built into the drone 12 or in a storage device such as a memory card that can be removably attached to the drone 12.

[0047] Furthermore, the captured image captured and generated using the camera 14 may be transferred to the remote controller 16 using wireless communication, or may be transferred to the information processing device 20, another terminal device 24, or the like.

[0048] The remote controller 16 is a transmitter that controls the operations of the camera 14 and the drone 12. Wireless communication is used for communication between the remote controller 16 and the camera 14 and drone 12. The wireless communication may be in the form of a wireless LAN, a communication format using radio waves in the 2.4 GHz band or 5.7 GHz band, or a format using a mobile communication network.

[0049] The communication format for the control signals for operating the drone 12 and the communication format for transferring images captured by the camera 14 may be different, or a common format may be used for both. Note that LAN is an abbreviation for Local Area Network.

[0050] The remote controller 16 includes left and right sticks for controlling the flight operations of the drone 12, a lever for operating the gimbal head 13, a shooting button for instructing the camera 14 to take a photograph, and a shooting mode button for switching between video shooting and still image shooting.

[0051] When a touch panel display is used for the display 16A of the remote controller 16, operation buttons such as a shooting button are realized using software buttons displayed on the touch panel display.

[0052] The live video captured by the camera 14 may be displayed on the display 16A of the remote controller 16. The remote controller 16 can also grasp the status of the drone 12, such as its flight position and flight speed, in real time based on data from various sensors equipped on the drone 12. Flight information indicating the status of the drone may be displayed on the display 16A.

[0053] 1 is an example of an image captured using the camera 14. The information processing system 10 captures at least one still image from the air and processes the captured image IM in the information processing device 20.

[0054] The information processing device 20 is configured using a computer. The computer applied to the information processing device 20 may be a server, a personal computer, or a workstation.

[0055] The information processing device 20 is electrically connected to the remote controller 16 and the terminal device 24 via a network 22 so as to be able to freely communicate data. The network 22 may be a local area network or a wide area network. The information processing device 20 acquires various information from the drone 12 and the camera 14. The information processing device 20 may also acquire map data of the survey area from a map information storage device via the network 22. Note that the map information storage device is not shown in FIG. 1. The map information storage device is designated by the reference numeral 30 and is illustrated in FIG. 2.

[0056] The information processing device 20 may include a wired input / output interface such as a universal serial bus, etc. The information processing device 20 may include one or more ports as the input / output interface.

[0057] The information processing device 20 includes a display device 20A and an input device 20B as user interfaces. The display device 20A displays various types of information in the information processing device 20. The input device 20B includes devices operated by the user, such as a keyboard and a mouse. The information processing device 20 includes a touch panel type display device 20A, and the display device 20A and the input device 20B may be configured as an integrated unit.

[0058] The terminal device 24 may be a mobile information terminal such as a smartphone or a tablet terminal. The terminal device 24 includes a display 24A. The terminal device 24 may have the functions of the remote controller 16. The terminal device 24 may have the processing functions of the information processing device 20.

[0059] 2 is a functional block diagram showing the electrical configuration of an information processing device according to an embodiment. The information processing device 20 includes a map information acquisition unit 30, a feature meta information acquisition unit 32, an aerial image acquisition unit 34, and a boundary information acquisition unit 36.

[0060] The map information acquisition unit 30 acquires map information of features, including information on the geographical location of each feature in the survey area, such as its address, and shape information of the feature, from an external map information storage system 38. The shape of the house may be a two-dimensional shape of the house, including the area of ​​the house on the map.

[0061] The feature meta information acquisition unit 32 acquires feature meta information including information about the house for which map information has been acquired from the map information storage system 38. The feature meta information may include structural information about the house, indicating whether the house is wooden or non-wooden, the age of the house, and the repair history of the house. The feature meta information may also include information about the damage status of features such as houses.

[0062] Here, a feature refers to all objects that exist on the ground, whether natural or artificial, such as buildings, trees, rocks, etc. A feature on a map refers to all objects that are depicted on a map, including physically existing and visible objects such as railways, roads, and street trees, as well as non-physically existing and invisible objects such as borders, place names, and bus routes.

[0063] The aerial image acquisition unit 34 acquires aerial images of a survey area captured using the camera 14 shown in Fig. 1. Fig. 2 illustrates an example of how a desired aerial image is acquired from an aerial image storage system 40 that stores aerial images captured and generated using the camera 14. The aerial image acquisition unit 34 may acquire the aerial images from the camera 14.

[0064] The aerial image acquisition unit 34 may acquire additional information about the aerial image as information about the shooting of the aerial image. The additional information about the aerial image may include information about the shooting position, information about the shooting direction, and information about the shooting conditions.

[0065] The boundary information acquisition unit 36 ​​acquires boundary information regarding the boundaries of the survey area from the map information storage system 38. The boundary information may be information representing geographical boundaries such as boundaries defined on a map. The boundary information acquisition unit 36 ​​may accept information representing geographical boundaries specified on a map by a user.

[0066] The information processing device 20 includes an analysis area determination unit 42. The analysis area determination unit 42 determines an analysis area in which analysis processing related to features is to be performed, based on the boundary information acquired using the boundary information acquisition unit 36.

[0067] The information processing device 20 includes an analysis unit 44. The analysis unit 44 acquires map information, feature meta information, aerial images, and analysis area information, and uses the map information, feature meta information, aerial images, and analysis area information to individually extract multiple features included in the analysis area, and derive a damage level for each of the individually extracted features.

[0068] The analysis unit 44 performs statistical processing on the degree of damage to features included in the analysis area. From the results of the statistical processing, representative information that represents the analysis area is derived. The analysis unit 44 outputs the representative information of the analysis area as the analysis result.

[0069] The information processing device 20 includes an image processing unit 46. The image processing unit 46 acquires map information, feature meta information, aerial images, and analysis area information, and performs processing to associate the map information of the analysis area with the aerial images of the analysis area using the map information, feature meta information, aerial images, and analysis area information.

[0070] The image processing unit 46 performs processing to extract features from the aerial images. The aerial images of each feature extracted from the aerial images of the analysis area are sent to the analysis unit 44. The analysis unit 44 uses the aerial images of each feature in the analysis area to derive the degree of damage for each feature.

[0071] The image processing unit 46 performs processing to transform the aerial photograph image of the analysis area. For example, the image processing unit 46 generates an orthoimage of the analysis area from the aerial photograph image of the analysis area.

[0072] The image processing unit 46 generates a registered composite image by superimposing the orthoimage of the analysis area at the position of the analysis area on the map of the survey area, and outputs the registered composite image.

[0073] The information processing device 20 includes a display information generation unit 48. The display information generation unit 48 generates display information that displays, on one screen, the representative information of the analysis area generated using the analysis unit 44 and the aligned composite image generated using the image processing unit 46. The information processing device 20 transmits a signal representing the display information to the display device 20A, and causes the display device 20A to display the display information.

[0074] [Hardware Configuration of Information Processing Device] Fig. 3 is a block diagram showing the hardware configuration of the information processing device shown in Fig. 2. Various processes of the information processing device 20 are implemented by applying an arbitrary computer. In the arbitrary computer, a processor may execute a program to execute various processes of the information processing device 20.

[0075] Any computer may be a general-purpose computer such as a personal computer, or a special-purpose computer such as a server computer. Any computer may be a system such as a workstation, or any other piece of hardware capable of running programs, such as a virtual machine.

[0076] At least some of the functions of the information processing device 20 may be realized using cloud computing. At least some of the functions of the information processing device 20 may be provided as SaaS. SaaS is an abbreviation for Software as a Service.

[0077] The information processing device 20 includes a processor 202, a computer-readable medium 204, a communication interface 206, an input / output interface 208, and a bus 210. The computer-readable medium 204 is a non-transitory tangible entity, and includes a memory 212 that is a main storage device and a storage 214 that is an auxiliary storage device.

[0078] The processor 202 is connected to a memory 212, a storage 214, an input / output interface 208, an input device 222, and a display device 224 via a bus 210. The display device 224 shown in Figure 3 corresponds to the display device 20A shown in Figure 1.

[0079] The memory 212 includes RAM. The memory 212 may also include ROM. The storage 214 may be, for example, a hard disk drive, a solid state drive, or a combination of these. The storage 214 may also include an external storage device such as removable media.

[0080] Note that RAM is an abbreviation for Random Access Memory, and ROM is an abbreviation for Read Only Memory. A hard disk drive may be referred to as an HDD, which is an abbreviation for Hard Disk Drive. A solid state drive may be referred to as an SSD, which is an abbreviation for Solid State Drive.

[0081] The computer-readable medium 204 stores programs, data, and the like that realize various functions of the information processing device 20. The processor 202 executes the programs stored in the memory 212 to realize various functions of the information processing device 20. The processor 202 comprehensively controls each part of the information processing device 20 and various devices and units connected to the information processing device 20, and performs various processes.

[0082] The input / output interface 208 includes a communication interface connectable to a telecommunications line such as a local area network, and a connection interface connectable to an external device. Examples of a connection interface connectable to an external device include a universal serial bus and HDMI (HDMI is a registered trademark). HDMI is an abbreviation for High-Definition Multimedia Interface.

[0083] The processor 202 communicates with various devices of the information processing device 20 via the input / output interface 208, and transmits and receives various types of information.

[0084] Examples of the input device 222 include a keyboard and a pointing device such as a mouse. The input device 222 may include a numeric keypad and various switch buttons. The input device 222 may include a voice input device. The input device 222 may be a touch panel type input device that is integrated with the display screen of the display device 224.

[0085] The display device 224 may be a liquid crystal display, an organic EL display, or a projector. The display device 224 may be an appropriate combination of liquid crystal displays, etc. Various information is displayed on the display device 224. The display device 224 is used as part of a UI when receiving input from the input device 222. The display device 224 is not limited to one, and a multi-display configuration having multiple display devices is also possible.

[0086] Organic EL is an abbreviation for organic electroluminescence, and UI is an abbreviation for User Interface.

[0087] In this embodiment, each process is executed by a computer. A processor, a program, or a combination thereof may be applied to the computer to execute the process. The computer may be a general-purpose computer, a computer for specific applications, a system such as a workstation, or any other hardware element capable of executing a program.

[0088] The processor 202 may be configured with one or more pieces of hardware, and the type of hardware is not limited. The hardware of the processor 202 may be a programmable logic device such as a CPU, an MPU, or an FPGA. The processor 202 may be configured with a dedicated circuit that executes specific processing, such as an ASIC. The hardware of the processor 202 may be configured with a GPU that performs processing specialized for image processing, an NPU that specializes in AI processing, or the like.

[0089] The processor 202 functions as Units, which are various processing units that execute various processes, and Means, which are various processing means that execute various processes. Note that CPU is an abbreviation for Central Processing Unit, MPU is an abbreviation for Micro-Processing Unit, and FPGA is an abbreviation for Field-Programmable Gate Array. Furthermore, GPU is an abbreviation for Graphics Processing Unit, AI is an abbreviation for Artificial Intelligence, and NPU is an abbreviation for Neural network Processing Unit.

[0090] The processor 202 may be configured by combining different types of hardware. The hardware of the processor 202 may be an electric circuit or the like that combines electric circuit elements such as semiconductor elements. The electric circuit may be referred to as a circuitry.

[0091] When multiple pieces of hardware execute any one or more processes of the processor 202, the multiple pieces of hardware may be located in devices physically separated from each other, or may be located in the same device. The order of processes executed by the processor 202 is not limited to the order disclosed in this specification and may be changed as appropriate. The hardware is configured using an electric circuit or the like that combines circuit elements such as semiconductor elements.

[0092] Furthermore, the present embodiment may be realized by applying hardware, software, firmware, microcode, or a combination thereof. The software, firmware, and microcode are configured by applying a program. For example, the program may be a group of program modules, and the functions of the software or the like may be realized by applying a processor that executes each function.

[0093] The program may be, for example, a program code and a plurality of code segments stored in one or more non-transitory computer-readable media such as a storage medium and storages. The program may be stored in a divided manner across a plurality of non-transitory computer-readable media that exist in devices that are physically separate from each other.

[0094] A program code or code segment may represent any combination of a procedure, a function, a subprogram, a routine, a subroutine, a module, a software package, a class, an instruction, a data structure, or a program statement. The program code or code segment may be connected to another code segment or a hardware circuit by sending or receiving information, data, arguments, parameters, or memory contents.

[0095] [Specific example of disaster investigation support] Fig. 4 is an explanatory diagram of information processing performed using an information processing system. Fig. 4 shows an example of information processing performed in formulating a disaster investigation plan that is implemented after a disaster occurs. Below, an example in which a building is used as a feature will be described.

[0096] When conducting a survey of the damage situation, first, a survey area 100 for the damage situation is defined. The survey area 100 is defined by the local government in charge of the damage survey. The survey area 100 may be an administrative area such as a city, ward, town, or village.

[0097] Once the disaster-stricken survey area 100 is defined, aerial photography of the survey area 100 is carried out, and aerial images 102 of the survey area 100 are acquired. Figure 4 illustrates aerial photography carried out using cameras 14 mounted on each of multiple drones 12. Each camera 14 carries out aerial photography at each of multiple shooting positions, generating multiple aerial images 102 taken at different shooting positions across the entire survey area 100.

[0098] The aerial images 102 are stored in an aerial image storage system 40 shown in Figure 2. The aerial image storage system 40 can search for the aerial images 102 using identification information assigned to each disaster survey as an index. The aerial image storage system 40 may be configured to be able to search for the aerial images 102 using the survey target area 100, the analysis area, etc. as indexes. The aerial image acquisition unit 34 shown in Figure 2 acquires multiple aerial images 102 corresponding to the survey target area 100 from the aerial image storage system 40.

[0099] Each of the multiple circles shown in Figure 4 represents the shooting range 103 of the drone 12 in one shooting session. The aerial images 102 at adjacent shooting positions may have a portion of the shooting range 103 overlapping. Figure 4 illustrates an example in which multiple cameras 14 are used to shoot the survey area 100, but a single camera 14 may be used to shoot the survey area 100.

[0100] The aerial image 102 may be supplemented with information about the shooting location and the shooting direction as information related to the shooting. The information about the shooting location may be the latitude, longitude, and altitude of the shooting location. The information about the shooting direction may be at least one of the orientation of the optical axis of the camera 14 and the direction of the optical axis of the camera 14 relative to the vertical direction. The additional information about the aerial image 102 may include shooting parameters applied to the camera 14.

[0101] In acquiring the map information, the map 104 of the survey area 100 and additional information of the map 104 are acquired as map information from the maps stored in the map information storage system 38. The map 104 of the survey area 100 may be a plurality of maps for each of a plurality of sub-areas into which the survey area 100 is divided. The map information includes, as additional information of the map 104, the geographic coordinates of buildings to which a combination of latitude, longitude, and altitude of a feature is applied, building shape information that represents the shape of each building, and building height information that represents the height of each building.

[0102] In determining the analysis area, a plurality of analysis areas 105 are determined from within the survey area 100. That is, in determining the analysis area, analysis area information indicating the boundaries of the analysis area 105 is acquired.

[0103] In extracting the positional relationships of buildings, map information is used to extract the positions and shapes of buildings from the aerial image 102. Specifically, a roof frame 106 representing the periphery of the roof of the building and an outer periphery frame 108 representing the periphery of the building on the ground are set for the aerial image 102. Figure 4 shows the aerial image 102 in which the roof frame 106 and outer periphery frame 108 are combined for each building.

[0104] In extracting the positional relationships of buildings, a correspondence relationship 110 between the coordinate values ​​of each building in the aerial image 102 and the geographic coordinates of each building is derived. A two-dimensional Cartesian coordinate system with one of the four corners of the aerial image 102 as the origin may be applied to the coordinates in the aerial image 102. Figure 4 illustrates an example of the correspondence relationship 110, which is a group of corresponding pairs between geographic coordinates and image coordinates.

[0105] In the extraction of buildings, a building image 102A is generated for each building included in the aerial image 102. The building image 102A may include the entire target building, or may include a portion of a building adjacent to the target building. The building image 102A may be generated by performing a cropping process on the aerial image 102.

[0106] In the analysis of the building images 102A, the damage level for each building is calculated. Statistical processing is performed on the damage level for each building, and representative information 112 for each analysis area is derived. Details of the representative information 112 for the analysis area will be described later.

[0107] In the damage level display selection, the display mode of the damage level for each building is selected. In Fig. 4, the damage levels of buildings are exemplified as "completely destroyed," "safe," and "unknown." The "unknown" damage level indicates a case where the building falls into neither "safe" nor "completely destroyed."

[0108] For buildings whose damage level is unknown, information is added as to whether they are wooden or non-wooden. Furthermore, for buildings whose damage level is unknown, information is added as to whether they comply with new earthquake resistance standards or old earthquake resistance standards. Note that the damage levels shown in FIG. 4 are merely examples, and other information such as the age of the building may be added.

[0109] In the image transformation to the geographical position, an orthoimage 114 is generated, which is an analysis area image corresponding to a portion of the aerial image 102 that corresponds to the analysis area. In addition, icons representing the degree of damage to each building are superimposed on the orthoimage 114. In the orthoimage 114 illustrated in FIG. 4, the icons are omitted.

[0110] [Specific Example of Damage Degree Deriving] Fig. 5 is an explanatory diagram of damage degree derivation. Fig. 5 illustrates the procedure for deriving the damage degree. In step S100, the map information acquisition unit 30 illustrated in Fig. 2 acquires the geographic coordinates of the features as information on the geographical position of each feature as map information for the features. Also in step S100, the shape of each feature is acquired as map information for each feature.

[0111] In step S102, the aerial image acquisition unit 34 acquires the aerial image 102 and information related to the shooting of the aerial image. The information related to the shooting of the aerial image may be information added to the aerial image 102. The information related to the shooting of the aerial image may include information on the shooting position, shooting direction, and shooting conditions.

[0112] In step S104, the image processing unit 46 matches the features extracted from the map information with the features extracted from the aerial image to identify each feature in the aerial image. Specifically, the coordinate values ​​of the multiple features extracted from the aerial image are associated with geographic coordinates, and a correspondence relationship 110 of the geographic coordinates for each building shown in Figure 4 is derived.

[0113] In step S106, the feature meta information acquisition unit 32 acquires, as feature meta information, information indicating whether the feature is a wooden or non-wooden structure, information indicating the age of the feature, etc. The information indicating the age of the feature may be information on the completion date of the feature.

[0114] In step S108, the image processing unit 46 uses the feature's ID (Identification), the feature's geographic coordinates, the feature's height, the aerial image, and matching information between the map and the aerial image to extract the range of each feature from the aerial image 102. Also in step S108, the analysis unit 44 derives the degree of damage for each type of feature extracted from the aerial image 102.

[0115] The damage level may be derived using a damage level derivation AI that derives the damage level for each feature. The damage level derivation AI is generated by performing learning using photographed images of each type of feature and labels indicating the damage level of the feature as learning data, and when a photographed image of each feature is input, the damage level of the feature is output. An example of a photographed image of each feature here is a building image 102A shown in Figure 4.

[0116] Examples of the type of feature include whether the feature is wooden or non-wooden, the age of the feature, etc. The photographed images and the type of feature for each feature are applied as input to the damage degree derivation AI.

[0117] In step S108, the output of the damage degree derivation AI, which is trained to recognize the visual damage degree and outputs the visual damage degree when a photographed image of a feature is input, may be converted into a damage degree value using information on the type of each feature, thereby deriving the damage degree for each feature. A score expressed using a numerical value between 0 and 1 may be applied to the damage degree. A percentage may be applied to the score.

[0118] The display manner of the damage degree for each feature may be determined from the damage degree for each feature and the type of feature derived in step S108. Examples of the display manner of the damage degree include display icons shown in FIG.

[0119] [Specific Example of Determining Analysis Area and Statistical Processing of Damage Degree] Figure 6 is an explanatory diagram of a specific example of determining an analysis area and statistical processing of damage degree. In step S110, the analysis area determination unit 42 shown in Figure 2 may apply an administrative district to determine the geographical location and range of the analysis area 105. In step S110, the geographical location and range of the analysis area 105 may be determined by applying an area bounded by roads, ridges, rivers, etc.

[0120] In step S112, the damage level of each feature and the geographic information of the features are used for each analysis area 105 to derive representative information for each analysis area 105. For example, in step S112, statistical values ​​such as rank-specific ratios, average values, and variance values ​​are calculated for values ​​representing the damage level of each type of feature in the analysis area 105, and output as representative information for each analysis area 105.

[0121] In an information processing method implemented by a computer functioning as an information processing device, each of multiple computers may implement some of the steps of the information processing method. The information processing method may include a step in which the computer acquires information input by a user operating an input device. The information processing method may also include a step in which the computer acquires information transmitted from an external device. Note that the term "user" is synonymous with "operator" and "operator."

[0122] [Specific example of aligned composite image] Figure 7 is an explanatory diagram showing a specific example of an aligned composite image. The figure illustrates an aligned composite image 120 synthesized with an orthoimage 114 of an analysis area 105. The aligned composite image 120 is synthesized by synthesizing the orthoimage 114 at the position of the analysis area 105 with respect to a map 104 including the survey target area. Note that the aligned composite image 120 is an example of an aligned composite image synthesized by synthesizing multiple analysis area images by aligning the positions and contour shapes of features of multiple analysis area images with respect to a map including the analysis area of ​​the present disclosure.

[0123] 7 shows an enlarged view of the orthoimage 114 to be synthesized into the aligned composite image 120. Icons 122 that visually represent the degree of damage to each feature are attached to the orthoimage 114. Five types of icons 122 that represent the degree of damage are shown in the figure.

[0124] On the screen where the aligned composite image 120 is displayed, when an operation such as clicking is performed on the position of the orthoimage 114, the orthoimage 114 may be enlarged and displayed. The enlarged orthoimage 114 may be displayed in a window separate from the aligned composite image 120, or a speech bubble may be applied.

[0125] 7, an icon marked with A represents total destruction, and an icon marked with B represents safety. Icons marked with C, D, and E represent unknown. Furthermore, an icon marked with C represents a non-wooden structure such as a reinforced concrete structure and a new earthquake-resistant structure, and an icon marked with D represents an old earthquake-resistant structure. An icon marked with D represents a wooden structure and a new earthquake-resistant structure. Each of the five types of icons 122 is applied in a manner that allows them to be visually distinguished, such as by having a different color or a different shape.

[0126] [Display example of representative information] Fig. 8 is an explanatory diagram showing an example of a display screen of representative information. The figure shows an enlarged portion of the aligned composite image 120 shown in Fig. 7. Fig. 8 also shows an example in which the representative information 112 is in the form of a bar graph 160.

[0127] 8 shows the number of completely destroyed buildings for each sub-analysis area 162 obtained by further dividing the analysis area 105. In the representative information 112, the number of completely destroyed buildings is classified by building type.

[0128] The sub-analysis areas 162 shown in Fig. 8 are defined using a 5x4 grid 164 on the aligned composite image 120. Of the sub-analysis areas 162 shown in Fig. 8, those that do not include any analysis areas 105 have the number of totally destroyed features set to zero. Furthermore, for sub-analysis areas 162 that include the analysis area 105 but also include areas that are not analysis areas 105, the number of totally destroyed features for the analysis areas 105 included in the sub-analysis area 162 is represented as representative information 112.

[0129] Each arrowhead line pointing from each grid 164 representing a sub-analysis area 162 to the horizontal axis of the bar graph 160 representing the representative information 112 indicates the correspondence between each sub-analysis area 162 and the horizontal axis of the graph. That is, the horizontal axis of the bar graph 160 represents each of the multiple sub-analysis areas 162, and the vertical axis represents the number of completely destroyed buildings in each sub-analysis area 162.

[0130] The aligned composite image 120 and the representative information 112 are displayed on the display device 20A. The aligned composite image 120 and the representative information 112 may be included in the same window, or may be displayed in separate windows. Figure 8 illustrates an example in which the aligned composite image 120 and the representative information 112 are displayed in the same window.

[0131] [Example of Priority Display] Fig. 9 is an explanatory diagram showing an example of a priority display. The figure schematically illustrates a priority display screen 140 that displays the priority of the damage investigation for each analysis area 105. The priority display screen 140 is illustrated on the display device 20A illustrated in Fig. 2. The priority display screen 140 may be displayed in response to a user input.

[0132] The priority order display screen 140 includes a map display section 142 and a priority order information section 144. The map display section 142 displays a map 104 representing the entire survey area 100. The priority order information section 144 displays priority order information 146 representing the priority order of damage surveys for each analysis area 105. The priority order information 146 shown in FIG. 9 is applied with a numerical value representing the priority order for each analysis area 105.

[0133] 9 shows a priority display screen 140 in which a map display section 142 and a priority information section 144 are included in the same window. The map display section 142 and the priority information section 144 may be displayed in separate windows.

[0134] 9 illustrates an example in which chome such as AA1-chome is applied as the analysis area 105. The same figure also illustrates an example in which a grid 164 representing a sub-analysis area 162 is superimposed on FF2-chome, which is the analysis area 105. Note that the town names illustrated in FIG. 9 are arbitrary names.

[0135] 9 illustrates priority information 146 in a list format. The priority information 146 may be arranged in ascending or descending order of priority. The priority information 146 illustrated in the figure illustrates analysis areas 105 with priorities ranging from 1 to 6, but the number of analysis areas 105 displayed as the priority information 146 can be specified as appropriate.

[0136] 9 illustrates an example in which the priority of damage investigations is derived from representative information obtained by statistically processing the degree of damage for each feature, but information representing at least one of the period required for the investigation, the number of investigators, the expertise of the investigators, and the proficiency of the investigators may be derived. The priority of damage investigations, etc. may be derived as investigation plan information.

[0137] [Application Examples to Programs and Program Products] The information processing methods according to the embodiments may be configured as programs or program products in which a processor or a computer including a processor implements the functions of the steps.

[0138] For example, a program or program product may be configured to enable a computer to perform the following functions: acquire map information, acquire classification information for features, acquire aerial images, determine an analysis area, statistically process the degree of damage for each feature in the analysis area to derive representative information for the analysis area, and associate the analysis area with the representative information.

[0139] The program or program product may be stored in a computer-readable medium that is a tangible, non-transitory information storage medium, or may be provided through an information storage medium.

[0140] [Operational Effects of the Embodiment] The information processing device 20, the information processing method, etc. according to the embodiment can obtain the following operational effects.

[0141] [1] An analysis area 105 is determined for the survey area 100. An aligned composite image 120 is generated by combining an orthoimage 114 of the analysis area 105 in the aerial image 102 with a map 104 of the survey area 100. In addition, representative information 112 is derived for the analysis area 105, representing the extent of damage to which the degree of damage to each feature has been statistically processed. The representative information 112 is associated with the aligned composite image 120 and displayed. This allows the extent of damage to each analysis area 105 to be understood.

[0142] [2] Based on the representative information 112 of the analysis area 105, priority information 146 indicating the priority of damage investigation is generated for each analysis area 105. This can support the formulation of a damage investigation plan.

[0143] [3] As representative information, the number of completely lost features for each type of feature is calculated for each analysis area. This makes it possible to grasp the number of investigators dispatched to each analysis area, their skills, etc.

[0144] [4] The orthoimage 114 to be synthesized into the aligned synthesized image 120 displays an icon 122 indicating the degree of damage for each feature, thereby allowing the distribution of the degree of damage to be grasped.

[0145] [5] Representative information 112 is derived for each sub-analysis area 162 obtained by subdividing the analysis area 105. This allows the distribution of damage conditions in the analysis area 105 to be grasped.

[0146] [6] The location and extent of the analysis area 105 are defined by boundaries such as administrative districts, roads, rivers, etc. This determines the analysis area 105 having geographical boundaries.

[0147] In this embodiment, an example is given of investigating the damage status of a building when a disaster such as an earthquake occurs, but a configuration similar to this embodiment may also be applied to inspecting changes in a building over time, investigating the asset value of a building, and completing inspections of new construction or renovations.

[0148] The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention.

[0149] 10 Information processing system 12 Drone 13 Gimbal head 14 Camera 16 Remote controller 16A Display 20 Information processing device 20A Display device 20B Input device 22 Network 24 Terminal device 24A Display 30 Map information acquisition unit 32 Feature meta information acquisition unit 34 Aerial image acquisition unit 36 ​​Boundary information acquisition unit 38 Map information storage system 40 Aerial image storage system 42 Analysis area determination unit 44 Analysis unit 46 Image processing unit 48 Display information generation unit 100 Survey area 102 Aerial image 102A Building image 103 Shooting range 104 Map 105 Analysis area 106 Roof frame 108 Peripheral frame 110 Correspondence relationship 112 Representative information 114 Orthoimage 120 Aligned composite image 122 Icon 140 Display screen 142 Map display section 144 Priority section 146 Priority information 160 Bar graph 162 Sub-analysis area 164 Grid 202 Processor 204 Computer-readable medium 206 Communication interface 208 Input / output interface 210 Bus 212 Memory 214 Storage 222 Input device 224 Display device MI Captured image S100 to S108 Damage degree derivation steps S110 to S112 Representative information derivation steps

Claims

1. An information processing device comprising: a processor; and a memory storing a program to be executed by the processor, wherein the processor executes the program stored in the memory to obtain map information representing the geographical position and shape of one or more features included in a survey area; obtain feature meta information for each of the one or more features; obtain aerial images of the survey area; derive a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images; statistically process the degree of damage derived for each of the one or more features for an analysis area included in the survey area; derive representative information representing the analysis area; and display the analysis area in association with the representative information.

2. The information processing device described in claim 1, wherein the processor extracts multiple analysis area images of the analysis area from the aerial image, aligns the positions of the multiple analysis area images and the contour shapes of the features with respect to a map including the analysis area, generates an aligned composite image by combining the multiple analysis area images, and associates the aligned composite image with the representative information and displays it.

3. The information processing device according to claim 2, wherein the processor statistically processes the damage level derived for each of the features for each sub-analysis area into which the analysis area is subdivided, generates representative information for each sub-analysis area, and associates the aligned composite image with the representative information for each sub-analysis area and displays it.

4. The information processing device according to claim 2, wherein the processor derives the degree of damage for each feature, applies a display mode according to the degree of damage, and displays the degree of damage superimposed on the aligned composite image.

5. The information processing device according to claim 1, wherein the processor determines the analysis area using boundary information representing boundaries included in the map information.

6. The information processing device according to claim 1, wherein the processor determines the analysis area that includes a plurality of the features based on the aerial image and the map information.

7. The information processing device according to claim 1, wherein the processor acquires information representing the analysis area designated by a user.

8. The information processing device of claim 1, wherein the processor acquires survey plan information for the analysis area that indicates at least one of the survey priority, the period required for the survey, the number of surveyors, the surveyor's expertise, and the surveyor's proficiency, and displays the survey plan information and the representative information.

9. An information processing method comprising the steps of: acquiring map information representing the geographical position and shape of one or more features included in a survey area; acquiring feature meta information for each of the one or more features; acquiring aerial images of the survey area; deriving a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images; statistically processing the degree of damage derived for each of the one or more features for an analysis area included in the survey area, and deriving representative information representing the analysis area; and displaying the representative information in association with the analysis area.

10. A program that enables a computer that functions as an information processing device to achieve the following functions: acquire map information representing the geographical position and shape of one or more features included in a survey area; acquire feature meta information for each of the one or more features; acquire aerial images of the survey area; derive a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images; statistically process the degree of damage derived for each of the one or more features for an analysis area included in the survey area, and derive representative information that represents the analysis area; and display the representative information in association with the analysis area.

11. A non-transitory computer-readable recording medium on which the program according to claim 10 is recorded.

12. An information processing system comprising an information processing device that can be freely connected to an external device, the information processing device comprising: a processor; and a memory that stores a program to be executed by the processor, wherein the processor executes the program stored in the memory to obtain map information representing the geographical position and shape of one or more features included in a survey area; obtain feature meta information for each of the one or more features; obtain aerial images of the survey area; derive a degree of damage for each of the one or more features based on the map information, the feature meta information, and the aerial images; statistically process the degree of damage derived for each of the one or more features for an analysis area included in the survey area; derive representative information that represents the analysis area; and display the analysis area in association with the representative information.

Images