Information processing device, information processing method, and program

Abstract

Provided are an information processing device, information processing method, and program that achieve selection of an image appropriate for an observed object. The information processing device performs the following: acquiring geographical information of an observed building; acquiring a first individual building image (DAI 11) that includes a first building collated using geographical information and that has been trimmed from an aerial photograph; displaying the first individual building image as an observed building image (ODAI) that includes the observed building; acquiring first candidate images (DAI12, DAI13, DAI21, DAI31) that serve as replacement candidates for the first individual building image when the observed building and the first building included in the first individual building image are confirmed as matching; and displaying the first candidate images.

Description

Information processing device, information processing method, and program

[0001] The present invention relates to an information processing device, an information processing method, and a program.

[0002] When a disaster such as an earthquake occurs, aerial images are used to survey the affected areas to understand the situation. In the survey of the damage situation, buildings such as houses are observed and the degree of damage is identified.

[0003] Patent Document 1 describes a system for classifying features using multiple aerial images. The system identifies building areas and road surface areas from satellite images of the ground. The system classifies building areas into schools, hospitals, etc., and road surface areas into parking lots, roads, etc.

[0004] Patent Document 2 describes an inspection management system that outputs a three-dimensional image of an object to be inspected, based on images of the object taken by a camera mounted on a flying object, showing symptoms of deterioration of the object. The system described in this document generates a three-dimensional model of the object based on the images of the object. The system searches for the location of the object on an online map on the Internet and obtains latitude and longitude information.

[0005] Patent No. 7418281 Patent No. 7046873

[0006] When surveyors conduct observations from the ground, it is difficult to observe the target building from above. To address this issue, a system has been proposed in which aerial images of the target building are separately prepared, and the latitude and longitude information of the target building is referenced to utilize the aerial images for observations from the ground.

[0007] However, the above-described system utilizing aerial images makes it difficult for an investigator to easily confirm whether the subject of the aerial image matches the target building when the investigator observes from the ground. Furthermore, the above-described system utilizing aerial images makes it difficult to switch to an appropriate aerial image when the subject of the aerial image does not match the target building. In other words, the above-described system utilizing aerial images makes it difficult to select an appropriate image for the target building.

[0008] Neither the invention described in Patent Document 1 nor the invention described in Patent Document 2 focuses on the problem of selecting an appropriate image for the building being observed, and does not solve this problem. Note that the above-mentioned problem is not limited to the observation of buildings in disaster damage investigations, and similar problems exist in the inspection of changes over time in buildings, investigations of the asset value of buildings, and completion inspections of new construction or extensions / renovations.

[0009] The present invention has been made in view of the above circumstances, and has as its object to provide an information processing device, an information processing method, and a program that realize the selection of an appropriate image for an object to be observed.

[0010] An information processing device according to a first aspect of the present disclosure includes one or more processors and one or more memories for storing instructions to be executed by the one or more processors, wherein the one or more processors acquire geographical information for a building to be observed, acquire a first individual building image including a first building matched using the geographical information, acquire a first individual building image cropped from an aerial image, display the first individual building image on a display screen as the building to be observed image including the building to be observed, and when confirming the consistency between the building to be observed and the first building included in the first individual building image, acquire a first candidate image that is a candidate for replacing the first individual building image, and display the first candidate image on the display screen.

[0011] According to the information processing device of the first aspect of the present disclosure, a first candidate image that is a replacement candidate for a first individual building image that includes the observed building is displayed, thereby realizing the selection of an appropriate individual building image for the observed object.

[0012] The individual building image includes one individual building that is a target, and may also include parts of individual buildings that are adjacent to the target individual building.

[0013] In the information processing device of the second aspect, in the information processing device of the first aspect, the first candidate image may include at least one of a second individual building image that includes a first building and is photographed in a different direction from the first individual building image, and a third individual building image that includes a second building that is different from the first building.

[0014] According to this aspect, a first candidate image to replace the first individual building image can be selected from at least one of a second individual building image that is photographed in a different direction from the first individual building image, and a third individual building image that includes a second building that is different from the first building.

[0015] The second building, which is different from the first building, may be an individual building within a specified distance from the first building.

[0016] An information processing device according to a third aspect is an information processing device according to the first or second aspect, wherein one or more processors acquire a first individual building image from an individual building image storage device in which one or more individual building images, each of which is an individual building image cropped from an aerial image, and to which geographical information of the first building is associated, are stored.

[0017] In this aspect, the information processing device may include an individual building image storage device.

[0018] An information processing device according to a fourth aspect is the information processing device of the third aspect, wherein the one or more processors may acquire the first candidate image from an individual building image storage device.

[0019] In this aspect, the information processing device may include an individual building image storage device.

[0020] In an information processing device according to a fifth aspect, in an information processing device according to any one of the first to fourth aspects, one or more processors may accept an operator's operation representing a replacement of the building image to be observed, and replace the first individual building image with a first candidate image in accordance with the operator's operation.

[0021] According to this aspect, the operator can manually replace the first individual building image with the first candidate image.

[0022] An information processing device according to a sixth aspect is the information processing device of the fifth aspect, wherein the one or more processors may store the first candidate image into which the first individual building image has been replaced as the target building image to be observed.

[0023] According to this aspect, when the observation target building and the observation target building image do not match, an appropriate observation target building image is stored.

[0024] An information processing device according to a seventh aspect is the information processing device according to any one of the first to sixth aspects, wherein the one or more processors may rotate an image of the building to be observed on the display screen.

[0025] According to this aspect, the visibility of the image of the building being observed by the operator can be improved.

[0026] In an information processing device according to an eighth aspect, in an information processing device according to any one of the first to seventh aspects, one or more processors may acquire observation direction information indicating the observation direction of the building to be observed, and display the observation direction information on a display screen.

[0027] According to this aspect, the operator can view the image of the building to be observed that is oriented in the same direction as the operator's own observation direction.

[0028] This reduces the burden on the operator in observing, and can relatively improve the accuracy of matching between the building to be observed and the first individual building image.

[0029] In an information processing device of a ninth aspect, in the information processing device of the eighth aspect, one or more processors may display a first candidate image to which a shooting direction corresponding to the observation direction has been applied, and may not display a first candidate image to which a shooting direction not corresponding to the observation direction has been applied.

[0030] According to this aspect, it is possible to narrow down the number of first candidate images to be displayed on the display screen.

[0031] In an information processing device according to a tenth aspect, in an information processing device according to any one of the first to ninth aspects, one or more processors may acquire a fourth individual building image including a first building and a fourth building adjacent to the first building, and display the fourth individual building image on a display screen as an image of the building to be observed.

[0032] According to this aspect, when checking the consistency between the building to be observed and the first individual building image, it is possible to use information about the fourth building adjacent to the first building.

[0033] In the information processing device of the 11th aspect, in the information processing device of the 10th aspect, one or more processors may acquire a second candidate image including a fifth individual building image that is photographed in a different direction from the fourth individual building image, and display the second candidate image on the display screen.

[0034] According to this aspect, it is possible to use information about adjacent buildings for the second candidate image as well.

[0035] In an information processing device according to the 12th aspect, in an information processing device according to any one of the first to 11th aspects, one or more processors may acquire shooting date information indicating the shooting date of the first individual building image and display the shooting date information on the display screen.

[0036] According to this aspect, when checking the consistency between the building to be observed and the first individual building image, it is possible to use the date on which the first individual building image was taken.

[0037] The shooting date information may be shooting date and time information that includes the shooting time on the shooting date.

[0038] An information processing device according to a thirteenth aspect is an information processing device according to the twelfth aspect, in which one or more processors acquire a plurality of first individual building images corresponding to each of a plurality of shooting dates, accept input specifying a shooting date from among the plurality of shooting dates, and display the first individual building image corresponding to the specified shooting date on the display screen.

[0039] According to this aspect, it is possible to select a first individual building image taken on a desired date from among a plurality of dates.

[0040] In the information processing device of the 14th aspect, in the information processing device of the 13th aspect, one or more processors may acquire multiple third candidate images corresponding to each of multiple shooting dates, and display the third candidate image corresponding to the identified shooting date on the display screen.

[0041] According to this aspect, it is possible to narrow down the candidate images to be displayed on the display screen based on the shooting date.

[0042] An information processing method according to a fifteenth aspect of the present disclosure is an information processing method in which a computer functioning as an information processing device executes the following steps: acquiring geographical information for a building to be observed; acquiring a first individual building image cropped from an aerial image, the first individual building image including a first building matched using the geographical information; displaying the first individual building image on a display screen as the building to be observed image including the building to be observed; acquiring a first candidate image that is a candidate for replacing the first individual building image when confirming the consistency between the building to be observed and the first building included in the first individual building image; and displaying the first candidate image on the display screen.

[0043] According to the information processing method according to the fifteenth aspect of the present disclosure, it is possible to obtain the same effects as those of the information processing device according to the first aspect of the present disclosure.

[0044] In the information processing method according to the fifteenth aspect, it is possible to appropriately combine the same items as those specified in the second to fourteenth aspects. In this case, the components that perform the processing or function specified in the information processing device can be understood as components of the information processing method that perform the corresponding processing or function.

[0045] An information processing method according to a sixteenth aspect may be the information processing method of the fifteenth aspect, wherein each step executed by the computer includes a step executed based on an operation by an operator of the computer.

[0046] A program relating to a 17th aspect of the present disclosure is a program that enables a computer functioning as an information processing device to realize the following functions: acquire geographical information for a building to be observed; acquire a first individual building image that is a first individual building image that includes a first building matched using the geographical information and that is cropped from an aerial image; display the first individual building image on a display screen as the building to be observed that includes the building to be observed; acquire a first candidate image that is a candidate for replacing the first individual building image when checking the consistency between the building to be observed and the first building included in the first individual building image; and display the first candidate image on a display screen.

[0047] According to the program according to the seventeenth aspect of the present disclosure, it is possible to obtain the same effects as those of the information processing device according to the first aspect of the present disclosure.

[0048] In the program according to the seventeenth aspect, it is possible to appropriately combine the same items as those specified in the second to fourteenth aspects. In this case, the components that perform the processes or functions specified in the information processing device can be understood as components of the program that perform the corresponding processes or functions.

[0049] An information processing device according to an 18th aspect of the present disclosure is an information processing device that includes one or more processors and one or more memories that store instructions to be executed by the one or more processors, wherein the one or more processors acquire geographical information for a building to be observed, acquire an aerial image including a first building that has been matched using the geographical information, display the aerial image on a display screen, and, when confirming the consistency between the building to be observed and the first building included in the aerial image, acquire a first candidate image that is a candidate for replacing the aerial image, and display the first candidate image on the display screen.

[0050] According to the information processing device according to the eighteenth aspect of the present disclosure, a first candidate image is displayed as a replacement candidate for an aerial image including the observed building, thereby realizing the selection of an appropriate aerial image for the observed object.

[0051] In the information processing device according to the eighteenth aspect, it is possible to appropriately combine the same items as those specified in the second to fourteenth aspects. In this case, the components that perform the processes or functions specified in the information processing device can be understood as the components of the information processing device that perform the corresponding processes or functions.

[0052] According to the present invention, a first candidate image that is a candidate for replacing a first individual building image that includes the building to be observed is displayed, thereby realizing the selection of an appropriate image for the object to be observed.

[0053] FIG. 1 is a schematic diagram showing an overview of a survey support system to which an investigator terminal device according to an embodiment is applied. FIG. 2 is a flowchart showing the steps of a survey support method applied to the survey support system shown in FIG. 1. FIG. 3 is a block diagram showing the electrical configuration of the investigator terminal device shown in FIG. 1. FIG. 4 is a block diagram showing the hardware configuration of the electrical configuration of the investigator terminal device shown in FIG. 3. FIG. 5 is a schematic diagram of a display screen according to a first example. FIG. 6 is a schematic diagram of a display screen according to a second example. FIG. 7 is a schematic diagram of a display screen according to a third example. FIG. 8 is a schematic diagram of a display screen according to a fourth example. FIG. 9 is a schematic diagram of a display screen according to a fifth example. FIG. 10 is a schematic diagram of a display screen according to a sixth example. FIG. 11 is a schematic diagram of a display screen according to a seventh example. FIG. 12 is an overall configuration diagram of a survey system to which a survey support system according to an embodiment is applied.

[0054] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification, identical components are designated by the same reference numerals, and duplicate descriptions will be omitted where appropriate. Furthermore, when multiple components are listed in the following embodiments, it can be interpreted as including at least one of the multiple components.

[0055] [Outline of a survey support system to which an information processing device is applied] Fig. 1 is a schematic diagram showing an outline of a survey support system to which an inspector terminal device according to an embodiment is applied. The survey support system 10 is a system that supports observation of buildings from the ground by an inspector.

[0056] The survey support system 10 uses individual building images, which include an image of one individual building cropped from an aerial image containing multiple buildings. The individual building images are associated with latitude and longitude information of the individual building. The latitude and longitude information may be geographical information of the individual building. The geographical information may be an address or coordinate values ​​in a two-dimensional coordinate system applied to a map.

[0057] The aerial images may be captured from a manned air vehicle such as an airplane or a helicopter, or may be captured from an unmanned air vehicle such as a drone. The aerial images may be satellite images captured from an artificial satellite.

[0058] FIG. 1 illustrates an individual building image DAI1 and an individual building image DAI2 associated with latitude and longitude information expressed as coordinate values ​​in a two-dimensional coordinate system applied to an aerial photograph image.

[0059] 1, an arbitrary character string is illustrated as the coordinate value assigned to the individual building image DAI1. The individual building image DAI1 and the individual building image DAI2 are stored in the image database 20. Hereinafter, when there is no need to distinguish between the individual building images DAI1, etc., they will be referred to as individual building images DAI.

[0060] The generation of the individual building images DAI associated with the latitude and longitude information and the transmission to the image database 20 may be performed using an image processing device to which a computer having an image processing function is applied.

[0061] A plurality of individual building images DAI that are cropped from a plurality of aerial photograph images are stored in the image database 20. The individual building images DAI stored in the image database 20 can be searched using latitude and longitude information.

[0062] The image database 20 may store multiple individual building images DAI for one building. In other words, the image database 20 may store multiple individual building images DAI for one piece of latitude and longitude information. The multiple individual building images DAI for one piece of latitude and longitude information may include individual building images DAI that include the same building but taken from different directions. The image database 20 is an example of an individual building image storage device of the present disclosure.

[0063] The inspector of the building being observed carries an inspector terminal device 30 on which an inspection support application is installed. The inspector terminal device 30 is a computer equipped with one or more processors and one or more memories in which a program containing one or more instructions executed by the one or more processors is stored. In FIG. 1, a tablet computer is shown as the inspector terminal device 30. The inspector terminal device 30 may also be a notebook computer, a smartphone, or the like.

[0064] The inspector terminal device 30 is equipped with a touch panel display 32. The display 32 displays various information when the inspection support application is executed. Figure 1 shows a display screen 34 on which the observation target building image ODAI is displayed.

[0065] The inspector may input latitude and longitude information of the building to be observed into the inspector terminal device 30. The inspector terminal device 30 acquires one or more individual building images DAI extracted using the input latitude and longitude information from among the multiple individual building images DAI stored in the image database 20. The inspector terminal device 30 applies predefined selection conditions to select one building image ODAI to be observed from among the one or more individual building images DAI. The display 32 displays the selected building image ODAI to be observed.

[0066] Furthermore, when checking the consistency between the observed building and the observed building image ODAI, the display screen 34 displays a candidate image CI that is a replacement candidate for the observed building image ODAI. Details of the candidate image CI will be described later. The inspector terminal device 30 is an example of an information processing device disclosed herein.

[0067] [Procedure of the Survey Support Method] Figure 2 is a flowchart showing the procedure of the survey support method applied to the survey support system shown in Figure 1. Each step shown in Figure 2 is executed by the investigator terminal device 30 shown in Figure 1.

[0068] The latitude and longitude information acquisition process S10 acquires the latitude and longitude information of the building to be observed input by the inspector. The latitude and longitude information may be acquired by inputting the building to be observed from pre-stored information indicating the correspondence between the building to be observed and latitude and longitude information. An inspector terminal device 30 with a GPS function may acquire GPS information of the inspector terminal device 30. Note that GPS is an abbreviation for Global Positioning System.

[0069] In the individual building image acquisition step S12, one or more individual building images DAI corresponding to the latitude and longitude information acquired in the latitude and longitude information acquisition step S10 are acquired from the image database 20.

[0070] In the individual building image acquisition step S12, individual building images DAI including buildings in the vicinity of the observed building, such as buildings adjacent to the observed building, are acquired from the image database 20. For example, the latitude and longitude of the vicinity of the observed building can be identified by defining a specified range from the latitude and longitude in the latitude and longitude information acquired in the latitude and longitude information acquisition step S10 as the vicinity of the observed building.

[0071] In the observation target building image selection step S14, one individual building image DAI to be the observation target building image ODAI is selected from the one or more individual building images DAI acquired in the individual building image acquisition step S12.

[0072] In the candidate image selection step S16, one or more candidate images are selected from the one or more individual building images DAI acquired in the individual building image acquisition step S12. In the candidate image selection step S16, an individual building image DAI that has not been selected as the observation target building image ODAI in the observation target building image selection step S14 may be selected as the candidate image CI.

[0073] In the candidate image selection step S16, individual building images of buildings in the vicinity of the building to be observed may be selected as candidate images CI. The building image ODAI to be observed and the candidate images CI are displayed on the display screen 34.

[0074] In the user input reception determination step S18, it is determined whether a user input to replace the observed building image ODAI with a candidate image CI has been received. For example, in the user input reception determination step S18, it may be determined whether a button that is displayed on the display screen 34 and that is operated by the user has been operated. Note that the user here is an inspector. The user is also an example of an operator in the present disclosure.

[0075] In the user input reception determination step S18, if it is determined that the user input has not been received, the determination is No. In the case of a No determination, the process proceeds to the termination determination step S24. On the other hand, in the user input reception determination step S18, if it is determined that the user input has been received, the determination is Yes. In the case of a Yes determination, the process proceeds to the replacement candidate selection step S20.

[0076] In the replacement candidate selection step S20, a candidate image CI is selected from one or more candidate images to replace the individual building image DAI selected as the target building image ODAI. The candidate image may be selected automatically or manually by the user.

[0077] In the observation target building image replacement step S22, the candidate image CI selected in the replacement candidate selection step S20 is replaced as the observation target building image ODAI and displayed on the display screen 34. After the observation target building image replacement step S22, the process proceeds to the user input reception determination step S18, where the user input reception determination step S18 and the replacement candidate selection step S20 are executed, and the candidate image CI may be reselected.

[0078] In the termination determination step S24, a determination is made as to whether the survey support method has been terminated. If the survey support method continues to be executed, the determination is No in the termination determination step S24. If the determination is No, the process proceeds to the latitude and longitude information acquisition step S10, and each step from the latitude and longitude information acquisition step S10 to the termination determination step S24 is repeatedly executed until the determination is Yes in the termination determination step S24.

[0079] On the other hand, if the investigation support method is to be terminated, the result is a Yes judgment. If the result is a Yes judgment, a specified termination process is executed and the investigation support method is terminated. Note that each of the above processes is an example of a step constituting the information processing method of the present disclosure. Furthermore, the functions realized by the steps constituting the information processing method of the present disclosure are an example of various functions of the present disclosure.

[0080] [Configuration example of inspector terminal device] Figure 3 is a block diagram showing the electrical configuration of the inspector terminal device shown in Figure 1. The inspector terminal device 30 includes a latitude and longitude information acquisition unit 36. The latitude and longitude information acquisition unit 36 ​​acquires latitude and longitude information of the building to be observed.

[0081] The inspector terminal device 30 includes an image acquisition unit 38. The image acquisition unit 38 searches the image database 20 using the latitude and longitude information and acquires individual building images DAI corresponding to the latitude and longitude information. The image acquisition unit 38 also acquires individual building images DAI of buildings included in the area surrounding the target building of observation, which is defined based on the latitude and longitude information.

[0082] The inspector terminal device 30 includes an image storage unit 40. The image storage unit 40 stores the individual building images DAI acquired via the image acquisition unit 38. The image storage unit 40 may be configured as an external device of the inspector terminal device 30.

[0083] The inspector terminal device 30 includes a selection unit 42. The selection unit 42 selects an observation target building image ODAI from the individual building images DAI stored in the image storage unit 40. The selection unit 42 selects a candidate image CI from the individual building images DAI stored in the image storage unit 40.

[0084] The inspector terminal device 30 includes a display screen generation unit 44. The display screen generation unit 44 generates a display screen 34 that displays the target building image ODAI and candidate image CI selected by the selection unit 42, and causes the display 32 to display the display screen 34.

[0085] The inspector terminal device 30 includes a user input receiving unit 46. The user input receiving unit 46 receives user input of latitude and longitude information, and user input when replacing the observation target building image ODAI with a candidate image CI.

[0086] When the user input accepting unit 46 accepts the latitude and longitude information, it transmits the latitude and longitude information to the image acquiring unit 38. When the user input accepting unit 46 accepts a signal representing replacement of the observation target building image ODAI with the candidate image CI, it transmits the signal to the selecting unit 42.

[0087] Figure 4 is a block diagram showing the hardware configuration of the electrical configuration of the investigator terminal device shown in Figure 3. The investigator terminal device 30 includes a processor 100, a computer-readable medium 102, a communication interface 104, an input / output interface 106, and a bus 108.

[0088] The processor 100 includes a CPU. The processor 100 may also include a GPU. Note that CPU is an abbreviation for Central Processing Unit, and GPU is an abbreviation for Graphics Processing Unit.

[0089] The processor 100 is connected to a computer-readable medium 102, a communication interface 104, and an input / output interface 106 via a bus 108. The input / output interface 106 is connected to a display 32 and an input device 33.

[0090] The input device 33 may include a keyboard, a mouse, etc. In FIG.

[0091] The inspector terminal device 30 realizes various functions by having the processor 100 execute a program stored in the computer-readable medium 102. Note that the term "program" is synonymous with the term "software."

[0092] The computer-readable medium 102 includes a memory serving as a main storage device and a storage serving as an auxiliary storage device. The memory and storage are not shown. The computer-readable medium 102 may be a semiconductor memory, a hard disk drive, a solid-state drive, or the like. The computer-readable medium 102 may also be any combination of multiple devices.

[0093] A hard disk drive may be referred to as an HDD, which is an abbreviation of the English term Hard Disk Drive, and a solid state drive may be referred to as an SSD, which is an abbreviation of the English term Solid State Drive.

[0094] The investigator terminal device 30 communicates data with external devices via a communication interface 104. The communication interface 104 may comply with various standards, such as USB. The communication format of the communication interface 104 may be either wired or wireless. USB is an abbreviation for Universal Serial Bus and is a registered trademark.

[0095] The computer-readable medium 102 stores various programs executed by the processor 100, such as a latitude and longitude information acquisition program, an individual building image acquisition program, an observation target building image selection program, a candidate image selection program, and a display screen generation program.

[0096] The various programs described above are executed by the various processing units shown in FIG. 3 in the various steps shown in FIG.

[0097] The various programs stored in the computer-readable medium 102 include one or more instructions. The computer-readable medium 102 stores various data, various parameters, etc. that are used when the various programs are executed. The computer-readable medium 102 may function as the image storage unit 40 shown in FIG. 3.

[0098] Examples of the hardware structure of the processor 100 include a CPU, a GPU, a PLD, and an ASIC. The CPU is a general-purpose processor that executes programs and functions as various functional units. The GPU is a processor specialized for image processing.

[0099] A PLD is a processor whose electrical circuitry can be reconfigured after the device is manufactured. An example of a PLD is an FPGA. An ASIC is a processor that includes dedicated electrical circuits specifically designed to perform a specific task.

[0100] It should be noted that PLD is an abbreviation for Programmable Logic Device, ASIC is an abbreviation for Application Specific Integrated Circuit, and FPGA is an abbreviation for Field Programmable Gate Array.

[0101] A processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types. Examples of combinations of various processors include a combination of one or more FPGAs and one or more CPUs, and a combination of one or more FPGAs and one or more GPUs. Another example of a combination of various processors is a combination of one or more CPUs and one or more GPUs.

[0102] A single processor may be used to configure multiple functional units. An example of using a single processor to configure multiple functional units is a configuration in which a single processor is configured by applying a combination of one or more CPUs and software, such as an SoC, as typified by a computer such as a client or server, and this processor operates as multiple functional units.

[0103] Another example of using one processor to configure multiple functional units is to use a processor that uses one IC chip to realize the functions of an entire system including multiple functional units. Note that SoC is an abbreviation for System On a Chip, and IC is an abbreviation for Integrated Circuit.

[0104] In this way, the various processing units are configured using one or more of the various processors described above as a hardware structure.More specifically, the hardware structure of the various processors described above is an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0105] 5 is a schematic diagram of a display screen according to Example 1. On the display screen 34, an individual building image DAI11, an individual building image DAI12, and an individual building image DAI13 of the house A are displayed.

[0106] The individual building image DAI11, the individual building image DAI12, and the individual building image DAI13 are photographed from different directions. Angle a indicates the photographing direction of the individual building image DAI11, angle b indicates the photographing direction of the individual building image DAI12, and angle c indicates the photographing direction of the individual building image DAI13.

[0107] On the display screen 34, the individual building image DAI11 selected as the observation target building image ODAI is displayed enlarged compared to the other individual building images DAI12, etc. The individual building image DAI11 selected as the observation target building image ODAI is placed in the center portion 34A of the display screen 34. The center portion 34A is defined as an area that includes the center position of the display screen 34.

[0108] The display screen 34 displays an individual building image DAI21 of a house B that is in the neighborhood of the house A, and an individual building image DAI31 of a house C that is in the neighborhood of the house A. The individual building image DAI21 and the individual building image DAI31 are arranged as candidate images CI in a peripheral portion 34B that is defined as the area below a central portion 34A on the display screen 34.

[0109] A selection mark SM is displayed for each of the individual building images DAI11, etc. A display mode different from that of the selection mark SMA of the individual building image DAI11 selected as the observation target building image ODAI is applied to the selection mark SM of the non-selected individual building images DAI12, etc. The selection mark SM may be configured as an operation button that is operated by the user when one of the individual building images DAI11, etc. is selected.

[0110] Figure 5 illustrates an example in which an individual building image DAI11 selected as the target building image ODAI is also displayed as a candidate image CI, but the individual building image DAI11 selected as the target building image ODAI may not be displayed as a candidate image CI.

[0111] The house A is an example of a first building of the present disclosure, and the individual building image DAI11 is an example of a first individual building image of the present disclosure. The individual building image DAI12, the individual building image DAI13, the individual building image DAI21, and the individual building image DAI31 are examples of one or more first candidate images that are candidates for replacing the first individual building image of the present disclosure.

[0112] The individual building image DAI12 and the individual building image DAI13 are examples of the second individual building image of the present disclosure. The house B and the house C are examples of the second building of the present disclosure, and the individual building image DAI21 and the individual building image DAI31 are examples of the third individual building image of the present disclosure.

[0113] [Effects of the information processing device and survey support method according to the embodiment] [1] The inspector terminal device 30 uses the latitude and longitude information of the observed building to acquire multiple individual building images DAI corresponding to the latitude and longitude information from the image database 20. The observed building image ODAI based on the latitude and longitude information of the observed building is identified from the multiple individual building images DAI and displayed on the display 32 of the inspector terminal device 30.

[0114] Furthermore, one or more candidate images CI that are candidates for replacing the observed building image ODAI are identified from the multiple individual building images DAI and displayed on the display 32 of the inspector terminal device 30 .

[0115] This allows an investigator looking at the display 32 displaying the target building image ODAI to replace the target building image ODAI with an individual building image DAI selected from one or more candidate images CI when comparing the target building image ODAI with the target building.

[0116] [2] The candidate images CI include individual building images DAI of the same building as the target building image ODAI, photographed at a different shooting angle. This allows the target building image ODAI to be replaced with an individual building image DAI photographed at a different shooting angle, and the surveyor can compare the individual building images DAI photographed at a different shooting angle with the target building.

[0117] [3] The candidate images CI include individual building images DAI of buildings that are different from the target building image ODAI. This allows the inspector to replace the target building image ODAI with the correct individual building image DAI when the target building image ODAI for the target building is incorrect, allowing the inspector to observe the correct target building.

[0118] [4] When the observation target building image ODAI is replaced with an individual building image DAI selected from the candidate images CI, the replaced individual building image DAI is stored as the observation target building image ODAI. As a result, the individual building image DAI that was actually used as the observation target building image ODAI when observing the observation target building is stored.

[0119] [5] An individual building image DAI selected as an observation target building image ODAI is applied with a selection mark SMA indicating that it has been selected, and the selection mark SMA is displayed in a manner different from that of a non-selected selection mark SM. This allows the surveyor to visually recognize that the individual building image DAI has been selected as the observation target building image ODAI.

[0120] In the present embodiment, an example is shown in which an individual building image DAI is acquired by cropping from an aerial image, but a first aerial image including the building to be observed may also be acquired. The first aerial image may be displayed as the building to be observed image ODAI on the display 32 of the inspector terminal device 30. A candidate image CI to replace the first aerial image may be a second aerial image taken in a different direction from the first aerial image, or a third aerial image including a different building from the first aerial image.

[0121] [Display Screen Variations] [Second Example] Fig. 6 is a schematic diagram of a display screen according to the second example. A central portion 342A of a display screen 342 according to the second example displays an individual building image DAI11A obtained by rotating the individual building image DAI11 selected as the target building image ODAI to be observed. As with the display screen 34 shown in Fig. 5, a peripheral portion 342B of the display screen 342 displays an individual building image DAI12, which is a candidate image CI, and the like.

[0122] A rotation button B1, which is operated when rotating the individual building image DAI11A, is displayed on the display screen 342. A selection mark SMA, which indicates that the rotation button B1 is being operated, is added to the rotation button B1.

[0123] Figure 6 shows an individual building image DAI11A that has been rotated 90 degrees clockwise relative to the individual building image DAI11 to which the standard display shown in Figure 5 is applied, but the direction and angle of rotation of the individual building image DAI11A can be specified as appropriate.

[0124] The display screen 342 according to the second example makes it possible to display the target building image ODAI in an orientation that is easy for the inspector to view. In addition, the inspector can rotate the target building image ODAI in any orientation.

[0125] 7 is a schematic diagram of a display screen according to Example 3. A display screen 343 according to Example 3 displays observation direction information ODI, which indicates the observation direction from which an investigator using the investigator terminal device 30 observes the building being observed.

[0126] 7 illustrates an arrow line representing the observation direction as the observation direction information ODI. The same figure also illustrates a mode in which the observation direction information ODI is superimposed on the individual building image DAI11B selected as the observation target building image ODAI.

[0127] An observation direction display button B2, which is operated when displaying the investigator's observation direction, is displayed on the display screen 343. When the investigator operates the observation direction display button B2, observation direction information ODI is displayed on the display screen 343.

[0128] The observation direction information ODI may be automatically hidden when a specified period of time has elapsed since the start of display. The observation direction information ODI may be hidden in accordance with a user input. For example, the observation direction information ODI may be displayed when the observation direction display button B2 is operated an odd number of times, and may be hidden when the observation direction display button B2 is operated an even number of times.

[0129] The investigator terminal device 30 includes an observation direction information acquisition unit that acquires observation direction information ODI that indicates the investigator's observation direction. The observation direction information acquisition unit may acquire observation direction information ODI entered by the investigator as user input. Note that reference numeral 343A denotes the center of the display screen 343. Reference numeral 343B denotes the periphery of the display screen 343.

[0130] The observation direction information acquisition unit is not limited to acquiring the observation direction information ODI based on user input, but may also automatically acquire the observation direction information ODI from a direction sensor or the like provided in the inspector terminal device 30. Furthermore, the observation direction information acquisition unit may acquire the observation direction information ODI calculated using an image of the observed building taken by an inspector at the site of the observed building.

[0131] According to the display screen 343 of the third example, an investigator looking at the target building image ODAI can understand the direction in which the target building is being viewed, and can be assisted in selecting an individual building image DAI that is suitable for the target building image ODAI.

[0132] Furthermore, when the researcher uses the observation direction information ODI obtained at the site of the building being observed to rotate the image ODAI of the building being observed on the display screen 343 to match the direction from which the researcher is observing the building being observed, the relationship between the front, back, left and right of the building being observed as seen by the researcher on the display screen 343 is maintained, thereby enabling the researcher to intuitively grasp the building being observed on the display screen 343.

[0133] 8 is a schematic diagram of a display screen according to Example 4. On a display screen 344 according to Example 4, individual building images DAI whose capture direction is the same as or close to the observation direction of the inspector are selectively displayed, and individual building images DAI that do not correspond to the observation direction of the inspector are not displayed.

[0134] 8 illustrates a case where the inspector's observation direction is the same as the photographing direction of the individual building image DAI11 illustrated as angle a, and is close to the photographing direction of the individual building image DAI12 illustrated as angle b. Although the photographing direction is not shown, the photographing direction of the individual building image DAI31 is the same as the photographing direction of the individual building image DAI11. Note that reference numeral 344A denotes the center of the display screen 344. Reference numeral 344B denotes the periphery of the display screen 344.

[0135] A shooting angle designation button B3 is displayed on the display screen 344. When the inspector operates the shooting angle designation button B3, the display screen 343 displays the individual building image DAI12 etc. having a specified shooting direction according to the inspector's observation direction, and the individual building image DAI21 etc. having a shooting direction different from the specified shooting direction are not displayed.

[0136] The display screen 344 according to the fourth example narrows down the candidate images CI to replace the observation target building image ODAI, and can assist in the selection of an individual building image DAI suitable for the observation target building image ODAI.

[0137] 9 is a schematic diagram of a display screen according to Example 5. A display screen 345 according to Example 5 uses an individual building image DAI14 that has been roughly cropped from an aerial image, etc. The individual building image DAI14, which is an individual building image DAI that has been roughly cropped from an aerial image, includes house A, which is an individual building, and adjacent house D.

[0138] The same applies to individual building images DAI15 and DAI16, which are cropped from the aerial image with house A as an individual building. When the individual building to be cropped is adjacent to two or more buildings, the individual building image DAI14 etc. may include multiple adjacent buildings. Note that house D adjacent to house A may be house B adjacent to house A, or may be house C adjacent to house A.

[0139] The individual building image DAI22 ​​in which house B is considered to be an individual building includes at least a portion of house E adjacent to house B. The individual building image DAI32 in which house C is considered to be an individual building includes at least a portion of house F adjacent to house C.

[0140] An enlarged individual building image display button B4 that the surveyor operates when displaying an enlarged individual building image DAI14 or the like instead of the standard individual building image DAI is displayed on the display screen 345. When the enlarged individual building image display button B4 is operated, the display screen 34 shown in Fig. 5 is switched to the display screen 345 shown in Fig. 9.

[0141] 3 may acquire enlarged individual building images DAI14 and the like corresponding to each individual building image DAI when acquiring the individual building images DAI. The image database 20 may store pairs of individual building images DAI and enlarged individual building images DAI14 and the like. Note that reference numeral 345A denotes the center of the display screen 345. Reference numeral 345B denotes the periphery of the display screen 345.

[0142] According to the display screen 345 of the fifth example, the inspector can grasp the buildings adjacent to the observed building image ODAI. This makes it possible to match the observed building image ODAI with the surrounding area of ​​the observed building.

[0143] 9 is an example of a fourth building of the present disclosure, and the individual building image DAI14 is an example of a fourth individual building image of the present disclosure. The individual building image DAI15 and the individual building image DAI16 are examples of a fifth individual building image of the present disclosure, and are examples of second candidate images including the fifth individual building image.

[0144] 10 is a schematic diagram of a display screen according to Example 6. On a display screen 346 according to Example 6, an individual building image DAI11 selected as the target building image ODAI is displayed with shooting date and time information DI indicating the shooting date and time.

[0145] FIG. 10 illustrates an example of the shooting date and time information DI indicating the shooting date of the individual building image DAI11. The shooting date and time information DI may include information on the shooting time. The shooting time may include information on the hour and minute, or may include information on the second. The shooting date and time information DI illustrated in FIG. 10 represents any year, month, and date. The same applies to the shooting date and time information DI illustrated in FIG. 11. Note that reference numeral 346A represents the center of the display screen 346. Reference numeral 346B represents the periphery of the display screen 346.

[0146] A photography date and time information display button B5 is displayed on the display screen 346. When the inspector operates the photography date and time information display button B5, the photography date and time information DI of the individual building image DAI11 selected as the observation target building image ODAI is displayed on the display screen 346.

[0147] According to the display screen 346 of the sixth example, the inspector can grasp the individual building image DAI11 selected as the target building image ODAI. Note that the shooting date and time information DI shown in FIG. 10 is an example of the shooting date information of the present disclosure.

[0148] [Seventh Example] Fig. 11 is a schematic diagram of a display screen according to the seventh example. In a central portion 347A of a display screen 347 according to the seventh example, an individual building image DAI captured on a specified date is selectively displayed. In a peripheral portion 347B of the display screen 347, an individual building image DAI13A and an individual building image DAI21A that have the same capture date as the individual building image DAI11 selected as the target building image ODAI are selectively displayed. On the other hand, the individual building image DAI12 and individual building image DAI31 shown in Fig. 5 are not displayed because their capture dates differ from those of the individual building image DAI11.

[0149] A drop-down menu DM, which is displayed when selecting a shooting date and time, is displayed on the display screen 347. When a tab displaying the shooting date and time is clicked, the drop-down menu DM displays a menu showing selectable shooting dates and times.

[0150] The investigator can select a desired photographing date and time from among the photographing dates and times displayed in the drop-down menu DM, thereby narrowing down the candidate images to be displayed on the display screen 347 .

[0151] The display screen 347 according to the seventh example narrows down the candidate images CI to replace the target building image ODAI, and can assist in the selection of an individual building image DAI that is suitable for the target building image ODAI. Note that the individual building image DAI13A and the individual building image DAI21A shown in FIG. 11 are examples of third candidate images in the present disclosure.

[0152] The display screens 34 and the like shown in Figures 5 to 11 can be combined as appropriate. For example, the various buttons shown in Figures 6 to 11 can be combined as appropriate.

[0153] [Configuration example of an investigation system to which the investigation support system is applied] Figure 12 is a diagram showing the overall configuration of an investigation system to which the investigation support system according to the embodiment is applied. The investigation system 200 shown in the figure includes an imaging drone 202, a camera 204, a controller 206, a terminal device 208, and a server 210. The imaging drone 202, the camera 204, the terminal device 208, and the server 210 can each perform data communication with each other via a network 212.

[0154] In Figure 12, the arrow representing data communication between the photography drone 202 and the network 212 and the arrow representing data communication between the camera 204 and the network 212 are illustrated using a single arrow.

[0155] The filming drone 202 and the camera 204 may individually perform data communication with the network 212, or the data communication between the filming drone 202 and the network 212 and the data communication between the camera 204 and the network 212 may be shared.

[0156] The photography drone 202 is an unmanned aerial vehicle that is remotely controlled using a controller 206. The photography drone 202 may have an autopilot function that flies according to a specified program.

[0157] The camera 204 is supported by a gimbal 214 at the bottom of the body 216 of the filming drone 202 and is mounted on the filming drone 202. The bottom of the body 216 of the filming drone 202 refers to the side opposite the rotor 218 when the position where the rotor 218 is attached is considered to be the top of the body 216 of the filming drone 202.

[0158] The gimbal 214 rotatably supports the camera 204. The gimbal 214 may support the camera 204 rotatably in two directions, with each of two mutually orthogonal directions being a rotation axis direction.

[0159] The camera 204 includes an optical system and an image sensor. The optical system includes one or more lenses, such as a focus lens. The image sensor may be a CCD image sensor or a CMOS image sensor.

[0160] The optical system and image sensor are not shown in Fig. 12. Note that CCD is an abbreviation for Charge Coupled Device, and CMOS is an abbreviation for Complementary Metal-Oxide Semiconductor.

[0161] The camera 204 forms an image of the subject light received from the imaging range FA on the imaging surface of the imaging element. The imaging element outputs an electrical signal representing the received subject light. The camera 204 generates an image signal of the subject from the electrical signal representing the subject light using a signal processing circuit, and outputs the image signal of the subject. The camera 204 may send an image signal representing a through image to the controller 206. The display 207 of the controller 206 may display the through image.

[0162] The camera 204 photographs the investigation area SA from above the investigation area SA. The camera 204 changes the photographing range FA in accordance with the movement of the photographing drone 202, and photographs the entire investigation area SA.

[0163] Camera 204 captures an image at least once for each imaging range FA and generates one or more still images for each imaging range FA. Camera 204 can capture images of multiple houses in a single capture. This results in multiple still images of each house captured from multiple different imaging directions. A change in the imaging range of camera 204 is understood as a change in the imaging position for each house.

[0164] The controller 206 transmits wireless control signals to the filming drone 202 to control the operation of the filming drone 202. That is, in response to the pilot's operation of the controller 206, a control signal indicating the rotation direction and rotation speed of the rotor 218 is transmitted from the controller 206 to the filming drone 202. Based on the control signals transmitted from the controller 206, the filming drone 202 performs operations such as ascending, descending, turning, moving forward, backward, left, and right, and hovering.

[0165] The controller 206 may be in data communication with a terminal device 208 and a server 210 via a network 212 .

[0166] The terminal device 208 functions as the investigator terminal device 30 shown in Fig. 1. The terminal device 208 has the survey support application installed and executes the survey support application.

[0167] The terminal device 208 includes a display 220. The display 220 of the terminal device 208 functions as the display of the investigator terminal device 30 shown in Fig. 1. Note that Fig. 12 shows a smartphone-type terminal device 208.

[0168] The terminal device 208 may have the functions of the controller 206. That is, the terminal device 208 may have a function of wireless communication with the photography drone 202 and a function of executing a control program for the photography drone 202.

[0169] A computer, such as a personal computer or a workstation, is used as the server 210. The server 210 acquires various pieces of information transmitted from the filming drone 202, the camera 204, and the terminal device 208.

[0170] The server 210 may have an image processing function for generating individual building images from aerial images. The image processing function for generating individual building images from aerial images may be provided in an image processing device (not shown). The server 210 may be provided with the image database 20 shown in FIG. 1.

[0171] The network 212 may apply wireless communication using a specified frequency band. The network 212 may apply any wired communication type. The network 212 may combine wireless and wired communication types. The network 212 may apply any communication standard. The network 212 may be electrically connected to computers and the like provided at the survey base or the like so as to be able to communicate freely.

[0172] In this embodiment, an example is given of investigating the damage status of a building when a disaster such as an earthquake occurs, but a configuration similar to this embodiment may also be applied to inspecting changes in a building over time, investigating the asset value of a building, and completing inspections of new construction or renovations.

[0173] The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention.

[0174] 10 Survey support system 20 Image database 20 30 Investigator terminal device 32 Display 33 Input device 34 Display screen 34A Central part 34B Peripheral part 36 Latitude and longitude information acquisition unit 38 Image acquisition unit 40 Image storage unit 42 Selection unit 44 Display screen generation unit 46 User input reception unit 100 Processor 102 Computer readable medium 104 Communication interface 106 Input / output interface 108 Bus 200 Survey system 202 Photography drone 204 Camera 206 Controller 207 Display 208 Terminal device 210 Server 212 Network 214 Gimbal 216 Main body 218 Rotor 220 Display 342 Display screen 342A Central part 342B Peripheral part 343 Display screen 343A Center 343B Peripheral area 344 Display screen 344A Center 344B Peripheral area 345 Display screen 345A Center 345B Peripheral area 346 Display screen 346A Center 346B Peripheral area 347 Display screen 347A Center 347B Peripheral area B1 Rotation button B2 Observation direction display button B3 Shooting angle designation button B4 Enlarged individual building image display button B5 Shooting date and time information display button CI Candidate image DAI Individual building image DAI1 Individual building image DAI2 Individual building image DAI11 Individual building image DAI11A Individual building image DAI11B Individual building image DAI12 Individual building image DAI13 Individual building image DAI13A Individual building image DAI14 Individual building image DAI15 Individual building image DAI16 Individual building image DAI21 Individual building image DAI21A Individual building image DAI22 ​​Individual building image DAI31 Individual building image DAI32 Individual building image DI Shooting date and time information DM Drop-down menu FA Shooting range ODAI Observation target building image ODI Observation direction information SA Investigation range SM Selection mark SMA Selection mark S10 to S24 Each step of the investigation support method

Claims

1. An information processing device comprising: one or more processors; and one or more memories storing instructions to be executed by the one or more processors, wherein the one or more processors: acquire geographical information for an observed building; acquire a first individual building image cropped from an aerial image, the first individual building image including a first building matched using the geographical information; display the first individual building image on a display screen as an observed building image including the observed building; acquire a first candidate image to be a replacement candidate for the first individual building image when confirming the consistency between the observed building and the first building included in the first individual building image; and display the first candidate image on the display screen.

2. The information processing device described in claim 1, wherein the first candidate image includes at least one of a second individual building image that includes the first building and is photographed in a different direction from the first individual building image, and a third individual building image that includes a second building that is different from the first building.

3. The information processing device described in claim 1, wherein the one or more processors acquire the first individual building image from an individual building image storage device that stores one or more individual building images in which one individual building is cropped from an aerial image and to which the geographical information of the first building is associated.

4. The information processing device according to claim 3, wherein the one or more processors acquire the first candidate image from the individual building image storage device.

5. The information processing device described in claim 1, wherein the one or more processors accept an operator's operation representing a replacement of the observed building image, and replace the first individual building image with the first candidate image in accordance with the operator's operation.

6. The information processing device according to claim 5, wherein the one or more processors store the first candidate image in which the first individual building image has been replaced as the target building image.

7. The information processing device according to claim 1, wherein the one or more processors rotate the image of the building being observed on the display screen.

8. The information processing device according to claim 1, wherein the one or more processors acquire observation direction information indicating the observation direction of the building to be observed, and display the observation direction information on the display screen.

9. The information processing device described in claim 8, wherein the one or more processors display the first candidate image to which a shooting direction corresponding to the observation direction has been applied, and do not display the first candidate image to which a shooting direction not corresponding to the observation direction has been applied.

10. The information processing device described in claim 1, wherein the one or more processors acquire a fourth individual building image including the first building and a fourth building adjacent to the first building, and display the fourth individual building image on the display screen as the building image to be observed.

11. The information processing device described in claim 10, wherein the one or more processors acquire a second candidate image including a fifth individual building image photographed in a different direction from the fourth individual building image, and display the second candidate image on the display screen.

12. The information processing device according to claim 1, wherein the one or more processors acquire photographing date information indicating the photographing date of the first individual building image, and display the photographing date information on the display screen.

13. The information processing device described in claim 12, wherein the one or more processors acquire a plurality of the first individual building images corresponding to each of a plurality of shooting dates, accept input specifying a shooting date from among the plurality of shooting dates, and display the first individual building image corresponding to the specified shooting date on the display screen.

14. An information processing device as described in claim 13, wherein the one or more processors acquire a plurality of third candidate images corresponding to each of a plurality of shooting dates, and display the third candidate image corresponding to the identified shooting date on the display screen.

15. An information processing method in which a computer functioning as an information processing device executes the following steps: acquiring geographical information on an observed building; acquiring a first individual building image cropped from an aerial image, the first individual building image including a first building matched using the geographical information; displaying the first individual building image on a display screen as an observed building image including the observed building; acquiring a first candidate image to be a replacement candidate for the first individual building image when confirming the consistency between the observed building and the first building included in the first individual building image; and displaying the first candidate image on the display screen.

16. The information processing method according to claim 15, wherein each step executed by the computer includes a step executed based on an operation by an operator of the computer.

17. A program that enables a computer functioning as an information processing device to realize the following functions: acquiring geographical information on an observed building; acquiring a first individual building image cropped from an aerial image, the first individual building image including a first building matched using the geographical information; displaying the first individual building image on a display screen as an observed building image including the observed building; acquiring a first candidate image that is a candidate for replacing the first individual building image when confirming the consistency between the observed building and the first building included in the first individual building image; and displaying the first candidate image on the display screen.

18. A non-transitory computer-readable recording medium on which the program according to claim 17 is recorded.

19. An information processing device comprising: one or more processors; and one or more memories storing instructions to be executed by the one or more processors, wherein the one or more processors: acquire geographical information on an observed building; acquire an aerial image including a first building matched using the geographical information; display the aerial image on a display screen; acquire a first candidate image to be a replacement for the aerial image when confirming the consistency between the observed building and the first building included in the aerial image; and display the first candidate image on the display screen.

Images