Survey support system, survey support program, and survey support method

Abstract

The objective of this invention is to provide a novel technology that enables accurate investigation of damage to buildings. [Solution] To solve the above problems, the present invention provides a survey support system for assisting in the survey of building damage in disaster-stricken areas, the survey support system comprising a registration unit and a generation unit, the registration unit registering a plurality of survey images of one or more of the buildings in a database, and the generation unit generating a group of orthomosaic images based on the plurality of survey images.

Description

Technical Field

[0001] The present invention relates to an investigation support system, an investigation support program, and an investigation support method.

Background Art

[0002] Conventionally, when a disaster occurs, a housing damage assessment survey is conducted to officially certify the degree of damage to houses caused by the disaster and issue a certificate (disaster damage certificate) used for receiving support funds and applying for occupancy in temporary housing. However, when there are problems with the road restoration status, etc., it has been very difficult to conduct a housing damage assessment survey because many survey teams cannot be sent to the site. An example of a technology for solving such problems is proposed in, for example, Patent Document 1.

[0003] Patent Document 1 discloses that an aerial image is acquired, a completely destroyed house is detected using machine learning, a user to be notified is selected based on the position of the house determined to be completely destroyed, a notification asking about the situation is sent to the selected user, and the damage level is determined using the auxiliary information received from the user as a response to the notification.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, the damage level of a building can be determined using an aerial image. However, in Patent Document 1, since an aerial image is used, the details of the building cannot be imaged, and there is a problem that an accurate investigation of the damage to the building cannot be conducted.

[0006] In view of the above circumstances, the problem to be solved by the present invention is to provide a novel technology that enables accurate investigation of damage to buildings. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides a survey support system for assisting in the survey of damage to buildings in disaster-stricken areas, the survey support system comprising a registration unit and a generation unit, the registration unit registering a plurality of survey images of one or more of the buildings in a database, and the generation unit processing to generate a group of orthomosaic images based on the plurality of survey images.

[0008] By using this configuration, it is possible to generate orthomosaic images that accurately assess the extent of damage to buildings in disaster-stricken areas by using an aircraft to take pictures of the buildings.

[0009] In a preferred embodiment of the present invention, the registration unit registers building location information relating to the location of the photographed building in addition to the plurality of survey images, and the generation unit processes the generation of an orthophoto image generated based on the plurality of photographed survey images, the plurality of photographed survey images, and the building location information to be linked.

[0010] By using this configuration, multiple survey images can be linked to orthomosaic images based on building location information. This allows users to view orthomosaic images that capture images of buildings over a wide area while simultaneously viewing survey images that capture details of individual buildings, making it easy to assess the overall damage situation in the affected area.

[0011] In a preferred embodiment of the present invention, the survey support system further comprises a display processing unit and a determination unit, wherein the display processing unit displays the survey orthomosaic image, receives a designation of building location information, and displays the survey image associated with the building location information, and the determination unit receives a designation of damaged parts of the building shown in the survey image and determines the level of damage to those parts.

[0012] This configuration allows for the selection and display of damaged areas in oblique photographs capturing details of a building, and enables the determination of the level of damage to those areas. This makes it easy for administrators to determine the level of damage to specific areas.

[0013] In a preferred embodiment of the present invention, the determination unit receives the designation of a building component and the damaged area for each building component, and determines the damaged area by inputting the designated building component and the damaged area to a learning model that estimates the level of damage to the area from a building damage image for each building component.

[0014] This configuration allows even inexperienced administrators to easily determine the level of damage to specific parts of a building by using a learning model to assess it. Furthermore, by accepting user-specified damaged areas and inputting these into the learning model, the system can reliably identify the damaged areas and determine their damage levels accurately. Additionally, since the damage level assessment differs for each building element, administrators can pre-specify the components to be assessed before having the learning model determine their damage levels. This reduces the processing load on the learning model and improves assessment accuracy.

[0015] In a preferred embodiment of the present invention, the display processing unit displays a damage area designation unit that allows the designation of the damaged area for each building component, and a damage level display unit that displays the determined damage level for each building component, on the same screen.

[0016] This configuration allows administrators to specify damaged areas while simultaneously checking the assessment results, and to compare the assessment results with their own knowledge at each step. This easily improves the reliability of the assessment results.

[0017] In a preferred embodiment of the present invention, the investigation support system further includes a calculation unit. The calculation unit receives, as the designation of the damaged part, the designation of a damaged area surrounding the damaged part, and calculates a building damage rate based on the investigation image in which the damaged area is designated and the part damage level.

[0018] In a preferred embodiment embodiment of the present invention, the calculation unit receives the designation of the damaged area for each building component, calculates the ratio of the damaged area for each building component based on the investigation image in which the damaged area is designated for each building component, and calculates the building damage rate based on the ratio of the damaged area and the part damage level for each building component.

[0019] With such a configuration, it is possible to calculate the damage rate of a building only by using an investigation image designating the damaged part, and it is possible to easily calculate the building damage rate even in a damage situation where calculation is difficult.

Effect of the Invention

[0020] The present invention can provide a novel technology that enables an accurate investigation of building damage.

Brief Description of the Drawings

[0021] <0000​​​​​​​​​​​​​​​​​​​​ [Figure 8] A block diagram of the hardware configuration of the survey terminal device in another embodiment of the present invention. [Figure 9] An example of a processing flowchart in another embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0022] (Embodiment 1) Hereinafter, a more detailed description will be given with reference to the attached screens. Preferred embodiments are shown in the drawings. However, it can be implemented in many different forms and is not limited to the embodiments described in this specification.

[0023] For example, in this embodiment, the configuration, operation, etc. of the survey support system will be described. However, the same operational effects can also be achieved by the method, apparatus, computer program, etc. executed. Also, the program may be stored in a recording medium. By using this recording medium, for example, the program can be installed in a computer, and thereby a survey support device and a survey support system can be configured. Here, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

[0024] <1. Outline of Embodiment 1> The present invention relates to a system for assisting in the investigation of building damage in disaster areas such as earthquakes and tsunamis. FIG. 1 is an example of a diagram showing the outline of the present invention. In the present invention, first, an image (survey image) of buildings in the disaster area is taken from the air using an aircraft (drone) (1). The taken survey image is stored in cloud storage together with aircraft position information etc. regarding the position of the aircraft when the survey image was taken (2). Here, the aircraft position information is information for specifying a position such as latitude and longitude or coordinates defined on a map.

[0025] A group of orthomosaic images generated based on the stored survey images is displayed on the administrator terminal device (3), and by specifying a building on the orthomosaic image, multiple survey images linked to building location information regarding the location of that building are displayed (4). The system accepts the selection of an image from among the displayed multiple survey images for which the damage level should be determined (5), and accepts the selection of an area surrounding the damaged part (hereinafter referred to as the damaged area) (6), and by inputting the image of the damaged part contained within the damaged area into a learning model such as an image recognition model, the damage level of the damaged part (hereinafter referred to as the part damage level) is determined (7).

[0026] In this embodiment, the survey images are oblique images of the building taken from each side, and are images of the building taken by the aircraft in order to generate orthomosaic images. In the following description of Embodiment 1, the survey images will be described as oblique images.

[0027] Furthermore, in this invention, an orthoimage group is generated, which includes an orthoimage generated based on multiple oblique images, and an investigation orthoimage linked to the multiple oblique images and building location information. Alternatively, an orthoimage may be generated as an orthoimage group. In addition, in this invention, an orthoimage is obtained by generating an orthoimage, but an orthoimage generated externally may also be obtained.

[0028] Furthermore, while the building damage survey in this embodiment is limited to what can be investigated visually from the outside (first-stage survey), it may also include an internal inspection (second-stage survey). In this case, the building damage rate described later may be calculated using the results of determining the damage level of the building's internal components (interior walls, floors (including stairs), columns or load-bearing walls, ceilings, fixtures, and equipment, etc.).

[0029] <1.1. System Configuration of Embodiment 1> Figure 2 is a block diagram showing the configuration of the system in Embodiment 1. As shown in Figure 2, the survey support system 0 comprises a survey support device 1, an aircraft 2, an administrator terminal device 3, a database 4, and an operation terminal device 5. The survey support device 1, the aircraft 2, the administrator terminal device 3, and the database 4 are configured to communicate with each other via a communication network NW. In the present invention, the communication network NW is an IP (Internet Protocol) network, but there are no restrictions on the type of communication protocol, and furthermore, there are no restrictions on the type or size of the network.

[0030] The investigation support device 1 can be a general-purpose server computer or a personal computer. The administrator terminal device 3 can be a smartphone, tablet, personal computer, wearable device, etc. Furthermore, the investigation support device 1 may consist of multiple computers capable of sending and receiving information via a communication network NW or another network.

[0031] The aircraft 2 comprises an aircraft body 20, such as a drone, which does not carry a person; an attitude control device 30 having sensors and a control module; and an imaging device 40 having a camera (imaging unit) and a gimbal for controlling the attitude of the camera.

[0032] Database 4 stores multiple oblique images of buildings in the disaster area, taken by the aircraft 2, and a group of orthophotos generated based on these multiple oblique images. In this embodiment, Database 4 stores orthophotos generated based on the oblique images, building location information, and survey orthophotos linked to the multiple oblique images. Specifically, Database 4 stores orthophotos linked to multiple oblique images as survey orthophotos for each survey target area (classification by prefecture, city, town, or village, etc.) based on building location information.

[0033] The operating terminal device 5 is used by the user to operate the aircraft 2 and to check videos captured by the imaging device 40 and the inspection status. In this embodiment, the operating terminal device 5 is a tablet terminal. The operating terminal device 5 can be appropriately used depending on the embodiment, for example, the display of a remote control for operating the drone, a smartphone, a wearable device, etc.

[0034] The control terminal device 5 has pre-installed applications for operating the aircraft 2 and setting up autonomous flight. The aircraft 2 and the control terminal device 5 are connected via wireless communication for aircraft, such as in the 2.4GHz band.

[0035] <1.2. Hardware Configuration of the Invention> Figure 3 is a block diagram of the hardware configuration of the investigation support system 0. As shown in Figure 3(a), the server 10 (investigation support device 1) comprises a processing unit 101, a storage unit 102, and a communication unit 103.

[0036] The processing unit 101 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation process of the investigation support device 1 by executing the investigation support program, OS, and other applications according to the present invention. The memory unit 102 includes a volatile memory such as RAM capable of storing instruction sets, and a non-volatile recording medium such as an HDD or SSD capable of recording the OS and the research support program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and performs communication control with the communication network NW to input and output information.

[0037] As shown in Figure 3(b), the terminal device 9 (administrator terminal device 3 and operation terminal device 5) comprises a processing unit 901, a storage unit 902, a communication unit 903, an input unit 904, and an output unit 905.

[0038] The processing unit 901 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation process of the terminal device 9 by executing the OS and other applications. The memory unit 902 includes volatile memory such as RAM that can store instruction sets, and non-volatile recording media such as HDDs or SSDs that can store the OS, etc. The communication unit 903 has a communication interface device for connecting to a network and performs communication control with the communication network NW to input and output information. The input unit 904 has an input device capable of input processing, such as a keyboard or a touch panel. The output unit 905 has a display device capable of display processing, such as a display.

[0039] As shown in Figure 3(c), the aircraft 2 comprises a main control unit 201 that controls the flight movements of the aircraft 2, a motor 202 that drives the wings of the aircraft 2 and causes it to fly, a motor controller 203 that adjusts the amount of power supplied to the motor 202 based on signals from the main control unit 201, a wireless communication unit 204 for communicating with the operation terminal device 5, a GPS module 205 for obtaining position information of the aircraft 2, and a storage unit 206.

[0040] Furthermore, the aircraft 2 includes a sensor 301 for acquiring attitude information of the aircraft, an attitude control unit 302 for controlling stable autonomous flight based on information from the sensor 301, an imaging unit 401 for imaging targets such as buildings, and a gimbal 402 for controlling the attitude of the imaging unit 401.

[0041] <1.3. System Functional Configuration in Embodiment 1> Figure 4 is a block diagram of the functional configuration of the investigation support device 1 in Embodiment 1. As shown in Figure 4, the investigation support device 1 comprises a generation unit 11, a learning unit 12, a determination unit 13, a calculation unit 14, and a display processing unit 15. This represents the concrete realization of information processing by software (stored in the storage unit 102) by hardware (processing unit 101).

[0042] In this embodiment, the system configuration is a so-called server-client type, where the client receives the processing results performed by the investigation support device 1 (server) in response to a request from the administrator terminal device 3 (client). Alternatively, it may be a so-called standalone type, where the investigation support program is launched on the client terminal. In this case, the administrator terminal device 3 may comprise some or all of the functional components (parts) of the investigation support device 1. For example, the administrator terminal device 3 may comprise a generation unit 11, a learning unit 12, a determination unit 13, a calculation unit 14, and a display processing unit 15, and the investigation support device 1 may be a cloud storage that stores oblique images, orthomosaic images, etc.

[0043] <1.3.1.Generation part 11> The generation unit 11 processes multiple oblique images to generate a group of orthomosaic images and registers them in the database 4. The generation unit 11 processes multiple oblique images that have been captured to generate orthomosaic images. The generation unit 11 also processes the generated orthomosaic images, the multiple oblique images that have been captured, and the building location information to generate survey orthomosaic images.

[0044] <1.3.2. Learning Section 12> The learning unit 12 learns a learning model for determining the level of damage to building parts. The learning unit 12 learns the learning model using images of damaged parts of the building and the level of damage to those parts as training data. In this embodiment, the learning unit 12 learns a learning model that estimates the level of damage to each building component from building damage images (oblique images) using building components including the foundation, walls (exterior walls), and roof, images of damaged parts of the building, and the level of damage to those parts as training data.

[0045] In a preferred embodiment of the present invention, the learning unit 12 may further use building materials as training data to learn a learning model that estimates the level of damage to a specific part of a building from images of building damage for each building material.

[0046] Furthermore, when determining the level of damage to internal building components, a learning model may be trained using internal building components, images of damaged areas of the building, and the level of damage to those components as training data.

[0047] <1.3.3. Judgment part 13> The determination unit 13 determines the level of damage to a specific part of a building based on the oblique image. The determination unit 13 inputs the oblique image into a learned model to determine the level of damage to the building parts shown in the oblique image.

[0048] Furthermore, the determination unit 13 determines the building damage level based on the building damage ratio calculated by the calculation unit 14. The determination unit 13 determines the building damage level based on the building damage ratio for each building element.

[0049] In a preferred embodiment of the present invention, the determination unit 13 determines the building damage level based on the building tilt and the building damage ratio for each building element calculated by the calculation unit 14.

[0050] <1.3.4. Calculation Section 14> The calculation unit 14 calculates the building damage ratio of buildings damaged by the disaster. The calculation unit 14 calculates the building damage rate using oblique images and the damage level of each part determined based on the oblique images. The calculation unit 14 calculates the building damage ratio for each building element based on the calculated building damage rate and the building composition ratio which indicates the weight of each building element.

[0051] In a preferred embodiment of the present invention, the calculation unit 14 calculates the building tilt of a building that has tilted due to the disaster. The calculation unit 14 calculates the building tilt using an image of the building (such as an oblique image) taken by an aircraft 2 or the like.

[0052] <1.3.5. Display Processing Unit 15> The display processing unit 15 processes a screen for determining the level of damage to a specific body part and displays the results of the processing to the administrator terminal device 3.

[0053] <1.4. Functional Configuration of Aircraft Unit 2> As shown in Figure 4, the aircraft 2 is equipped with a registration unit 21. This is a system where information processing by software (stored in the memory unit 206) is concretely realized by hardware (main control unit 201).

[0054] <1.4.1. Registration Section 21> The registration unit 21 registers multiple oblique images captured by the imaging unit 401 into the database 4. The registration unit 21 registers the multiple oblique images and the aircraft position information at the time the oblique images were taken. In this embodiment, the registration unit 21 registers multiple oblique images for each survey area based on the aircraft position information.

[0055] In this embodiment, the registration unit 21 provided on the aircraft 2 registers the captured oblique images in the database 4. However, the survey support device 1 may also be equipped with a registration unit 21, and the registration unit 21 may register the captured oblique images received from the aircraft 2 in the database 4.

[0056] <1.5. Processing Flowchart in Embodiment 1> Referring to Figure 5, the survey support method using the survey support system 0 in Embodiment 1 will be described. Figure 5 is a flowchart showing the process from when the survey support device 1 acquires oblique images taken by the aircraft 2, registers a group of orthomosaic images based on the oblique images, to when it determines the building damage level in response to an operation from the administrator. The process related to the survey support method in this embodiment is broadly divided into image acquisition process (A) and determination process (B).

[0057] <1.5.1. Image Acquisition Process (A)> In the image acquisition process (A), the oblique image taken by the aircraft 2 is registered in the database 4, an orthomosaic image is generated based on the oblique image, and a survey orthomosaic image is generated based on the orthomosaic image.

[0058] First, in step SA1 (hereinafter, "step SXX" will simply be referred to as "SXX"), the imaging unit 401 of the aircraft acquires an oblique image. In this embodiment, the aircraft 2, via wireless communication with the operating terminal device 5, flies over the disaster area in response to the operation of the user (investigator) of the operating terminal device 5, thereby photographing one or more buildings and acquiring multiple oblique images.

[0059] In SA2, the registration unit 21 registers the shooting-related information in the database 4. In this embodiment, the registration unit 21 receives, as shooting-related information, a plurality of oblique images acquired in SA1, the position information of the aircraft at the time the oblique images were taken, and the attitude information of the aircraft 2 acquired by the sensor 301 of the aircraft 2.

[0060] In SA3, the generation unit 11 processes to generate an orthomosaic image. In this embodiment, the generation unit 11 generates building location information for one or more buildings based on a plurality of oblique images acquired in SA1, and processes to generate an orthomosaic image based on the building location information. Specifically, the generation unit 11 uses two or more overlapping oblique images (so-called stereo pair images) of the same building taken from different viewpoints to generate building location information represented by a 3D point cloud. Then, the generation unit 11 processes to generate an orthomosaic image based on the 3D point cloud.

[0061] In a preferred embodiment of the present invention, the building location information of one or more photographed buildings is identified based on multiple oblique images, as well as aircraft position information and attitude information, and an orthomosaic image is generated.

[0062] In SA4, the generation unit 11 processes the orthophotos for the survey. In this embodiment, the generation unit 11 processes the orthophotos generated in SA3, and for each building location information identified in SA3, it processes the orthophotos by linking multiple oblique images acquired in SA1. Specifically, the generation unit 11 processes the orthophotos for the survey by linking multiple oblique images used to identify a building location information to that building location information. Then, the generation unit 11 registers the generated orthophotos for each survey target area based on the building location information in the database 4.

[0063] In a preferred embodiment of the present invention, survey orthomosaic images may be registered in association with a disaster ID for identifying the disaster that occurred, the date of the disaster, and surveyor identification information (such as a surveyor ID or surveyor name, or a survey group ID or survey group name) for uniquely identifying the survey group or surveyor.

[0064] <1.5.2. Judgment Processing (B)> In the determination process (B), the administrator specifies the area to be investigated and the buildings for which the damage level will be determined, and the process up to the point where the damage level is determined is executed.

[0065] <1.5.2.1. Display of Survey Orthomosaic Images> First, in SB1-SB3, the survey orthomosaic image relating to the survey area is displayed on the administrator terminal device 3. In this embodiment, the administrator terminal device 3 receives the designation of the survey area from the administrator, and the display processing unit 15 receives a display instruction for the survey orthomosaic image including the designated survey area (SB1). The display processing unit 15 then identifies and displays the survey orthomosaic image associated with the designated survey area (SB2), and transmits the display processing result to the administrator terminal device 3 (SB3).

[0066] In a preferred embodiment of the present invention, in addition to specifying the area to be surveyed, the system may also accept the specification of a disaster ID, the date of the disaster, and the identification information of the surveyor.

[0067] <1.5.2.2. Displaying Multiple Diagonal Images> Next, in SB4-7, oblique images of the buildings for which the damage level is to be determined are displayed on the administrator terminal device 3. In this embodiment, the administrator terminal device 3 receives a specification from the administrator of building location information where the building for which the damage level is to be determined is located (SB4). The display processing unit 15 receives a display instruction for oblique images associated with the building location information, including the building location information (SB5). The display processing unit 15 then identifies and displays multiple oblique images associated with the building location information (SB6), and transmits the display processing result to the administrator terminal device 3 (SB7).

[0068] <1.5.2.3. Judgment Process> Then, in SB8-13, the level of damage to the building is determined. First, in SB8, the administrator terminal device 3 receives the designation of building components and damaged areas from the administrator. In this embodiment, the display processing unit 15 receives the designation of one of the multiple oblique images transmitted in SB7, and displays a damaged area designation section that allows the designation of the damaged area for that oblique image. Then, the administrator terminal device 3 receives the designation of the damaged areas of the building and the building components from the administrator based on the designated area.

[0069] In this embodiment, the system accepts input from the administrator of a closed curve enclosing the damaged area as the designation of the damaged area. Alternatively, a part identification unit (not shown) may input an oblique image to an image recognition model to distinguish between the damaged area and the undamaged area, and the display processing unit 15 may display the oblique image in which the damaged area can be designated, thereby accepting the designation of the distinguished damaged area as the designation of the damaged area. Furthermore, the area of ​​the damaged area distinguished by the part identification unit may be designated as the damaged area.

[0070] Then, in SB9, the determination unit 13 receives a damage level determination instruction that includes an oblique image of the specified damaged area, and determines the area damage level in SB10. In this embodiment, the determination unit 13 inputs the oblique image and the damaged area specified in SB8 into a learning model learned by the learning unit 12, and determines the area damage level corresponding to the damaged area.

[0071] In a preferred embodiment of the present invention, the determination unit 13 determines the part damage level corresponding to the damaged area by further inputting building components into a learning model. Specifically, the determination unit 13 determines the part damage level corresponding to the damaged area by inputting oblique images in which building components and damaged areas are specified into a learning model that has been learned by the learning unit 12, which estimates the part damage level from building damage images for each building component.

[0072] In a preferred embodiment of the present invention, the determination unit 13 further inputs the building materials of the building into a learning model to determine the damage level corresponding to the damaged area. Specifically, the determination unit 13 inputs an image of the designated damaged area into an image recognition model to identify the building material of the damaged area. Then, the determination unit 13 inputs the identified building material and a diagonal image of the damaged area into a learning model, which has been learned by the learning unit 12 to estimate the damage level from building damage images for each building material, to determine the damage level corresponding to the damaged area.

[0073] Then, after the processes SB8-SB10 are executed for each building component, the processes from SB11 onwards are executed.

[0074] In SB11, the determination unit 13 determines the building damage level. In this embodiment, first, the calculation unit 14 calculates the building damage rate using the oblique image in SB8 in which the damaged area is specified, and the area damage level determined in SB10. Specifically, the calculation unit 14 identifies the total perimeter area of ​​the building components (foundation, walls, roof, etc.) specified in SB8, and the area of ​​the specified damaged part, and calculates the building damage rate based on the damaged area ratio, which is the ratio of the damaged area area to the total perimeter area, and the area damage level. More specifically, the calculation unit 14 identifies the total perimeter area of ​​the specified building components using multiple oblique images linked to the specified building location information, and calculates the building damage rate by multiplying the damaged area ratio calculated using the total perimeter area by the area damage level.

[0075] In this embodiment, the multiple oblique images associated with the specified building location information are images of the same building taken from different directions. The calculation unit 14 receives a specification from the user for a region that encloses the entire perimeter of the building components as seen from each direction, and determines the total perimeter area of ​​the building components based on the area of ​​each specified region. Alternatively, the calculation unit 14 may use well-known image recognition technology to determine the total perimeter area of ​​the building components in each direction from each oblique image, and then determine the total perimeter area of ​​the building components by summing the total perimeter areas in each direction.

[0076] The calculation unit 14 then calculates the building damage ratio for each building component by multiplying the calculated building damage rate by the building composition ratio for each building element. For example, the building composition ratios are set as follows: roof "15%", walls "75%", and foundation "10%".

[0077] Finally, the determination unit 13 sums up the building damage ratios for each calculated building component and determines the building damage level based on the total value and the threshold of the building damage level index. The building damage level index is set as follows: for example, "partial damage" if the damage ratio is less than 10%, "near-partial damage" if the damage ratio is 10% or more, and "partial damage" if the damage ratio is 20% or more.

[0078] In a preferred embodiment of the present invention, the building tilt is calculated, and the building damage level is determined based on conditions derived from the building tilt. Specifically, the calculation unit 14 calculates the building tilt for each of the four corner pillars of the building based on images of the pillars taken by the flying object 2, and calculates the average value thereof. The determination unit 13 uses the condition of whether the average value of the building tilt exceeds a predetermined threshold as a condition derived from the building tilt. If the average value of the building tilt exceeds the predetermined threshold, it sums the building damage ratio calculated for the "roof" of the building components and a predetermined value set as the building damage ratio for the building tilt. The determination unit 13 then compares this sum with the sum of the building damage ratios for all building components and determines the building damage level based on the higher value and the threshold of the building damage level index. If the average value of the building tilt does not exceed the predetermined threshold, the determination unit 13 determines the building damage level based on the sum of the building damage ratios for all building components and the threshold of the building damage level index.

[0079] The damage levels determined in SB12-SB13 are displayed to the administrator terminal device 3. In this embodiment, the display processing unit 15 displays the damaged area designation unit and the damage level display unit that displays the damage levels of each building component determined in SB10 on the same screen (SB12), and transmits the display processing results to the administrator terminal device 3 (SB13). Specifically, each time a damage level is determined for a building component and a damaged area designated in the damaged area designation unit, the display processing unit 15 updates and displays the damage level of the part corresponding to that building component. Then, the display processing unit 15 displays the building damage level determined by the determination unit 13 based on the updated damage level of the part.

[0080] As described above, by executing the processes SA1 to SB13 in Embodiment 1, it becomes possible to remotely assess the damage situation in a disaster area using a drone. Furthermore, by allowing the user to specify the damaged areas and having the learning model determine the damage level for the specified areas, it is possible to accurately determine the damage level for the identified damaged areas, thereby enabling a more accurate assessment of the damage situation.

[0081] In this embodiment, the damaged area is an area that encloses each damaged part individually, but it may also be an area that encloses multiple damaged parts together. In such a case, if the damaged area is specified to span multiple building components, a notification unit (not shown) may, for example, notify that there is an error in the specification of the damaged area (e.g., please specify the damaged area for each building component). Alternatively, a boundary determination unit (not shown) may determine the boundaries of the building components and divide the damaged area at those boundaries to determine the damage level of each building component.

[0082] (Embodiment 2) In Embodiment 1, a drone is used to determine the level of damage to buildings in the disaster area. On the other hand, in Embodiment 2, images taken with a 360-degree camera owned by an investigator who actually went to the disaster area are used to determine the level of damage to buildings. In the following description, the system according to this embodiment will perform each functional configuration, and parts that are common with Embodiment 1 will be omitted.

[0083] <2. Overview of Embodiment 2> Figure 6 is an example of a schematic diagram in Embodiment 2 of the present invention. In this embodiment, first, an investigator who goes to the disaster area takes images of buildings in the disaster area from above using an investigation terminal device equipped with a 360-degree camera and a 5m monopod (1). The captured investigation images are stored in cloud storage along with location information regarding the location where the investigation images were taken (2).

[0084] A map of the disaster area is displayed on the administrator terminal device (3), and by specifying the shooting location information on the map, survey images associated with the shooting location information are displayed (4). In the displayed survey images, the system accepts the specification of the damaged area (5), and by inputting the images of the damaged parts contained within the damaged area into a learning model such as an image recognition model, the part damage level of the damaged part is determined (6).

[0085] In this embodiment, the survey image is a panoramic image, which is a single image created by synthesizing images continuously taken over 360 degrees around the survey terminal device. In the following description of Embodiment 2, the survey image will be described as a panoramic image.

[0086] <2.1. System Configuration of Embodiment 2> Figure 7 is a block diagram showing the configuration of the system in Embodiment 2. As shown in Figure 7, the survey support system 0 comprises a survey support device 1, an administrator terminal device 3, a database 4, and a survey terminal device 6, and the survey support device 1, administrator terminal device 3, database 4, and survey terminal device 6 are configured to communicate with each other via a communication network NW.

[0087] The survey terminal device 6 can be a smartphone, tablet, personal computer, wearable device, or the like.

[0088] <2.2. Hardware configuration of the survey terminal device 6> Figure 8 is a block diagram of the hardware configuration of the survey terminal device 6. As shown in Figure 8, the terminal device 9 (survey terminal device 6) includes a processing unit 901, a storage unit 902, a communication unit 903, an input unit 904, an output unit 905, an imaging unit 906 for imaging targets such as buildings, and a GPS module 907 for obtaining the position of the terminal device 9.

[0089] <2.3. System Functional Configuration in Embodiment 2> In this embodiment, the survey support device 1 has the same mechanical configuration as in Embodiment 1. Furthermore, the survey terminal device 6 includes a registration unit having the same function as the registration unit 21 provided by the aircraft 2. In the following description, the registration unit of the survey terminal device 6 will also be treated as the registration unit 21.

[0090] <2.3.1.Generation part 11> In this embodiment, the generation unit 11 processes the image to generate an oriented image. The generation unit 11 processes the image to generate an oriented image based on the panoramic image and the shooting direction information.

[0091] <2.4. Processing flowchart in Embodiment 2> Referring to Figure 9, the survey support method using the survey support system 0 in Embodiment 2 will be described. Figure 9 is a flowchart showing the process from when the survey support device 1 acquires a panoramic image taken by the survey terminal device 6, registers an oriented image based on the panoramic image, and determines the building damage level in response to an operation from the administrator.

[0092] <2.4.1. Image Acquisition Processing (C)> In the image acquisition process (C), the panoramic image captured by the survey terminal device 6 is registered in the database 4, and the process is executed until an oriented image is generated based on the panoramic image.

[0093] First, in step SC1, the imaging unit 906 of the survey terminal device 6 acquires a panoramic image. In this embodiment, the survey terminal device 6 photographs one or more buildings from above the disaster area in response to the surveyor's operation, and acquires one panoramic image for each shooting location.

[0094] In SC2, the registration unit 21 registers the shooting-related information in the database 4. In this embodiment, the registration unit 21 receives as shooting-related information the panoramic image for each shooting location acquired in SC1, the shooting location information in which the panoramic image was taken, and the shooting direction information indicating the orientation of the camera when the subject was photographed.

[0095] In SC3, the generation unit 11 processes the image with orientation. In this embodiment, the generation unit 11 aligns the viewpoint of the panoramic image acquired in SC1 based on the shooting direction information acquired in SC2, and processes the image with orientation.

[0096] In SC4, the generation unit 11 processes the data to generate images with survey directions. In this embodiment, the generation unit 11 processes the data to generate survey images with survey directions that are linked to the directional images generated in SC3 and the shooting location information acquired in SC2, based on the map data of the disaster area. The generation unit 11 then registers the generated survey images with survey directions for each survey target area based on the shooting location information in the database 4.

[0097] <2.4.2. Judgment Process (D)> In the determination process (D), the administrator specifies the area to be investigated and the buildings for which the damage level will be determined, and the process up to the point where the damage level is determined is executed.

[0098] <2.4.2.1. Displaying Panoramic Images> First, in SD1-SD3, a map of the survey area is displayed on the administrator terminal device 3. In this embodiment, the administrator terminal device 3 receives a designation of the survey area from the administrator, and the display processing unit 15 receives an instruction to display the map of the designated survey area (SD1). The display processing unit 15 then processes the map of the designated survey area to display (SD2) and transmits the display processing result to the administrator terminal device 3 (SD3).

[0099] <2.4.2.2. Determination process from panoramic image display> Next, in SD4-7, a panoramic image for determining the level of damage is displayed on the administrator terminal device 3. In this embodiment, the map of the area to be surveyed displayed in SD3 displays the shooting location information registered in SC4 in a selectable format, and the administrator terminal device 3 receives the specification of the shooting location information from the administrator (SD4). The display processing unit 15 receives a display instruction for the panoramic image associated with the shooting location information, including the specified shooting location information (SD5). The display processing unit 15 then identifies the panoramic image associated with the shooting location information and processes it for display (SD6), and transmits the display processing result to the administrator terminal device 3 (SD7).

[0100] Then, the display processing unit 15 displays the damage area specification section, which displays a panoramic image in which the damaged area can be specified in SD7, and displays it to the administrator terminal device 3. Then, in SD8, the administrator terminal device 3 receives operations from the administrator regarding rotation and zooming of the panoramic image and accepts the specification of the damaged area. Then, the processing in SD9 to SD13 is executed in the same way as in SB9 to SB13 of Embodiment 1, and the determined building damage level is displayed on the administrator terminal device 3.

[0101] As described above, by executing the processes SC1 to SD13 in Embodiment 2, it becomes possible to grasp the extent of the damage by actually going to the disaster area using a 360-degree camera. This makes it possible to take photographs even in places where it is difficult to photograph with a drone, and to grasp the extent of the damage in the disaster area more accurately.

[0102] In Embodiment 2, the damage level determination process using only the survey terminal device 6 was described, but a damage level determination process combining Embodiments 1 and 2 may also be performed.

[0103] Furthermore, in Embodiments 1 and 2, the display process refers to the process by which the display processing unit 15 executes a process to generate information necessary for display, and transmits the generated information to the terminal device 9, thereby causing the terminal device 9 to display the generated information. On the other hand, if the display processing unit 15 is provided in the terminal device 9 (in the case of a standalone type), the display process may refer to the process by which the display processing unit 15 executes a process to generate the necessary information, and transmits the generated information to the output unit 905 of the terminal device 9, thereby causing the output unit 905 to display the generated information.

[0104] Furthermore, in Embodiments 1 and 2, the generation process is the process of generating an image in the survey support device 1, but it may also be a process of transmitting information necessary for image generation to an external image generation system, causing the system to generate an image, and acquiring the generated image. [Explanation of symbols]

[0105] 0: Survey support system 1: Survey support device 2: Flying object 3: Administrator terminal device 4: Database 5: Operating terminal device 6: Survey terminal device 10: Server 101: Processing Unit 102: Storage section 103: Communications Department 9: Terminal device 901: Processing Unit 902: Storage section 903: Communications Department 904: Input section 905: Output section 906: Imaging Unit 907: GPS module 20: Main body of the flying object 30: Attitude control device 40: Imaging device 201: Main Control Unit 202: Motor 203: Motor Controller 204: Wireless Communication Department 205: GPS module 206: Storage section 301: Sensor 302: Attitude Control Unit 401: Imaging Unit 402: Gimbal 11: Generation part 12: Learning Department 13: Judgment section 14: Calculation Unit 15: Display Processing Unit 21: Registration Department NW: Communication Network

Claims

1. This is a survey support system for assisting in the survey of building damage in disaster-stricken areas. The aforementioned survey support system comprises a registration unit and a generation unit, The registration unit registers multiple survey images of one or more of the aforementioned buildings into a database. The generation unit processes the multiple survey images to generate a group of orthomosaic images. Survey support system.

2. The registration unit registers building location information relating to the location of the photographed building, in addition to the multiple survey images. The generation unit processes the generation of an orthomosaic image based on the multiple captured survey images, and an orthomosaic image linked to the multiple captured survey images and the building location information. The survey support system according to claim 1.

3. The aforementioned survey support system further comprises a display processing unit and a determination unit, The display processing unit processes the survey orthomosaic image for display, receives the building location information, and processes the survey image associated with the building location information for display. The determination unit receives the designation of the damaged parts of the building shown in the survey image and determines the level of damage to those parts. The survey support system according to claim 2.

4. The determination unit receives the designation of building components and the damaged areas for each building component, and inputs the designated building components and damaged areas to a learning model that estimates the level of damage to each building component from building damage images to determine the level of damage to each building component. The survey support system according to claim 3.

5. The display processing unit displays, on the same screen, a damage area specification unit that allows the specified damage area for each building component, and a damage level display unit that displays the determined damage level for each building component. The survey support system according to claim 3.

6. The aforementioned survey support system further comprises a calculation unit, The calculation unit receives a designation of a damaged area as the designation of the damaged area, and calculates the building damage rate based on the survey image in which the damaged area is designated and the damage level of the area. The survey support system according to claim 3.

7. The calculation unit receives the designation of the damaged area for each building component, calculates the percentage of the damaged area for each building component based on the survey image in which the damaged area has been designated for each building component, and calculates the building damage rate based on the percentage of the damaged area and the damage level for each building component. The investigation support system according to claim 6.

8. This is a survey support program to assist in surveying damage to buildings in disaster-stricken areas. The aforementioned survey support program uses a computer as a registration unit and a generation unit, The registration unit registers multiple survey images of one or more of the aforementioned buildings into a database. The generation unit processes the multiple survey images to generate a group of orthomosaic images. Research support program.

9. A survey support method for assisting in the assessment of building damage in disaster-stricken areas, Computers A process of registering multiple survey images of one or more of the aforementioned buildings into a database, A process to generate a group of orthomosaic images based on the aforementioned multiple survey images, A method for supporting investigations to carry out such investigations.

Images