Information processing systems, information processing methods, and programs

Abstract

We provide an information processing system that can accurately determine the state of damage. [Solution] According to one aspect of the present invention, an information processing system is provided comprising one or more processors, wherein the processors, in a building image acquisition step, acquire one or more images of a damaged building; in a generation step, generate 3D data representing the building in three-dimensional space based on the acquired one or more images; and in a determination step, determine the state of damage to the building based on the generated 3D data.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In Patent Document 1, a display unit maintains and displays an operation button corresponding to switching to each of a plurality of screens including an issuance management screen for receiving a designation of an investigation date and time of a building for which an investigation application has been made, and an investigator screen for receiving registration of an investigator in charge of the investigation of the building, regardless of which screen is displayed. An investigation plan generation unit receives an input of a target period for generating an investigation plan on any of the displayed screens, and based on information on buildings for which an investigation date and time is specified within the target period among the buildings to be investigated specified on the issuance management screen, and information on the investigators received for registration on the investigator screen, generates an investigation plan for which investigator investigates which building at what time during the target period.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In disaster damage assessment operations during disasters, support staff from other departments without knowledge often handle the operations, and assessment errors lead to claims from residents, generation of re-assessment operations, etc., which has been a burden on local governments.

[0005] In view of the above circumstances, the present invention aims to provide an information processing system or the like that can accurately determine the damaged state.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system is provided comprising one or more processors, wherein the processors, in a building image acquisition step, acquire one or more images of a damaged building; in a generation step, generate 3D data representing the building in three-dimensional space based on the acquired one or more images; and in a determination step, determine the state of damage to the building based on the generated 3D data.

[0007] According to this embodiment, the state of damage can be determined with high accuracy. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the overall configuration of disaster damage assessment system 1. [Figure 2] This figure shows an example of the hardware configuration of the disaster damage assessment server 10. [Figure 3] This figure shows an example of the hardware configuration of user terminal 30. [Figure 4] This is an activity diagram showing an example of damage detection processing. [Figure 5] This figure shows an example of the disaster damage assessment screen that is displayed. [Figure 6] This figure shows an example of the displayed shooting instruction screen. [Figure 7] This figure shows an example of the displayed guidance. [Figure 8] This figure shows an example of the displayed damage status screen. [Figure 9] This figure shows another example of the displayed damage status screen. [Figure 10] This figure shows an example of the displayed evaluation summary screen. [Figure 11] This figure shows an example of the application form screen that is displayed. [Figure 12] This figure shows an example of the displayed evidence information screen. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0010] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model that can output a desired result by inputting a prompt (these models include parameters that construct the correlation relationship between input and output) or a visual language model.

[0012] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0013] Furthermore, a circuit in a broad sense is a circuit realized by appropriately combining at least a circuit, circuitry, a processor, a memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes an application specific integrated circuit (ASIC), programmable logic devices (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0014] <Embodiment> 1. System Configuration Hereinafter, the system configuration according to the embodiment will be described. FIG. 1 is a diagram showing an example of the overall configuration of the disaster damage assessment system 1. In FIG. 1, an overview of each device included in the disaster damage assessment system 1 and an overview of users who use these devices are shown. Each overview will be described as needed while referring to other figures.

[0015] The disaster damage assessment system 1 is a system for assisting in the assessment of the damage state of damaged buildings. The disaster damage here includes not only damage caused by earthquakes but also damage or deformation to buildings caused by various disasters including water disasters such as heavy rain, floods, inundation or landslides, and wind disasters such as typhoons, tornadoes, strong winds or blizzards. The disaster damage assessment system 1 targets the assessment of the inclination, subsidence, displacement, deformation, damage to exterior materials or the depth of inundation, etc. of buildings caused by these disasters. These assessment targets vary depending on the type of disaster.

[0016] The disaster damage assessment system 1 includes a communication line 2, a disaster damage assessment server 10, a building information server 20, and a user terminal 30. The building information server 20 includes a building information DB 3 (DB: Data Base). Note that the configuration shown in FIG. 1 is an example, and each server such as the disaster damage assessment server 10 may be distributed among two or more devices, and may also be provided in the form of SaaS (Software as a Service) or a cloud computing system. Further, two or more user terminals 30 and databases such as the building information DB 3 may be provided respectively.

[0017] The communication line 2 is not particularly limited, but for example, it is constituted by the Internet network. Further, the communication line 2 may include a local area network, a mobile communication network, a VPN (Virtual Private Network), etc. The communication line 2 mediates the exchange of data between devices connected to its own line. In the example of FIG. 1, each server is connected to the communication line 2 by wire, and each client terminal is connected wirelessly. Note that the connection of each device to the communication line 2 may be wired or wireless.

[0018] The user terminal 30 is a terminal used by a user who uses the disaster damage assessment system 1, and for example, is a smartphone, a tablet terminal, a notebook personal computer, etc. The user terminal 30 has display means and input means, and functions as a UI (User Interface) for operations by the user. The user of the user terminal 30 is a person who checks the damage state of a damaged building and makes an application or report based on the evaluation result, and for example, is the owner of the building, the administrator, the insurance applicant, or a person engaged in the business of confirming building damage.

[0019] The building information server 20 is an information processing device that stores and maintains building information, which is information about a building, in the building information DB3. The building information includes, for example, building drawings, building photographs, building location, structural type, number of floors, and building attribute information such as the year of construction. The building information server 20 may be a server managed by, for example, a department that manages buildings in a local government, or a server managed by the building owner, management company, or insurance company.

[0020] Furthermore, the disaster assessment server 10 performs, for example, an authentication process to authenticate the user and a display process to display images on a client terminal such as the user terminal 30. The disaster assessment server 10 stores authentication information (for example, user ID and password) for authenticating the user and authenticates the user who enters the authentication information as a user of the disaster assessment system 1. By authenticating the user, the disaster assessment server 10 determines which data the user can access and adds identification information to the data entered by the user to make it identifiable.

[0021] The disaster assessment server 10 performs display processing such as generating and sending HTML (Hyper Text Markup Language) files using a web program, for example, to generate a web page that displays the system screen, and displays the generated system screen on the client terminal using the browser's functions. Alternatively, the client terminal may install an application program to use the disaster assessment system 1, and the disaster assessment server 10 may generate and display a system screen that can be displayed by that application. The disaster assessment server 10 controls the display on the client terminal by performing these display processing operations.

[0022] The authentication and display control processes described above are executed not only on the disaster assessment server 10 but also on the building information server 20. The user authentication information used by each server may be the same or different. If they are different, a correspondence table showing the correspondence between these authentication pieces of information is used to manage users on each server as the same user.

[0023] 2. Hardware Configuration The hardware configuration according to this embodiment will be described below. Figure 2 shows an example of the hardware configuration of the disaster damage assessment server 10. The disaster damage assessment server 10 comprises a control unit 101, a storage unit 102, a communication unit 103, and a bus 104. The bus 104 electrically connects each part of the disaster damage assessment server 10.

[0024] (Control Unit 101) The control unit 101 has at least one processor. The at least one processor may consist of, for example, a central processing unit (CPU), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), one or more integrated circuits, one or more discrete circuits, or a combination thereof.

[0025] The control unit 101 is a computer that realizes various functions related to the disaster damage assessment system 1 by reading predetermined programs stored in the memory unit 102. In other words, information processing by software stored in the memory unit 102 is concretely realized by the control unit 101, which is an example of hardware, and can be executed as each functional unit included in the control unit 101. Note that the control unit 101 is not limited to being a single unit, and may be implemented with multiple control units 101 for each function, or a combination thereof.

[0026] (Storage unit 102) The memory unit 102 stores various types of information as defined above. This can be done, for example, as a storage device such as a solid-state drive (SSD) or hard disk drive (HDD) that stores various programs related to the disaster assessment system 1 executed by the control unit 101, or as memory such as random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to program calculations. The memory unit 102 stores various programs and variables related to the disaster assessment system 1 executed by the control unit 101.

[0027] (Communications Department 103) The communication unit 103 is composed of a communication module. The communication module may be a wireless communication module compliant with standards such as IEEE 802.11a / b / g / n / ac / ax, LTE, 5G, 6G, or a wired communication module compliant with standards such as IEEE 802.3. The communication unit 103 is configured to transmit various electrical signals from the disaster assessment server 10 to external components. The communication unit 103 is also configured to receive various electrical signals from external components to the disaster assessment server 10. More preferably, the communication unit 103 has a network communication function, thereby enabling the communication of various information between the disaster assessment server 10 and external devices via the communication line 2.

[0028] The building information server 20 has the same hardware configuration as the disaster damage assessment server 10. For the building information server 20, only the control unit 201 will be described using a different reference numeral than the control unit 101 of the disaster damage assessment server 10.

[0029] Figure 3 shows an example of the hardware configuration of a user terminal 30. The user terminal 30 comprises a control unit 301, a storage unit 302, a communication unit 303, an input unit 304, an output unit 305, and a bus 306. The bus 306 electrically connects each part of the user terminal 30. The control unit 301, storage unit 302, and communication unit 303 are similar hardware to the control unit 101, storage unit 102, and communication unit 103 shown in Figure 2, although their specifications and models may differ.

[0030] (Input section 304) The input unit 304 has input receiving means such as keys, buttons, touchscreens, and mice, and accepts input from the user. The input unit 304 may also have sound receiving means such as a microphone, and may have the function of collecting the user's voice and accepting the collected voice as input. The input unit 304 may also have location information acquisition means such as a GPS (Global Positioning System) receiver, and may acquire the location information of the user terminal 30.

[0031] (Output section 305) The output unit 305 has display means such as a display and sound emission means such as a speaker, and outputs visual information and auditory information. For example, the output unit 305 displays visual information such as screens, images, icons, and text on the display surface of the display unit in a manner that is visible to the user. The output unit 305 also outputs audible sound such as speech or synthesized sound from the speaker.

[0032] 3. Information Processing The following describes the information processing according to the embodiment. In the following description, the disaster assessment server 10, the building information server 20, and the user terminal 30 are described as the main entities of each information processing, but these information processing are executed by at least one processor provided in the disaster assessment system 1, that is, the processors in the control units 101, 201, and 301 of each device. The following describes the damage determination process for determining the damage status of a disaster-stricken building.

[0033] Figure 4 is an activity diagram showing an example of the damage assessment process. The damage assessment process is initiated, for example, when an operator logs in to the disaster assessment server 10 by operating the user terminal 30. First, the disaster assessment server 10 generates a disaster assessment screen (activity A11) and displays the generated disaster assessment screen on the user terminal 30 (activity A12).

[0034] Figure 5 shows an example of a disaster damage assessment screen. In the disaster damage assessment screen G1 shown in Figure 5, the text "Please enter the address of the building to be assessed for disaster damage" is displayed, along with an input field C11, a display field C12, a current location input button B11, and an input completion button B12.

[0035] Input field C11 is for entering the address of the damaged building. The address may be entered manually in input field C11, or the current location of the user terminal 30 may be entered as the address of the damaged building by operating the current location input button B11. A map showing the current location is displayed in display field C12. After the user confirms that the displayed map is the location of the damaged building, they can confirm that the current location is entered as the building's address by operating the current location input button B11.

[0036] When the input completion button B12 is pressed, the user terminal 30 accepts the address information entered in the input field C11 (Activity A13) and sends the entered address information to the disaster damage assessment server 10. When the disaster damage assessment server 10 receives the transmitted address information (Activity A14), it sends request data to the building information server 20 requesting building information for the address indicated by the received address information.

[0037] When the building information server 20 receives the request data, it reads the building information for the address indicated by the request data from the building information DB3 (Activity A15) and sends the read building information to the disaster damage assessment server 10. The disaster damage assessment server 10 acquires the transmitted building information as building information for the damaged building (Activity A16). Next, the disaster damage assessment server 10 generates a shooting instruction screen that instructs the shooting of the damaged building (Activity A21) and displays the generated shooting instruction screen on the user terminal 30 (Activity A22).

[0038] Figure 6 shows an example of a displayed shooting instruction screen. In the shooting instruction screen G2 shown in Figure 6, the text "Please take pictures according to the guidance" and shooting buttons B21 to B26 are displayed. Shooting buttons B21 to B26 are images used to initiate shooting for each target: "Exterior," "Slope," "Structure," "Foundation," "Walls," and "Roof." When each shooting button is operated, guidance for shooting the corresponding target is displayed.

[0039] Figure 7 shows an example of the displayed guidance. The shooting screen G3 shown in Figure 7 is displayed when the shooting button B21, which is set to "exterior," is pressed. On the shooting screen G3, the shooting end button B31, the text indicating the guidance "Please take the picture so that the building fits within the frame," and the captured image C31 are displayed. The user moves around the building and takes pictures while ensuring that the building fits within the captured image C31, and after taking pictures from as many different directions as possible, they press the shooting end button B31 to end the shooting. Images taken from multiple directions in this way are used in the subsequent 3D data generation process.

[0040] Furthermore, the disaster damage assessment system 1 may be configured to display guidance information during shooting so that even users unfamiliar with the shooting process can ensure shooting quality suitable for generating 3D data of the building. For example, the user terminal 30 determines whether the shooting distance, shooting angle, shooting direction, or shooting range is appropriate based on the images, videos, or metadata acquired during the shooting of the building. Then, according to the determination result, it displays guidance on the shooting method, such as "Please get a little closer to the building to take the picture," "Please take the picture so that the entire building fits in the frame," or "Please take the picture from a different angle."

[0041] Furthermore, if the user terminal 30 determines that the shooting conditions necessary for generating 3D data of the building have been met, it may display an icon, mark, or text indicating that the shooting method is appropriate. By providing visual feedback to the user regarding the suitability of the shooting status in this way, the user can easily understand whether the shooting method is appropriate.

[0042] According to this configuration, regardless of whether or not the user has experience in photography, it is possible to stably acquire images suitable for generating 3D data of buildings, thereby improving the accuracy of 3D data generation and the accuracy of damage state determination based on it.

[0043] The user also takes pictures of other objects by operating the capture button. When the user terminal 30 captures images of each object in the building (Activity A23), it sends the captured images of the building to the disaster damage assessment server 10. At this time, one image or multiple images of the building may be taken. The multiple images may be a video or multiple still images.

[0044] When the disaster damage assessment server 10 receives one or more images of a building from the transmitted data (Activity A24), it generates 3D data of the building based on the acquired images (Activity A31). 3D data is data that represents a building in three-dimensional space, and is, for example, point cloud data that represents the position of each part that makes up the building using three-dimensional coordinates, or mesh data generated based on said point cloud data.

[0045] When the disaster damage assessment server 10 acquires multiple building images, it extracts feature points (corner points, edges, etc.) from the acquired images and associates identical feature points between images. Next, the disaster damage assessment server 10 estimates the camera orientation and the 3D position of the feature points based on the association results and generates a sparse point cloud. Furthermore, the disaster damage assessment server 10 uses the sparse point cloud and the estimated camera orientation to increase the density of the point cloud using multi-view stereo or the like, generating a dense point cloud. The disaster damage assessment server 10 generates a mesh of the building surface from the generated point cloud and applies image-based textures as needed. This provides 3D data that three-dimensionally represents the position, shape, and slope of walls, roofs, foundations, etc.

[0046] Furthermore, when the disaster damage assessment server 10 acquires a single building image, it can use that image as input to perform a depth estimation model. The depth estimation model is a model that estimates the distance to the target object for each pixel in the image, and is constructed using, for example, a trained convolutional neural network. Based on the depth information of each pixel estimated by the depth estimation model, the disaster damage assessment server 10 converts each pixel on the image into a three-dimensional coordinate system and generates point cloud data representing the building. This makes it possible to represent the three-dimensional shape of a building in three dimensions, even from a single image.

[0047] Next, the disaster damage assessment server 10 determines the extent of damage to the building based on the generated 3D data (Activity A32). For example, the disaster damage assessment server 10 determines the extent of damage to the building based on the inclination of the building's walls or foundation as shown in the generated 3D data. Specifically, the disaster damage assessment server 10 determines that the greater the inclination of the building's walls or foundation, the greater the extent of damage. The inclination of the building's walls or foundation is calculated as an angle of inclination relative to the vertical.

[0048] Furthermore, the disaster damage assessment server 10 determines the extent of damage to the building based on the displacement of the building's roof, walls, or foundation as indicated by the generated 3D data. For example, the disaster damage assessment server 10 determines that the greater the displacement of the building's roof, walls, or foundation as indicated by the generated 3D data, the greater the extent of damage to the building. The disaster damage assessment server 10 calculates the building's displacement as the difference between the design position of the building, which is set based on the building drawings included in the building information acquired in A16, and the current position indicated by the 3D data. Alternatively, the disaster damage assessment server 10 may calculate the building's displacement based on reference parts set from the 3D data or relative positional relationships with respect to the vertical direction, without using drawings.

[0049] When using an AI module, the disaster assessment server 10 inputs, for example, the generated 3D data of the building as input data to the AI ​​module, and determines the state of damage to the building based on the output of the AI ​​module. The AI ​​module is, for example, a model that has been pre-trained using training data to which the state of damage to the building has been assigned. Specifically, the AI ​​module extracts feature quantities related to the position, shape, inclination, or displacement of each part of the building, such as walls, roofs, and foundations, from the point cloud data or mesh data included in the 3D data. Then, based on the extracted feature quantities, the AI ​​module estimates the state of damage to the entire building or to each part, and outputs a determination result indicating whether or not there is damage or the degree of damage.

[0050] In this way, by using the AI ​​module, it is possible to determine the state of damage by comprehensively considering the three-dimensional geometric information of the building shown in the 3D data, and it becomes possible to evaluate complex shape changes and damage that spans multiple parts with high accuracy.

[0051] On the other hand, when using a rule-based algorithm, the disaster damage assessment server 10 may determine the state of damage to a building by evaluating the inclination angle, displacement, or shape change amount calculated from 3D data based on pre-set judgment rules. For example, the disaster damage assessment server 10 may use rules that determine "minor damage" if the inclination angle of the building's wall or foundation relative to the vertical is less than a first threshold, "moderate damage" if it is greater than or equal to the first threshold but less than a second threshold, and "major damage" if it is greater than or equal to the second threshold.

[0052] Furthermore, the disaster damage assessment server 10 may determine that there is damage to a roof, wall, or foundation if the displacement in the height or horizontal direction exceeds a predetermined standard value, and may use a rule to determine the degree of damage in stages according to the displacement. In addition, the disaster damage assessment server 10 may use a complex judgment rule such that if both the tilt angle and the displacement exceed a predetermined threshold, the damage is determined to be more severe than if only one indicator exceeds the threshold.

[0053] The disaster assessment server 10 generates a damage status screen showing the building's damage status using the damage status determination results (Activity A33). Then, the disaster assessment server 10 displays the generated damage status screen on the user terminal 30 (Activity A34).

[0054] Figure 8 shows an example of a displayed damage status screen. The damage status screen G4 in Figure 8 displays an example of the results of evaluating the damage status of each part of the building based on the building's 3D data. The upper part of Figure 8 shows the correspondence between the generated 3D model of the building and the multiple planes (hereinafter referred to as "planes") that constitute the 3D model. Each plane represents a specific part of the building, such as a wall or roof.

[0055] The upper left of Figure 8 shows schematic diagram E41, which divides the 3D model of the building into planar units. For example, plane00, plane01, and plane02 each correspond to different wall or roof surfaces. The upper right of Figure 8 shows model D41, which is a 3D reconstruction of the actual building, allowing for visual confirmation of which part of the building each plane corresponds to.

[0056] The lower part of Figure 8 shows evaluation table C41, which displays the damage evaluation results for each of the aforementioned planes. Each row in evaluation table C41 corresponds to each plane, such as plane00, plane01, and plane02, and each column shows information regarding the accuracy index of damage detection and the damage ratio. In evaluation table C41, IoU (Intersection over Union) is an index that shows the degree of agreement between the detected damage area and the correct area, and is calculated for each type of damage, such as window, spall, and crack. A larger IoU value indicates that a higher detection accuracy has been obtained for that type of damage. If there is no corresponding damage, IoU is not calculated and may be displayed as NaN in the evaluation table.

[0057] Furthermore, evaluation table C41 shows the damage ratio based on the ground truth data (GT Damage Ratio) and the damage ratio estimated by 3D data analysis or AI (Pred Damage Ratio). In addition, the difference between these two (Pred-GT) is shown, allowing for a quantitative assessment of whether the estimated result is overestimated or underestimated compared to the ground truth value.

[0058] In this way, the disaster damage assessment server 10 divides the building's 3D data into plane units and quantitatively evaluates the type and percentage of damage for each plane. This makes it possible to grasp the damage status of not only the entire building but also each part with high accuracy, and can be effectively used for subsequent damage status determination, input into application forms, or presentation of the basis for determination.

[0059] Figure 9 shows another example of the displayed damage status screen. The damage status screen G5 shown in Figure 9 displays a model D51 that is a 3D reconstruction of the actual building. Damage locations E51, E52, and E53 are displayed on model D51. Damage locations E51, E52, and E53 indicate areas that deviate from the standard building shape (building shape calculated from drawings or 3D data) by a certain amount or more. By looking at the damage status screen G5, the user can intuitively understand the damage locations.

[0060] Furthermore, on the damage status screen G5, the assessment results for damaged areas E51, E52, and E53 may be superimposed. For example, for each damaged area, numerical information, symbols, or string information indicating the degree of damage, such as the amount of tilt of the wall or foundation, the amount of displacement of the roof, wall, or foundation, may be superimposed near the corresponding area on the model D51.

[0061] In this way, by overlaying the assessment results with the damaged areas, users can visually and quantitatively understand which parts are tilted or damaged to what extent. Furthermore, the overlaid assessment results may be toggled on or off according to the user's operation, and users may be able to select between detailed or simplified display.

[0062] Next, the disaster damage assessment server 10 generates an assessment summary screen using the damage status determination results (Activity A35). The assessment summary screen is a screen that summarizes the contents of the disaster damage assessment. The disaster damage assessment server 10 displays the generated assessment summary screen on the user terminal 30 (Activity A36).

[0063] Figure 10 shows an example of a displayed evaluation summary screen. In the evaluation summary screen G6 shown in Figure 10, the evaluation summary C61 is displayed. The evaluation summary screen G6 displays information that allows for an overview of the results and progress of the disaster damage assessment. Specifically, the evaluation summary C61 aggregates and displays information about the building being investigated, the survey results, the progress of the survey, and the history.

[0064] The evaluation summary C61 displays the <Information of Survey Subjects>, including the address, name, and owner information of the building being surveyed. The <Survey Results> section displays whether it is a primary, secondary, or re-survey, whether the survey method is AI evaluation or staff evaluation, and the confidence level corresponding to the assessment of the damage condition. This allows for an understanding of the content of the assessment results and their reliability.

[0065] The <Investigation Status> section displays the progress of the investigation, such as "Not Investigated," "Initial Investigation Completed," or "Disaster Victim Certificate Issued." This allows users to see at a glance what stage the case is currently at. The <Investigation Log> section displays the identification information of the investigator who conducted the investigation, the date and time of the investigation, and the investigation history in chronological order. This makes it possible to review the progress of the investigation and the history of responses at a later date. Furthermore, the <Investigation Comments> section allows users to input or display supplementary explanations, special notes, or precautions regarding the investigation. In addition, the <Evaluation / Approval Log> may display historical information regarding the confirmation or approval of evaluation results.

[0066] In this way, by using the evaluation summary screen G6, information related to disaster damage assessment can be centrally managed and confirmed, and users, investigators, or administrators can grasp the evaluation content and streamline the procedures. The string "Primary Survey" in <Survey Results> is the operation image B61 for displaying the application form screen of the application form that includes the results of the "Primary Survey". When operation image B61 is operated, the user terminal 30 requests the disaster damage assessment server 10 to display the application form screen.

[0067] Upon receiving this request, the disaster assessment server 10 generates an application form screen using the damage assessment results (Activity A37). The application form is a form for applying for or reporting the damage status of a disaster-stricken building, and may be an application for a disaster certificate, a declaration of damage to a building for property tax purposes, or a damage claim form for fire insurance or earthquake insurance. The application form includes string information indicating the building's location, building type, and the assessed damage status. The application form also has fields for entering the location of the damage, the extent of the damage, or the damage classification. The disaster assessment server 10 displays the generated application form screen on the user terminal 30 (Activity A38).

[0068] Figure 11 shows an example of the application form screen. Application form C71 is displayed on application form screen G7 in Figure 11. The date and time of the survey are entered, which is the date and time when the building images were taken. The address and head of household are entered, for example, based on the housing information obtained in A16. In addition, application form screen G7 has fields for entering the damage status of each item: "exterior," "slope," "structure," "foundation," "walls," and "roof."

[0069] In the "Exterior" input field, the disaster assessment server 10 inputs the damage status of the building's exterior determined based on the generated 3D data. For example, the disaster assessment server 10 inputs the damage status of the exterior based on the presence and extent of irregularities, steps, cracks, or missing exterior materials on the exterior wall surface as shown by the 3D data. Specifically, the disaster assessment server 10 inputs that there is damage to the exterior if the amount of disturbance or step in the three-dimensional shape of the exterior wall surface exceeds a predetermined standard value.

[0070] In the "slope" input field, the disaster damage assessment server 10 inputs based on the inclination of the entire building or its main structural components as shown by the generated 3D data. For example, the disaster damage assessment server 10 calculates the angle of inclination of a wall or foundation relative to the vertical direction from the 3D data and inputs that the larger the angle of inclination, the greater the damage caused by the inclination. The disaster damage assessment server 10 may also input the amount of inclination as a numerical value or as a predetermined category (e.g., "large," "medium," "small" inclination).

[0071] In the "Structural Frame" input field, the disaster damage assessment server 10 inputs the damage status of the building's structural frame, such as columns and beams. The disaster damage assessment server 10 determines the damage status of the structural frame based on, for example, the displacement, bending, or deformation of columns or beams indicated by 3D data, and inputs the result. Specifically, the disaster damage assessment server 10 inputs that there is damage to the structural frame if the three-dimensional position of a structural member is displaced by a predetermined amount or more from its reference position.

[0072] In the "Foundation" input field, the disaster damage assessment server 10 inputs the damage status of the building's foundation or base. The disaster damage assessment server 10 determines the damage status of the foundation based, for example, on the amount of settlement, differential settlement, or tilt of the foundation as shown by the 3D data. If the displacement of the foundation in the height direction or horizontal direction is large, the disaster damage assessment server 10 inputs that the damage status of the foundation is large.

[0073] In the "Wall" input field, the disaster assessment server 10 inputs the damage status of the wall surface as indicated by the generated 3D data. The disaster assessment server 10 determines the damage status of the wall based, for example, on the inclination of the wall surface relative to the vertical direction, the out-of-plane tilt of the wall surface, or the amount of step difference of the wall surface, and inputs the result. The disaster assessment server 10 may also classify and input the degree of damage according to the inclination angle or displacement of the wall surface.

[0074] In the "Roof" input field, the disaster assessment server 10 inputs the damage status of the roof portion as shown by the generated 3D data. The disaster assessment server 10 determines the damage status of the roof based, for example, on the amount of settlement of the roof surface, localized distortion, or the amount of displacement of the ridge. If the amount of displacement in the height direction of the roof or the change in the shape of the roof surface is large, the disaster assessment server 10 inputs that the damage status of the roof is large.

[0075] Next, the user terminal 30 accepts user modifications to each input field (Activity A41). The user reviews the damage status assessment results displayed in A34 and modifies the application form displayed in A38 as needed. The damage status screen G4 shown in Figure 8, the damage status screen G5 shown in Figure 9, and the application form screen G7 shown in Figure 11 can be switched sequentially. When the user terminal 30 accepts the application form confirmation operation (Activity A42), it sends the confirmed application form to the disaster assessment server 10.

[0076] The disaster assessment server 10 submits the submitted application to a designated recipient (Activity A43). The designated recipient could be, for example, the server of the department responsible for issuing disaster certificates in a local government, the insurance claim acceptance server of an insurance company, a server managed by a building management company, or an information processing device used by the application agent. In this way, the extent of damage to the affected building is determined, and an application reflecting the assessment results is submitted.

[0077] 4. Summary of the structure The parts and steps described below are in a corresponding relationship with one another. That is, each step shows the procedure or example of operation of the processing performed by each part, and each part is an example of a functional module that is functionally realized when the processor of the control unit 101 of the disaster damage assessment server 10 executes a program stored in the storage unit 102.

[0078] <Determining the state of damage using 3D data> As described above, the disaster assessment server 10 acquires one or more images of the damaged building. In the example above, the disaster assessment server 10 acquires an image of the building taken from the user terminal 30 at A24 in Figure 4. A24 is an example of a building image acquisition step executed by the disaster assessment server 10 as an example of a building image acquisition unit. The images acquired in A24 are moving images or still images taken in sequence.

[0079] Next, the disaster damage assessment server 10 generates 3D data representing the building in three-dimensional space based on one or more acquired images. In the example above, the disaster damage assessment server 10 generates 3D data of the building based on one or more acquired images of the building in A31 of Figure 4. A31 is an example of a generation step executed by the disaster damage assessment server 10 as an example of a generation unit.

[0080] The disaster damage assessment server 10 then determines the extent of damage to the building based on the generated 3D data. In the example above, the disaster damage assessment server 10 determines the extent of damage to the building based on the generated 3D data at A32 in Figure 4. A32 is an example of a determination step executed by the disaster damage assessment server 10 as an example of a determination unit.

[0081] In this configuration, since the 3D data includes three-dimensional geometric information such as the position, shape, and inclination of each part of the building, it becomes possible to quantitatively grasp the displacement and inclination of damaged areas compared to using only 2D images, and to determine the state of damage with higher accuracy. Furthermore, by determining the state of damage based on 3D data, the results are less affected by differences in shooting conditions and viewpoints, improving the objectivity and reproducibility of the determination results. Moreover, in this configuration, even when it is difficult to acquire multiple images, 3D data can be generated from a single image to determine the state of damage, thus reducing the burden of shooting. This makes it possible to determine the state of damage even in situations where the shooting environment is limited immediately after a disaster, or in situations where a rapid initial assessment is required.

[0082] In detail, the disaster assessment server 10 (an example of a judgment unit) determines the state of damage to a building based on the tilt of the building's walls or foundations indicated by the generated 3D data. This is because the tilt of walls or foundations is an indicator that directly reflects ground subsidence, foundation failure, and structural strength reduction, and has a strong correlation with the state of damage to a building and the risk of collapse. With this configuration, damage with a high risk of collapse can be determined with high accuracy.

[0083] Furthermore, by determining the inclination of a wall or foundation based on 3D data, the angle of inclination relative to the vertical can be quantitatively calculated regardless of the shooting direction or viewpoint. Therefore, compared to visual inspection or assessment using 2D images, it is possible to perform damage condition assessment with higher objectivity and reproducibility. Moreover, by determining the damage condition based on the inclination of a wall or foundation, structurally dangerous damage can be detected early, even if the visible damage is minor, contributing to safety assurance and rapid response decisions.

[0084] In more detail, the disaster assessment server 10 (an example of a judgment unit) determines the state of damage to a building based on the displacement of the building's roof, walls, or foundation indicated by the generated 3D data. This is because the roof displacement represents the amount of roof settlement, local distortion of the roof surface, and ridge displacement, the wall displacement represents horizontal displacement (pushing), out-of-plane tilting, and steps due to cracks, and the foundation displacement represents sinking due to ground subsidence, tilt (overall inclination), and distortion of the foundation.

[0085] This configuration allows for highly accurate determination of three-dimensional damage, which is difficult to determine with two-dimensional image analysis. Furthermore, by determining displacement based on 3D data, the position of the roof, walls, or foundation can be obtained as three-dimensional coordinates, allowing for quantitative understanding of the displacement amount. This makes it possible to objectively and comparatively evaluate the extent of the damage. In addition, damage that is not easily noticeable from the outside, such as local distortion of the roof or walls, or tilting in the out-of-plane direction, can be detected as changes in 3D shape, allowing for the determination of the damage state, including damage that is easily overlooked in two-dimensional images. Moreover, by using both tilt-based and displacement-based determinations in combination, it is possible to perform a more accurate damage state determination that reflects both the overall deformation state of the building and the local deformation state.

[0086] <Comparison with the drawing> Furthermore, the disaster damage assessment server 10 acquires building drawings. In the example above, the disaster damage assessment server 10 acquires building information, including building drawings, from the building information server 20 at A16 in Figure 4. A16 is an example of a drawing acquisition step executed by the disaster damage assessment server 10 as an example of a drawing acquisition unit.

[0087] The disaster damage assessment server 10 (an example of a judgment unit) then compares the generated 3D data with the acquired drawings to determine the extent of damage to the building. This is because the building drawings show the original shape and structure, and by comparing them with the 3D data, it is possible to quantitatively evaluate the degree of deviation from the original design state (positional displacement, deformation of shape, tilt, and settlement, etc.).

[0088] This configuration allows for a more precise determination of the damage state. Furthermore, by comparing 3D data with drawings as a reference, the influence of individual differences between buildings and differences in shooting conditions can be suppressed, and the damage state can be determined based on the design state specific to that building. In addition, by evaluating the damage state as a difference from the original design shape, even minor deformations and early-stage damage that are not easily noticeable externally can be detected as numerical deviations.

[0089] <Indication of damaged area> Furthermore, the disaster assessment server 10 displays a 3D model image of the building indicated by the generated 3D data. In this case, the disaster assessment server 10 displays the damaged areas in a different manner from other areas in the 3D model image. This display process is an example of a 3D display step executed by the disaster assessment server 10 as an example of a 3D display unit.

[0090] In the example above, the disaster assessment server 10 displays damaged areas E51, E52, and E53 in a different manner (different color) on the damage status screen G5 shown in Figure 9. Note that the different manner is not limited to this; for example, it could be blinking, highlighted, or have a different texture.

[0091] This configuration allows for an intuitive understanding of the damaged areas. Furthermore, by displaying the damaged areas on a 3D model image, the spatial relationships and extent of the damage can be grasped three-dimensionally, making it easy to understand the distribution of damaged areas within the entire building. In addition, even spatial relationships in the depth direction and deformations in the out-of-plane direction, which are difficult to grasp with 2D images, can be understood while suppressing misidentification by displaying the damaged areas on the 3D model image.

[0092] <Output of judgment result> Furthermore, the disaster damage assessment server 10 displays string information indicating the determined damage status in an editable format on the application form for reporting the building's damage status. In the above example, the disaster damage assessment server 10 displays string information indicating the building's damage status in an editable format on the application form, as shown in A35 and A36 of Figure 4, similar to the application form screen G7 in Figure 11. A35 and A36 are examples of judgment result display steps executed by the disaster damage assessment server 10 as an example of a judgment result display unit.

[0093] This configuration allows the determined damage status to be automatically reflected in the application form, thereby reducing the manual data entry required to create the application form and improving the efficiency of the application process. Furthermore, since it is editable, users can modify the contents of the application form. Additionally, by automatically reflecting the determination results as text information in the application form, transcription errors and omissions of the determination results can be suppressed, ensuring consistency between the determination and the application content.

[0094] <Difference: Comparison with past images> Building information is not limited to the above information (building drawings). The disaster damage assessment server 10 may acquire one or more past images of the building taken in the past as building information. The acquisition process in this case is an example of a past image acquisition step executed by the disaster damage assessment server 10 as an example of a past image acquisition unit.

[0095] Past images may include, for example, images taken routinely by the building owner or resident, or images taken during inspections by the building management company or property management company. They may also include images taken by insurance companies at the time of insurance contract or renewal, or images taken during surveys by local governments or public institutions. Furthermore, they may include record images taken at the time of construction, renovation, or sale of the building.

[0096] Next, the disaster damage assessment server 10 (an example of a generation unit) generates historical 3D data based on one or more acquired historical images. The disaster damage assessment server 10 generates 3D data of a building based on one or more images of the building, similar to A31 shown in Figure 4.

[0097] The disaster damage assessment server 10 (an example of a determination unit) then compares the generated current 3D data with the generated past 3D data to determine the extent of damage to the building. Specifically, the disaster damage assessment server 10 aligns the current 3D data with the past 3D data and calculates the positional difference, tilt difference, or shape difference of the building's roof, walls, or foundation. If these differences exceed a predetermined threshold, it determines that damage has occurred in that part. The differences may be calculated as the amount of displacement in the height direction, the amount of displacement in the horizontal direction, the amount of change in the tilt angle relative to the vertical direction, etc.

[0098] This embodiment allows for the determination of damage conditions that more accurately reflect the actual building. Furthermore, this embodiment allows for the determination of damage conditions based on past images, even when building drawings do not exist, making it applicable to older buildings or buildings for which drawings have been lost. Moreover, by using past 3D data as a basis, it is possible to determine the damage conditions while considering the building's unique shape and changes over time, enabling an evaluation that more accurately reflects the actual condition of the building compared to a comparison with design values.

[0099] <Variation example: Confidence level> The disaster damage assessment server 10 may display information other than the determination result of the building's damage status. For example, the disaster damage assessment server 10 (an example of a determination result display unit) may further display the confidence level of the damage status indicated by the string information. The confidence level of the damage status is an index that indicates the likelihood of the determination result of the damage status. The disaster damage assessment server 10 may, for example, calculate the confidence level of the damage status based on the output result of the AI ​​model used in the damage status determination process based on 3D data.

[0100] Specifically, the disaster assessment server 10 can use the estimated probability, likelihood, or score value for each damage state as the confidence level. The disaster assessment server 10 may also calculate the confidence level by considering the number of input images, the diversity of shooting directions, the accuracy of 3D data generation, or the consistency of damage detection results. Furthermore, the disaster assessment server 10 may calculate the confidence level based on the degree of agreement between multiple judgment results, consistency with past data, or variability of judgment results.

[0101] In this configuration, the user can grasp the likelihood of the displayed damage assessment result, making it easier to decide whether to adopt the assessment result as is or modify it in the application compared to when the confidence level is not displayed. As a result, the user can simplify the verification process for assessment results with a high confidence level and focus on verifying or modifying those with a low confidence level, thereby improving the overall efficiency of the application preparation process. Furthermore, by displaying the confidence level, the reliability of the damage assessment result can be presented as supplementary information to the recipient of the application, making it easier to understand and verify the application content.

[0102] <Variation: Basis for judgment> Information other than the judgment result is not limited to the confidence level mentioned above. The disaster assessment server 10 (an example of a judgment result display unit) may also display supporting information that shows the basis for the judgment of the damage state indicated by the string information.

[0103] Figure 12 shows an example of a displayed evidence information screen. The evidence information screen G8 in Figure 12 displays the evaluation details and evidence information C81 showing the basis for the evaluation. The evidence information screen G8 is a screen that presents the user with specific reasons for the damage condition determination result. The <Evaluation Basis Summary> of evidence information C81 displays an overview of the damage condition determined for the building, as well as the main reasons that led to that determination, as text or summary information. This allows the user to easily grasp the overall picture of the damage condition determination result.

[0104] The <Evaluation Basis Photos> section displays images of the building used to determine the extent of damage, or images showing the areas where damage was detected. These images may be user-submitted or generated from 3D data, and may be displayed in a manner that makes the damaged areas easily visible. The <Evaluation Criteria Compliance> section displays a list of evaluation items, evaluation criteria, evaluation scores, and evaluation basis used to determine the extent of damage. In addition, the confidence level of the AI's judgment result is displayed for each evaluation item. This allows the user to see which evaluation criteria were used for each evaluation item and what score or evaluation it received.

[0105] The <Comments> section allows users to input or display supplementary explanations, notes, or observations regarding the judgment result. The <Correction History> section displays the history of any corrections made to the judgment result or evaluation, along with the date and time, the person who made the correction, and the content before and after the correction. By displaying the basis information screen G8 in this way, the transparency and explainability of the damage condition judgment result can be improved, allowing users to objectively assess the validity of the judgment result.

[0106] The supporting information is not limited to the information described above. The disaster damage assessment server 10 may, for example, display the angle of inclination of the building's walls or foundations relative to the vertical direction, or the amount of displacement of the roof, walls, or foundations in the height or horizontal direction, as supporting information, calculated from the generated 3D data.

[0107] Furthermore, the disaster damage assessment server 10 may display the difference between the design position set based on the building drawings and the current position shown by the 3D data as supporting information, or it may display the difference between the position shown by past 3D data and the position shown by current 3D data as supporting information. In addition, the disaster damage assessment server 10 may display numerical information, graphs, or highlighted 3D model images (for example, model D51 on the damage status screen G5 shown in Figure 9) as supporting information for each part where damage has been detected.

[0108] This configuration makes it easier to judge the validity of the damage status compared to when the supporting information is not displayed. Furthermore, by displaying specific numerical or differential information that serves as the basis for determining the damage status, users can objectively understand the reasoning behind the determination. This makes it easier for users to clearly explain the reasons for any corrections they make to the determination result, improving the explainability of the application. In addition, since the basis for determining the damage status can be presented to the recipient of the application, the verification process for the application content is made easier, and the application processing is streamlined.

[0109] <Differentiation: Corrective learning> As described above, the disaster damage assessment server 10 (an example of a determination unit) may be configured to determine the damage state based on the output of an AI module that has received 3D data as input. In that case, the disaster damage assessment server 10 may have the AI ​​module learn the damage state edited by the user in the application form. This learning process is an example of a learning step that the disaster damage assessment server 10 executes as an example of a learning unit.

[0110] For example, the disaster assessment server 10 may perform learning by updating the parameters of the AI ​​module using data that associates 3D data with the damage state corrected by the user for that 3D data as training data. The disaster assessment server 10 may also perform the above learning as online learning or batch learning, or it may be performed all at once after a certain number of correction results have been accumulated.

[0111] In this configuration, the AI ​​module's judgment model can be adaptively improved based on user corrections, enabling damage state determination that closely resembles human judgment. Furthermore, individual or regional differences in judgment criteria can be reflected through learning, making it possible to determine damage state in a way that is adapted to the judgment tendencies of users and operators. In addition, as learning data accumulates with repeated use of the system, the accuracy of damage state determination can be continuously improved.

[0112] <Variation: Image of a building> The building image is not limited to images taken by the user terminal 30, i.e., photographs. In addition to two-dimensional images taken by a visible light camera, the building image may also include data acquired by a distance measurement sensor capable of acquiring three-dimensional information of buildings, etc., using infrared or laser light reflection.

[0113] For example, the building image may be point cloud data acquired using a smartphone, tablet, or handheld LiDAR sensor equipped with a LiDAR sensor. The disaster damage assessment server 10 may directly use such point cloud data as part or all of the 3D data representing the building, or it may integrate the point cloud data with images captured by a camera to generate 3D data.

[0114] According to this configuration, compared to using only visible light images, the system is less affected by the shooting environment, can acquire the building's shape, tilt, or displacement with higher accuracy, and enables stable determination of the damage state.

[0115] <Variation: Use of metadata> The information used to generate 3D data of a building is not limited to video information such as still images and videos. The disaster damage assessment server 10 may generate 3D data using various metadata acquired by the camera during shooting, in addition to video information.

[0116] Metadata may include, for example, location information indicating the shooting location, angle information indicating the shooting direction or orientation, shooting time, shooting speed, type of shooting equipment, lens information, exposure time, focal length, or other shooting setting information. The disaster damage assessment server 10 may use this metadata to improve the accuracy of estimating the relative positional relationship between images or the camera orientation, and improve the accuracy of corresponding point search, camera orientation estimation, or point cloud generation when generating 3D data.

[0117] This configuration improves the accuracy and stability of 3D data generation compared to using only video information, and as a result, makes it possible to understand the shape, inclination, or displacement of a building with greater precision.

[0118] <Variation example: Self-assessment method> Regarding the output of the assessment results, the system may be configured to output a message indicating that a self-assessment method, which allows for the omission of on-site inspections, can be applied if the building damage is determined to be minor. The self-assessment method is a method in which a user, such as a resident, assesses the damage to a building based on images of the building taken by the user themselves and submits an application based on the assessment result. The disaster assessment server 10 performs the aforementioned damage assessment process based on the building images transmitted from the user terminal 30, and determines whether the self-assessment method is appropriate based on the result. The criteria for determining whether the self-assessment method is applicable may be changed according to the criteria set by each local government or system.

[0119] For example, the disaster damage assessment server 10 may output one of the following judgment results based on the judgment result of the building damage status. (1) If it is determined that there are no problems as per the self-assessment application (2) When the damage to the building is minor and it is determined that it does not fall under the scope of self-assessment. (3) When the damage to the building is severe and it is determined that an on-site inspection and assessment by the local government or other relevant body is necessary, rather than relying on a self-assessment method.

[0120] The disaster assessment server 10 may output to the user terminal 30, as text information or a screen display, whether it is possible to apply using the self-assessment method, whether an application is not required, or whether an on-site inspection by the local government is necessary, depending on the above assessment result.

[0121] This configuration reduces the burden on local government officials for on-site inspections when the damage is minor, and also enables swift processing of applications by residents. On the other hand, when the damage is severe, the application of the self-assessment method can be restricted, and appropriate on-site inspections can be guided, thereby ensuring fairness and reliability of the assessment.

[0122] <Modification example: Automatic building identification> The disaster damage assessment server 10 may automatically identify which building a photographed corresponds to. For example, the disaster damage assessment server 10 automatically identifies a building based on location information and captured images obtained from the user terminal 30. Specifically, the disaster damage assessment server 10 identifies the shooting location based on GPS information obtained from the user terminal 30 and extracts candidate buildings located around that shooting location. Alternatively, the disaster damage assessment server 10 may identify the target building by comparing the photographed building's exterior features, roof shape, window arrangement, building contour shape, or surrounding environment with building information registered in the building information DB. If the building identification is narrowed down to multiple candidates, the system may be configured to present the candidates to the user and allow them to select one.

[0123] The disaster damage assessment server 10 acquires building information corresponding to the identified building, such as the building's address, real estate ID, fixed asset number, building management number, or similar identification information, and automatically enters this identification information into the application form or assessment report. This configuration reduces the effort required for users to manually enter building information, prevents misidentification of buildings or errors in description, and improves the accuracy and efficiency of disaster damage assessment and application procedures.

[0124] <Variations: Variations in composition> The configuration shown in Figure 1, etc. (overall configuration, hardware configuration, and functional configuration, etc.) is just an example, and other configurations can be used as long as they do not cause inconvenience in implementation. For example, a client terminal called a user terminal 30 may perform some of the processing that a server device such as the disaster damage assessment server 10 performs, or a server device may perform some of the processing that a client terminal performs. Also, processing that is performed in one activity may be performed in two or more activities, or processing that is performed in two or more activities may be performed in one activity. In short, as long as the necessary information processing is performed throughout the disaster damage assessment system 1, the devices that perform that information processing are not limited.

[0125] The output destination for information or data (hereinafter referred to as "information, etc.") may be other devices, displays, storage units (including built-in and external storage units), email addresses, or accounts in other systems. Acquisition of information, etc. includes not only acquiring information, etc. transmitted from other devices, but also acquiring information, etc. generated by the device itself or information, etc. stored by the device itself. The database may be stored by an information processing device such as the disaster assessment server 10, or it may be stored in an external device. Furthermore, the information stored in the database may be used in a manner that temporarily stores it in memory.

[0126] The embodiments described above included an information processing system (disaster damage assessment system 1) comprising a server device equipped with a processor (disaster damage assessment server 10, etc.) and a terminal (user terminal 30, etc.) that can access the server device, but are not limited to this. The embodiments described above may also be, for example, an information processing method. The information processing method comprises steps executed by one or more processors in the information processing system. Furthermore, the embodiments described above may also be a program. The program causes a computer to execute the same steps as those executed by the information processing system.

[0127] <Note> Furthermore, they may be provided in the following embodiments.

[0128] (1) An information processing system comprising one or more processors, wherein the processors, in a building image acquisition step, acquire one or more images of a damaged building; in a generation step, generate 3D data representing the building in three-dimensional space based on the acquired one or more images; and in a determination step, determine the state of damage to the building based on the generated 3D data.

[0129] According to this embodiment, the state of damage can be determined with high accuracy.

[0130] (2) An information processing system as described in (1) above, wherein the processor determines the state of damage to the building based on the inclination of the building's walls or foundation as indicated by the generated 3D data in the determination step.

[0131] This configuration allows for highly accurate determination of damage that poses a high risk of collapse.

[0132] (3) An information processing system according to (1) or (2) above, wherein the processor determines the state of damage to the building based on the displacement of the roof, walls, or foundation of the building indicated by the generated 3D data in the determination step.

[0133] According to this configuration, it is possible to accurately determine the three-dimensional state of damage, which was difficult to determine with two-dimensional image analysis.

[0134] (4) An information processing system according to any one of (1) to (3) above, further comprising a drawing acquisition step, wherein the processor acquires a drawing of the building in the drawing acquisition step, and in the determination step compares the generated 3D data with the acquired drawing to determine the state of damage to the building.

[0135] This configuration allows for a more precise determination of the damage state.

[0136] (5) An information processing system according to any one of (1) to (4) above, further comprising a past image acquisition step, wherein the processor acquires one or more past images of the building taken in the past in the past image acquisition step, generates past 3D data based on the acquired one or more past images in the generation step, and determines the damage state of the building in the determination step by comparing the generated current 3D data with the generated past 3D data.

[0137] This configuration allows for the determination of damage conditions that more accurately reflect those of actual buildings.

[0138] (6) An information processing system according to any one of (1) to (5) above, further comprising a 3D display step, wherein the processor in the 3D display step displays a 3D model image of the building shown by the generated 3D data, and in the 3D model image displays the damaged parts in a manner different from other parts.

[0139] This configuration allows for an intuitive understanding of the damaged area.

[0140] (7) An information processing system according to any one of (1) to (6) above, further comprising a determination result display step, wherein the processor in the determination result display step displays string information indicating the determined damage state in an editable manner on an application form for reporting the damage state of the building.

[0141] This configuration makes it possible to reduce the manual data entry work required to prepare application forms.

[0142] (8) An information processing system as described in (7) above, wherein the processor further displays the degree of confidence of the corruption state indicated by the string information in the determination result display step.

[0143] With this configuration, it is easy to determine whether to adopt the judgment result as is or to modify it in the application.

[0144] (9) An information processing system according to either (7) or (8) above, wherein the processor further displays, in the determination result display step, basis information indicating the basis for determining the damaged state indicated by the string information.

[0145] This configuration makes it easier to determine the appropriateness of the damage.

[0146] (10) An information processing system according to any one of (7) to (9) above, further comprising a learning step, wherein the processor determines the damage state based on the output of an AI module that has been input with the 3D data in the determination step, and causes the AI ​​module to learn the damage state edited in the application form in the learning step.

[0147] This configuration makes it possible to determine the state of damage in a way that closely resembles human judgment.

[0148] (11) An information processing system according to any one of (1) to (10) above, comprising a server device equipped with the processor and a terminal that can access the server device.

[0149] According to this configuration, the information processing system can be implemented in various ways.

[0150] (12) An information processing method comprising each of the steps described in any one of (1) to (10) above, which is performed by a processor.

[0151] According to this embodiment, one embodiment of the information processing method can be provided.

[0152] (13) A program that causes a computer to perform any one of the steps described in (1) to (10) above.

[0153] According to this embodiment, one embodiment can be provided in the form of a program. Of course, this is not always the case. Furthermore, the embodiments and modifications described above may be implemented in any combination.

[0154] Finally, various embodiments of the present invention have been described, but these are presented as examples only and are not intended to limit the scope of the invention. Novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0155] 1: Disaster Damage Assessment System 2: Communication lines 10: Disaster Damage Assessment Server 20: Building Information Server 30: User terminal 101: Control Unit 201: Control Unit 301: Control Unit

Claims

1. An information processing system comprising one or more processors, wherein the processors In the building image acquisition step, one or more actual images of the damaged building are acquired. In the generation step, based on the acquired one or more images, measured 3D data representing the building in three-dimensional space is generated. In the drawing acquisition step, the drawings of the aforementioned building are acquired. In the determination step, the current position of each part indicated by the measured 3D data and the design position set based on the drawing are compared based on geometric information, and the difference in coordinates between the current position and the design position is calculated as displacement to determine the state of damage to the building. Information processing system.

2. In the information processing system according to Claim 1, the processor is: In the determination step, the state of damage to the building is determined based on at least the displacement of the roof ridge, local distortion of the roof surface, wall step, or sinking of the foundation. Information processing system.

3. In the information processing system according to claim 1, the processor is In the determination step, the damage status of the building is determined based on the inclination of the building's walls or foundation as shown by the generated measured 3D data. Information processing system.

4. In the information processing system described in claim 1, The process further includes a step for acquiring past images, and the processor, In the aforementioned past image acquisition step, one or more past images of the building taken in the past are acquired. In the generation step, past measured 3D data is generated based on the acquired one or more past images. In the determination step, the current measured 3D data generated is compared with the past measured 3D data generated to determine the damage status of the building. Information processing system.

5. In the information processing system described in claim 1, The 3D display step is further provided, and the processor, In the 3D display step, a 3D model image is displayed in which the building is divided into multiple planar units, and for each planar unit, at least one of the determined type of damage, the percentage of damage, and the calculated displacement amount is superimposed on the 3D model image and displayed in association with that planar unit. Information processing system.

6. An information processing system comprising one or more processors, wherein the processors In the building image acquisition step, one or more images of the damaged building are acquired. In the generation step, based on the acquired one or more images, measured 3D data representing the building in three-dimensional space is generated. In the determination step, the damage status of the building is determined based on the output of the AI ​​module to which the generated measured 3D data is input. In the judgment result display step, the string information indicating the determined damage state is displayed in an editable manner on the application form for reporting the damage state of the building, and the confidence level of the damage state calculated based on the output result of the AI ​​module is further displayed in association with the application form as information to support the user's decision regarding the adoption or modification of the string information. Information processing system.

7. In the information processing system described in claim 1, The process further includes a step of displaying the judgment result, wherein the processor In the judgment result display step, the string information indicating the determined damage state is displayed in an editable manner on the application form for reporting the damage state of the building. Information processing system.

8. In the information processing system according to claim 7, the processor is In the judgment result display step, the basis for the judgment of the damage state indicated by the string information is further displayed. Information processing system.

9. In the information processing system described in claim 7, The processor further includes a learning step, In the determination step, the damage state is determined based on the output of the AI ​​module that has received the measured 3D data. In the learning step, the AI ​​module is made to learn the damage state edited in the application form. Information processing system.

10. In the information processing system according to any one of claims 1 to 9, A server device equipped with the aforementioned processor, An information processing system comprising a terminal capable of accessing the aforementioned server device.

11. Information processing method, The processor comprises each step according to any one of claims 1 to 9, Information processing methods.

12. It is a program, Cause the computer to perform each of the steps described in any one of claims 1 to 9. program.

Images