Systems and methods for three-dimensional rendering of building structure outside walls to facilitate fire inspection

Abstract

Systems and methods for facilitating a fire inspection of a building structure. Multiple digital images (e.g., a video) of the outside walls of the building structure are processed to generate a three-dimensional (3D) sparse point cloud. An estimated perimeter of the building structure is determined based on the sparse point cloud and aligned with a geospatial perimeter to generate a geospatially-aligned 3D sparse point cloud. The geospatially-aligned cloud is then volume-rendered to generate a 3D reconstruction of the outside walls. One or more portions of this 3D reconstruction are displayed on a display screen, based on a position, orientation, and / or field of view (FOV) of an observation / inspection point relative to the building structure. A fire inspection may be performed based on respective displayed portions of the 3D reconstruction, in which one or more displayed objects that pose an ignition risk may be tagged and annotated for wildfire mitigation actions.

Description

Attorney Docket No. FWIG-002W001SYSTEMS AND METHODS FOR THREE-DIMENSIONAL RENDERING OF BUILDING STRUCTURE OUTSIDE WALLS TO FACILITATE FIRE INSPECTIONCross-Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 716,443, titled ‘"System and Method for Machine-Learning Based Feature Extraction in Sparse Point Cloud Reconstructions,” filed on November 5, 2024 and to U.S. Provisional Application No. 63 / 716,469, titled “Image Processing and User Interfaces for Virtual Fire Inspections,” filed on November 5, 2024. Each of the aforementioned provisional applications is hereby incorporated herein by reference.Background

[0002] Wildfires are an increasingly important factor for homeowners to consider in various parts of the world, including in North America. In high-risk areas, wildfires present a real danger of total loss of a building structure, such as a residential house.

[0003] Assessing a building structure and corresponding land for w ildfire requires that a skilled technician travel to the property7to take measurements of the building structure including those of structural features on the building structure, such as windows, doors, vents, soffits, and decks. The technician also needs to evaluate the condition of the structural features, for example to determine whether wildfire mitigation measures have been implemented to minimize wildfire risks. The technician also takes measurements of potential fuel sources on the property and their respective distances from the building structure. All of this data can then be used to assess the wildfire risk of the building structure. This is a timeintensive and expensive process for the technical to travel to the property and to perform the measurement w ork onsite.Summary

[0004] Example embodiments described herein have innovative features, no single one of which is indispensable or solely responsible for their desirable attributes. The following description and drawings set forth certain illustrative implementations of the disclosure in detail, which are indicative of several exemplary7w ays in which the variousAttorney Docket No. FWIG-002W001 principles of the disclosure may be carried out. The illustrative examples, however, are not exhaustive of the many possible embodiments of the disclosure. Without limiting the scope of the claims, some of the advantageous features will now be summarized. Other objects, advantages, and novel features of the disclosure will be set forth in the following detailed description of the disclosure when considered in conjunction with the drawings, which are intended to illustrate, not limit, the invention.Brief Description of the Drawings

[0005] For a fuller understanding of the nature and advantages of the concepts disclosed herein, reference is made to the detailed description and the accompanying drawings.

[0006] Fig. 1 is a flow chart of a computer-implemented method for generating a three-dimensional rendering of a building structure according to one or more embodiments.

[0007] Fig. 2 shows an example of a 3D sparse point cloud reconstruction of a building structure.

[0008] Fig. 3 is a flow chart of a method for determining an estimated perimeter of a building structure using a 3D sparse point cloud reconstruction.

[0009] Fig. 4 shows an example of an estimated perimeter determined from projecting high-density vertical columns of points onto a ground plane according to one or more embodiments.

[0010] Fig. 5 shows a simplified example of a volume-rendered scaled 3D sparse point cloud reconstruction of a building structure according to one or more embodiments.

[0011] Fig. 6 is a flow chart of a computer-implemented method for performing a virtual wildfire inspection of a building structure according to one or more embodiments.

[0012] Fig. 7A is an overhead view of the building structure and an observ ation / inspecti on point (e.g., of a virtual wildfire inspector) according to one or more embodiments.Attorney Docket No. FWIG-002W001

[0013] Fig. 7B is a simplified view of a display screen that displays a viewable portion of the building structure corresponding to or representing the position, field-of-view (FOV), and orientation of the observation / inspection point shown in Fig. 7A.

[0014] Fig. 8A is an overhead view of the building structure showing an example updated position and updated FOV of the observation / inspection point compared to the position and FOV of the observation / inspection point shown in Fig. 7A.

[0015] Fig. 8B is a simplified view of an updated viewable portion of the building structure on the display screen corresponding to or representing the position, FOV, and orientation of the observation / inspection point shown in Fig. 8A.

[0016] Fig. 9A is an overhead view of the building structure showing an example updated orientation of the observation / inspection point compared to the orientation of the observation / inspection point shown in Figs. 8A and 9A.

[0017] Fig. 9B is a simplified view of an updated viewable portion of the building structure on the display screen corresponding to or representing the position, FOV, and orientation of the observation / inspection point shown in Fig. 9A.

[0018] Fig. 10 is a flow chart of a computer-implemented method for performing a virtual wildfire inspection of a building structure according to one or more embodiments.

[0019] Fig. 11 shows the updated viewable portion of the building structure in Fig. 9B where the w ood pile has been tagged as a tagged object.

[0020] Fig. 12 is a block diagram representing the data for a tagged object that can be stored.

[0021] Fig. 13 is a flow chart of a computer-implemented method for performing a virtual wildfire inspection of a building structure according to one or more embodiments.

[0022] Fig. 14A shows an example of a virtual inspection report that includes a table.

[0023] Fig. 14B show s an example of a virtual inspection report that includes one or more sections.

[0024] Fig. 15 is a flow' chart of a computer-implemented method for detecting and / or segmenting structural features in digital images according to one or more embodiments.Attorney Docket No. FWIG-002W001

[0025] Fig. 16 is an example 3D sparse point cloud rendering of a building structure that includes masked colors for the structural features according to one or more embodiments

[0026] Fig. 17 is an example 3D sparse point cloud rendering of a building structure that includes bounding boxes corresponding to masked structural features according to one or more embodiments.

[0027] Fig. 18 is a simplified view of a display screen that displays a viewable portion of a building structure where each type of structural feature is masked in a different color according to one or more embodiments.

[0028] Fig. 19 is a flow chart of a method for remotely monitoring a building structure for compliance of wildfire mitigation actions.

[0029] Fig. 20 is a flow chart of a method for performing a virtual fire inspection according to one or more embodiments.

[0030] Fig. 21 shows an example overhead image of a property' including a building structure according to one or more embodiments.

[0031] Fig. 22 illustrates a networked computing system according to one or more embodiments.

[0032] Fig. 23 is an example block diagram of a computing device that may incorporate one or more embodiments of the present disclosure.Detailed Description

[0033] Images representing all outside walls of a building structure, including any structural features on / in the outside walls, are used to generate a three-dimensional (3D) computer rendering of the building structure. In one example implementation, respective images of outside walls of a building structure may be acquired as a video of the exterior of the building structure (e.g., taken with one or more cameras of a smartphone held by a person and directed at the exterior of the building structure as the person walks around the exterior of the building structure). The images are first transformed into a 3D sparse point cloud to form a 3D sparse point cloud reconstruction of the building structure. Such a 3D sparse point cloud reconstruction of the building structure is then volumetrically-rendered to form the 3DAttorney Docket No. FWIG-002W001 rendering. In one aspect, the 3D sparse point cloud is geospatially registered. In other aspects, a viewable portion of the 3D rendering can be displayed according to a position, a field-of-view, and / or an orientation of an observation / inspection point (e.g., corresponding to a location of a virtual wildfire inspector) relative to the building structure. The viewable portion of the 3D rendering can be iteratively updated as the position, the field-of-view, and / or the orientation of the observation / inspection point changes.

[0034] Objects such as structural features and / or fuel sources represented in the viewable portion of the 3D rendering can be tagged. A geospatial position of a tagged object is automatically stored with each tag. In one or more embodiments, an image of the tagged object can be automatically generated from the 3D rendering and stored. In one or more embodiments, an image representing a position of the tagged object relative to the building structure can be automatically generated and stored. A virtual inspection report can be created and generated. Compliance with the virtual inspection report can be audited by performing one or more (follow-up) virtual inspections using a new / updated 3D rendering generated from new images representing all outside walls of the building structure (e.g., after one or more risk mitigation actions have been taken associated with the building structure and its environs).

[0035] Fig. 1 is a flow chart of a computer-implemented method 10 for generating a three-dimensional rendering of a building structure according to one or more embodiments. Method 10 can be performed by one or more hardware-based processors in a computer for example by executing computer-readable instructions stored in the non-volatile memory of the computer.

[0036] In optional step 100, image data representing digital images that collectively represent or show all outside walls of a building structure are received. Though the digital images collectively represent all outside walls of a building structure, any one digital image may only represent one outside wall on the building structure or even a portion (e g., a segment) of one outside wall on the building structure. In one or more embodiments, some or all of the digital images can have a field-of-view sufficient to show the entire height (vertical span) of the structure.

[0037] The digital images can comprise or represent digital photographs (e.g., digital photo data) captured with a portable (e.g.. handheld) digital camera, such as a digital cameraAttorney Docket No. FWIG-002W001 in a portable computer such as smartphone or a tablet, or a dedicated digital camera. Additionally or alternatively, the digital images can comprise or represent digital video(s) (e.g., digital video data) and / or sampled digital video images (e.g., sampled digital video image data) from one or more digital videos. The digital video(s) can be captured with a portable (e.g., handheld) digital video camera such as a digital video camera in a portable computer such as smartphone or a tablet, or a dedicated digital video camera. Additionally or alternatively, some or all of the digital photographs and / or some or all of the digital videos can be captured with a digital camera and / or a digital video camera, respectively, that can be included in or mounted on a drone or robot. In one or more embodiments, the digital video(s) can be created while panning and / or phy sically moving the digital video camera up and / or along one or more sides and / or outside walls of the building structure, such as in a multi-story structure, to capture all or substantially all of the structural features on the outside walls. In one or more embodiments, a digital video can represent all the outside walls of a building structure while the digital video camera is moved in a single loop about the perimeter of the building structure. The digital video camera can be held by a person who walks in a single loop about the perimeter of the building structure to capture the digital video. Alternatively, the digital video camera can placed or mounted on / in a drone or robot that moves in a single loop about the perimeter of the building structure to capture the digital video.

[0038] The building structure can be or comprise a residential structure such as a house, a condominium, a townhouse, or a residential building, or can be or comprise a commercial structure such as an office building, a retail store, or a factory. The building structure is located on property that can include objects such as natural objects (e.g., trees, shrubs, grass, rocks, and / or other natural objects) and / or artificial (man-made) objects (e.g., vehicles, secondary structures such as a shed and / or a pool house, furniture, fences, signs, industrial storage containers, pavement, and / or other artificial objects).

[0039] The digital images can be received or provided from computer memory (e.g., non-volatile memory) in the computer or in another computer in communication the computer. In one or more embodiments, the digital images are received directly or indirectly from a portable computer, such as a smartphone, that includes a digital camera to capture the digital images.Attorney Docket No. FWIG-002W001

[0040] In step 101, a plurality7of digital images that collectively represent all outside walls of a building structure are transformed into a three-dimensional (3D) sparse point cloud representation (“reconstruction”) of the building structure.

[0041] By way of example, a sparse point cloud of the digital images can be created using sparse point cloud reconstruction and / or Structure-from-Motion methods such as COLMAP or Hierarchical Localization. COLMAP is a general-purpose Structure-from- Motion and Multi-View Stereo pipeline, created by Johannes L. Schoenberger, available at https: / / colmap.github.io / , which is hereby incorporated by reference. HierarchicalLocalization can be performed using hierarchical localization toolbox, a modular toolbox that leverages image retrieval and feature matching, created by Paul-Edouard Sarlin, available at https: / / github.com / cvg / Hierarchical-Localization, which is hereby incorporated by reference.

[0042] The inventors have recognized and appreciated, however, that various aspects of the digital images used to create a sparse point cloud can significantly improve the integrity and utility of the sparse point cloud for ultimately generating a 3D rendering of the building structure to facilitate a fire inspection, as set forth in further detail below.

[0043] For example, the respective digital images (e.g., or video) of the outside walls of the building structure should contain as much of the vertical span of a given outside wall of the building structure as possible (and preferably the entire vertical span of the outside wall). Accordingly, when the digital images are acquired (e.g., via a camera, smartphone or other image acquisition device), as much of the vertical span of the outside wall as possible should be kept within the viewing frame of the device (and optionally some additional space above and below the vertical span of the outside wall may be kept in the viewing frame).

[0044] Additionally, during image acquisition, quick movements or panning of the camera, smartphone or other image acquisition device should be avoided, so as to mitigate motion blur across multiple digital images. Furthermore, the digital images should be acquired by making at least one entire loop around the perimeter of the building structure so as to acquire digital images of all portions of all outside walls. In some implementations, at least one entire loop must be made around the perimeter of the building structure to ensure sufficient coverage of the digital images to effectively generate a sparse point cloud. In one aspect, the end point of a loop around the building may continue past the start point of the loop such that an ending portion of the loop overlaps with a starting portion of the loop; thisAttorney Docket No. FWIG-002W001 ensures that all of the outside walls of the building structure are represented in the digital images (even if some portions of a given outside wall are repeated in multiple images). In another aspect, if image processing power permits, multiple loops can be made around the perimeter of the building structure to acquire the digital images used to generate the sparse point cloud. In yet another aspect, if the camera, smartphone or other image acquisition device is carried / operated by a person, the systems and methods disclosed herein may provide to the person, via the device, one or more prompts instructing the user to: 1) capture the entire vertical span of the building structure during image acquisition; 2) avoid quick movements or panning with the device during image acquisition; and 3) make at least one full loop around the perimeter of the building structure while acquiring the digital images.

[0045] Fig. 2 shows an example of a 3D sparse point cloud reconstruction 20 of a building structure. The 3D sparse point cloud reconstruction 20 includes a sparse point cloud 200. Each point 210 in the sparse point cloud 200 has a respective color 215 and voxel location according to the digital images. In one aspect, the sparse point cloud 200 is ’‘non-dimensional” in that a distance between any two points in the cloud does not represent any physical value (such as a distance between physical elements of the building structure or its environs). Fig. 2 also illustrates an estimated ground plane 410 shown with respect to the sparse point cloud 200 of the spare point cloud reconstruction 20. Generation of the estimated ground plane 410 is discussed in detail below.

[0046] In acquiring the digital images used to generate the sparse point cloud 200, it should be appreciated that, in one aspect, the orientation of the image acquisition device with respect to the building structure during at least the first digital image (or first frame of a video) acquired of an outside wall of the building structure determines a three-dimensional (3D) coordinate system or '‘grid” of voxels on which the sparse point cloud is generated. The respective x, y and z dimensions of each voxel do not necessarily correspond to any actual physical dimension, but instead represent some unit of resolution between minimum and maximum values for the respective dimensions in the grid space. The sparse point cloud that is generated based on processing of the subsequent digital images is going to be constructed with reference to the 3D coordinate system established by at least the first digital image. Accordingly, if the image acquisition device is tilted with respect to the outside wall of the building structure during acquisition of at least the first digital image (e.g., an optical axis ofAttorney Docket No. FWIG-002W001 the image acquisition device is not essentially normal to the building structure, or not essentially parallel to the ground surface below the building structure), the sparse point cloud that is generated based on processing of the subsequent digital images is also going to be similarly tilted with respect to the perspective of the actual building structure and the ground surface below the building structure. In view of the foregoing, respective additional steps of the method 10 shown in Fig. 1 and discussed further below apply inventive processing concepts to account for an arbitrary tilting of the 3D coordinate system on which the sparse point cloud initially is generated, to in turn generate a geospatially-calibrated spare point cloud that accurately represents the building structure (and in some instances the immediate surroundings of the building structure).

[0047] To this end, and with reference again to Fig. 1, in step 102, the 3D sparse point cloud 200 generated in step 101 and shown in Fig. 2 is further processed to determine an estimated ground plane, and in step 103 an estimated perimeter of the building structure is determined based on the estimated ground plane. In one or more embodiments, steps 102 and 103 of Fig. 1 can be performed according to steps 301-303 in Fig. 3. Turning now to Fig. 3, in step 301 the 3D grid on which the sparse point cloud 200 is generated is divided into vertical columns, in which the z dimension (or height dimension) of each vertical column is taken as infinite (or the maximum available value in the z dimension), and in which the x and y dimensions of each vertical column are the same (e.g., to form a square, using an equal number of voxels; in one example implementation, the number of voxels corresponds to approximately one foot by one foot in physical distance). Density-based scanning is then performed on each vertical column to determine the lowest point of the sparse point cloud 200 in each vertical column.

[0048] In step 302, a plane is then fit across the lowest points of the respective vertical columns to define the estimated ground plane 410 shown in Fig. 2. Once the ground plane is defined, in step 303 all the points in the 3D sparse point cloud are projected onto that plane. In step 303, the projected points correspond to the estimated perimeter of the building structure. Fig. 4 shows an example of the projected 3D sparse points resulting in the estimated perimeter 400 (i.e., all the points in the 3D sparse point cloud projected onto the ground plane 410 shown in Fig. 2.Attorney Docket No. FWIG-002W001

[0049] Fig. 4 also shows a geospatially calibrated polygon 420 that represents the geospatial perimeter or geospatial footprint of the building structure. In one or more embodiments, the geospatially calibrated polygon 420 can be provided by and / or received from a third party such as Microsoft (e.g., Microsoft’s open footprint initiative), OpenStreetMaps (e.g., OpenStreetMaps’ communit -enhanced footprint data), Google, and / or other third-party (ies). Additionally or alternatively, a trained machine-learning (ML) structure footprint detection model can be used to detect the structure footprint in one or more overhead, aerial, and / or satellite images of the building structure. The trained ML structure footprint detection model can be trained with sample overhead, aerial, and / or satellite images and known footprints of building structures.

[0050] Returning for the moment to the method 10 shown in Fig. 1, in step 104 the estimated perimeter 400 is registered and / or aligned with a geospatially calibrated polygon 420 that represents the geospatial perimeter or geospatial footprint of the building structure. In one example implementation, the estimated perimeter 400 and the geospatially calibrated polygon 420 can be spatially aligned by rotating, translating, and scaling the estimated ground plane 410 to create a best overlay fit of the estimated perimeter 400 and the geospatially calibrated structure perimeter 420 as determined by iterative closest point (ICP) alignment or another method. The axes and values shown in Fig. 4 are provided as examples and are not intended to be limiting.

[0051] As a result of alignment and / or registration, the estimated perimeter 400 is placed in a known or common coordinate space of the geospatially calibrated polygon 420.

[0052] Returning to Fig. 1, in step 105 the 3D sparse point cloud 200 can be scaled according to the alignment / registration performed in step 104 so as to generate a scaled sparse point cloud reconstruction. When the estimated polygon 420 is rotated and / or translated during alignment / registration. the 3D locations (e.g., voxel locations) of the points in the 3D sparse point cloud 200 are rotated and / or translated, respectively, relative to their original locations. For example, if the rotational orientation of the estimated polygon 420 is changed as a result of alignment / registration, the corresponding location (e.g., voxel location) of each point in the 3D sparse point cloud 200 is rotationally varied accordingly. Likewise, if the estimated polygon 420 is translated parallel to the ground plane 410 as a result ofAttorney Docket No. FWIG-002W001 alignment / registration, the corresponding location (e.g., voxel location) of each point in the 3D sparse point cloud 200 is translated laterally accordingly.

[0053] If the size of the estimated polygon 420 is changed as a result of alignment / registration. the 3D locations (e.g., voxel locations) of the points in the 3D sparse point cloud 200 are varied accordingly, relative to their original locations. For example, if the size of the estimated polygon 420 is increased as a result of alignment / registration, then a given point in the 3D sparse point cloud 200 (e.g., representing a portion of a window) may be located further away from the ground plane and / or laterally further away from a centroid of the estimated polygon than that point was originally (prior to the increase in size of the estimated polygon). Likewise, if the size of the estimated polygon 420 is decreased as a result of alignment / registration, then a given point in the 3D sparse point cloud 200 (e.g., representing a portion of a window) may be located closer to the ground plane 410 and / or laterally closer to a centroid of the estimated polygon than that point was originally (prior to the decrease in size of the estimated polygon).

[0054] As a result of scaling in step 105 of Fig. 1, the 3D locations (e.g., voxel locations) of the points in the 3D sparse point cloud 200 are in the known or common coordinate space of the geospatially calibrated polygon 500. The voxel location of each point in the 3D sparse point cloud 200 has a corresponding geospatial position that can be stored in non-volatile memory.

[0055] Returning to Fig. 1. in step 106 a 3D computer rendering of the building structure is generated. The 3D rendering can comprise or can be the scaled 3D sparse point cloud reconstruction discussed above in connection with step 105. Alternatively, the scaled 3D sparse point cloud reconstruction can be a “volumetrically -rendered reconstruction” to generate a more realistic 3D rendering which can be or can be close to photorealistic in one or more embodiments. In one aspect, volumetric rendering can be used to “paint” the colors represented in the respective points of the scaled 3D sparse point cloud so as to generate a volumetrically-rendered reconstruction.

[0056] In one or more embodiments, the scaled 3D sparse point cloud reconstruction can be volumetrically-rendered using Gaussian Splatting. Gaussian Splatting can be performed as described in “3D Gaussian Splatting for Real-Time Radiance Field Rendering” by Bernhard Kerbl et al, ACM Trans. Graph., Vol. 42, No. 4. published August 2023 and / orAttorney Docket No. FWIG-002W001 at https: / / repo-sam.inria.fr / fungraph / 3d-gaussian-splatting / , which are hereby incorporated by reference.

[0057] In one or more other embodiments, the scaled 3D sparse point cloud reconstruction can be volumetrically-rendered using Neural Radiance Field (NeRF). NeRF can be performed as described in "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis” by Ben Mildenhall et al., European Conference on Computer Vision (ECCV) Oral Presentation, arXiv:2003.08934v2, August 3, 2020 and / or at https: / / www.matthewlancik.com / nerf, which are hereby incorporated by reference.

[0058] Fig. 5 shows a simplified example of a volume-rendered scaled 3D sparse point cloud reconstruction (volume-rendered reconstruction 60) of a building structure 600 according to one or more embodiments. The volume-rendered reconstruction 60 includes the outside walls 610 of the building structure 600 and structural features 620 defined on, in, and / or connected to the outside walls 610. Examples of structural features 620 include windows 622, doors 624, decks 626. balconies, vents 628 (e.g., air vents, laundry vents, and / or other vents), soffits, and / or garage doors 630. Though the roof 640 may not be viewable in the digital images representing the outside walls of the building structure (e.g., that are transformed in step 101 to a sparse point cloud representation), the volume-rendered reconstruction 60 can include a roof 640 (though the rendering of the roof 640 may differ in appearance from the roof of the actual building structure).

[0059] The volume-rendered reconstruction 60 can include additional objects 650 that are viewable in the digital images representing the outside walls of the building structure (e.g., that are transformed in step 101 to a sparse point cloud representation). Examples of these objects 650 include patio furniture 652, foundation plantings 654, fences 656, raised beds 658, and / or wood piles 660. One, some, or all of the additional objects 650 may represent fuel sources that can increase the wildfire ignition risk of the building structure 600.

[0060] Fig. 6 is a flow chart of a computer-implemented method 70 for performing a virtual wildfire inspection of a building structure according to one or more embodiments. Method 70 can be performed by one or more hardw are-based processors in a computer for example by executing computer-readable instructions stored in the non-volatile memory7of the computer.Attorney Docket No. FWIG-002W001

[0061] In step 701, only a portion of the 3D rendering (e.g., the volume-rendered reconstruction 60) is displayed on a display screen. The display screen can be coupled to the same computer that generated the 3D rendering (e.g., in step 106 of Fig. 1) such that method 10 and method 70 are performed on the same computer. Alternatively, the display screen can be coupled to a different computer that generated the 3D rendering, such as in a client-server relationship. The portion of the 3D rendering that is displayed can correspond to or represent a field of view (FOV), orientation, and position of an observation / inspection point (e.g., a location of a virtual wildfire inspector) relative to the building structure.

[0062] Fig. 7A is an overhead view of the building structure 600 showing an example observation / inspection point 800 (e.g., of a virtual wildfire inspector) having a position 810, a FOV 820, and an orientation 830 relative to the building structure 600. In Fig. 7A, the position 810 of the observation / inspection point 800 is in the middle of outside wall 610A. The orientation 830 of the observation / inspection point 800 is directed towards the middle of the outside wall 610A. The FOV 820 can be defined by a FOV angle 822.

[0063] Fig. 7B is a simplified view of a display screen 80 that displays a viewable portion 840 of the building structure 600 corresponding to or representing the position 810, FOV 820, and orientation 830 of the observation / inspection point 800 in Fig. 7A. As shown in Fig. 7B, the viewable portion 840 includes outside wall 610A but does not include outside walls 610B-D. The structural features 620 (e.g.. windows 622, door 624, and garage door 630) defined on, in, and / or connected to the outside wall 610A are displayed in the viewable portion 840. To the extent any additional objects 650, that were captured in the digital images transformed in step 101, are located in front of the outside wall 610A and within the FOV 820, those objects can be shown as part of the viewable portion 840.

[0064] Returning to Fig. 6, in step 702 one or more user inputs is / are received to change the position, FOV. and / or orientation of the observation / inspection point 800. The user inputs can be provided by gesture (e g., on a touch screen), a mouse, a keyboard, voice command, and / or other user input. In one or more embodiments, the user input can be provided through a graphical user interface that can be configured to represent movement of the observation / inspection point (e.g., the location of a virtual wildfire inspector) relative to the building structure.Attorney Docket No. FWIG-002W001

[0065] In one or more embodiments, the virtual wildfire inspector and / or the position of the virtual wildfire inspector relative to the building structure can be displayed on the display screen. In one or more other embodiments, the virtual wildfire inspector and / or the position of the virtual wildfire inspector relative to the building structure is / are not shown on the display screen.

[0066] In step 703, an updated portion of the 3D rendering is displayed on the display screen. The updated portion corresponds to or represents the new / updated FOV, orientation, and position of the observation / inspection point relative to the building structure (e.g., according to the user input(s) received in step 702).

[0067] Fig. 8A is an overhead view of the building structure 600 showing an example updated position 910 and updated FOV 920 of the observation / inspection point 800 compared to the position 810 and FOV 820 of the observation / inspection point 800 shown in Fig. 7A. The observation / inspection point 800 has moved closer to the outside wall 10A in front of the door 624. The updated position 910 and the updated FOV 920 can be updated, relative to the initial position 810 and initial FOV 820 of the observation / inspection point 800 shown in Fig. 7A, in response to one or more user inputs received in step 702. The updated FOV 920 can be defined by an updated FOV angle 922. The size or scope of the updated FOV 920 (e.g., the updated FOV angle 922) is smaller than the size or scope of the FOV 820 (e.g., than the FOV angle 822). In one or more other embodiments, the size or scope of the updated FOV 920 (e g., the updated FOV angle 922) can be larger than the size or scope of the FOV 820 (e.g., than the FOV angle 822).

[0068] Fig. 8B is a simplified view of an updated viewable portion 940 of the building structure 600 on the display screen 80. The updated view able portion 940 corresponds or represents the updated position 910 and the updated FOV 920.

[0069] In one or more embodiments, steps 702 and 703 can be repeated in a loop 704 to iteratively update the position, FOV, and / or orientation of the observation / inspection point and to iteratively update the displayed portion of the 3D rendering. Each update to the displayed portion of the 3D rendering (step 703) corresponds to or represents a respective update to the position, FOV, and / or orientation of the observ ation / inspection point (in response to the user input(s) received in step 702).Attorney Docket No. FWIG-002W001

[0070] Fig. 9A is an overhead view of the building structure 600 showing an example updated orientation 1030 of the observation / inspection point 800 compared to orientation 830 of the observation / inspection point 800 shown in Figs. 8A and 9A. The observation / inspection point 800 has turned to the right in Fig. 9 A, compared to Figs. 8A and 9 A, such that a portion of the door 624, one of the windows 620, and the wood pile 660 are within an updated FOV 1020 (having an updated FOV angle 1022) of the observation / inspection point 800. The observation / inspection point 800 is in the same position 910 in Figs. 9A and 10A. The updated orientation 1030 is determined according to the user input(s) received in step 702. The updated FOV angle 1022 can be the same as the updated FOV angle 922.

[0071] Fig. 9B is a simplified view of an updated viewable portion 1040 of the building structure 600 on the display screen 80. The updated viewable portion 1040 corresponds or represents the updated orientation 1030 and the updated FOV 1020.

[0072] Fig. 10 is a flow chart of a computer-implemented method 1100 for performing a virtual wildfire inspection of a building structure according to one or more embodiments. Method 1100 can be performed by one or more hardware-based processors in a computer for example by executing computer-readable instructions stored in the non-volatile memory of the computer.

[0073] Method 1100 adopts some of the same preliminary steps as shown in method 70 of Fig. 6, and includes additional steps beginning with step 1104 of receiving a user input to tag an object. The tagged object can represent a policy violation (e.g., as specified by one or more terms of a w ildfire insurance policy). Additionally or alternatively, the tagged object can represent a fuel source that may be located too close to (e.g., within a predetermined distance of) the building structure, which may increase the wildfire risk to the building structure. The wildfire risk to the building structure can be determined using a property ignition model (PIM), for example as described in U.S. Application Publication No. 2023 / 0023808, titled “System And Method For Wildfire Risk Assessment, Mitigation And Monitoring For Building Structures,” which is hereby incorporated by reference.

[0074] In optional step 1105, the state or appearance of an object (e.g., as it appears on a display of a user device) can change when the object is tagged (e.g., from an untagged state to a tagged state). With reference for the moment to Fig. 11, a symbol such as checkAttorney Docket No. FWIG-002W001 mark can be overlaid on the tagged object 1200 to represent that the object has been tagged. In another example, one or more of the color, shading, or fill pattern (or other displayed visible attribute) of the tagged object 1200 can be different than that of an untagged object.

[0075] Returning to Fig. 10, in optional step 1106, one or more GUI elements for the tagged object can be displayed. The GUI element(s) can include one or more text fields, one or more drop-down lists, one or more radio buttons, and / or one or more other GUI elements. The GUI element(s) can be configured to allow a user to input information relating to the tagged object (e.g.. so as to annotate the tagged object in some manner). For example, the GUI element(s) can include one or more object description GUI elements, one or more object category GUI elements, one or more remedial instructions GUI elements, one or more policy violation category GUI elements, one or more notes GUI elements, and / or one or more object count GUI elements.

[0076] The object description GUI element(s) can allow a user to enter or provide a description of the tagged object. The object category GUI element(s) can allow a user to enter or provide a category for the tagged object. The remedial instructions GUI element(s) can allow a user to enter or provide remedial instructions for reducing or eliminating the wildfire risk to the building structure caused by the tagged object and / or for eliminating the policy violation associated with the tagged object. The policy violation category GUI element(s) can allow a user to enter or provide a category for the policy violation associated with the tagged object. The notes GUI element(s) can allow a user to enter or provide notes relating to the tagged object, the remedial instructions, and / or the policy violation, and / or other notes. The object count GUI element(s) can allow a user to enter or provide the number of items associated with the tagged object. Generally speaking, any of the foregoing GUI elements relating to the tagged object may be referred to and considered as an "annotation" associated with the tagged object.

[0077] Fig. 11 shows the updated viewable portion 1040 of the building structure 600 where the wood pile 660 has been tagged (in response to user input in step 1104 of Fig. 10) as a tagged object 1200. GUI elements 1210 for the tagged object 1200 are displayed on the side of the display 80. There can be additional or fewer GUI elements 1210 (e.g.. at least one GUI element 1210) in one or more embodiments.Attorney Docket No. FWIG-002W001

[0078] Using the wood pile 660 as an example of a tagged object 1200, the GUI elements 1210 can include object description GUI element(s) that allow a user to enter or provide a description (e.g., wood pile), an object category (e.g., combustible material), remedial instructions (e.g., move more than 30 feet from building structure), the policy violation category (e.g., combustible material within 30 feet of building structure), notes, and / or the object count (e.g., 6 logs in wood pile).

[0079] Returning to Fig. 10, in optional step 1107 input data for the GUI element(s) displayed in step 1106 is received.

[0080] In step 1108, data for the tagged object is stored in non-volatile memory. The stored data includes a unique identifier for the tag, the data input (if any) by user for the GUI element(s) received in optional step 1107, and the geospatial location of the tagged object. The geospatial location of the tagged items is determined by the user within the geospatially aligned point cloud space. Any point specified in the aligned point cloud represents an actual geospatial location (i.e., the point cloud can include things that are not just the structure which can be tagged). The point cloud includes any object in the video frame. In one or more embodiments, the stored data for the tagged object can include an image of the tagged object and / or an image representing the location of the tagged object relative to the building structure.

[0081] Fig. 12 is a block diagram representing data 1300 for a tagged object 1200 that can be stored in step 1108 of Fig. 10. The data 1300 can include unique identifier data 1301. input data 1302 describing the tagged object 1200 (e.g., received in optional step 1107), geospatial location data 1303 representing the geospatial location of the tagged object 1200, tagged object image data 1304 that represents an image 1310 (e.g., a zoomed-in image) of the tagged object 1200, and / or position image data 1305 that represents an image 1320 (e.g., an overhead image) that shows the location of the tagged object 1200 relative to the building structure 600.

[0082] The image 1310 can comprise some of or all of a viewable portion (e.g., updated viewable portion 1040) of the building structure 600 (e.g., as shown in Fig. 11) in which the tagged object 1200 is viewable. Alternatively, the image 1310 can comprise another image of the tagged object 1200 that can be generated or created from the volume- rendered reconstruction 60.Attorney Docket No. FWIG-002W001

[0083] The image 1320 can comprise an annotation of the estimated polygon 420 or an annotation of the geospatially calibrated polygon 500 where the annotation indicates the position or location of the tagged object 1200 relative to the estimated polygon 420 or relative to the geospatially calibrated polygon 500, respectively.

[0084] Returning to Fig. 10, steps 702, 703, 1 104, and 1108 and optional step(s) 1105, 1006, and / or 1107 can be repeated in a loop 1109 such that multiple objects can be tagged as the virtual inspector is iteratively moved with respect to the building structure (e.g., with respect to the volume-rendered reconstruction 60 of the building structure). An object may not be tagged in a given iteration through the loop 1109 (e.g., in a given iteration of the loop, the steps 1104 and 1008 can be optional, such as when no policy violations are detected).

[0085] Fig. 13 is a flow chart of a computer-implemented method 1400 for performing a virtual wildfire inspection of a building structure according to one or more embodiments. Method 1400 can be performed by one or more hardware-based processors in a computer for example by executing computer-readable instructions stored in the non-volatile memory of the computer.

[0086] In step 1401, multiple (e.g., a plurality of) portions of the 3D rendering are iteratively displayed on a display. Each portion of the 3D rendering can be displayed according to a corresponding FOV, a corresponding orientation, and a corresponding position of a virtual inspector relative to the building structure, for example according to steps 701- 703 (see Fig(s). 7 and / or 11).

[0087] In step 1402, user inputs are received to tag one or more objects shown in one or more portions of the 3D rendering displayed in step 1401. A given user input to tag an object can be performed according to step 1104.

[0088] In optional step 1403, input data for one or more GUI element(s) for each tagged object are received. Input data for one or more GUI element(s) for a given tagged object can be performed according to optional step 1107.

[0089] In step 1404, data for the tagged object(s) are stored in non-volatile memory'. The data stored for each tagged object can be the same as described in step 1108.Attorney Docket No. FWIG-002W001

[0090] In step 1405, a request is received for a virtual inspection report for the building structure. The request can be received in response to user input such as through a GUI element on the display.

[0091] In step 1406, virtual inspection report data are generated in response to the request received in step 1405. The virtual inspection report data represent a virtual inspection report.

[0092] In step 1407, the virtual inspection report is displayed on a display. In one or more embodiments, the virtual inspection report can comprise a table that can include a row for each tagged object. The table can include column(s) that represent respective GUI element(s) and the respective input data for each tagged object. The table can also include an image (e.g., image 1310) of the tagged object and / or an image (e.g., image 1320) of an annotated polygon or footprint of the building structure that represents the location of the tagged object with respect to the building structure. An example of a virtual inspection report 1500 that includes a table 1510 is shown in Fig. 14A. Additionally or alternatively, the virtual inspection report can include multiple sections with each section including tagged objects in the same category and / or with each section including tagged objects on or near the same outside wall of the building structure. The virtual inspection report can include an image (e.g., image 1310) of each tagged object and / or an image (e.g., image 1320) of an annotated polygon or footprint of the building structure that represents the location of each tagged object with respect to the building structure. An example of a virtual inspection report 1520 that includes one or more sections 1530 is shown in Fig. 14B.

[0093] Fig. 15 is a flow chart of a computer-implemented method 1600 for detecting and / or segmenting structural features in digital images according to one or more embodiments. Method 1600 can be performed by one or more hardware-based processors in a computer for example by executing computer-readable instructions stored in the nonvolatile memory of the computer.

[0094] In step 1601, digital images that collectively represent all outside walls of a building structure are received or provided. The digital images can be stored in and / or received from a local or remote computer memory (e.g., non-volatile memory ). The digital images comprise digital image data that represent the digital images. The digital images canAttorney Docket No. FWIG-002W001 comprise or represent digital photographs or sampled digital images from one or more digital videos.

[0095] In step 1602, the digital images are fed into one or more trained models that is / are configured to segment and / or detect the structural features shown or represented in the digital images. The trained model(s) can comprise one or more trained computer vision segmentation model(s) and / or one or more separate trained machine-learning (ML) model(s). The computer vision segmentation model(s) and / or trained ML model(s) is / are trained with (a) digital images with known locations and shapes of structural features in example building structures and (b) digital images of example building structures that do not include any structural features. The computer vision segmentation model(s) and / or the trained ML model(s) can distinguish between each type of structural feature (e.g., between doors, windows, decks, balconies, vents, soffits, garage doors, and / or other type(s) of structural features).

[0096] In one or more embodiments, the structural features are segmented and / or detected using multiple (e.g., a plurality of) trained computer vision segmentation models and / or multiple (e.g., a plurality of) trained ML models. In one or more embodiments, a separate trained computer vision segmentation model and / or a separate trained ML model can be used to detect and / or segment each type of structural feature on / in a building structure. For example, a first trained computer vision segmentation model and / or a first trained ML model can be used to detect and / or segment windows on a building structure. Additionally or alternatively, a second trained computer vision segmentation model and / or a second trained ML model can be used to detect and / or segment doors on a building structure. Additionally or alternatively, a third trained computer vision segmentation model and / or a third trained ML model can be used to detect and / or segment soffits in a building structure. Each separate trained computer vision segmentation model and / or each separate trained ML model can be trained using (a) first sample images that include the type of feature and (b) second sample images that do not include the type of feature.

[0097] In step 1603, the structural features that are segmented and / or detected in each digital image can be masked (e.g., with pixels in a particular solid color). In one aspect, each different type of feature can be masked with a different color. For example, windows can beAttorney Docket No. FWIG-002W001 masked with a first color (e.g., red), and doors can be masked with a second color (e.g., blue). The masked digital images can be stored in computer memory’ (e.g., non-volatile memory).

[0098] In step 1604, a 3D rendering of the building structure is generated using the masked digital images. The 3D rendering of the building structure can be generated according to method 10. The masked color(s) of the structural features in the 3D rendering can visually highlight their respective locations during a virtual inspection.

[0099] Fig. 16 is an example 3D sparse point cloud rendering 1700 of a building structure 600 that includes one or more masked colors 1710 for the structural features 1720 according to one or more embodiments. The masked color(s) 1710 is / are represented as clusters of specifically colored points making it possible to find the 3D location for the structural features 1720 in space by scanning for dense regions of points with that specific color(s). In one or more embodiments, as noted above, each different type of structure feature 1720 can be represented in a different masked color 1710.

[0100] With reference now to Fig. 17, in one or more embodiments bounding boxes 1810 can be estimated or determined from the clusters of specifically colored points 1820 (e.g., red or another color) corresponding to structural features 1830 in a 3D sparse point cloud rendering 1800 of a building structure 600, for example as shown in Fig. 17. The bounding boxes 1810 can be used to automatically estimate the size and geospatial location of each structural feature 1830. The size and location of each structure feature 1830 can be stored in computer memory (e.g., non-volatile memory).

[0101] Fig. 18 is a simplified view of a display screen 1900 that displays a viewable portion 1940 of a building structure 600 where each type of structural feature 620 is masked in a different color, shade and / or fill pattern (e.g., has a different state). For example, the windows 622 are masked in a first color (or have a first state), the door 624 is masked in a second color (or has a second state), the vent 628 is masked in a third color (or has a third state), and the garage door 630 is masked in a fourth color (or has a fourth state). The viewable portion 1940 is the same as the viewable portion 840 except for the masked colors of the structural feature 620 shown in the viewable portion 1940.

[0102] Fig. 19 is a flow chart of a method 2000 for remotely monitoring a building structure for compliance with wildfire mitigation recommendations.Attorney Docket No. FWIG-002W001

[0103] In step 2001, first digital images that collectively represent all outside walls of a building structure are received or provided. The first digital images represent the state of the building structure on a first time or a first date. The first digital images can be stored in and / or received from a local or remote computer memory (e.g., non-volatile memory). The first digital images comprise first digital image data that represent the first digital images. The first digital images can comprise or represent digital photographs or sampled digital images from one or more digital videos.

[0104] In step 2002, a first 3D rendering of the building structure is generated using the first digital images. The first 3D rendenng of the building structure can be generated according to method 10 of Fig. 1.

[0105] In step 2003, a first virtual inspection of the building structure is performed using the first 3D rendering. The first virtual inspection can be performed according to method 70 of Fig. 6, method 1100 of Fig. 10, and / or method 1400 of Fig. 13.

[0106] Based at least in part on the first virtual inspection performed in step 2003, a virtual inspection report can be provided to an occupant, homeowner, and / or ser ice provider to perform wildfire mitigation actions detailed in the virtual inspection report (e.g.. in the remedial instructions).

[0107] In step 2004, second digital images that collectively represent all outside walls of the building structure are received or provided. The second digital images represent the state of the building structure on a second time or a second date that occurs after the first time / date (and presumably after at least some of the wildfire mitigation actions are completed). The second digital images can be stored in and / or received from a local or remote computer memory (e.g., non-volatile memory). The second digital images comprise second digital image data that represent the second digital images. The second digital first images can comprise or represent digital photographs or sampled digital images from one or more digital videos.

[0108] In step 2005, a second 3D rendering of the building structure is generated using the second digital images. The second 3D rendering of the building structure can be generated according to method 10 of Fig. 1.Attorney Docket No. FWIG-002W001

[0109] In step 2006, a second virtual inspection of the building structure is performed using the second 3D rendering. The second virtual inspection can be performed according to method 70, method 11 0, and / or method 1400. The second virtual inspection can be performed to confirm (e.g., remotely confirm) whether or not respective wildfire mitigation actions have been performed (e.g., pursuant to the virtual inspection report following the first virtual inspection).

[0110] In step 2007, it is determined whether any wildfire mitigation actions still need to be performed. For example, there may be one or more wildfire mitigation actions that have not been performed and / or additional wildfire mitigation actions identified during the second virtual inspection. If there are no wildfire mitigation actions that still need to be performed for the building structure (i.e., step 2007=no), then in step 2008 it is determined that the building structure is in compliance with wildfire mitigation recommendations (e.g., provided in a virtual inspection report). If there is / are one or more wildfire mitigation actions that still need to be performed for the building structure (i.e., step 2007=yes, then the method 2000 can return to step 2004 in a loop 2008 to receive new digital images that collectively represent all outside walls of the building structure. The new digital images can be received preferably after any remaining wildfire mitigation action(s) for the building structure are completed. An additional virtual inspection report can be provided to an occupant, homeowner, and / or service provider to confirm whether or not any remaining wildfire mitigation action(s) (e.g., as detailed in the remedial instructions of one or more previous virtual inspection reports) have been sufficiently addressed.

[0111] Fig. 20 is a flow chart of a method 2100 for performing a virtual fire inspection according to one or more embodiments.

[0112] In step 2101, a geospatially calibrated aerial digital image of a property including a building structure is acquired or received. The aerial image can be a satellite image, an aerial image (e.g., captured by plane, drone, or balloon) or another aerial images of the property7including the building structure. The aerial images can be high-quality and / or can be scaled to determine the geospatial location and dimensions of the structure and objects on the property such as trees and / or shrubs and the respective distance between the nearest outside wall of the property and each object (e.g., each tree and / or shrub).Attorney Docket No. FWIG-002W001

[0113] Fig. 21 shows an example overhead image 2200 of a property 2210 including a building structure 2220. The overhead image 2200 shows large vegetation such as trees 2230 and shrubs 2240. In one or more embodiments, the overhead image 2200 can show one or more detached (e.g., ancillary) structures 2225 on the property 2210. The detached structure(s) 2225 can include a detached garage, a shed, a pool house, and / or another detached structure. The dimensions of the overhead image 2200 overall, as well as respective sizes and dimensions of the objects represented in the overhead image 2200 (such as the building structure 2220, the trees 2230, the shrubs 2240, and any ancillary structure(s) 2225) can be measured and / or determined as described, for example, in U.S. Application Publication No. 2023 / 0023808, previously incorporated by reference. In addition, the respective distances between the building structure 2220 and other objects (e.g. , each tree 2230, each shrub 2240, and each ancillary structure 2225) can be measured and / or determined as described, for example, in U.S. Application Publication No. 2023 / 0023808, previously incorporated by reference.

[0114] Returning to Fig 21, in optional step 2102 a geospatial perimeter of the building structure can be determined using the geospatially calibrated aerial digital image. The geospatial perimeter of the building structure can be determined using a trained ML structure footprint detection model that is configured to detect the structure footprint in an overhead image of a building structure (for example as described in step 104 of Fig. 1).

[0115] In step 2103, fuel sources represented in the aerial image are detected. The fuel sources can be detected with one or more trained ML models that is / are configured to detect fuel sources, such as trees, shrubs, and / or detached structures in aerial images.

[0116] In step 2104, the property ignition risks, due to a wildfire, for the building structure are determined or estimated. The property ignition risks can be determined or estimated using a PIM, for example as disclosed in U.S. Application Publication No. 2023 / 0023808, previously incorporated by reference.

[0117] In step 2105, digital images that collectively represent all outside walls of a building structure are received or provided. The digital images can be stored in and / or received from a local or remote computer memory (e.g., non-volatile memory ). The digital images comprise digital image data that represent the digital images. The digital images canAttorney Docket No. FWIG-002W001 comprise or represent digital photographs or sampled digital images from one or more digital videos.

[0118] In step 2106, a 3D rendering of the building structure is generated using the digital images received in step 2105. The first 3D rendering of the building structure can be generated according to method 10 of Fig. 1.

[0119] In step 2107, a virtual inspection of the building structure is performed using the 3D rendering. The virtual inspection can be performed according to method 70, method 1100, and / or method 1400. The virtual inspection can identify one or more wildfire mitigation actions to be performed as recommendations for compliance with wildfire insurance policy requirement; for example, the wildfire mitigation action(s) can represent wildfire insurance policy violations and / or exclusions.

[0120] In step 2108, the property ignition risks, due to a wildfire, for the building structure are updated (e g., determined a second time) based on the assumption or expectation that the wildfire mitigation action(s) are performed.

[0121] In step 2109, a virtual inspection report can be generated. The virtual inspection report can be shared, displayed, and / or printed.

[0122] Fig. 22 illustrates a networked computing system 2400 according to one or more embodiments. At least a portion of the networked computing system 2400 may be used to implement one or more methods and / or systems as described herein. The networked computing system 2400 may comprise a network 2402, at least one user 2404, at least one stationary computing device 2406, at least one mobile computing device 2408, and / or a sen' er 2410, in any of various combinations as may be readily apprehended by one of ordinary skill in the art.

[0123] Network 2402 may be any one or more networks that allow elements of the present system to communicate with each other, as may be known in the art, for example wide area networks (WANs), LANs, and the like, that may be wired or wireless, and may include Bluetooth™, WiFi, and other approaches to local or remote communication.

[0124] There may be one or more stationary computing devices 2406 that may be one or more computing systems that may be used various users, such as homeowners (ow ners of building structures), appraisers, mitigation consultants or companies, monitoring consultantsAttorney Docket No. FWIG-002W001 or companies, insurance companies or brokers and adjusters, structure data providers (such as images of structures, weather data, elevation data, material data, and the like).

[0125] One or more mobile computing devices 2408 may provide access to various functionality, similar to stationary computing device(s) 2406. In addition, mobile computing device(s) 2408 can include one or more digital cameras that can allow one or more users to capture digital images of the outside walls of a building structure, such as digital photographic images and / or video(s).

[0126] One or more stationary7computing devices 2406, one or more mobile computing devices 2408, and / or one or more servers 2410 may be used to perform one or more methods as described herein - for example to generating a 3D sparse point cloud representation of a building structure, generating a 3D rendering of a building structure, performing a virtual inspection of a building structure, and / or estimating a wildfire risk for a building structure. A stationary7computing device 2406, a mobile computing device 2408, or a server 2410 may be referred to simply as a computing device or a computer.

[0127] Fig. 23 is an example block diagram of a computing device 2500 that may incorporate one or more embodiments of the present disclosure. Fig. 23 is merely illustrative of a machine system to cany7out aspects of the technical processes described herein, and does not limit the scope of the claims. One of ordinary skill in the art would recognize other variations, modifications, and alternatives. In one or more embodiments, the computing device 2500 typically includes a display and / or graphical user interface 2502, a data processing system 2520, a communication network interface 2512, input device(s) 2508, output device(s) 2506, and the like.

[0128] The data processing system 2520 may include one or more processors 2504 that communicate with a number of peripheral devices via a bus subsystem 2518. These peripheral devices may include the input device(s) 2508, the output device(s) 2506, the communication network interface 2512, and / or a storage subsystem, such as a volatile memory 2510 and a nonvolatile memory72514.

[0129] The volatile memory 2510 and / or the nonvolatile memory 2514 may store computer-executable instructions and thus forming logic 2522 that when applied to and executed by the processor(s) 2504 implement embodiments of the methods disclosed herein.Attorney Docket No. FWIG-002W001The nonvolatile memory 2514 can store one or more trained ML models, one or more trained computer vision segmentation model, and / or one or more trained ML engines.

[0130] The input device(s) 2508 include devices and mechanisms for inputting information to the data processing system 2520. These may include a keyboard, a keypad, a touch screen incorporated into the monitor / display and / or graphical user interface 2502, audio input devices such as voice recognition systems, microphones, and other types of input devices. In various embodiments, the input device(s) 2508 may be embodied as a computer mouse, a trackball, a track pad, a joystick, wireless remote, drawing tablet, voice command system, eye tracking system, and the like. The input device(s) 2508 typically allow a user to select objects, icons, control areas, text and the like that appear on the monitor / display and / or graphical user interface 2502 via a command such as a click of a button or the like.

[0131] The output device(s) 2506 include devices and mechanisms for outputting information from the data processing system 2520. These may include the monitor / display and / or graphical user interface 2502, speakers, printers, infrared light emitting diodes (LEDs), and so on as understood in the art.

[0132] The communication network interface 2512 provides an interface to communication networks (e.g., one or more communication networks 2516) and devices external to the data processing system 2520. The communication network interface 2512 may serve as an interface for receiving data from and transmitting data to other systems. Embodiments of the communication network interface 2512 may include an Ethernet interface, a modem (telephone, satellite, cable, Integrated Services Digital Network (ISDN)), (asynchronous) digital subscriber line (DSL), FireWire, USB, a wireless communication interface such as BlueTooth or WiFi, a near-field communication wireless interface, a cellular interface, and / or another communication network interface.

[0133] The communication network interface 2512 may be coupled to the communication network(s) 2516 via an antenna, a cable, or the like. In one or more embodiments, the communication network interface 3512 may be physically integrated on a circuit board of the data processing system 2520, or in some cases may be implemented in software or firmware, such as "soft modems" or the like.Attorney Docket No. FWIG-002W001

[0134] The computing device 2500 may include logic that enables communications over a network using protocols such as HTTP, TCP / IP, RTP / RTSP, IPX, UDP and the like.

[0135] The nonvolatile memory 2514 is an example of tangible (e.g., non-transitory) media configured to store computer-readable data and instructions to implement various embodiments of the processes described herein. The volatile memory 2510 is another example of tangible (e.g., non-transitory ) media configured to store computer-readable data and instructions to implement various embodiments of the processes described herein. Other types of tangible media include removable memory (e.g., pluggable USB memory devices, mobile device SIM cards), optical storage media such as CD-ROMS, DVDs, semiconductor memories such as flash memories, non-transitory read-only-memories (ROMS), battery- backed volatile memories, networked storage devices, and the like. The volatile memory' 2510 and / or the nonvolatile memory 2514 may be configured to store the basic programming and data constructs that provide the functionality of the disclosed methods and other embodiments thereof that fall within the scope of the present disclosure.

[0136] The logic 2522 that implements embodiments of the present disclosure may be stored in the volatile memory 2510 and / or in the nonvolatile memory^ 2514. The logic 2522 may be read from the volatile memory' 2510 and / or non-volatile memory' 2514 and executed by the processor(s) 2504. The volatile memory' 2510 and / or the nonvolatile memory 2514 may also provide a repository for storing data used by the logic 2522.

[0137] The volatile memory 2510 and / or the nonvolatile memory 2514 may include a number of memories including a main random access memory (RAM) for storage of instructions and data during program execution and a read only memory (ROM) in which read-only non-transitory' instructions are stored. The volatile memory' 2510 and / or the nonvolatile memory 2514 may include a fde storage subsystem providing persistent (nonvolatile) storage for program and data files. The volatile memory 2510 and / or the nonvolatile memory 2514 may include removable storage systems, such as removable flash memory.

[0138] The bus subsystem 2518 provides a mechanism for enabling the various components and subsystems of data processing system 2520 to communicate with each other. Although the communication network interface 2512 is depicted schematically as a single bus, some embodiments of the bus subsystem 2518 may utilize multiple distinct busses.Attorney Docket No. FWIG-002W001

[0139] The computing device 2500 may be a device such as a smartphone, a desktop computer, a laptop computer, a rack-mounted computer system, a computer server, or a tablet computer device. The computing device 2500 may be implemented as a collection of multiple networked computing devices, for example in a distributed computing system. Further, the computing device 2500 may typically include operating system logic the types and nature of which are known in the art.

[0140] The invention should not be considered limited to the particular embodiments described above. Various modifications, equivalent processes, as well as numerous structures to which the invention may be applicable, will be readily apparent to those skilled in the art to which the invention is directed upon review of this disclosure. The above-described embodiments may be implemented in numerous ways. One or more aspects and embodiments involving the performance of processes or methods may utilize program instructions executable by a device (e.g.. a computer, a processor, or other device) to perform, or control performance of, the processes or methods.

[0141] In this respect, various inventive concepts may be embodied as a non- transitory computer readable storage medium (or multiple non-transitory computer readable storage media) (e.g., a computer memory of any suitable type including transitory7or non- transitory digital storage units, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement one or more of the various embodiments described above. When implemented in software (e.g., as an app), the software code may be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0142] Further, it should be appreciated that a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer, as non-limiting examples. Additionally, a computer may be embedded in a device not generally regarded as a computer but with suitable processing capabilities, including a Personal Digital Assistant (PDA), a smartphone or any other suitable portable or fixed electronic device.Attorney Docket No. FWIG-002W001

[0143] Also, a computer may have one or more communication devices, which may be used to interconnect the computer to one or more other devices and / or systems, such as. for example, one or more networks in any suitable form, including a local area network or a wide area network, such as an enterprise network, and intelligent network (IN) or the Internet. Such networks may be based on any suitable technology and may operate according to any suitable protocol and may include wireless networks or wired networks.

[0144] Also, a computer may have one or more input devices and / or one or more output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that may 778 be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound generating devices for audible presentation of output. Examples of input devices that may be used for a user interface include keyboards, and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or in other audible formats.

[0145] The non-transitory computer readable medium or media may be transportable, such that the program or programs stored thereon may be loaded onto one or more different computers or other processors to implement various one or more of the aspects described above. In some embodiments, computer readable media may be non- transitory media.

[0146] The terms "program," 'app." and "software" are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that may be employed to program a computer or other processor to implement various aspects as described above. Additionally, it should be appreciated that, according to one aspect, one or more computer programs that when executed perform methods of this application need not reside on a single computer or processor but may be distributed in a modular fashion among a number of different computers or processors to implement various aspects of this application.

[0147] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that performs particular tasks or implement particular abstract data types. The functionality7of the program modules may be combined or distributed as desired in various embodiments.Attorney Docket No. FWIG-002W001

[0148] Also, data structures may be stored in computer-readable media in any suitable form. For simplicity of illustration, data structures may be show n to have fields that are related through location in the data structure. Such relationships may likewise be achieved by assigning storage for the fields with locations in a computer-readable medium that convey relationship between the fields. However, any suitable mechanism may be used to establish a relationship between information in fields of a data structure, including through the use of pointers, tags or other mechanisms that establish relationship between data elements.

[0149] Thus, the disclosure and claims include new and novel improvements to existing methods and technologies, which were not previously known nor implemented to achieve the useful results described above. Users of the method and system will reap tangible benefits from the functions now made possible on account of the specific modifications described herein causing the effects in the system and its outputs to its users. It is expected that significantly improved operations can be achieved upon implementation of the claimed invention, using the technical components recited herein.

[0150] Also, as described, some aspects may be embodied as one or more methods. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0151] What is claimed is:

Claims

Attorney Docket No. FWIG-002W001Claims1. A computer-implemented method for generating a three-dimensional rendering of outside walls of a building structure for a virtual wildfire inspection, comprising: a. receiving images of the outside walls of the building structure on a property, the images collectively representing all outside walls of the building structure and structural features on and / or in the outside walls; b. transforming the images into a three-dimensional sparse point cloud to form a three-dimensional sparse point cloud reconstruction of an outside of the building structure; c. projecting the three-dimensional sparse point cloud onto a ground plane to determine a projected perimeter of the building structure; d. forming a projected polygon that corresponds to the projected perimeter; e. receiving a geospatially calibrated overhead image, the geospatially calibrated overhead image including an overhead view of the property including the building structure; f. segmenting, using one or more trained machine-learning models, the geospatially calibrated overhead image to define a geospatially calibrated polygon that represents a geospatial perimeter of the building structure, g. registering the projected polygon with the geospatially calibrated polygon so as to generate a geospatially aligned three-dimensional sparse point cloud; and h. generating the three-dimensional rendering from the geospatially aligned three- dimensional sparse point cloud, the three-dimensional rendering including the outside walls and the structural features, wherein the structural features rendered on the outside of the building structure have respective geospatial positions corresponding to the geospatial perimeter of the building structure.

2. The method of claim 1, wherein the three-dimensional rendering is generated by a volume rendering of the geospatially aligned three-dimensional sparse point cloud to form a three- dimensional reconstruction of the outside walls and the structural features of the building structure.

3. The method of claim 2, wherein the volume rendering is performed using a three- dimensional Gaussian splat or a neural radiance field.Attorney Docket No. FWIG-002W0014. The method of claim 2, further comprising receiving a user input to tag an object shown in the three-dimensional reconstruction, the object representing a policy violation, the tag including a tagged geospatial position of the object, a description of the object, and remedial instructions for eliminating the policy violation.

5. The method of claim 1, further comprising: i. displaying only a portion of the three-dimensional rendering on a display screen, the portion displayed representing a field of view from a position relative to the building structure in the three-dimensional rendering; j. receiving a user input to update an orientation of the field of view relative to the building structure in the three-dimensional rendering, a size of the field of view relative to the building structure in the three-dimensional rendering, and / or the position relative to the building structure in the three-dimensional rendering; k. updating, in response to the user input received in step j, the portion of the three- dimensional rendering displayed on the display screen according to an updated orientation of the field of view, an updated size of the field of view, and / or an updated position.

6. The method of claim 1, further comprising: i. displaying only a portion of the three-dimensional rendering on a display screen, the portion displayed according to a field of view from a position relative to the building structure in the three-dimensional rendering; j. receiving a plurality of user inputs, each input received sequentially to update an orientation of the field of view relative to the building structure in the three-dimensional rendering, a size of the field of view relative to the building structure in the three-dimensional rendering, and / or the position relative to the building structure in the three-dimensional rendering; k. iteratively updating, in response to each user input received in step j, the portion of the three-dimensional rendering displayed on the display screen according to a respective updated orientation of the field of view, a respected updated size of the field of view, and / or a respective updated position from a respective user input so to create an interactive three- dimensional rendering of the outside walls of the building structure.Attorney Docket No. FWIG-002W0017. A method for performing a virtual fire inspection of a building structure that includes a plurality of outside walls, the method comprising: a. transforming a plurality of digital images into three-dimensional sparse point cloud data, the digital images collectively representing all of the outside walls of the building structure; b. determining, with the three-dimensional sparse point cloud data, an estimated perimeter of the building structure; c. aligning the estimated perimeter with a geospatial perimeter of the building structure to form a geospatially aligned three-dimensional sparse point cloud; d. volume rendering the geospatially aligned three-dimensional sparse point cloud to generate a three-dimensional reconstruction of the outside walls of the building structure; and e. displaying only a portion of the three-dimensional reconstruction on a display screen, the portion displayed according to a field of view from a position relative to the building structure.

8. The method of claim 7, further comprising: receiving a user input to tag an object shown in the portion of the three-dimensional reconstruction displayed on the display screen, the object representing a policy violation; generating, in response to the user input, a tag for the object; and automatically associating a geospatial position of the object with the tag.

9. The method of claim 8, further comprising automatically displaying a first graphical user interface (GUI) element configured to receive a description of the object and a second GUI element configured to receive remedial instructions for eliminating the respective policy violation.

10. The method of claim 7, further comprising: receiving digital video data that represents a video showing all of the outside walls of the building structure; and sampling the video to form the digital images.

11. The method of claim 7, further comprising, prior to step a:Attorney Docket No. FWIG-002W001 segmenting the digital images to detect structural features of the building structure represented in the digital images; and masking the detected structural features so as to visually highlight the structural features in the three-dimensional reconstruction.

12. The method of claim 11, wherein the digital images are segmented using one or more trained computer vision segmentation models and / or one or more trained machine-learning models.

13. The method of claim 11, further comprising: classifying each detected structural feature into one of a plurality of structural-feature types; and masking the detected structural feature(s) classified in each structural -feature type with a different mask color so as to visually distinguish each structural-feature type.

14. The method of claim 7, further comprising: f. receiving a plurality of user inputs, each input received sequentially to update an orientation of the field of view relative to the building structure, a size of the field of view relative to the building structure, and / or the position relative to the building structure; and g. iteratively updating, in response to each user input received in step f, the portion of the three-dimensional reconstruction displayed on the display screen according to a respective updated orientation of the field of view, a respected updated size of the field of view, and / or a respective updated position from a respective user input so to simulate a movement of a virtual inspector with respect to the building structure.

15. The method of claim 14, further comprising: receiving a plurality of tag inputs to tag a plurality of objects shown during the movement of the virtual inspector with respect to the building structure, each object representing a respective policy violation; generating, in response to the tag inputs, a plurality of tags, each tag associated with a respective object;Attorney Docket No. FWIG-002W001 automatically associating a respective geospatial position of each object with a respective tag; and automatically storing each tag and the respective geospatial position of each object.

16. The method of claim 15, further comprising automatically displaying, for each tag, a respective first graphical user interface (GUI) element configured to receive a respective description of the respective object and a respective second GUI element configured to receive respective remedial instructions for eliminating the respective policy violation.

17. The method of claim 1 , further comprising: receiving, for each tag: the respective description of the respective object and the respective remedial instructions; automatically storing the respective description of the respective object and the respective remedial instructions; receiving a virtual inspection report request to generate a virtual inspection report that includes for each tag: the respective description of the respective object, the respective remedial instructions, and the respective image of the respective object.

18. The method of claim 17, further comprising: automatically capturing, for each tag, a respective virtual image representing, in the three-dimensional reconstruction, the respective object associated with the respective tag: automatically associating each virtual image with the respective tag; and automatically storing the virtual images in the non-volatile memory, wherein the virtual inspection report includes the respective virtual image for each tag.

19. The method of claim 18, further comprising: automatically generating, for each tag, a respective annotated image of the geospatial perimeter of the building structure, each annotated image including a respective annotation that indicates a relative position of the respective object with respect to the geospatial perimeter of the building structure, the relative position corresponding to the respective geospatial position of the respective object,Attorney Docket No. FWIG-002W001 wherein the virtual inspection report includes the respective annotated image for each tag.

20. A computer configured to generate a three-dimensional rendering of outside walls of a building structure for a virtual wildfire inspection, comprising: one or more processors; computer memory in communication with the processors, the computer memory including non-volatile memory storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: a. receive images of the outside walls of the building structure on a property, the images collectively representing all outside walls of the building structure and structural features on and / or in the outside walls; b. transform the images into a three-dimensional sparse point cloud to form a three-dimensional sparse point cloud reconstruction of an outside of the building structure; c. project the three-dimensional sparse point cloud onto a ground plane to determine a projected perimeter of the building structure; d. form a projected polygon that corresponds to the projected perimeter; e. receive a geospatially calibrated overhead image, the geospatially calibrated overhead image including an overhead view of the property including the building structure; f. segment, using one or more trained machine-learning models, the geospatially calibrated overhead image to define a geospatially calibrated polygon that represents a geospatial perimeter of the building structure, g. register the projected polygon with the geospatially calibrated polygon so as to generate a geospatially aligned three-dimensional sparse point cloud; and h. generate the three-dimensional rendering from the geospatially aligned three-dimensional sparse point cloud, the three-dimensional rendering including the outside walls and the structural features, wherein the structural features rendered on the outside of the building structure have respective geospatial positions corresponding to the geospatial perimeter of the building structure.Attorney Docket No. FWIG-002W00121. A computer configured for performing a virtual fire inspection of a building structure that includes a plurality of outside walls, comprising: one or more processors; computer memory in communication with the processors, the computer memory including non-volatile memory storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: a. transform a plurality of digital images into three-dimensional sparse point cloud data, the digital images collectively representing all of the outside walls of the building structure; b. determine, with the three-dimensional sparse point cloud data, an estimated perimeter of the building structure; c. align the estimated perimeter with a geospatial perimeter of the building structure to form a geospatially aligned three-dimensional sparse point cloud; d. volume render the geospatially aligned three-dimensional sparse point cloud to generate a three-dimensional reconstruction of the outside walls of the building structure; and e. display only a portion of the three-dimensional reconstruction on a display screen, the portion displayed according to a field of view from a position relative to the building structure.

Images