Automatically determining building information from plan structure analysis

Abstract

Techniques are described for performing, using a computing device, automatic operations for determining matching buildings and using corresponding information in other automatic ways, such as by determining buildings having similarity to an indicated target building based at least in part on a building plan and / or other attributes, and automatically determining building modifications implemented for the target building by based at least in part on differences from similar buildings and / or building features prioritized or de-prioritized from analysis of one or more types of user activity data, optionally combined with one or more other specified criteria. Information about such determined buildings may be used in a variety of automated ways, including for controlling device navigation (e.g., autonomous vehicles), for display on client devices in corresponding graphical user interfaces, for further analysis to identify shared and / or aggregated features, and the like.

Description

Technical Field

[0001] The following disclosure generally relates to techniques for automatically determining and using information about structural features and other properties of a building based on analysis of a building floor plan and other building information, and for subsequently using the determined information in one or more automated manners to determine room connectivity and visibility / roominess and accessibility information for the building (e.g., for an indicated type of user, such as a user in a wheelchair), and to providing corresponding navigation data for the building based on the determined information, and by automatically generating descriptions of similarities. Background Art

[0002] In various situations, such as building analysis, property inspections, real estate acquisition and development, general contracting, improvement cost estimation, etc., it may be desirable to know the interior of a house or other building without physically traveling to the building and entering the building. However, it may be difficult to effectively capture, represent, and use such building interior information, including identifying buildings that meet criteria of interest, and displaying visual information captured within the building interior to a user at a remote location (e.g., enabling the user to understand the layout and other details of the interior, including controlling the display in a manner selected by the user). Moreover, even if the user is present at the building, it is difficult to effectively navigate the building and determine information about the building, which is not readily apparent. Although a floor plan of a building can provide some information about the layout and other details of the building interior, such use of a floor plan has some disadvantages, including that the floor plan is difficult to construct and maintain, difficult to accurately scale and populate with information about the interior of a room, difficult to visualize and otherwise use, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 Included are diagrams depicting exemplary building interior environments and computing system(s) used in embodiments of the present disclosure, including generating and presenting information representing the interior of a building, and / or determining and further using information regarding assessment of structural attributes of the building based on analysis of a floor plan of the building.

[0004] Figure 2A-2J Examples of determining information about structural features and other properties of a building based on analysis of at least a building floor plan, and subsequently using the determined building information in one or more automated manners are shown.

[0005] Figure 3 is a block diagram illustrating a computing system suitable for implementing an embodiment of a system that performs at least some of the techniques described in this disclosure.

[0006] Figure 4A-4BAn exemplary embodiment of a flow chart for a Building Plan Attribute Determination / Usage Manager (BFPADUM) system routine is shown in accordance with an embodiment of the present disclosure.

[0007] Figure 5 An exemplary implementation of a flow chart for an image capture and analysis (ICA) system routine is shown in accordance with an implementation of the present disclosure.

[0008] Figure 6A-6B An exemplary embodiment of a flow chart for a Mapping Information Generation Manager (MIGM) system routine is shown in accordance with an embodiment of the present disclosure.

[0009] Figure 7A-7B An exemplary implementation of a flow chart for a building information access system routine is shown in accordance with an implementation of the present disclosure. DETAILED DESCRIPTION

[0010] The present disclosure describes techniques for performing automatic operations using a computing device, the automatic operations involving automatically determining information about structural features and other attributes of a target building based on an analysis of existing building information, and for using the determined building information in various ways (e.g., providing corresponding updated navigation data for the target building, such as for controlling navigation of an autonomous vehicle or other mobile device in the target building). The automatic determination of information about structural features and other attributes of a building can be based in part or in whole on analyzing a floor plan of the building in some embodiments and scenarios. Such a floor plan can be used in at least some embodiments for a constructed multi-room building (e.g., a house, an office building, etc.), and includes at least two-dimensional (2D) room shapes positioned relative to each other, the at least two-dimensional (2D) room shapes generated from a panoramic image or other image or otherwise associated with a panoramic image or other image (e.g., a rectilinear stereo image) acquired at an acquisition location in and around the building (e.g., without or using information from any depth sensor or other distance measurement device about the distance from the acquisition location of the image to walls or other objects in the surrounding building). In at least some embodiments, such determination of building attribute information for the target building includes one or more of the following: determining one or more values ​​of one or more accessibility attributes for each of a plurality of locations in the building based at least in part on the structural characteristics of the building, such as with respect to an indicated user type (e.g., a user in a wheelchair, other type of wheeled or tracked vehicle, a user not in a wheelchair but with limited mobility, etc.); determining one or more values ​​of one or more connectivity attributes reflecting room-to-room connectivity and optionally other room adjacency information for each of a plurality of locations in the building based at least in part on the structural characteristics of the building, such as the structural characteristics of the building between a particular room (e.g., from a center of a room, or a room doorway or other room location) or other locations; determining one or more values ​​of one or more isovist attributes reflecting at least one of visibility and / or spaciousness for each of a plurality of locations in the building based at least in part on the structural characteristics of the building, the one or more values ​​reflecting at least one of visibility and / or spaciousness, etc. In addition, in various embodiments, such determined building attribute information for the target building may be used in various ways to assist in navigation of the building, for display or other presentation to one or more users on one or more client devices in a corresponding GUI (graphical user interface) to enable virtual navigation of the target building, etc. Additional details regarding the automation and use of determined information about structural features and other attributes of a target building are included below, and in at least some embodiments, some or all of the techniques described herein may be performed via automated operation of a Building Floor Plan Attribute Determination / Usage Manager (“BFPADUM”) system, as further discussed below.

[0011] As described above, in at least some embodiments and scenarios, automatic operation of the BFPADUM system may include determining one or more values ​​of one or more accessibility attributes for each of multiple locations in a building based at least in part on structural characteristics of the building, such as with respect to an indicated user type (e.g., a user in a wheelchair, other type of wheeled or tracked vehicle, a user not in a wheelchair but with limited mobility, etc.). Determining associated values ​​for one or more accessibility attributes of a building can include analyzing at least a floor plan of the building (and optionally additional building information, such as images captured at the building) to evaluate various structural characteristics of the building, such as the widths of doorways and other non-doorway wall openings, the lengths and / or heights of various structural elements (e.g., walls, countertops, cabinets, appliances, fixtures, etc.), elevation changes and associated structural elements (e.g., stairs, ramps, etc.), the locations of doors and other movable components when opened or otherwise moved, and the like, and using the structural characteristics to evaluate the accessibility of each of a plurality of locations in some or all of the building (e.g., in some or all of the building) (e.g., for each location in a grid of locations throughout the floor plan and, optionally, in the surrounding property in which the building is located, e.g., for each grid cell having a dimension of one or more inches and / or one or more feet or other distance measurement on each side of the cell). As a non-exclusive example, assessing the accessibility of a location relative to a user in a wheelchair and / or other wheeled or tracked vehicle may include assessing any structural barriers to entry into a building (e.g., stairs, an undersurface of a path of travel that is not paved or otherwise sufficiently rigid to support the weight of a wheelchair or other vehicle when loaded, the width of an exterior doorway, etc.), assessing any structural barriers to travel in the building between an entrance to the building and the location, assessing at least one of the length or height of one or more structural elements (e.g., walls) of the building in a defined area (e.g., within a defined radius) around the location and the distance of (multiple) elements from the location, etc., and combining information from multiple such assessments (e.g., in a weighted manner) to determine an accessibility score or other accessibility value for the location that reflects the degree of accessibility of a wheelchair or other vehicle to reach the location from outside the building, and / or to interact with other nearby building objects or other elements from the location from the wheelchair or other vehicle. Evaluating at least one accessibility attribute for this user type and / or other types of users (e.g., walking users with limited mobility) may also include using one or more other factors (e.g., factors specific to that type of user), such as measuring travel distance and / or elevation change from the location to other building locations (e.g., to a bathroom or kitchen or bedroom), the presence or absence of assistive installed items (e.g., grab bars, handrails, seats, elevators, etc.), and the like.

[0012] Furthermore, such determined accessibility scores or other accessibility values ​​for building locations may be expressed in various ways, such as one of a plurality of enumerated accessibility values, as a percentage or other value on a continuous scale, etc. Furthermore, accessibility values ​​for a plurality of locations may be grouped or otherwise aggregated in various ways, such as for each room and / or other area, for each floor or other group of rooms (e.g., rooms of the same type or function), for the building as a whole and / or other buildings on the property (e.g., outbuildings), for the property as a whole, etc., as well as utilizing an aggregated accessibility value for one or more rooms or other areas that may be expressed in various ways, such as an absolute value (e.g., a measure or other quantity of sub-areas within the room(s) or other area(s) that meet one or more defined criteria, such as the amount of square footage or other area measurement of locations having associated accessibility values ​​above or below a defined accessibility value threshold), a ratio or other relative value (e.g., a ratio of the amount of sub-areas within the room(s) or other area(s) that meet one or more defined criteria and expressed relative to the total amount of the room(s) or other area(s), etc. As non-exclusive examples, such aggregated accessibility values ​​may include the number or number of inaccessible rooms within a building (or other room grouping, such as for a floor), the ratio of the number of such inaccessible rooms to the total number of rooms in the building (or other room grouping, such as for a floor), the amount (e.g., square feet) of inaccessible areas within a building (or other room grouping or defined area, such as for a floor or an entire property), the percentage of one or more such inaccessible areas to the total amount of areas of that type, and the like, with “inaccessible” locations or groups of locations in this example having associated accessibility values ​​that are below a defined accessibility threshold. Furthermore, the determined accessibility values ​​for groups or other aggregations of locations may be presented or otherwise provided in a variety of ways, such as in textual and / or numerical form (e.g., with an indication of the associated locations, such as displayed on a display of a floor plan or property), as a displayed heat map (e.g., in which each location is assigned one of a plurality of enumerated visual attributes corresponding to the accessibility value for that location and optionally overlaid on a visual representation of a floor plan and / or property boundary, or displayed in a manner associated with the locations on the floor plan and / or on the property), etc. The following includes additional details regarding automated operations for determining and using one or more accessibility attribute values ​​for locations in a building, including Figure 2E-2F .

[0013] As described above, in at least some embodiments and scenarios, the automatic operation of the BFPADUM system may include determining one or more values ​​of one or more connectivity attributes for each of a plurality of locations in a building based at least in part on the structural characteristics of the building, the one or more values ​​reflecting inter-room connectivity and optionally other room adjacency information. The determination of the values ​​of the one or more connectivity attributes may include analyzing at least one floor plan of the building (and optionally additional building information, such as images captured at the building) to determine information about inter-room connections between rooms (e.g., doorways, non-doorway wall openings, etc.) and optionally about other adjacent rooms that are not directly connected (e.g., having a path of travel between rooms that includes one or more intermediate rooms, such as corridors, stairs, other rooms, etc.). The one or more assessments of connectivity may then include determining, for each of the one or more locations in the room (e.g., the center of the room, a doorway or other wall opening, etc.), one or more measures of connectivity of the location to one or more other locations in the building and / or on the property, such as one or more of the following: an actual travel distance to each of the one or more other defined locations (e.g., the center or other location of other defined rooms or other areas, such as a kitchen, bathroom, bedroom, family room, etc.) via intermediate room(s), if any; a certain number of other directly connected and / or adjacent rooms or other areas for the location; the presence or absence of structural obstacles (e.g., stairs) between the location and each of the one or more other rooms or other areas or other locations, etc. In at least some embodiments and scenarios, determining values ​​of one or more connectivity attributes of the target building includes generating an adjacency graph representing room-to-room connections and optionally other room adjacencies, and also storing other attributes of the target building, and optionally also generating one or more vector-based embeddings to compactly represent the adjacency graph and information of the target building attributes, e.g., for use in comparing the target building with other buildings using such information. Additional details regarding the generation and use of such adjacency graphs and vector-based embeddings are included below.

[0014] Furthermore, such determined connectivity scores or other connectivity values ​​for one or more building locations (e.g., rooms or other areas) may be expressed in a variety of ways, such as one of a plurality of enumerated connectivity values, as a percentage or other value on a continuous scale, etc. As non-exclusive examples, such connectivity assessments may include the distance between two locations (e.g., in feet or other distance measurements, in terms of the number of room-to-room connections and / or intervening rooms or other areas, etc.), the number of directly connected and / or adjacent rooms or other areas of a location, etc. Furthermore, the determined connectivity values ​​may be presented or otherwise provided in a variety of ways, such as in textual and / or numerical form (e.g., with an indication of an associated location, such as displayed on a floor plan or display of the property), as a displayed adjacency graph (e.g., overlaid on a visual representation of a floor plan and / or property boundary, or otherwise displayed in a manner associated with a location on the floor plan and / or property), and optionally displayed with particular graph nodes (e.g., corresponding to rooms or other areas) and / or with graph edges (e.g., corresponding to interconnectivity and / or adjacency between two rooms or other areas) highlighted (e.g., corresponding to a path of travel between two rooms or other areas) and / or displayed with associated textual information (e.g., distances) displayed, as a displayed path of travel separate from an adjacency graph, etc. Additional details regarding automated operations for determining and using values ​​of one or more connectivity attributes for locations in a building are included below, including with respect to Fig.2I .

[0015] As described above, in at least some embodiments and scenarios, automatic operation of the BFPADUM system may include determining one or more values ​​of one or more viewshed attributes for locations in a building, the one or more values ​​reflecting at least one of visibility and / or spaciousness from those locations (e.g., a 2D spatial area and / or a 3D or three-dimensional spatial volume visible from a location, optionally including sight lines through doorways and / or past other movable structural elements when positioned to increase visibility) based at least in part on the structural characteristics of the building. Determining the values ​​of one or more viewshed attributes for the building may include analyzing at least a floor plan of the building (and optionally additional building information, such as images captured at the building) to assess sight lines based on the structural characteristics of the building in at least a 2D plane and optionally in 3D space, and determining an assessment of visibility and / or spaciousness for each of a plurality of locations in some or all of the building (e.g., for each location in a grid of locations throughout the floor plan, and optionally throughout the surrounding property in which the building is located, such as each grid cell having a size of one or more inches and / or one or more feet, or other distance measurement per side).

[0016] Furthermore, such determined viewshed scores or other viewshed values ​​for building locations may be expressed in various ways, such as one of a plurality of enumerated viewshed values, as a percentage or other value on a continuous scale, etc. Furthermore, viewshed values ​​for a plurality of locations may be grouped or otherwise aggregated in various ways, such as for each room and / or other area, for each floor or other group of rooms (e.g., rooms of the same type or function), for the building and / or one or more other structures on the property as a whole (e.g., an outbuilding), for the property as a whole (e.g., including a plurality of structures on the property, etc.), and for aggregated viewshed values ​​for one or more rooms or other areas that may be expressed in various ways, such as an absolute value (e.g., a measure or other quantity of sub-areas within the room(s) or other area(s) that meet one or more defined criteria, such as the number of square feet having associated viewshed values ​​above or below a defined viewshed value threshold), a ratio or other relative value (e.g., a ratio of the quantity of sub-areas within the room(s) or other area(s) that meet one or more defined criteria and expressed relative to the total quantity of the room(s) or other area(s), etc. As non-exclusive examples, such aggregated viewshed values ​​may include the size of one or more sub-areas within a defined area (e.g., one or more rooms, the building as a whole, etc.) that have viewshed values ​​above a defined threshold, the ratio of such sub-area size(s) to the total size of the area, the amount of such sub-area size(s) (e.g., square feet) within an area (e.g., one or more rooms, a building, a floor, a total estate, etc.), a percentage of one or more such sub-area sizes relative to the total size of the area, etc. In addition, the determined viewshed values ​​for the one or more locations may be presented or otherwise provided in a variety of ways, such as in textual and / or numerical form (e.g., with an indication of the associated location, such as displayed on a floor plan or estate display), as a displayed heat map (e.g., with each location given one of a plurality of enumerated visual attributes corresponding to the viewshed value for the location, and optionally overlaid on a visual representation of the floor plan and / or estate boundaries, or displayed in a manner associated with the location on the floor plan and / or estate), etc. The following includes additional details regarding automated operations for determining and using values ​​of one or more viewshed attributes for locations throughout a building, including information regarding Figure 2G-2H .

[0017] Determination of values ​​for accessibility attribute(s) and / or connectivity attribute(s) and / or viewshed attribute(s) may be performed in various ways in various embodiments, including in some embodiments using one or more machine learning models trained to determine one or more attribute values ​​of this type from an input floor plan and optionally additional input building information (e.g., a building image). In some embodiments, computer vision techniques are used to determine one or more attribute values ​​of this type based on a visual analysis of a floor plan and optionally additional building information (e.g., a building image), for example instead of or in addition to using one or more trained machine learning models. In addition, in at least some embodiments, machine learning techniques can be used to learn specific accessibility attribute(s) and / or connectivity attribute(s) and / or viewshed attribute(s) associated with specific types of users, for example by evaluating data about multiple interactions involving multiple users of multiple types and multiple buildings, wherein the multiple interactions include at least one of acquisition interactions by at least some of the multiple users of at least some of the multiple buildings, or reconstruction interactions by at least some of the multiple users involving at least some of the multiple buildings, or browsing interactions by at least some of the multiple users having online information about at least some of the multiple buildings. In addition, in various embodiments, such determined building information for a target building can be used in various ways to assist in navigation of the building, to present to one or more users, to determine whether the target building meets one or more defined criteria (e.g., search criteria), to identify other buildings similar to the target building, as part of an evaluation of the value or other characteristics of the target building, as part of personalizing information about the target building for the user or otherwise providing user-specific information about one or more buildings including the target building (e.g., by using user-specific thresholds for determining areas that are sufficiently accessible or connected or have viewshed visibility, information about buildings and / or building areas having determined values ​​or other evaluations of one or more of accessibility attributes or connectivity attributes or viewshed attributes, etc.), to generate recommendations including the target building and / or other buildings similar to the target building, to generate a textual description of the target building, and the like.

[0018] The described techniques provide various benefits in various embodiments, including allowing floor plans of multi-room buildings and other structures to be identified and used more efficiently and quickly and in a manner that was not previously accessible, including automatically determining one or more types of building attributes (accessibility attribute(s), connectivity attribute(s), viewshed attribute(s), etc.) based on analysis of the building floor plans and optionally other building information (e.g., images of the building captured at the building). In addition, such automated techniques allow such determination of building attributes to be performed using information obtained from the actual building environment (rather than from plans of how the building should theoretically be constructed), as well as allowing changes to structural elements and / or visual appearance elements that occur after the building is initially constructed to be captured. Such described techniques also provide the benefit of allowing improved automated navigation of buildings by mobile devices (e.g., semi-autonomous or fully autonomous vehicles) based at least in part on the identification of building floor plans that match specified criteria, including significantly reducing the computing power and time used to attempt to otherwise learn the layout of the building. Furthermore, in some embodiments, the described techniques can be used to provide an improved GUI in which a user can more accurately and quickly identify one or more building floor plans that match specified criteria, and obtain information about one or more such buildings (e.g., for navigating the interior of one or more buildings), including in response to a search request, as part of providing personalized information to a user, as part of providing a user with value estimates and / or other information about a building (e.g., after analyzing information about one or more target building floor plans that are similar to one or more initial floor plans or otherwise match specified criteria), etc. Various other benefits are also provided by the described techniques, some of which are further described elsewhere herein.

[0019] As described above, the automatic operation of the BFPADUM system may include generating and using an adjacency graph for a building floor plan in at least some embodiments, while generating building adjacency information in other formats in other embodiments. Such a floor plan of a building may include a 2D (two-dimensional) representation of various information about the building (e.g., rooms, doorways between rooms and other room-to-room connections, external doorways, windows, etc.), and may be further associated with various types of supplementary or additional information about the building (e.g., data of multiple other building-related attributes). Such additional building information may, for example, include one or more of the following: a 3D or three-dimensional model of a building including height information (e.g., for building walls and other vertical areas); a 2.5D or 2.5-dimensional model of a building, which includes a visual representation of walls and / or other vertical surfaces when reproduced, without explicitly modeling the measured heights of those walls and / or other vertical surfaces; images and / or other types of data captured in the rooms of the building, including panoramic images (e.g., 360° panoramic images), etc., as discussed in more detail below. Such an adjacency graph may store or otherwise include some or all such data for a building, e.g., at least some such data being stored in or otherwise associated with nodes of the adjacency graph, the nodes of the adjacency graph representing some or all of the rooms of the floor plan (e.g., each node containing information about properties of the room represented by the node), and / or at least some such property data being stored in or otherwise associated with edges between nodes, the edges between nodes representing connections between adjacent rooms via doorways or other inter-room wall openings, or in some cases further representing adjacent rooms that share at least a portion of, and optionally all of, at least one wall without any direct inter-room openings connecting the two rooms (e.g., each edge containing information about the state of the connection between the rooms represented by the nodes interconnected by the edge, such as whether an inter-room opening exists between the two rooms, and / or one type of inter-room opening or other type of adjacency between the two rooms, such as without any direct inter-room wall opening connection). In some embodiments and scenarios, the adjacency graph may further represent at least some information about the exterior of a building, such as exterior areas adjacent to doorways or other wall openings between a building and exterior and / or other accessory structures located on the same property as the building (e.g., a garage, shed, pool house, detached guest house, mother-in-law unit or other accessory dwelling unit, swimming pool, patio, deck, sidewalk, garden, yard, etc.), or more generally, some or all exterior areas of a property that includes one or more buildings (e.g., a house and one or more accessory buildings or other accessory structures).Such external areas and / or other structures may be represented in the adjacency graph in various ways, such as by separate nodes for each such external area or other structure, or as attribute information associated with a corresponding node or edge, or in lieu of attribute information associated with the adjacency graph as a whole (for the building as a whole). In at least some embodiments, the adjacency graph may also have associated attribute information for corresponding rooms and connections between rooms, so that some or all of the information available on the floor plan and otherwise associated with the floor plan (or, in some embodiments and situations, information in and associated with the 3D model of the building) is represented within the adjacency graph. For example, if there is an image associated with a particular room of a floor plan or other associated area (e.g., an external area), the corresponding visual attributes may be included within the adjacency graph, either as part of the associated room or other area, or as a separate node layer representing the image within the graph. In embodiments having adjacency information in a form other than an adjacency graph, some or all of the above types of information may be stored in or otherwise associated with the adjacency information, including information about rooms, information about adjacencies between rooms, information about the state of connections between adjacent rooms, information about attributes of the building, and the like. Additional details on the generation and use of adjacency graphs are included below, including with respect to . Fig.2I Examples and their associated descriptions.

[0020] As described above, the automatic operation of the BFPADUM system may also include generating and using one or more vector-based embeddings (also referred to herein as "vector embeddings") to concisely represent information in an adjacency graph of a floor plan of a building so as to summarize the semantic meaning and spatial relationships of the floor plan in a manner that enables some or all of the floor plan to be reconstructed from the vector embeddings. In various embodiments, such vector embeddings may be generated in various ways, such as by using representation learning and one or more trained machine learning models, and in at least some such embodiments, such vector embeddings may be encoded in a format that is not easily discernible to a human reader. Non-exclusive examples of techniques for generating such vector embeddings are included in the following, which are incorporated herein by reference in their entirety: Jiwoong Park et al., “Symmetric Graph Convolution Autoencoder For Unsupervised Graph Representation Learning,” 2019 International Conference on Computer Vision, August 7, 2019; William L Hamilton et al., “Inductive Representation Learning On Large Graphs,” 2017 31st Conference on Neural Information Processing Systems, June 7, 2017; and Thomas N. Kipf et al., “Variational Graph Auto-Encoders,” 2017 30th Conference on Neural Information Processing Systems (Bayesian Deep Learning Workshop), November 21, 2016. Additional details on the generation and use of vector embeddings are included below, including information about Fig.2I Examples and descriptions of .

[0021] Furthermore, as described above, a floor plan may have various information associated with individual rooms and / or with connections between rooms and / or with a corresponding building and / or the containing property as a whole, and a corresponding adjacency graph and / or vector embedding for such a floor plan may include some or all of such associated information (e.g., properties of nodes represented as rooms in an adjacency graph and / or properties of edges of connections between rooms in an adjacency graph and / or properties of the adjacency graph represented as a whole, such as in a node representing the entire building, and having corresponding information encoded in the associated vector embedding(s). Such associated information may include various types of data, including information regarding one or more of the following non-exclusive examples: room type, room dimensions, location of windows and doors and other inter-room openings in the room, room shape, type of view from each exterior window, information and / or copies of images taken in the room, information and / or copies of audio or other data captured in the room, various types of information regarding characteristics of one or more rooms (e.g., automatically identified based on image analysis, provided by an operator user of a BFPADUM system and / or by an end-user viewing information about a floor plan and / or by an operator user of an ICA and / or MIGM system as part of capturing information about a building and generating a floor plan for a building, etc.), attributes of structures and objects (e.g., color, shape, material, age, condition, quality, etc.), types of inter-room connections, dimensions of inter-room connections, etc. In addition, in at least some embodiments, one or more additional subjective attributes may be determined for and associated with a floor plan, such as by analyzing floor plan information (e.g., an adjacency graph of a floor plan) by one or more trained machine learning models (e.g., classification neural network models) to identify floor plan characteristics of a building as a whole or for a particular building floor (e.g., open floor plan; typical / normal vs. atypical / odd / unusual floor plans; standard vs. non-standard floor plans; accessibility-friendly floor plans, such as by being accessible in terms of one or more characteristics such as for people with disabilities and / or advanced age, etc.). In at least some such embodiments, the one or more classification neural network models are part of the BFPADUM system and are trained by supervised learning using labeled data that identifies floor plans having each of the possible characteristics, while in other embodiments, such classification neural network models may instead use unsupervised clustering. Additional details regarding determining and using attribute information for floor plans are included below.

[0022] After determining attribute information for one or more accessibility attributes and / or one or more connectivity attributes and / or one or more viewshed attributes of a building from its floor plan (optionally including an adjacency graph and one or more vector embeddings), the determined information (optionally combined with other determined building attributes) can be used by the BFPADUM system as a specified criterion to automatically determine one or more other similar or otherwise matching floor plans of other buildings in various ways in various embodiments. For example, in some embodiments, an initial floor plan is identified, and one or more corresponding vector embeddings (with details about the determined attribute information) are generated for the initial floor plan and compared with the generated vector embeddings for other candidate floor plans to determine the difference between the vector embedding of the initial floor plan and the vector embeddings of some or all of the candidate floor plans, with a smaller difference between the two vector embeddings corresponding to a higher similarity between the building information represented by those vector embeddings. In various embodiments, the difference between two such vector embeddings may be determined in various ways, including, as non-exclusive examples, by using one or more of the following distance metrics: Euclidean distance, cosine distance, graph edit distance, a custom distance metric specified by a user, etc.; and / or determining similarity in other ways without using such distance metrics. In at least some embodiments, multiple such initial floor plans may be identified and used in the manner described to determine a combined distance between a set of vector embeddings for the multiple initial floor plans and a vector embedding for each of multiple other candidate floor plans, such as by determining a separate distance from each of the initial floor plans to a given other candidate floor plan, and by combining multiple individually determined distances in one or more ways (e.g., a mean or other average, cumulative total, etc.) to generate a combined distance from the set of vector embeddings for the multiple initial floor plans to a given other candidate floor plan. Additional details about comparing vector embeddings of floor plans to determine floor plan similarity are included below.

[0023] Additionally, in some embodiments, one or more target floor plans that are similar to specified criteria associated with a particular end-user are identified (e.g., based on one or more initial target floor plans selected by the end-user and / or identified as previously of interest to the end-user, whether based on explicit and / or implicit activity of the end-user to specify such floor plans; based on one or more search criteria explicitly and / or implicitly specified by the end-user; etc.), based at least in part on one or more determined accessibility attributes and / or connectivity attributes and / or viewshed attributes, and used in further automated activities to personalize interactions with the end-user. In various embodiments, such further automated personalized interactions may be of various types, and in some embodiments may include displaying or otherwise presenting to the end-user information about the target floor plans and / or additional information associated with those floor plans. The following includes additional details regarding the use of one or more identified target floor plans for further end-user personalization and / or presentation.

[0024] Furthermore, in at least some embodiments and circumstances, some or all of the images acquired for a building and associated with a floor plan of the building may be panoramic images, each acquired at one of a plurality of acquisition locations within or around the building, such that a panoramic image is generated at each such acquisition location from one or more videos at the acquisition location (e.g., 360° video acquired from a smartphone or other mobile device held by a user turning toward the acquisition location), or multiple images acquired in multiple directions from the acquisition location (e.g., from a smartphone or other mobile device held by a user turning toward the acquisition location), or all image information is captured simultaneously (e.g., using one or more fisheye lenses), etc. Such images may include visual data, and in at least some embodiments and circumstances, acquisition metadata regarding the acquisition of such panoramic images may be obtained and used in various ways, such as data acquired from an IMU (inertial measurement unit) sensor or other sensor of the mobile device (e.g., compass heading data, GPS location data, etc.) as the mobile device is carried by a user or otherwise moved between acquisition locations. It should be understood that such panoramic images may in some cases be represented in a spherical coordinate system and provide up to 360° coverage around the horizontal and / or vertical axis, so that a user viewing the starting panoramic image can move the viewing direction within the starting panoramic image to different directions so that different images (or "views") are presented within the starting panoramic image (including, if the panoramic image is represented in a spherical coordinate system, converting the image being rendered to a planar coordinate system). The following includes additional details related to obtaining and using panoramic images or other images of buildings.

[0025] In at least some embodiments, the BFPADUM system can operate with one or more separate ICA (image capture and analysis) systems and / or with one or more separate MIGM (mapping information and generation manager) systems to obtain and use the floor plan and other associated information of the building from the ICA and / or MIGM systems, while in other embodiments, such a BFPADUM system can incorporate some or all of the functionality of such an ICA and / or MIGM system as part of the BFPADUM system. In other embodiments, the BFPADUM system can operate without using some or all of the functionality of the ICA and / or MIGM systems, for example, if the BFPADUM system obtains information about the building floor plan and related information from other sources (e.g., by manually creating or providing such a building floor plan and / or related information by one or more users).

[0026] With respect to the functionality of such an ICA system, it can, in at least some embodiments, perform automated operations to acquire images (e.g., panoramic images) at various acquisition locations associated with a building (e.g., inside multiple rooms of a building), and optionally further acquire metadata related to the image acquisition process (e.g., compass heading data, GPS location data, etc.) and / or metadata related to the movement of the capture device between acquisition locations. In at least some embodiments, such acquisition and subsequent use of the acquired information can be performed without having or using information from a depth sensor or other distance measuring device regarding the distance from the acquisition location of the image to walls or other objects in the surrounding building or other structure. For example, in at least some such embodiments, such techniques may include capturing visual data from a series of multiple acquisition locations within multiple rooms of a house (or other building) using one or more mobile devices (e.g., a camera having one or more fisheye lenses and mounted on a rotatable tripod or otherwise having an automatic rotation mechanism; a camera having one or more fisheye lenses sufficient to capture 360 ​​degrees horizontally without rotation; a smartphone held and moved by a user so as to rotate the user's body and holding the smartphone in a 360° circle about a vertical axis; a camera held by or mounted on a user or the user's clothing; a camera mounted on an air-based and / or ground-based drone or other robotic device; etc.). Additional details regarding the operation of devices implementing the ICA system are included elsewhere herein so as to perform such automatic operations and, in some cases, further interact in one or more ways with one or more ICA system operator users to provide further functionality.

[0027] With respect to the functionality of such a MIGM system, it may, in at least some embodiments, perform automated operations to analyze a plurality of 360° panoramic images (and optionally other images) that have been acquired for the interior of a building (and optionally the exterior of a building), and determine room shapes and locations of connecting room passages for some or all of these panoramic images, as well as wall elements and other elements of some or all of the rooms of the building in at least some embodiments and scenarios. Types of connecting passages between two or more rooms may include one or more doorway openings and other inter-room non-doorway wall openings, windows, stairways, non-room corridors, etc., and the automated analysis of the images may identify such elements based at least in part on identifying the outlines of passages, identifying content within passages that is different from that outside of them (e.g., different colors or shading), etc. The automated operations may also include generating a floor plan of the building using the determined information, and optionally generating other mapping information of the building, such as by using the inter-room passage information and other information to determine the relative locations of related room shapes with respect to each other, and optionally adding distance scaling information and / or various other types of information to the generated floor plan. Additionally, in at least some embodiments, the MIGM system can perform further automated operations to determine and associate additional information with a building floor plan and / or a specific room or location within the floor plan, to analyze images and / or other environmental information (e.g., audio) captured within the building interior to determine specific attributes (e.g., color and / or material type and / or other characteristics of specific features or other elements (such as floors, walls, ceilings, countertops, furniture, fixtures, appliances, cabinets, islands, fireplaces, etc.); the presence and / or absence of specific features or other elements; etc.), or otherwise determine relevant attributes (e.g., the direction facing a building feature or other element (e.g., a window); the view from a specific window or other location; etc.). The following includes additional details regarding the operation of the computing device(s) implementing the MIGM system to perform such automated operations, and in some cases further interact with one or more MIGM system operator users in one or more ways to provide further functionality.

[0028] For the purpose of illustration, some embodiments are described below in which specific types of information are obtained, used and / or presented in a specific manner for specific types of structures and by using specific types of devices. However, it will be understood that the described technology can be used in other embodiments in other ways, and therefore the present invention is not limited to the exemplary details provided. As a non-exclusive example, although specific types of data structures (e.g., floor plans, adjacency graphs, vector embeddings, etc.) are generated and used in a specific manner in some embodiments, it should be understood that other types of information describing floor plans and other associated information can be similarly generated and used in other embodiments, including buildings (or other structures or layouts) separated from houses, and in other embodiments, floor plans identified as matching specified criteria can be used in other ways. In addition, the term "building" refers to any partially or completely enclosed structure in this article, typically but not necessarily including one or more rooms that visually or otherwise separate the interior space of the structure. Non-limiting examples of such structures include a house, an apartment building or an individual apartment therein, a condominium, an office building, a commercial building or other wholesale and retail structure (e.g., a shopping mall, a department store, a warehouse, etc.), a supplemental structure having another primary structure on the property (e.g., a detached garage or shed with a house on the property), etc. The terms "acquire" or "capture" as used herein with respect to a building interior, acquisition location, or other location (unless the context clearly indicates otherwise) may refer to any recording, storage, or recording of media, sensor data, and / or other information related to spatial characteristics and / or visual characteristics and / or otherwise perceptible characteristics of the building interior or a subset thereof, such as by a recording device or by another device that receives information from a recording device. As used herein, the term "panoramic image" may refer to a visual representation that is based on, includes, or is separable into multiple discrete component images that originate from substantially similar physical locations in different directions and depict a larger field of view than any discrete component image depicted alone, including images with sufficiently wide-angle viewing angles from the physical location to include angles in a single direction that are beyond what is perceptible from a person's gaze. As used herein, the term "series" of acquisition locations generally refers to two or more acquisition locations, each of which is visited at least once in a corresponding order, regardless of whether other non-acquisition locations are visited between them, and regardless of whether visits to the acquisition locations occur during a single continuous time period, or at multiple different times, or by a single user and / or device, or by multiple different users and / or devices. In addition, various details are provided in the drawings and text for illustrative purposes, but are not intended to limit the scope of the invention.For example, the sizes and relative positions of elements in the drawings are not necessarily drawn to scale, with some details omitted and / or greater prominence provided (e.g., by size and positioning) to enhance readability and / or clarity. In addition, the same reference numerals may be used in the drawings to identify the same or similar elements or actions.

[0029] Figure 1 Included are example block diagrams of various computing devices and systems that can participate in the described techniques in some implementations, such as with respect to an example building 198 (a house in this example) and an example building floor plan property determination / usage manager ("BFPADUM") system 140 executing on one or more server computing systems 180 in this example implementation. An internal capture and analysis ("ICA") system (e.g., an ICA system 160 executing on one or more server computing systems 180 such as part of a BFPADUM system; an ICA system application 154 executing on a mobile image acquisition device 185; etc.) may first capture information 165 about one or more buildings or other structures (e.g., by capturing one or more 360° panoramic images and / or other images of multiple acquisition locations 210 in an example house 198), and a MIGM (Mapping Information Generation Manager) system 160 executing on one or more server computing systems 180 (e.g., as part of a BFPADUM system) may further use the captured building information and optional additional supporting information (e.g., provided by a system operator user via a computing device 105 through an intervening computer network 170) to generate and provide building floor plans 155 and / or other mapping-related information (not shown) for the building(s) or other structure(s). Additional details related to the automated operation of the ICA and MIGM systems are included elsewhere herein, including with respect to, respectively, Figure 2A-2D and about Figure 5 and Figure 6A-6B .

[0030] In the illustrated embodiment, the ICA and MIGM system 160 operates as part of a BFPADUM system 140 that analyzes building information 142 (e.g., images 165 acquired by the ICA system, floor plans 155 generated by the MIGM system, etc.) and determines building structural attribute information 141 (e.g., building attributes and their associated values, associated visualizations for subsequent presentation and / or other use, etc.) and optionally additional corresponding building information 145 (e.g., adjacency graphs, hierarchical vector embeddings, etc.), optionally using supporting information provided by a system operator user via the computing device 105 through an intervening computer network 170. The BFPADUM system 140 in the illustrated embodiment may also optionally use one or more evaluation criteria 143 as part of determining the building structure attribute value and / or associated visualization (e.g., specifying a threshold value for accessibility attributes to be considered "accessible" or "inaccessible", to specify a threshold value for viewshed attributes to be considered high viewshed values ​​or low viewshed values, to specify a threshold value for which types of room-to-room connections and / or paths are considered to be connections for connection attributes, etc.), including in some embodiments and situations using user-specific criteria (e.g., based on the user's preferences, personalizing the information presented or otherwise provided to the user, to use the type of information associated with the user type to determine and provide attribute values ​​and / or visualization to the user's type, etc.). In addition, the BFPADUM system 140 in the illustrated embodiment may also include one or more trained machine learning models 144 (e.g., one or more trained neural networks), and use the trained (multiple) machine learning models in various ways, including in some embodiments as part of analyzing building information and determining building structure attribute information, to generate an adjacency graph and vector embedding of the building, to learn the association between specific building structure attributes and the user's type, etc. In addition, in at least some embodiments and scenarios, one or more users of the BFPADUM client computing device 105 can further interact with the BFPADUM system 140 via the network 170 to assist in some automated operations of the BFPADUM system in determining building structure attribute information, and subsequently use the determined building structure attribute information in one or more other automated ways. Other details related to the automated operations of the BFPADUM system are included elsewhere herein, including with respect to Figure 2D-2J and Figure 4A-4B .

[0031] Although in this exemplary embodiment, the ICA and MIGM systems 160 are shown as executing on the same server computing system 180 as the BFPADUM system (e.g., all systems are operated by a single entity or are otherwise executed in cooperation with one another, such as by integrating some or all of the functionality of all systems together), in other embodiments, the ICA system 160 and / or the MIGM system 160 and / or the BFPADUM system 140 may be operated on one or more other systems separate from the system 180 (e.g., on the mobile device 185; one or more other computing systems not shown; etc.), either instead of copies of those systems executing on the system 180 or in addition to the system 180. In other embodiments, the BFPADUM may alternatively operate without the ICA system and / or the MIGM system, and alternatively obtain the panoramic images (or other images) and / or the building floor plans from one or more external sources.

[0032] The various components of the mobile computing device 185 are also Figure 1, including one or more hardware processors 132 (e.g., CPU, GPU, etc.) that execute software (e.g., ICA application 154, optional browser 162, etc.) using executable instructions stored and / or loaded on one or more memory / storage components 152 of the device 185, and optionally one or more types of one or more imaging systems 135 to obtain visual data of one or more panoramic images 165 and / or other images (not shown, such as rectilinear stereo images). In some embodiments, some or all of such images may be provided by one or more separate associated camera devices 184 (e.g., via wired / cabled connections, via Bluetooth or other inter-device wireless communications, etc.), whether in addition to or in lieu of images captured by the mobile device 185. The illustrated embodiment of the mobile device 185 also includes: one or more sensor modules 148, which in this example include a gyroscope 148a, an accelerometer 148b, and a compass 148c (for example, as part of one or more IMU units (not shown separately) on the mobile device); one or more control systems 147 that manage the I / O (input / output) and / or communications and / or networking of the device 185 (for example, receiving instructions from a user and presenting information to the user), such as for other device I / O and communication components 146 (for example, a network interface or other connection, a keyboard, a mouse or other pointing device, a microphone, a speaker, a GPS receiver, etc.), a display system 149 (for example, with a touch-sensitive screen), optionally one or more depth sensing sensors or one or more types of other ranging components 136, optionally a GPS (or global positioning system) sensor 134 or other location determination sensor (not shown in this example), and the like. The other computing devices / systems 105, 175, and 180 and / or camera device 184 may include various hardware components and stored information in a manner similar to mobile device 185, which are not shown in this example for the sake of brevity and are described below with reference to Figure 3 discussed in more detail.

[0033] One or more users (not shown) of one or more client computing devices 175 may also interact with the BFPADUM system 140 (and optionally the ICA system 160 and / or the MIGM system 160) via one or more computer networks 170 to assist in determining building structure attribute information and subsequently use the determined building structure attribute information in one or more other automated ways. Such user interactions may include, for example, specifying target criteria for searching corresponding floor plans or otherwise providing information to the user about target criteria of interest, or obtaining and optionally interacting with one or more specifically identified floor plans and / or with additional associated information (e.g., changing between a floor plan view and a view of a particular image at an acquisition location within or near the floor plan; changing the horizontal and / or vertical viewing direction of a corresponding view of a displayed panoramic image to determine the portion of the panoramic image to which the current user viewing direction is directed, etc.). In addition, the floor plan (or portion thereof) may be linked to or otherwise associated with one or more other types of information, including for a floor plan of a multi-story or multi-story building having multiple associated sub-floors for interconnecting different floors or layers (e.g., by connecting stairways), for a two-dimensional ("2D") floor plan of a building to be linked to or otherwise associated with a three-dimensional ("3D") rendering of a building, and the like. Furthermore, while in Figure 1 Not shown, but in some embodiments, client computing device 175 (or other devices, not shown) may receive and use information about the identified floor plan and / or other mapping-related information in an additional manner to control or assist in automated navigation activities of those devices (e.g., autonomous vehicles or other devices), either in lieu of or in addition to displaying the identified information.

[0034] exist Figure 1 In the computing environment shown, network 170 can be one or more publicly accessible linked networks, possibly operated by various different parties (e.g., the Internet). In other implementations, network 170 can have other forms. For example, network 170 can instead be a private network, such as a company or university network that is completely or partially inaccessible to non-privileged users. In other implementations, network 170 can include private networks and public networks, wherein one or more private networks can access one or more public networks and / or be accessed from one or more public networks. In addition, network 170 can include various types of wired and / or wireless networks in various situations. In addition, client computing device 175 and server computing system 180 can include various hardware components and stored information, as described below with reference to Figure 3 discussed in more detail.

[0035] exist Figure 1 In an example of , the ICA system can perform automated operations, such as using visual data acquired via the mobile device 185 and / or associated camera device 184, that involve generating multiple 360° panoramic images at multiple associated acquisition locations (e.g., in multiple rooms or other locations within a building or other structure, and optionally around some or all of the exterior of the building or other structure), and for generating and providing a representation of the interior of a building or other structure. For example, in at least some such embodiments, such techniques may include using one or more mobile devices (e.g., a camera having one or more fisheye lenses and mounted on a rotatable tripod or otherwise having an automatic rotation mechanism, a camera with a fisheye lens that is sufficiently horizontal to capture 360° without rotation, a smartphone held and moved by a user, a camera held by a user or mounted on the user or on the user's clothing, etc.) to capture data from a series of multiple acquisition locations within multiple rooms of a house (or other building), and optionally further capture movement involving the acquisition device (e.g., movement between acquisition locations, such as rotation; movement between some or all acquisition locations, such as for linking multiple acquisition locations together; etc.), in at least some cases without measured distances between acquisition locations or other measured depth information to objects in the environment surrounding the acquisition locations (e.g., without using any depth sensing sensors). After information about the acquisition location is captured, the technology may include generating a 360 degree panoramic image having 360 degrees of horizontal information about a vertical axis from the acquisition location (e.g., a 360 degree panoramic image showing the surrounding rooms in an equirectangular format), and then providing the panoramic image for subsequent use by the MIGM and / or BFPADUM system.

[0036] One or more end users (not shown) of one or more building information access client computing devices 175 may also interact with the BFPADUM system 140 (and optionally with the MIGM system 160 and / or the ICA system 160) via the computer network 170 to obtain, display, and interact with the generated floor plans (and / or other generated mapping information) and / or determined building structure attribute information and / or associated images, such as by providing information about one or more indicated buildings of interest and / or other criteria and receiving information about one or more corresponding matching buildings, as discussed in more detail elsewhere herein. Figure 7A-7B In addition, although Figure 1Not shown, but a floor plan (or portion thereof) may be linked to or otherwise associated with one or more additional types of information, such as one or more associated and linked images or other associated and linked information, including a two-dimensional ("2D") floor plan for a building being linked to or otherwise associated with a separate 2.5D model floor plan rendering of a building and / or a 3D model floor plan rendering of a building, etc., and including a floor plan for a multi-story or other multi-story building having multiple associated sub-floors for interconnecting different floors or layers (e.g., via connecting stairways), or being portions of a common 2.5D and / or 3D model. Thus, non-exclusive examples of end-user interaction with a displayed or otherwise generated 2D floor plan of a building may include one or more of the following: requesting display or other presentation of one or more indicated types of determined building structural attribute information, and optionally interacting with such displayed or otherwise presented determined building structural attribute information in various ways; changing between views of a particular image at an acquisition location within or near the floor plan and a floor plan view; changing between a 2D floor plan view and a 2.5D or 3D model view, the 2.5D or 3D model view optionally including images of walls of the displayed model texture-mapped; changing the horizontal and / or vertical viewing direction of a corresponding subset view of a displayed panoramic image (or an entrance into the panoramic image) to determine the portion of the panoramic image in the 3D coordinate system to which the current user viewing direction points, and rendering a corresponding plan image showing that portion of the panoramic image without the curvature or other deformation present in the original panoramic image, etc. Furthermore, although in Figure 1 Not shown, but in some embodiments, the client computing device 175 (or other devices, not shown) may receive and use the generated floor plan and / or other generated mapping-related information in an additional manner to control or assist in the automated navigation activities of those devices (e.g., autonomous vehicles or other devices), either in lieu of or in addition to displaying the generated information.

[0037] Figure 1 Also described is an exemplary building interior environment in which 360° panoramic images and / or other images are acquired, such as by an ICA system, and used by a MIGM system (e.g., under control of a BFPADUM system) to generate and provide one or more corresponding building floor plans (e.g., multiple incremental partial building floor plans), and such building information is further used by the BFPADUM system as part of an automatic building match determination operation. In particular, Figure 1One floor of a multi-story house (or other building) 198 is shown having an interior that is captured at least in part by a plurality of panoramic images, such as by a mobile image capture device 185 having image capture capabilities and / or one or more associated camera devices 184 as they move through the building interior to a series of multiple acquisition locations 210 (e.g., starting at acquisition location 210A, moving along a travel path 115 to acquisition location 210B, etc., and ending at acquisition location 210-O or 210P outside the building. Path 115 may not include straight-line movement between locations, as shown for acquisition locations 210A-210C relative to lines 215-AB, 215-AC, and 215-BC). An embodiment of an ICA system may automatically perform or assist in capturing data representing the building interior (and further analyzing the captured data to generate a 360° panoramic image to provide a visual representation of the building interior), and an embodiment of a MIGM system may analyze the visual data of the acquired images to generate one or more building floor plans (e.g., multiple incremental building floor plans) for the house 198. Although such a mobile image acquisition device may include various hardware components, such as a camera, one or more sensors (e.g., gyroscopes, accelerometers, compasses, etc., such as part of one or more IMUs, or inertial measurement units of the mobile device; altimeters; light detectors; etc.), a GPS receiver, one or more hardware processors, memory, a display, a microphone, etc., in at least some embodiments, the mobile device may not have access to or use a device to measure the depth of objects in the building relative to the location of the mobile device, so that in such embodiments, the relationship between different panoramic images and their acquisition locations can be determined based in part or in whole on features in the different images, but without using any data from any such depth sensor, while in other embodiments, such depth data can be used. In addition, although in Figure 1 A direction indicator 109 is provided for reference to the reader relative to the exemplary house 198, but in at least some embodiments, the mobile device and / or the ICA system may not use such absolute direction information and / or absolute location so that in such embodiments, the relative direction and distance between the acquisition locations 210 are determined without regard to the actual geographic location or direction, while in other embodiments, such absolute direction information and / or absolute location may be obtained and used.

[0038] In operation, the mobile device 185 and / or camera device 184 arrives at a first acquisition location 210A within a first room within a building (in this example, in a living room accessible via an exterior door 190-1) and captures or acquires a view of a portion of the building interior visible from the acquisition location 210A (e.g., some or all of the first room, and optionally, small portions of one or more other adjacent or neighboring rooms, such as through a doorway wall opening, a non-doorway wall opening, a hallway, a stairway, or other connecting passageway from the first room). View capture can be performed in various ways as discussed herein, and can include multiple objects or other features (e.g., structural details) that are visible in the image captured from the acquisition location. Figure 1 In the example of FIG. 1 , these objects or other features within the building 198 include doorways 190 (including 190-1 to 190-6, such as having swinging and / or sliding doors), windows 196 (including 196-1 to 196-8), corners or edges 195 (including corner 195-1 at the northwest corner of the building 198, corner 195-2 at the northeast corner of the first room, corner 195-3 at the southwest corner of the first room, corner 195-4 at the southeast corner of the first room, corner 195-5 at the north edge of the inter-room passage between the first room and the corridor, etc.), furniture 191 to 193 (e.g., recliner 191; chair 192; table 193, etc.), pictures or paintings hung on the wall or televisions or other hanging objects 194 (e.g., 194-1 and 194-2), lamps ( Figure 1 Not shown), various built-in household appliances or lamps or other structural elements ( Figure 1 ), etc. The user may also optionally provide a text or auditory identifier to be associated with the acquisition location and / or surrounding room (e.g., a "living room" for one of acquisition locations 210A or 210B or for a room including acquisition locations 210A and / or 210B), while in other embodiments, the ICA and / or MIGM system may automatically generate such an identifier (e.g., by automatically analyzing images and / or videos of the building and / or other recorded information to perform a corresponding automatic determination, such as by using machine learning; based at least in part on input from an ICA and / or MIGM system operator user, etc.) or may not use an identifier.

[0039] After the first acquisition location 210A has been captured, the mobile device 185 and / or camera device 184 can move or move to the next acquisition location (such as acquisition location 210B) during the movement between acquisition locations, optionally recording images and / or video and / or other data from hardware components (e.g., from one or more IMUs, from cameras, etc.). At the next acquisition location, the mobile device 185 and / or camera device 184 can similarly capture 360° panoramic images and / or other types of images from the acquisition location. The process can be repeated for some or all rooms of the building and in some cases outside the building, as shown in this example for additional acquisition locations 210C-210P, in which images from acquisition locations 210A to 210-O are acquired in a single image acquisition period (e.g., in a substantially continuous manner, such as within a total of 5 minutes or 15 minutes), and images from acquisition location 210P are optionally acquired at different times (e.g., from a street adjacent to a building or a building's forecourt). In this example, multiple acquisition locations 210K-210P are external to but associated with building 198, including acquisition locations 210L and 210M in one or more additional structures on the same property (e.g., ADU 189 or accessory dwelling unit; garage; shed; etc.), acquisition location 210K on exterior deck or patio 186, and acquisition locations 210N-210P at multiple yard locations on property 241 (e.g., backyard 187, side yard 188, front yard including acquisition location 210P, etc.). The acquired images for each acquisition location can be further analyzed, including rendering or otherwise placing each panoramic image in an equirectangular format in some embodiments, either at the time of image acquisition or later, and further analyzed by the MIGM and / or BFPADUM system in the manner described herein.

[0040] refer to Figure 1 Various details are provided, but it is to be understood that the details provided are non-exclusive examples included for purposes of illustration and that other implementations may be performed in other ways without some or all of such details.

[0041] Figure 2A-2J Examples are shown of automatically determining information about structural features and other properties of a building based on analysis of at least a floor plan of the building and subsequently using the determined building information in one or more automated ways, e.g. Figure 1 The building discussed in 198.

[0042] In particular, Figure 2A An example image 250a is shown, for example, Figure 1250a (or a subset view of a 360° panoramic image facing northeast, taken from the acquisition location 210B in the living room of the house 198 of FIG. 250a). A direction indicator 109a is further displayed in this example to show the direction in the northeast direction in which the image was taken. In the example shown, the displayed image includes built-in elements (e.g., light fixture 130a), furniture (e.g., chair 192-1), two windows 196-1, and a picture 194-1 hanging on the north wall of the living room. No inter-room passages (e.g., doorways or other wall openings) entering or leaving the living room are visible in this image. However, multiple room boundaries are visible in image 250a, including a horizontal boundary between a visible portion of the north wall of the living room and the ceiling and floor of the living room, a horizontal boundary between a visible portion of the east wall of the living room and the ceiling and floor of the living room, and an inter-wall vertical boundary 195-2 between the north wall and the east wall.

[0043] Figure 2B continue Figure 2A of examples, and shows that Figure 1 198 of house 198 taken in a northwest direction (or a subset view of the same 360° panoramic image taken from the acquisition location and formatted in a rectilinear manner). Direction indicator 109b is further displayed to show the northwest direction from which the image was taken. In this example image, a small portion of one of windows 196-1, as well as a portion of window 196-2 and new lighting fixture 130b continue to be visible. In addition, in image 250b, a portion of window 196-2 is shown in a manner similar to that of FIG. Figure 2A Horizontal and vertical room boundaries are visible in a .

[0044] Figure 2C continue Figure 2A-2B of examples, and shows that Figure 1 198, e.g., taken from acquisition location 210B (or a southwest-facing subset view of the same 360° panoramic image taken from that acquisition location and formatted in a rectilinear manner). Direction indicator 109c is further displayed to show the southwest direction in which the image was taken. In this example image, a portion of window 196-2 continues to be visible, similar to Figure 2A and Figure 2B The example image also shows two inter-room passages for the living room, which in this example include a doorway 190-1 with a swinging door for entering and leaving the living room ( Figure 1It is identified as a door to the exterior of the house, such as a front yard), and a doorway 190-6 having a sliding door for moving between the living room and the side yard 188. Figure 1 . Additional non-doorway wall opening 263a exists in the east wall of the living room to move between the living room and the hallway, but is not visible in images 250a to 250c. It should be understood that various other stereo images can be acquired from acquisition location 210B and / or other acquisition locations and displayed in a similar manner.

[0045] Figure 2D continue Figure 2A-2C , and shows a 360° panoramic image 255d (e.g., acquired from acquisition location 210B) that displays the entire living room in an equirectangular format. Figure 2A-2C The stereo images are oriented in the same way, so Figure 2D No direction indicator 109 is displayed in the panoramic image, even though the pose of the panoramic image may include one or more associated directions (e.g., the starting and / or ending directions of the panoramic image, such as if acquired by rotation). A portion of the visual data of the panoramic image 255d corresponds to the first stereoscopic image 250a (shown approximately in the center portion of the image 250d), while the left portion of the image 255d and the far-right portion of the image 255d contain visual data corresponding to the visual data of the stereoscopic images 250b and 250c. The exemplary panoramic image 255d includes windows 196-1, 196-2, and 196-3, furniture 191-193, doorways 190-1 and 190-6, and a non-doorway wall opening 263a leading to a corridor room (the opening showing a portion of the door 190-3 visible in the adjacent corridor). Image 255d also shows various room boundaries in a manner similar to the stereoscopic images, but the horizontal boundaries are shown in a gradually curved manner the farther they are from the horizontal centerline of the image. Visible boundaries include vertical wall-to-wall boundaries 195-1 to 195-4, a vertical boundary 195-5 at the left / north side of the corridor opening, a vertical boundary at the right side of the corridor opening, and horizontal boundaries between the walls and the floor and between the walls and the ceiling.

[0046] Figure 2D Also shown is an example 260d of a 2D floor plan for house 198, such as may be presented to an end user in a GUI, where the living room is the westernmost room of the house (as reflected by direction indicator 209). It should be appreciated that in some embodiments, a 3D or 2.5D floor plan with rendered wall height information may be similarly generated and displayed. An example of such a 3D floor plan is provided in addition to or in lieu of such a 2D floor plan. Figure 2JIn this example, various types of information are shown on the 2D floor plan 260d, such as one or more of the following: room labels added to some or all rooms (e.g., "Living Room" for a living room); room dimensions added to some or all rooms; visual indications of features, such as installed fixtures or appliances (e.g., kitchen appliances, bathroom items, etc.) or other built-in elements added to some or all rooms (e.g., a kitchen island); additional types of associated and linked information (e.g., other panoramic images and / or stereo images that the end user can select for further display, audio annotations and / or sound recordings that the end user can select for further presentation); , etc.); visual indications added to some or all rooms of locations of buildings (e.g., buildings, structures, buildings, etc.); visual indications added to some or all rooms of structural features such as doors and windows; visual indications of visual appearance information (e.g., color and / or material type and / or texture of installed items, such as floor coverings or wall coverings or surface coverings); visual indications of views from a particular window or other building location and / or other information about the exterior of the building (e.g., type of exterior space; items present in the exterior space; other related buildings or structures, such as sheds, garages, pools, decks, patios, walkways, gardens, etc.); keywords or legends 269 identifying visual indicators for one or more types of information, etc. When displayed as part of a GUI, some or all of such displayed information may be user-selectable controls (or associated with such controls) that allow an end-user to select and display some or all of the associated information (e.g., selecting an indicator to obtain a 360° panoramic image of location 210B to view a portion or all of the panoramic image (e.g., in a manner similar to FIG. 214 ). Figure 2A-2D In addition, in this example, a user-selectable control 261 is added to indicate the current floor displayed for the floor plan, and to allow the end user to select a different floor to be displayed. In some embodiments, changes to floors or other levels may also be made directly from the floor plan, such as via selection of a corresponding connecting path in the illustrated floor plan (e.g., stairs to floor 2). It should be understood that in some embodiments various other types of information may be added, that in some embodiments some of the illustrated types of information may not be provided, and that in other embodiments visual indications and user selections of linked and associated information may be displayed and selected in other ways.

[0047] Figure 2E continue Figure 2A-2D, and shows information 255e including a floor plan 230e of a building 198, and various information regarding structural aspects of the building floor plan, which is analyzed as part of determining one or more accessibility attributes and corresponding accessibility values ​​in order to evaluate width, height, length, elevation gain or loss, and other physical characteristics used for attribute determination. In the example shown, the types of structural building information analyzed include one or more of the following: entrance areas 211 for obtaining access to the building (including stairs 211a and ramp 211b in this example); stairs 212 (including stairs 212a to the second floor); doorway widths 213 (including entrance doorways 213a and doorways 213b from the hallway to bedroom 2, doorways 213c between bedroom 2 and the bathroom, doorways 213d between the bathroom and the hallway, sliding doorways 213e between the kitchen / dining room and the exterior wooden deck, doorways 213f between the hallway and bedroom 1, etc.); non-doorway wall openings 214 (including openings 214a between the living room and the hallway, openings 214b between the kitchen / dining room and the family room, etc.); and wall openings 215 and 216 for various household appliances and fixtures (including openings such as flush toilets). , bath items such as tubs or showers, sinks, cabinets, and light switches (not shown), etc., and kitchen items such as islands, countertops, ovens, stoves, refrigerators, dishwashers, sinks, cabinets, and light switches (not shown), etc.) and surrounding areas; the location and surrounding areas of movement of movable elements 217 (e.g., the swinging or sliding of doors when opened, the movement of drawers when opened, etc.); the widths and lengths of various structural elements 218 (including walls, such as the width of the north wall 218a of the living room and the width of the east wall 218b of the living room for assessing the accessibility of the northeast corner of the living room, etc.); exterior surfaces 219 on which movement may occur (e.g., wooden decks 219a, the area between the deck stair exit and the ADU entrance 219b, paver rooms 219c, other areas of the yard shown as grass in this example 219d, etc.), etc. It should be understood that various other structural elements within the building may also be measured or otherwise assessed for other areas of the building (e.g., the second floor), etc., including additional elements of the shown type that are not explicitly indicated. As discussed in more detail elsewhere herein, various measurement data for aspects of a building structure may be combined to determine accessibility values ​​for one or more accessibility attributes at each of one or more locations in the building, including, in some embodiments, accessibility values ​​for each location in a grid spanning some or all of the building and / or surrounding property (e.g., as described in detail elsewhere herein). Figure 2F shown).

[0048] Figure 2F continue Figure 2A-2E and illustrates an example that can be based at least in part on Figure 2EThe types of building structure information shown in the figure are used to determine and display the types of accessibility attributes and corresponding values ​​for one or more users in a GUI (not shown). In particular, Figure 2F Information 255f is included, which includes a visualization 255f1 of accessibility attributes associated with a wheelchair user, including using a grid of locations of the entire floor plan and surrounding property overlaid on a visual representation of the floor plan and property with a heat map displayed on the grid to reflect the degree of accessibility for such wheelchair user. In this example, the visual attribute of each cell of the grid includes one of 7 listed accessibility values ​​ranging from least accessible to most accessible, and an additional 8th value for areas of the building and / or property where a corresponding accessibility value is not determined. In this example, although the wheelchair user cannot access the stairs, the wheelchair user is able to gain entrance to the building via the front ramp, with the living room, hallway, bedroom 2, kitchen / dining room, and family room all generally having accessibility values ​​of the second highest accessibility value (except for corners and other limited areas of lower accessibility). In contrast, bedroom 1 has a low accessibility score due to the narrow doorway between the hallway and bedroom 1 being difficult or impossible to maneuver in a wheelchair. Similarly, the bathroom had an intermediate accessibility score due to only a small area for a wheelchair to maneuver around fixtures and other objects, and a lack of grab bars or other accessibility aids installed in the room. Areas in the small square in the kitchen / dining room and its swinging door were also rated low in accessibility, for example due to limited space for a wheelchair user to maneuver with the door open between the island and countertops, and areas near the corners of the room were not easily accessible due to the proximity of multiple walls. On the property outside the building, the wooden deck and paved patio areas were relatively accessible due to the easy access provided by the sliding doors and the hard and easy traversal of the exterior surface and being substantially flush with the room level, but the rest of the side yard was relatively inaccessible due to the grass surface and narrow width, and the back yard was very difficult to access due to the lack of easy access from the deck down to the grass (due to stairs) or from the side yard (due to the grass surface and the door between the side yard and the back yard having a high latching mechanism not shown). Although the interior of the ADU would otherwise be accessible if a wheelchair user were located inside the ADU, the interior of the ADU similarly has a low accessibility score for the same reasons as the backyard and because the gravel between the ADU and the landing stairs is difficult to reach. In addition to the heat map shown, additional accessibility attributes are shown in text / numerical form, including the number of inaccessible rooms, the ratio of inaccessible rooms to total rooms, the size of the inaccessible area of ​​the property (in square feet), the percentage of the property in this example that includes the inaccessible area, etc.

[0049] Figure 2FAdditional accessibility data 255f2 corresponding to additional accessibility attributes for a user who can walk but has limited mobility is further illustrated, wherein portions of the building floor plan and property are illustrated with separate heat maps corresponding to accessibility values ​​determined for the respective accessibility attributes. In this example, a large portion of the floor plan is more accessible to such a walking user than to a wheelchair user due to the greater maneuverability of such a walking user and the fact that the backyard and side yard are somewhat accessible.

[0050] It should be understood that Figure 2F Examples of accessibility data can differ in various ways in other implementations, including displaying information for other types of accessibility attributes (whether in addition to or in place of some or all of the types of accessibility data shown), displaying accessibility data in other ways (e.g., displaying accessibility values ​​in a manner other than a heat map or text / numeric format), providing accessibility information in response to user interaction (e.g., enabling a user to select one or more rooms or other areas and / or select one or more types of accessibility attributes for which accessibility values ​​are displayed), and the like.

[0051] As a non-exclusive example of a technique for determining a value for an accessibility attribute, a home's convenience with respect to accessibility may be determined in order to evaluate aspects including the home's structure and objects that are difficult and / or expensive to change / modify. Such accessibility determination may be used in a variety of ways, such as by homeowners improving their homes, by buyers evaluating their convenience (e.g., senior personnel), by agents conducting home advertising and evaluations, and the like. Non-exclusive examples of aspects for evaluation include the following: inaccessible structures that are too narrow to be accessed, such as the number of inaccessible doorways, one or more types of corresponding rooms (e.g., closets, bathrooms, etc.), since the impact can be evaluated differently for rooms with different functions, the ratio of the number of accessible doors to the number of inaccessible doors, measurements such as minimum and / or maximum and / or average doorway widths, etc.; inaccessible non-doorway wall openings (e.g., non-doorway boundaries across different rooms, such as the number of inaccessible openings, the ratio of the number of accessible openings to the number of inaccessible openings, measurements such as minimum and / or maximum and / or average opening widths, etc.); corridors (typically the most used area in a house, but also the narrowest area), so that the width and / or length and / or number of corners / turns can be measured the number of doors and / or exits from hallways, etc.; specific rooms with important functions and their objects, such as the location of bathtubs / showers and / or sinks / vanities and / or toilets in bathrooms, the location of sinks and / or islands and / or countertops and / or appliances in kitchens, etc.; the presence of stairs and their characteristics / characteristics between floors and / or levels, such as whether the house is split-level and / or has multiple floors, whether the stairs have handrails, stair measurements (e.g., height and / or width and / or slope), etc.; the presence of stairs to enter the house; the presence of elevators and their characteristics / characteristics (e.g., size) between floors and / or levels; the heights of various objects, such as windows, door handles, toilet housings, countertops, sinks, stoves, bathtubs, switches, dishwashers, etc.; the connectivity between rooms in a house (e.g., assessed by one or more connectivity attributes), etc.Non-exclusive examples of evaluations of these aspects include the following: for a given location in a room, the distance to the room's door and / or corners and / or walls and / or stairs and / or the location of any immovable objects (e.g., an air conditioner), so as to reduce the accessibility value of the location as one or more such distances decrease (e.g., different rooms have different sensitivities to the weighting of some or all of these factors, so that the impact of a small distance on inaccessibility to a bathroom is increased due to its functional importance, high frequency of use, and immovable fixtures, and similarly the impact of a kitchen and / or other multi-purpose rooms is increased), and if available, combining multiple such distances (giving importance to particular factors in a weighted manner); with respect to object heights, the difference from one or more defined target heights (e.g., one or more "convenient" heights, such as with respect to a relevant indicated type of user), so as to reduce the accessibility value of the object location as such differences increase (e.g., having different sensitivities for different objects to give such differences, such as increasing the impact of such differences on the inaccessibility of particular objects due to their functional importance and / or high frequency of use and / or location in a kitchen or bathroom); etc. In at least some embodiments, to determine an accessibility value for a room or other area or an entire house, a combination of the factors and aspects discussed above are used to evaluate the accessibility value of each of some or all locations in the room / other area / house, and the evaluated accessibility values ​​of those some / all locations are combined to determine an aggregate accessibility value for the room / other area / house (e.g., using an average, accumulation, etc.). It should be understood that other factors, aspects, and determination techniques may be used in other embodiments.

[0052] As another non-exclusive example of a technique for determining the value of an accessibility attribute, such an accessibility attribute may include one or more of the following: the number of inaccessible rooms or other areas or zones (e.g., used as a general indicator of accessibility); the ratio of inaccessible rooms / other areas / zones to the total number of rooms / other areas / zones (e.g., standardized for buildings of different sizes); the dimensions of the inaccessible rooms / other areas / zones (e.g., in square feet, cubic feet, etc.); the ratio of the dimensions of the inaccessible rooms / other areas / zones to the overall dimensions of the floor plan or rooms / other areas / zones of the same type (e.g., for bedrooms, living rooms, bathrooms, etc.), etc. Determination of an evaluation value for such an accessibility attribute of a location may include, for example, the following: first evaluating whether there is a doorway within a specified distance; if there is no doorway, measuring the distance from the wall to the location, and if the distance on the opposite side is less than the specified distance (e.g., for a narrow hallway), reducing the accessibility value; if there is a doorway, considering the wall distance and the doorway width; if there are stairs or other steps to reach the location (e.g., for a location in a sunken living room), reducing the accessibility value, for example, based on the height and number of steps; etc. It should be understood that other factors, aspects, and determination techniques may be used in other embodiments.

[0053] Figure 2G continue Figure 2A-2F , and shows information 255g including a floor plan 230e for a building, and various information regarding structural aspects of the floor plan of the building, which is analyzed as part of determining one or more viewshed attributes and corresponding viewshed values. In this example, determining the value of the viewshed includes determining at least two-dimensional lines of sight for each of at least some locations for performing a determination of the amount of square footage and / or volume of space visible from the location. As will be appreciated, if only a two-dimensional floor plan is available, without any information regarding the heights of rooms and other areas, the viewshed value may be determined to indicate an amount of square footage, whereas if a three-dimensional floor plan or other building model is available (e.g., as described with respect to FIG. 1 ), the viewshed value may be determined to indicate an amount of square footage. Figure 2J If the location is not shown) or other information about the height of rooms and other areas is available (e.g., from analysis of visual data of building images), then the viewshed value can be determined relative to square footage and / or volume of space. In other embodiments and situations, other measures of visibility and / or spaciousness can be used, whether in addition to or in lieu of the shown examples of using square footage and / or volume of space visible from the location.

[0054] exist Figure 2GIn the example shown, for location 221a on the east side of the living room, the line of sight shown includes substantially all of the living room, portions of the hallway, and additional portions of bedroom 2 and the bathroom, which are visible through the doorway between the hallway and bedroom 2 and the doorway between bedroom 2 and the bathroom if they are open. In contrast, the line of sight shown from nearby location 221b in the living room includes little to no visibility into the room along the hallway, while also including substantially all of the living room, and also including all of the hallway and kitchen / dining room through non-doorway wall openings at the east and west ends of the hallway. Therefore, the viewshed values ​​for both locations 221a and 221b will be relatively high, given the large square footage and / or volume of the living room and the additional areas that are visible. In contrast, location 221c in the bathroom may have a relatively small viewshed value due to the relatively small square footage and / or volume of the bathroom itself, and only limited visibility of other areas (e.g., bedroom 2, bedroom 1, and hallway) through the doorway between the bathroom and bedroom 2 or the hallway. It should be understood that similar viewshed values ​​can be analyzed for different locations throughout a building, and in some embodiments, visibility through windows into exterior areas can be further included as part of the viewshed determination for a given location, and viewshed values ​​for locations on the property outside the building can be similarly determined.

[0055] Figure 2H continue Figure 2A-2G and illustrates an example that can be based at least in part on Figure 2G The types of building structure information shown in the figure are non-exclusive examples of the types of viewshed attributes and corresponding values ​​that are determined and displayed for display to one or more users in a displayed GUI (not shown). Specifically, Figure 2H Information 255h is included regarding visualization of viewshed attributes based on the degree of visibility / comfort of the viewshed for the locations from the grid using a grid of locations overlaid on a visual representation of a building plan, and a heat map overlaid on the visual representation of the building plan. In this example, the viewshed attributes for each cell of the grid include one of seven enumerated viewsheds from minimum visibility / comfort to maximum visibility / comfort, and an additional eighth value for areas of the building and / or property where the viewshed value is uncertain. Figure 2G The locations 221a and 221b shown in the living room are between the various locations with the highest visibility / spaciousness in the living room and kitchen / dining room, while Figure 2GLocation 221c and other areas of the bathroom and bedroom 2 are shown as having below average viewshed values, small areas with limited sight lines, such as pantry and locker with the lowest visibility / spaciousness. In addition to the heat map shown, additional viewshed attributes are shown in text / numerical form, including the square footage size of areas of the living room and bathroom that have viewshed values ​​above a defined threshold (e.g., the top 2 or top 3 listed viewshed values), the percentage of those rooms that have viewshed values ​​above a defined threshold, etc. It should be understood that similar information can be determined and displayed for other groups of one or more rooms or other areas.

[0056] It will be further understood that Figure 2H Examples of viewshed data may differ in various ways in other embodiments, including displaying information of other types of viewshed attributes (either in addition to or in place of some or all of the types of viewshed data shown), displaying viewshed data in other ways (e.g., displaying viewshed data values ​​in a manner other than as a heat map), providing viewshed information in response to user interaction (e.g., causing a user to select one or more rooms or other areas and / or select one or more types of viewshed attributes for displaying viewshed data).

[0057] As a non-exclusive example of a technique for determining and using values ​​for viewshed attributes, viewshed values ​​may be used to provide information to a user about the privacy or openness of a home, such as a home with a high viewshed value, indicating that the home has a larger area of ​​visual overlap, which implies spaciousness, and a home with a low viewshed value, indicating that the home has more enclosed / private areas. Even if two homes have the same overall size, their total viewshed size and distribution in different rooms / areas may be significantly different. Furthermore, different types of viewshed values ​​may be desired and / or preferred for different room types, such that a larger viewshed size / ratio in a living room (e.g., if used for social activities) and a smaller viewshed size / ratio in a bedroom (e.g., to provide privacy) may be desired / preferred. Thus, viewshed values ​​for specific room types (e.g., bedrooms and dining rooms) may be determined, as well as viewshed values ​​for viewshed size and viewshed ratio. Thus, non-exclusive viewshed attributes may include the following: viewshed sizes for different room types (e.g., bedroom, storage, playroom, kitchen, hallway, balcony, pool, cabinet, staircase, dining room, garage, etc.); viewshed ratios for different room types (viewshed size / room size); viewshed ratio distribution, etc. As a non-exclusive example of determining viewshed values, a viewshed value may be determined for each floor, such as by treating walls as opaque (no visibility), and treating windows and doorways and non-doorway wall openings as transparent, setting the location of each room at its center, and then checking the visibility area over the entire floor plan at least at the center of each room. It should be understood that in other embodiments, other factors, aspects, and determination techniques may be used.

[0058] Fig.2I continue Figure 2A-2H, and shows information 255i including a floor plan 230e of a building, and various information about structural aspects of the floor plan of the building, the structural aspects of the floor plan of the building being analyzed as part of determining one or more connectivity attributes and corresponding connectivity values, such as shown in information 255i. In this example, the determination of the connectivity value includes generating an adjacency graph showing information about direct connections 235 between adjacent rooms having doorways or other non-doorway wall openings (e.g., direct room-to-room connections 235a-b between a location 244b near the center of the living room and a location 244a near the center of the hallway) and room adjacencies 236 for indirect connections between adjacent rooms lacking doorways or other non-doorway wall openings therebetween (e.g., room adjacencies 236b-f between a location 244b in the living room and a location 244f in bedroom 1, the actual indirect room-to-room connections between these locations including an intermediate passage through the hallway). In the example shown, connections 235 are further shown between rooms of the building and exterior areas through doorways or other openings, such as connections 235b-j between location 244b in the living room and location 244j outside the front entry door, although such exterior connections may not be used in some embodiments and scenarios. Moreover, in other embodiments and scenarios, connectivity between other areas may be further determined, such as between two or more exterior areas, between locations in the building 198 and one or more exterior buildings (e.g., an ADU in a backyard), etc. Moreover, in other embodiments and scenarios, other measures of connectivity may be used, either in addition to or in lieu of using the illustrated examples regarding direct connections and / or other adjacent room locations.

[0059] Fig.2I The information 255i2 further illustrates that the Fig.2I2 and 3. A non-exclusive example of types of connectivity attributes and corresponding values ​​that may be determined and displayed based on the type of building structure information shown in FIG. 2 for one or more users in a displayed GUI (not shown). In particular, in this example, some connectivity attributes with associated connectivity values ​​are shown in text / numeric form, including the distance between two or more locations and / or the number of inter-room connections, where the paths 237 shown correspond to some pairs of locations (e.g., paths 237c-d between bedroom 2 and bathroom 2). It should be understood that similar information may be determined and displayed for other groups of two or more locations (including groups of 2 or more rooms, other indications of paths, etc.). Additionally, in other embodiments, the connectivity information may be displayed in other ways, such as by overlaying some or all of a visual representation of an adjacency graph showing direct room-to-room connections and / or indirect connected room adjacencies (e.g., having a straight line between two locations, such as through one or more walls or other structural barriers, and optionally having associated values ​​to show straight-line distance), and / or an overlaying heat map showing connectivity values ​​for each of some or all locations on a grid throughout the building (e.g., regarding connectivity from the location to a determined destination, regarding relative connectivity from the location to multiple other locations of interest in the building, etc.).

[0060] It should be understood that Fig.2I Examples of connectivity data may differ in various ways in other embodiments, including displaying information for other types of connectivity properties (either in addition to or in place of some or all of the shown types of connectivity data), displaying connectivity data in other manners (e.g., displaying connectivity data values ​​in a manner other than as lines overlaid on a floor plan), providing connectivity information in response to user interaction (e.g., for a user to select one or more rooms or other areas and / or select one or more types of connectivity properties for which connectivity data is to be displayed, or otherwise selecting two or more locations and providing information about connectivity between those locations), etc.

[0061] As a non-exclusive example of a technique for determining and using values ​​for connectivity attributes, connectivity values ​​can provide an effective convenience indicator when walking through different rooms, especially connectivity across specific types of rooms (e.g., between bedrooms and bathrooms, between kitchens and dining rooms, etc.). Connectivity attributes may include or be based at least in part on various metrics, non-exclusive examples of which include: walking / skeleton distance to reflect the walking length of moving from a particular room (e.g., a bedroom) or location to another target room (e.g., a bathroom) or location; straight-line or empty-line distance to reflect the direct distance between two rooms or other locations if there are no structural barriers between the two rooms or other locations; hop number to reflect the number of rooms passed through when moving from one room (e.g., a bedroom) or other location to another target room (e.g., a bathroom) or location, without counting the starting room (e.g., if one can walk directly from a bedroom to a bathroom without passing through one or more other rooms, the hop number is 1, and if passing through a hallway or living room, the hop number is 2); room-to-room connectivity to reflect the number of doorways or non-doorway wall openings passed through when a person moves from one room (e.g., a bedroom) or other location to another target room (e.g., a bathroom) or location; etc. Non-exclusive examples of specific connectivity attributes that may be determined and used include the following: bedroom to bathroom; bedroom to living room; living room to bathroom; living room to kitchen; dining room to kitchen; living room to stairway; etc. In addition, multiple such metrics may be aggregated to determine an overall connectivity attribute for a particular room or other area or particular location, where non-exclusive examples of such aggregate connectivity attributes include the following: overall bedroom connectivity score; overall living room connectivity score; overall bedroom connectivity score; overall dining room connectivity score; overall whole home connectivity score; etc. As a non-exclusive example of determining the value of a connectivity attribute, given a floor plan, an adjacency graph is determined by analyzing the floor plan structure and indicating which rooms are connected and how they are connected (e.g., by doorways, non-doorway wall openings, windows, etc.). The adjacency graph is then used to determine other information, non-exclusive examples of which include: determining the number of hops and / or inter-room connectivity values ​​from each room to one or some or all other rooms; determining the walking distance between some or all rooms (e.g., rooms with direct connections) by selecting the center points of two rooms and calculating the distance between these two points, including determining the path used in at least some embodiments and situations (e.g., using the center path of the hallway), and optionally combining direct room connection information in some situations (e.g., determining the walking distance from bedroom 2 in building 198 to the kitchen / dining room by combining the walking distance from bedroom 2 to the hallway and the walking distance from the hallway to the kitchen / dining room) and using the shortest path if multiple paths are available; etc. It should be understood that other factors, aspects, and determination techniques may be used in other embodiments.

[0062] As part of determining the value of one or more accessibility attributes and / or one or more viewshed attributes and / or one or more connectivity attributes, information about various other building attributes may be considered and used, non-exclusive examples of such other building attributes including the following: number of floors; number of windows; number of doors; window widths and heights; doorway widths and heights; different room types (e.g., bedroom, warehouse, playroom, kitchen, hallway, balcony, swimming pool, toilet, staircase, dining room, garage, etc.) and / or locations (e.g., rooms on the first floor, rooms on other floors, etc.) and / or specific types (e.g., first floor, rooms on other floors, etc.); the number of rooms (e.g., master bedroom on a particular floor); average room size, such as for all rooms or rooms of a particular type or in a particular location; room size ratios, such as for all rooms or rooms of a particular type or in a particular location and compared to the overall size of the building or group of rooms (e.g., rooms on a particular floor); bathroom-to-bedroom ratio; window size; daylight score (e.g., the amount or level of daylight reaching a given room, such as based on window size, home orientation, etc.); presence of a "kitchen triangle" (e.g., a sink, oven / stove, and refrigerator that are within a defined distance of each other and not obstructed by other objects);

[0063] Figure 2J continue Figure 2A-2I and illustrates optional floor plans and property representations that may be used in some embodiments as part of determining values ​​for accessibility attributes and / or viewshed attributes and / or connectivity attributes, to be analyzed to determine values ​​for such attributes, as discussed in greater detail elsewhere herein. In particular, Figure 2J Information 255j including a 3D floor plan model 265j of a building 1 (in this example only the first floor is shown) is shown, for example, which may be displayed on a screen similar to Figure 2D GUI 260j. Figure 2J Not shown, but Figure 2D and / or Figure 2F and / or Figure 2H and / or Fig.2I Some or all of the additional types of information shown in the 2D plan view may be similarly displayed in the Figure 2J In the 3D floor plan model shown. Figure 2J Also not shown, but in some embodiments, additional information can be added to the displayed walls, such as from images taken during video capture (e.g., by "texture mapping" the walls by rendering and illustrating the actual paint, wallpaper, or other surfaces of the house on the rendering model 265), and / or can otherwise be used to add specified colors, textures, or other visual information to walls and / or other surfaces. Figure 2JAlso shown is an additional visual representation 266j showing an alternative 3D floor plan model of the building 198 and some or all of the surrounding property 241 to provide a "field plan" view showing exterior paths and surfaces, vegetation, etc. It should be understood that other types of floor plan information may be presented and / or analyzed in other embodiments and scenarios.

[0064] Already referred to Figure 2A-2J Various details are provided, but it is to be understood that the details provided are non-exclusive examples included for purposes of illustration and that other implementations may be performed in other ways without some or all of such details.

[0065] As described above, in some embodiments, the described techniques include using machine learning to learn attributes and / or other characteristics of an adjacency graph to encode in a generated corresponding vector embedding, such as the attributes and / or other characteristics that are most capable of subsequently automatically identifying a building floor plan having attributes that meet a target criterion, and using the vector embedding in at least some embodiments to identify a target building floor plan being encoded based on such learned attributes or other characteristics. In particular, in at least some such embodiments, graph representation learning is used to search for a mapping function that can map nodes in a graph to a d-dimensional vector such that similar nodes in the graph have similar embeddings in the learned space. Unlike traditional methods (such as graph kernel methods) (see, for example, SVNVishwanathan et al., "Graph Kernels", Journal of Machine Learning Research, 11:1201-1242, 2010; and Nils M. Kriege et al., "A Survey On Graph Kernels", arXiv:1903.11835, 2019), graph neural networks eliminate the process of manually designing features and directly learn high-level embeddings from the raw features of nodes or (sub)graphs. There are various techniques for extending and redefining convolutions in the graph domain, which can be classified as spectral methods and spatial methods.Spectral methods employ spectral representations of graphs and are specialized for specific graph structures, such that a model trained on one graph is not applicable to graphs with different structures (e.g., see Joan Bruna et al., “Spectral Networks And Locally Connected Networks On Graphs,” International Conference on Learning Representations 2014, 2014; Michael Defferrard et al., “Convolutional Neural Networks On Graphs With Fast Localized Spectral Filtering,” Proceedings of Neural Information Processing Systems 2016, 2016, pp. 3844–3852; and Thomas N. Kipf et al., “Semi-Supervised Classification With Graph Convolutional Networks,” International Conference on Learning Representations 2014 (International Conference on Learning Representations), 2017). The convolution operation of spectral methods is defined in the Fourier domain by computing the eigendecomposition of the graph Laplacian, and the filters can be approximated by a Chebyshev expansion of the graph Laplacian to reduce the expensive eigendecomposition, resulting in local filters, where the filters are optionally restricted to work on neighboring nodes one step away from the current node. Regarding spatial methods, it consists in learning an embedding of a node by recursively aggregating information from its local neighbors. Various numbers of neighboring nodes and corresponding aggregation functions can be handled in various ways.For example, a fixed number of neighbors of each node can be sampled, and different aggregation functions such as average, maximum, and long short-term memory networks (LSTM) can be used (e.g., see Will Hamilton et al., "Inductive Representation Learning On Large Graphs", Proceedings of Neural Information Processing Systems 2017, 2017, pp. 1024–1034). Alternatively, each neighboring node can be considered to contribute differently to the central node, and the contribution factors can be learned through a self-attention model (e.g., see P. Velickovic et al., "Graph Attention Networks", International Conference on Learning Representations 2018, 2018). Furthermore, each attention head captures feature correlations in a different representation subspace and can be treated differently, for example by using a convolutional subnetwork to weight the importance of each attention head (e.g., see Jiani Zhang et al., “GaAN: Gated Attention Networks For Learning On Large And Spatiotemporal Graphs,” Proceedings of Uncertainty in Artificial Intelligence 2018, 2018).

[0066] Furthermore, in some embodiments, creation of an adjacency graph and / or associated vector embedding for a building may also be based in part on partial information provided for the building (e.g., by an operator user of the BFPADUM system, by one or more end users, etc.). Such partial information may include, for example, one or more of the following: providing some or all room names of the rooms of the building, where connections between the rooms are to be automatically determined or otherwise established; providing some or all inter-room connections between the rooms of the building, with possible room names used to automatically determine or establish the rooms; providing some room names and inter-room connections, with other inter-room connections and / or possible room names automatically determined or otherwise established. In such embodiments, the automation technique may include using the partial information as part of completing or otherwise generating a floor plan for the building, where the floor plan is then used to create the corresponding adjacency graph and / or vector embedding.

[0067] Figure 3 300 and 380, which are block diagrams showing an embodiment of one or more server computing systems 300 executing an implementation of a BFPADUM system 340 and one or more server computing systems 380 executing an implementation of an ICA system 388 and a MIGM system 389. The server computing system(s) and the BFPADUM and / or ICA and / or MIGM systems may be implemented using a plurality of hardware components forming electronic circuits adapted and configured to perform at least some of the techniques described herein when in combined operation. The one or more computing systems and devices may also optionally execute a building information access system (e.g., server computing system(s) 300) and / or optional other programs 335 and 383 (e.g., server computing systems(s) 300 and 380, respectively, in this example). In the illustrated embodiment, each server computing system 300 includes one or more hardware central processing units ("CPUs") or other hardware processors 305, various input / output ("I / O") components 310, storage devices 320, and memory 330, and the illustrated I / O components include a display 311, a network connection 312, a computer-readable media drive 313, and other I / O devices 315 (e.g., I / O devices, keyboards, mice or other pointing devices, microphones, speakers, GPS receivers, etc.). Each server computing system 380 can have similar components, although for simplicity, only one or more hardware processors 381, memory 387, storage devices 384, and I / O components 382 are shown in this example.

[0068] In the illustrated embodiment, server computing system(s) 300 and executing BFPADUM system 340, server computing system(s) 380 and executing ICA system 388 and MIGM system 389, and optionally executing a building information access system (not shown) can communicate with each other and other computing systems and devices, for example via one or more networks 399 (e.g., the Internet, one or more cellular telephone networks, etc.), including interacting with user client computing devices 390 (e.g., for viewing floor plans, and optionally associated images and / or other related information, such as by interacting with or executing a copy of the building information access system), and / or mobile image acquisition devices 360 (e.g., for acquiring images and / or other information of a building or other environment to be modeled), and / or other navigable devices 395 that optionally receive and use floor plans and optionally other generated information for navigation purposes (e.g., for use by a semi-autonomous or fully autonomous vehicle or other device). In other embodiments, some of the described functionalities may be combined in fewer computing systems to combine the BFPADUM system 340 and the building information access system in a single system or device, to combine the BFPADUM system 340 and the image acquisition functionality of device 360 ​​in a single system or device, to combine the ICA system 388 and the MIGM system 389 and the image acquisition functionality of device 360 ​​in a single system or device, to combine the BFPADUM system 340 and the ICA system 388 and the MIGM system 389 in a single system or device, to combine the BFPADUM system 340 and the ICA system 388 and the MIGM system 389 and the image acquisition functionality of device 360 ​​in a single system or device, and the like.

[0069] In the illustrated embodiment, an embodiment of the BFPADUM system 340 is executed in the memory 330 of the server computing system 300 to perform at least some of the described techniques, such as by using the processor 305 to configure the processor 305 and the computing system 300 to execute the software instructions of the system 340 in a manner that implements the described techniques. The illustrated embodiment of the BFPADUM system may include one or more components not shown to each perform a portion of the functionality of the BFPADUM system, such as in the manner discussed elsewhere herein, and the memory may further optionally execute one or more other programs 335. As a specific example, in at least some embodiments, a copy of the ICA and / or MIGM system may be executed as one of the other programs 335, for example, instead of or in addition to the ICA system 388 or MIGM system 389 on the server computing system 380, and / or a copy of the building information access system may be executed as one of the other programs 335. The BFPADUM system 340 may also store and / or retrieve various types of data on the memory 320 (e.g., in one or more databases or other data structures) during its operation, such as floor plans and other associated building information 324 (e.g., generated and saved 2.5D and / or 3D models, building and room dimensions, for use with associated floor plans, and / or other associated building information 324). Figure 1 The invention also provides the present invention with reference to the accompanying drawings, the accompanying drawings and / or annotation information, etc.), determined structural property data 328 (e.g., property values ​​at specific locations, associated visualizations, etc.), optionally one or more trained machine learning (ML) models 325, optionally generated floor plan adjacency graph data structures and / or associated vector embedding data structures 326, evaluation criteria 327 for determining building structure property values ​​and / or visualizations, various types of user information 322, and / or various types of optional additional information 329 (e.g., various analytical information related to the presentation or other use of one or more building interiors or other environments).

[0070] In addition, embodiments of the ICA system 388 and MIGM system 389 in the illustrated embodiment are executed in the memory 387 of the server computing system 380 to perform techniques related to generating panoramic images and floor plans of buildings, such as by using the processor 381 to execute software instructions of the systems 388 and / or 389 in a manner that configures the processor 381 and the computing system 380 to perform automated operations that implement these techniques. The illustrated embodiments of the ICA and MIGM systems may include one or more components not shown to perform portions of the functionality of the ICA and MIGM systems, respectively, and the memory may further optionally execute one or more other programs 383. The ICA system 388 or MIGM system 389 may also store and / or retrieve various types of data on the storage device 384 (e.g., in one or more databases or other data structures) during operation, such as video and / or image information 386 acquired for one or more buildings (e.g., for analysis to generate floor plans, to provide 360° video or images for display to a user of the client computing device 390, etc.), floor plan and / or other generated mapping information 387, and optional other information 385 (e.g., for use with associated floor plans). Figure 1 Additional images and / or annotation information used with the associated plane Figure 1 dimensions of buildings and rooms used together, various analytical information related to the presentation or other use of one or more building interiors or other environments, etc.). Figure 3 Not shown, the ICA and / or MIGM system may also store and use additional types of information, such as information about other types of buildings to be analyzed and / or provided to the BFPADUM system, about ICA and / or MIGM system operator users and / or end users, etc.

[0071] Some or all of the user client computing device 390 (e.g., mobile device), mobile image acquisition device 360, optional other navigable device 395, and other computing systems (not shown) may similarly include some or all of the same types of components shown for the server computing system 300. As a non-limiting example, the mobile image acquisition device 360 ​​is each shown to include one or more hardware CPUs 361, I / O components 362, memory and / or storage devices 367, one or more imaging systems 365, IMU hardware sensors 369 (e.g., for acquiring video and / or images, associated device movement data, etc.), and optional other components 364. In the illustrated example, one or both of a browser and one or more client applications 368 (e.g., applications dedicated to the BFPADUM system and / or ICA system and / or MIGM system) are executed in the memory 367 to participate in communications with the BFPADUM system 340, the ICA system 388, the MIGM system 389, and / or other computing systems. Although specific components are not illustrated for other navigable devices 395 or other computing devices / systems 390, it will be appreciated that they may include similar and / or additional components.

[0072] It should also be understood that Figure 3 The computing systems 300 and 380 and other systems and devices included in the are merely illustrative and are not intended to limit the scope of the invention. The system and / or device may instead each include multiple interactive computing systems or devices, and may be connected to other devices not specifically shown, including via Bluetooth communication or other direct communication, via one or more networks such as the Internet, via the Web, or via one or more dedicated networks (e.g., mobile communication networks, etc.). More generally, the device or other computing system may include any combination of hardware that can interact and perform the functions of the type described, optionally when programmed or otherwise configured with specific software instructions and / or data structures, including but not limited to desktop computers or other computers (e.g., input boards, tablet computers, etc.), database servers, network storage devices and other network devices, smart phones and other cellular phones, consumer electronic devices, wearable devices, digital music player devices, handheld gaming devices, PDAs, wireless phones, Internet devices, and various other consumer products including appropriate communication capabilities. Furthermore, in some implementations, the functionality provided by the illustrated BFPADUM system 340 may be distributed among various components, some of the described functionality of the BFPADUM system 340 may not be provided, and / or other additional functionality may be provided.

[0073] It should also be understood that, although various items are shown as being stored in memory or in storage devices when used, these items or parts thereof may be transferred between memory and other storage devices for the purpose of memory management and data integrity. Alternatively, in other embodiments, some or all of the software components and / or systems may be executed in memory on another device and communicate with the illustrated computing system via inter-computer communication. Therefore, in some embodiments, when configured by one or more software programs (e.g., by the BFPADUM system 340 executed on the server computing system 300, by the building information access system executed on the server computing system 300 or other computing systems / devices, etc.) and / or data structures, some or all of the described techniques may be performed by a hardware device including one or more processors and / or memories and / or storage devices. Such as by executing software instructions of one or more software programs and / or by storing such software instructions and / or data structures, and in order to perform algorithms as described in the flowcharts and other disclosures herein. Additionally, in some embodiments, some or all of the systems and / or components may be implemented or provided in other ways, such as by being comprised of one or more devices implemented in part or in whole in firmware and / or hardware (e.g., rather than devices implemented in whole or in part by software instructions that configure a particular CPU or other processor), including, but not limited to, one or more application specific integrated circuits (ASICs), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), etc. Some or all of the components, systems, and data structures may also be stored (e.g., as software instructions or structured data) on a non-transitory computer-readable storage medium, such as a hard disk or flash drive or other non-volatile storage device, volatile or non-volatile memory (e.g., RAM or flash RAM), network storage device, or portable media article (e.g., DVD disk, CD disk, optical disk, flash memory device, etc.) to be read by an appropriate drive or via an appropriate connection. In some embodiments, systems, components, and data structures may also be transmitted via generated data signals (e.g., as part of a carrier wave or other analog or digital propagation signal) on various computer-readable transmission media, including wireless-based and wire / cable-based media, and may take various forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). In other embodiments, such computer program products may also take other forms. Therefore, embodiments of the present disclosure may be implemented with other computer system configurations.

[0074] Figure 4A-4BAn exemplary embodiment of a flow chart for a Building Plan Attribute Determination / Usage Manager (BFPADUM) system routine 400 is shown. The routine may be executed, for example, Figure 1 BFPADUM system 140, Figure 3 BFPADUM system 340, and / or as referenced Figure 2E-2J and the BFPADUM system described elsewhere herein to perform automated operations related to determining information about structural characteristics and other attributes of a target building based on an analysis of existing building information, and using the determined building information in a variety of ways. Figure 4A-4B In an exemplary embodiment of the routine, floor plans are used for houses or other buildings, but in other embodiments, other types of data structures and analyses may be used for other types of structures or for non-structure locations, and the identified buildings and / or their floor plans may be used in other ways than discussed with respect to routine 400 (as discussed elsewhere herein). In addition, while exemplary embodiments of the routine may use adjacency graphs and / or vector embeddings and / or other specified criteria (e.g., search terms) as part of determining building structural attribute information (e.g., for one or more connectivity attributes) and / or identifying buildings that match or are otherwise similar to the information, other embodiments of the routine may use only one such type of information and / or may use other additional types of information and analyses (e.g., to identify matching properties).

[0075] The illustrated implementation of the routine begins at box 405, where information or instructions are received. The routine continues to box 407 to determine whether the instructions or other information received in box 405 indicate determining information related to building properties, such as which properties to track and / or which properties to associate with a particular type of user, and if not, continues to box 408 to retrieve information about the building properties to track. Otherwise, the routine continues to box 409 to analyze user activity information related to the building using one or more trained machine learning models to determine one or more building properties to track and / or determine the association of a particular building property with a particular user type based on those activities. In at least some embodiments and scenarios, the information retrieved or otherwise determined in box 408 or 409 can be specific to a particular user and / or user type (e.g., as shown in box 405), for example, whether the information subsequently determined in the routine is for such a user or users of such a user type.

[0076] After either block 408 or 409, the routine continues to block 410, where it is determined whether the instruction or other information received in block 405 is to determine building attribute data for the target building, and if not, proceeds to block 490. Otherwise, the routine continues to block 415 to retrieve available information about the target building. In block 420, the routine determines whether a building floor plan is available for the target building, and if so, continues to block 422 to retrieve the stored floor plan and any additional information about the building (e.g., building features determined from analyzing images captured at the building, information about the building from a public data source, etc.), and then continues to block 445. If instead it is determined in block 420 that a building floor plan is not available for the target building, the routine instead continues to execute blocks 425-440, including optionally retrieving any other existing building information (e.g., building images, from public sources, etc.) in block 425, and then retrieving a building image of the target building, if available, or if not executing the ICA routine to acquire such an image, in block 430. Figure 5 An example of such a routine is provided. After block 430, the routine executes the MIGM routine to determine the floor plan and additional information about the target building from the image, wherein Figure 6A-6B An example of such a routine is provided.

[0077] After either box 422 or 440, the routine continues to box 445 to analyze the floor plan and optionally other building data (e.g., one or more building images to determine height information for particular rooms or other areas) to determine accessibility attribute values ​​for one or more accessibility attributes for each of one or more locations in the target building (e.g., for each location in a grid covering part or all of the building and optionally surrounding properties) (e.g., as determined in box 408 or 409), such as by using one or more trained machine learning models and / or computer vision techniques, and as described in more detail elsewhere herein, covering some or all of the building and optionally surrounding properties). After box 445, the routine continues to box 455, where it analyzes the floor plan and optionally other building data (e.g., one or more building images) to determine viewshed values ​​(e.g., as determined in box 408 or 409) for one or more viewshed attributes for each of the one or more locations in the target building (e.g., for each location in a grid covering part or all of the building and optionally surrounding real estate), such as by using one or more trained machine learning models and / or computer vision techniques, and as described in more detail elsewhere herein. After box 455, the routine continues to box 465, where it analyzes the floor plan and optionally other building data (e.g., one or more building images) to determine connectivity values ​​for one or more connectivity attributes (e.g., as determined in box 408 or 409) for each of the one or more locations in the target building (e.g., for each location in a grid covering part or all of the building and optionally surrounding real estate), such as by using one or more trained machine learning models and / or computer vision techniques, and as described in more detail elsewhere herein. In at least some embodiments and scenarios, the determination in box 465 includes generating an adjacency graph for the building, the adjacency graph representing information about at least interconnected rooms and optionally other adjacent rooms, wherein at least some connectivity values ​​are determined based at least in part on analysis or other use of the generated adjacency graph.

[0078] After box 465, the routine continues to box 470, where the two optionally analyze the acquired building information to determine one or more additional building attributes and associated values ​​of the building to determine room features from the image analysis, identify objects and other structural elements, determine one or more subjective attributes using one or more corresponding trained neural networks, and associate any such additional attributes with corresponding rooms or other areas and with portions of the adjacency graph. After box 470, the routine continues to box 475, optionally generating one or more spectrum beddings to represent at least some of the adjacency graph information (e.g., using representation learning and one or more trained machine learning models), such as for subsequent use in comparing buildings and their attribute information. Although not shown in the exemplary embodiment, the routine can also determine other buildings similar to the current target building based at least in part on comparing their respective spectrum beddings and determining other buildings with minimum distances or other differences. After block 475, the routine continues to block 485 where it generates one or more visualizations or other visual representations for at least some of the determined building attributes, such as a heat map, text / numeric visual representation, etc. for each of one or more accessible attributes and / or viewshed attributes and / or connectivity attributes. After block 485, the routine continues to block 488 where it stores some or all of the information determined and generated in blocks 409-485.

[0079] If it is determined in box 410 that the information or instructions received in box 405 are not to determine building property data for a target building, the routine continues to box 490 to perform one or more other indicated operations, as appropriate. Such other operations may include, for example, receiving and responding to requests for previously identified building information (e.g., requests for such information for display on one or more client devices, requests for such information to provide it to one or more other devices for use in automatic navigation, etc.), training one or more neural networks or other machine learning models to perform the types of operations described herein, using machine learning techniques to learn properties and / or other features of the adjacency graph to be encoded in the corresponding vector embeddings generated (e.g., to allow subsequent automatic identification of optimal properties and / or other characteristics of building floor plans with properties that meet target criteria), obtaining and storing information about users of the routine (e.g., the current user's search and / or selection preferences, the user's role and / or user type and / or relevant criteria for determining building modifications), generating and storing representative information of the building (e.g., floor plans, adjacency graphs, vector embeddings, property value visualizations, etc.) for later use, etc.

[0080] After frame 488 or 490, routine continues to frame 495, to determine whether to continue, for example, until receiving explicit instruction to terminate, or only when receiving explicit instruction to continue. If it is determined to continue, routine returns to frame 405 to wait and receive additional instructions or information, and otherwise continues to frame 499 and ends.

[0081] Figure 5 An example flow chart of an implementation of an ICA (image capture and analysis) system routine 500 is shown. The routine may be implemented by, for example, Figure 1 ICA system 160, Figure 3 ICA system 388, and / or as described in Figure 2A-2J and the ICA systems described elsewhere herein to acquire 360° panoramic images and / or other images at an acquisition location within a building or other structure, e.g., for subsequent generation of associated floor plans and / or other mapping information. Although portions of the example routine 500 are discussed with respect to acquiring particular types of images at particular acquisition locations, it will be appreciated that this routine or similar routines may be used to acquire video (having video frame images) and / or other data (e.g., audio), either in lieu of or in addition to such panoramic or other stereo images. Furthermore, while the illustrated embodiments acquire and use information from the interior of a target building, it will be appreciated that other embodiments may perform similar techniques on other types of data, including information on non-building structures and / or on the exteriors of one or more target buildings of interest. Furthermore, some or all of the routines may be performed on a mobile device used by a user to acquire the image information, and / or some or all of the routines may be performed by a system remote from such a mobile device. In at least some embodiments, the image information may be acquired from a mobile device that is used by a user to acquire the image information. Figure 4A-4B Block 430 of routine 400 of the embodiment of the present invention calls routine 500, provides corresponding information from routine 500 to routine 400 as part of the implementation of block 430, and in such case, returns processing control to routine 400 after blocks 577 and / or 599. In other embodiments, routine 400 may perform additional operations in an asynchronous manner without waiting for such a return of processing control (e.g., performing other processing activities while waiting for corresponding information from routine 500 to be provided to routine 400).

[0082] The illustrated implementation of the routine begins at block 505, where instructions or information are received. At block 510, the routine determines whether the received instructions or information indicate acquisition of visual data and / or other data representative of a building interior (optionally based on information provided about one or more additional acquisition locations and / or other guiding acquisition instructions), and if not, continues to block 590. Otherwise, the routine proceeds to block 512 to receive an instruction to begin an image acquisition process at a first acquisition location (e.g., from a user of a mobile image acquisition device that will perform the acquisition process). After block 512, the routine proceeds to block 515 to perform an acquisition location image acquisition activity for acquiring a 360° panoramic image of an acquisition location of a target building interior of interest, such as via one or more fisheye lenses and / or non-fisheye rectilinear lenses on a mobile device, and providing horizontal coverage of at least 360° around a vertical axis, although other types of images and / or other types of data may be acquired in other embodiments. As a non-exclusive example, the mobile image acquisition device may be a rotating (scanning) panoramic camera equipped with a fisheye lens (e.g., with 180° horizontal coverage) and / or other lenses (e.g., with less than 180° horizontal coverage, such as a conventional lens or a wide-angle lens or an ultra-wide lens). The routine may also optionally obtain annotations and / or other information about the acquisition location and / or surroundings from the user, for example, for later use in presenting information about the acquisition location and / or surroundings.

[0083] After the completion of frame 515, the routine continues to frame 520 to determine whether there are more acquisition locations to acquire images, such as based on the corresponding information provided by the user of the mobile device and / or received in frame 505. In some embodiments, the ICA routine will only acquire a single image and then proceed to frame 577 to provide the image and corresponding information (e.g., before receiving additional instructions or information to acquire one or more next images at one or more next acquisition locations, the image and corresponding information are returned to the BFPADUM system and / or the MIGM system for further use). If there are more acquisition locations to acquire additional images at the current time, the routine continues to frame 522 to optionally start the capture of link information (e.g., acceleration data) during the movement of the mobile device along the path of travel away from the current acquisition location and toward the next acquisition location within the building. The captured link information may include additional sensor data recorded during such movement (e.g., from one or more IMUs or inertial measurement units, on the mobile device or otherwise carried by the user) and / or additional visual information (e.g., images, videos, etc.). Initiating the capture of such link information may be performed in response to an explicit instruction from a user of the mobile device or based on one or more automatic analyses of information recorded from the mobile device. In addition, in some embodiments, during movement to the next acquisition location, the routine may also optionally monitor the motion of the mobile device and provide one or more guidance prompts (e.g., to the user) regarding the motion of the mobile device, the quality of the sensor data and / or visual information being captured, the relevant lighting / environmental conditions, the desirability of capturing the next acquisition location, and any other appropriate aspects of capturing link information. Similarly, the routine may optionally obtain annotations and / or other information from the user regarding the path of travel, such as for later use in presenting information about the path of travel or the result of the panoramic image inter-connection link. In block 524, the routine determines that the mobile device has arrived at the next acquisition location (e.g., based on an instruction from the user, based on stopping the forward movement of the mobile device for at least a predetermined amount of time, etc.), serving as the new current acquisition location, and returns to block 515 to perform acquisition location image acquisition activities for the new current acquisition location.

[0084] If it is determined in block 520 that there are not any more acquisition locations at which to acquire image information of the current building or other structure at the current time, the routine proceeds to block 545 to optionally pre-process the acquired 360° panoramic images prior to subsequent use thereof (e.g., for generating associated mapping information, for providing information about features of a room or other enclosed area, etc.) in order to produce images of a particular type and / or particular format (e.g., performing an equirectangular projection on each such image having straight vertical data, such as the sides of a typical rectangular door frame or a typical border between two adjacent walls that remains straight, and having straight horizontal data, such as the top of a typical rectangular door frame or a border between a wall and a floor that remains straight at the horizontal midline of the image, but gradually curves in a convex manner relative to the horizontal midline in the equirectangular projected image as the distance from the horizontal midline in the image increases. In block 577, the images and any associated generated or obtained information are stored for later use and optionally provided to one or more recipients (e.g., provided to block 430 of routine 400 if called from this block). Figure 6A-6B One example of a routine for generating a floor plan representation of a building interior from generated panoramic information is shown.

[0085] If it is determined in block 510 that the instruction or other information received in block 505 is not to acquire images and other data representing the interior of a building, the routine continues to block 590 to perform any other indicated operations, as appropriate, to configure parameters to be used in various operations of the system (e.g., based at least in part on information specified by a user of the system (e.g., based at least in part on information specified by a user of the system, such as a user of a mobile device capturing one or more building interiors, an operator user of the ICA system, etc.), respond to requests to generate and store information (e.g., identify one or more sets of interconnected linked panoramic images, each of which represents a building or a portion of a building that matches one or more specified search criteria), to identify one or more panoramic images that match one or more specified search criteria, to generate and store panoramic image connections between panoramic images of buildings or other structures (e.g., for each panoramic image, determining a direction within the panoramic image toward one or more other acquisition locations of one or more other panoramic images, so as to enable later display of an arrow or other visual representation with the panoramic image, and for each such determined direction from the panoramic image, enabling an end user to select one of the displayed visual representations to switch to display of another panoramic image at another acquisition location corresponding to the selected visual representation), to obtain and store other information about users of the system, to perform any housekeeping tasks, etc.

[0086] After frame 577 or frame 590, routine proceeds to frame 595 to determine whether to continue, for example, until receiving explicit instruction to terminate, or only when receiving explicit instruction to continue. If it is determined to continue, routine returns to frame 505 to wait for additional instructions or information, and if not, proceeds to step 599 and ends.

[0087] Figure 6A-6B An exemplary embodiment of a flow chart of a MIGM (Mapping Information Generation Manager) system routine 600 is shown. The routine may be executed by, for example Figure 1 MIGM system 160, Figure 3 MIGM system 389, and / or as regards Figure 2A-2J and the MIGM system described elsewhere herein to determine a room shape of a room (or other defined area) by analyzing information from one or more images captured in the room (e.g., one or more 360° panoramic images), to generate a partial or complete floor plan of a building or other defined area based at least in part on the one or more images of the area and optionally additional data captured by a mobile computing device, and / or to generate other mapping information of a building or other defined area based at least in part on the one or more images of the area and optionally additional data captured by a mobile computing device. Figure 6A-6B In the example of , the determined room shape of the room can be a 2D room shape representing the locations of the walls of the room or a 3D fully closed combination of planar surfaces representing the locations of the walls, ceiling, and floor of the room, and the generated mapping information for the building (e.g., a house) can include a 2D floor plan and / or a 3D computer model floor plan. However, in other embodiments, other types of room shapes and / or mapping information can be generated and used in other ways, including for other types of structures and defined areas, as discussed elsewhere herein. In at least some embodiments, the room shape can be generated from Figure 4A-4B Block 440 of routine 400 of the embodiment of the present invention calls routine 600, provides corresponding information from routine 600 to routine 400 as part of the implementation of such block 440, and in such case, returns processing control to routine 400 after blocks 688 and / or 699. In other embodiments, routine 400 may perform additional operations in an asynchronous manner without waiting for such a return of processing control (e.g., proceed to block 445 once corresponding information from routine 600 is provided to routine 400, perform other processing activities while waiting for corresponding information from routine 600 to be provided to routine 400, etc.).

[0088] The illustrated implementation of the routine begins at box 605, where information or instructions are received. The routine continues to box 610 to determine whether image information is already available for analysis of one or more rooms (e.g., for some or all of the indicated building, such as based on one or more such images received in box 605 as previously generated by the ICA routine), or whether such image information is currently to be acquired. If it is determined in box 610 that some or all of the image information is currently to be acquired, the routine continues to box 612 to acquire such information, optionally waiting for one or more users or devices to move throughout one or more rooms of the building and acquire panoramic images or other images at one or more acquisition locations in the one or more rooms (e.g., multiple acquisition locations in each room of the building), optionally together with metadata information about the acquisition and / or interconnect information related to movement between acquisition locations, as discussed in more detail elsewhere herein. Implementation of box 612 may, for example, include calling an ICA system routine to perform such an activity, wherein Figure 5 One exemplary embodiment of an ICA system routine for performing such image acquisition is provided. If it is determined in block 610 that an image is not currently being acquired, the routine continues to block 615 to obtain one or more existing panoramic images or other images from one or more acquisition locations in one or more rooms (e.g., multiple images acquired at multiple acquisition locations including at least one image and acquisition location in each room of a building), optionally along with metadata information about the acquisition and / or interconnected information related to movement between acquisition locations, such as, in some cases, has been provided in block 605 along with corresponding instructions.

[0089] After either box 612 or 615, the routine continues to box 620 where it determines whether mapping information is generated that includes a linked set of target panoramic images (or other images) for a building or other group of rooms (sometimes referred to as a "virtual tour" to enable an end user to move from any one of the images in the linked set to one or more other images linked to the starting current image, including, in some embodiments, by selecting a user-selectable control for each such other linked image displayed with the current image, optionally by overlaying a visual representation of such user-selectable controls and corresponding inter-image directions over the visual data of the current image, and similarly moving from the next image to one or more additional images linked to the next image, and so on), and if so, continues to box 625. The routine in box 625 selects at least some pairs of images (e.g., based on images of the pair having overlapping visual content), and based on shared visual content and / or based on other captured linked interconnection information (e.g., movement information) associated with the images of the pair (whether moving directly from an acquisition location of one image in a pair to an acquisition location of another image in the pair, or moving between these starting and ending acquisition locations via one or more other intermediate acquisition locations of other images). The routine in box 625 may further optionally use relative orientation information of at least the paired images to determine the global relative positions of some or all images relative to each other in a common coordinate system, and / or generate inter-image links and corresponding user-selectable controls as described above. Additional details on creating such linked sets of images are included elsewhere herein.

[0090] Following block 625, or if it is determined in block 620 that the instructions or other information received in block 605 is not to determine a linked set of images, the routine continues to block 635 to determine whether the instructions received in block 605 indicate to generate other mapping information (e.g., a floor plan) for the indicated building, and if so, the routine continues to perform some or all of blocks 637-685 to do so, and otherwise continues to block 690. In block 637, the routine optionally obtains additional information about the building, such as from activities performed during acquisition and optionally analysis of the images, and / or from one or more external sources (e.g., online databases, information provided by one or more end users, etc.). Such additional information may include, for example, the exterior dimensions and / or shape of the building, additional imagery and / or annotated information acquired corresponding to specific locations on the exterior of the building (e.g., surrounding the building and / or for other structures on the same property, from one or more overhead locations, etc.), additional imagery and / or annotated information acquired corresponding to specific locations within the building (optionally, for locations different from the location at which the panoramic image or other image was acquired), etc.

[0091] After box 637, the routine continues to box 640 to select the next room (starting with the first one) for which one or more images (e.g., 360° panoramic images) acquired in the room are available, and analyze the visual data of the images for the room to determine the room shape (e.g., by determining at least the wall locations), optionally together with determining uncertainty information about the walls and / or other portions of the room shape, and optionally including identifying other wall and floor and ceiling elements (e.g., wall structural elements / features such as windows, doorways and stairs and other inter-room wall openings and connecting passages, wall boundaries between a wall and another wall and / or ceiling and / or floor, etc.) and their locations within the determined room shape of the room. In some embodiments, room shape determination may include determining a 2D room shape using the boundaries of the walls to one another and at least one of the floor or ceiling (e.g., using one or more trained machine learning models), while in other embodiments, room shape determination may be performed in other ways (e.g., by generating a 3D point cloud of some or all of the room walls and optionally the ceiling and / or floor. For example, by analyzing at least visual data of the panoramic image and optionally additional data captured by the image acquisition device or an associated mobile computing device, optionally using one or more of SFM (Structure from Motion) or SLAM (Simultaneous Localization and Mapping) or MVS (Multi-View Stereo) analysis. In addition, the activity of block 645 Initial pose information for each of those panoramic images (e.g., provided with acquisition metadata for the panoramic images), and / or additional metadata for each panoramic image (e.g., acquisition height information for a camera device or other image acquisition device used to acquire the panoramic images relative to the floor and / or ceiling) may also be optionally determined and used. Additional details regarding determining room shapes and identifying additional information for rooms are included elsewhere herein. After box 640, the routine continues to box 645, where it determines whether there are more rooms for which room shapes have been determined based on the images acquired in those rooms, and if so, returns to box 640 to select the next such room for which room shape has been determined.

[0092] If it is determined in block 645 that there are no more rooms for which room shapes are generated, the routine continues to block 660 to determine whether to further generate at least a partial floor plan of the building (e.g., based at least in part on the determined room shapes from block 640, and optionally further information about how the determined room shapes are positioned relative to each other). If not, such as when only one or more room shapes are determined without generating further mapping information for the building (e.g., the room shape of a single room is determined based on one or more images acquired in the room by the ICA system), the routine continues to block 688. Otherwise, the routine continues to block 665 to retrieve one or more room shapes (e.g., the room shapes generated in block 645), or otherwise obtain one or more room shapes for the rooms of the building (e.g., based on manually provided input), whether 2D or 3D room shapes, and then continues to block 670. In block 670, the routine uses the one or more room shapes to create an initial floor plan (e.g., an initial 2D floor plan using 2D room shapes and / or an initial 3D floor plan using 3D room shapes), such as a partial floor plan that includes one or more room shapes but less than all room shapes of the building, or a complete floor plan that includes all room shapes of the building. If multiple room shapes are present, the routine further determines in block 670 the positioning of the room shapes relative to each other, such as by using visual overlap between images from multiple acquisition locations to determine the relative positions of those acquisition locations and room shapes surrounding those acquisition locations, and / or by using other types of information (e.g., using inter-room pathways connecting rooms, optionally applying one or more constraints or optimizations, etc.). In at least some embodiments, the routine in block 670 further refines some or all of the room shapes by generating a binary segmentation mask covering the relatively positioned room shapes, extracting polygons representing the outlines or contours of the segmentation mask, and separating the polygons into refined room shapes. Such a floor plan may include, for example, relative position and shape information for various rooms without providing any actual size information for the individual rooms or the building as a whole, and may also include multiple linked or associated sub-maps of the building (e.g., to reflect different floors, levels, sections, etc.). The routine also optionally associates the locations of doors, wall openings, and other identified wall elements on the floor plan.

[0093] After box 670, the routine optionally performs one or more steps 680 to 685 to determine and associate additional information with the floor plan. In box 680, the routine optionally estimates the dimensions of some or all rooms, such as from analysis of the images and / or their acquisition metadata or from general dimensional information obtained for the exterior of the building, and associates the estimated dimensions with the floor plan. It should be understood that if sufficiently detailed dimensional information is available, a building map, blueprint, etc. can be generated from the floor plan. After box 680, the routine continues to box 683 to optionally associate further information with the floor plan (e.g., with a specific room or other location within the building), such as additional existing images with specified locations and / or annotation information. In box 685, if the room shape from box 645 is not a 3D room shape, the routine also optionally estimates the height of the walls in some or all rooms, such as based on analysis of the image and optionally the size of known objects in the image, and height information about the camera when the image was acquired, and uses the height information to generate a 3D room shape for the room. The routine further optionally uses the 3D room shapes (whether from block 640 or block 685) to generate a 3D computer model floor plan of the building, wherein the 2D and 3D floor plans are associated with each other. In other embodiments, only the 3D computer model floor plan may be generated and used (including, if desired, by using a horizontal slice of the 3D computer model floor plan to provide a visual representation of the 2D floor plan).

[0094] Following box 685, or if it is determined in box 660 that a floor plan is not to be determined, the routine continues to box 688 to store the determined room shape and / or the generated mapping information and / or other generated information, optionally provide some or all of the information to one or more recipients (e.g., to box 440 of routine 400 if called from that box), and optionally further use some or all of the determined and generated information to provide a generated 2D floor plan and / or 3D computer model floor plan for display on one or more client devices and / or to one or more other devices, for use in automating navigation of such devices and / or associated vehicles or other entities, to similarly provide and use information about the determined room shape and / or linked collection of panoramic images and / or about determined additional information about room contents and / or pathways between rooms, etc.

[0095] If it is determined in box 635 that the information or instructions received in box 605 are not to generate mapping information for the indicated building, the routine continues to box 690 to perform one or more other indicated operations as appropriate. Such other operations may include, for example, receiving and responding to requests for previously generated floor plans and / or previously determined room shapes and / or other generated information (e.g., requests for such information used by the ILDM system, requests for such information to be displayed on one or more client devices, requests for such information to be provided to one or more other devices for use in automatic navigation, etc.), obtaining and storing information about the building for use in subsequent operations (e.g., information about the size, number or type of rooms, total square length, other buildings adjacent or nearby, vegetation adjacent or nearby, external images, etc.), etc.

[0096] After frame 688 or frame 690, routine continues to frame 695, to determine whether to continue, for example, until receiving explicit instruction to terminate, or only when receiving explicit instruction to continue. If it is determined to continue, routine returns to frame 605 to wait and receive additional instructions or information, and otherwise continues to frame 699 and ends.

[0097] Although not targeted Figure 6A-6BThe automated operations shown in the exemplary embodiments of the present invention are described, but in some embodiments, a human user can further help facilitate some operations of the MIGM system, such as providing one or more types of input to an operator user and / or end user of the MIGM system, which one or more types are further used in subsequent automated operations. As non-exclusive examples, such a human user can provide one or more types of input as follows: provide input to help link a collection of images so as to provide input in box 625, which is used as part of the automated operation of that box (e.g., specifying or adjusting an initially automatically determined orientation between one or more pairs of images, specifying or adjusting an initially automatically determined final global position of some or all images relative to each other, etc.); provide input in box 637, which is used as part of subsequent automated operations, such as information of one or more of the shown types about a building; provide input with respect to box 640 used as part of subsequent automated operations to specify or adjust initially automatically determined element locations and / or estimated room shapes, and / or manually combine information from multiple estimated room shapes of a room (e.g., providing input to box 670 used as part of subsequent operations to specify or adjust the initial automatically determined location of the room shape within the generated floor plan and / or to specify or adjust the initial automatically determined room shape itself within such floor plan; providing input to one or more of boxes 680 and 683 and 685 used as part of subsequent operations to specify or adjust initial automatically determined information of one or more types discussed with respect to those boxes; and / or specifying or adjusting initial automatically determined pose information (either initial pose information or subsequently updated pose information) for one or more of the panoramic images; etc. Additional details regarding embodiments in which one or more human users provide input that is further used in additional automated operations of the BFPADUM system are included elsewhere herein.

[0098] Figure 7A-7B An exemplary embodiment of a flow chart for a building information access system routine 700 is shown. The routine may be executed by, for example, Figure 1 The building information access client computing device 175 and its software system (not shown), Figure 3Executed by a client computing device 390 of the invention, and / or a building information access viewer or presentation system as described elsewhere herein, to receive and display information about one or more building structure attributes and their values ​​at one or more locations in a building (e.g., to overlay information about such attribute values ​​on a displayed floor plan of the building and / or a displayed visual representation of a property on which the building is located, such as for one or some or all of the locations on the displayed floor plan and / or visual representation of the property), to receive and display generated floor plans and / or other mapping information (e.g., determined room structure layouts / shapes, etc.), for a defined area optionally including a visual indication of one or more determined image acquisition locations, to obtain and display information about images that match one or more indicated target images, to display additional information (e.g., images) associated with a particular acquisition location in the mapping information, to obtain and display guided acquisition instructions provided by the BFPADUM system and / or other sources (e.g., regarding other images acquired during the acquisition period and / or regarding an associated building, such as a portion of a displayed GUI), to obtain and display an explanation or other description of a match between two or more buildings or properties, etc. Figure 7A-7B In the example shown, the mapping information presented is for a building (such as the interior of a house), but in other embodiments, other types of mapping information may be presented for other types of buildings or environments and used in other ways, as discussed elsewhere herein.

[0099] The illustrated implementation of the routine begins at block 705, where an instruction or information is received. At block 710, the routine determines whether the instruction or information received at block 705 is to display determined information for a target building, and if so, continues to block 715. The routine in block 715 determines whether the instruction or information received in block 705 is to select a target building using specified criteria (e.g., based at least in part on the indicated building), and if not, continues to block 725 to obtain an indication from the user of a target building to use (e.g., based on a current user selection, such as from a displayed list or other user selection mechanism; based on information received in block 705; etc.). Otherwise, if it is determined in block 715 that the target building is selected from the specified criteria (e.g., based at least in part on the indicated building), the routine continues to block 720 where it obtains an indication of one or more search criteria to be used, such as from the current user selection or as indicated in the information or instructions received in block 705, and then searches the stored information about buildings to determine one or more buildings that meet the search criteria or otherwise obtains an indication of one or more such matching buildings, such as information currently or previously generated by the BFPADUM system (an example of the operation of such a system will be referred to in detail below). Figure 4A-4B 705 ), while in other embodiments the routine may instead present multiple candidate buildings that satisfy the search criteria (e.g., in a ranked order based on the degree of match) and receive a user-selected target building from the multiple candidates.

[0100] After either box 720 or box 725, the routine continues to box 730 to retrieve building information for the target building, the building information including a floor plan of the building and / or other generated mapping information (e.g., one or more determined building structural attributes having associated values ​​for one or more building or property locations and optionally associated generated visualizations or other visual representations, a set of interlinked images for use as part of a virtual tour), and optionally associated link information indicating surrounding locations for the building interior and / or building exterior, and / or information regarding one or more generated explanations or other descriptions as to why the target building was selected as matching specified criteria (e.g., based in part or in whole on one or more other indicated buildings). In block 732, the routine then determines whether to display visualizations and / or other information about one or more determined building structure attributes (e.g., based on input received in block 705), and if not, continues to block 735 to select an initial view of the building information retrieved in block 730 (e.g., a view of a floor plan, a specific room shape, a specific image, etc., optionally along with a generated explanation or other description of why the target building was selected as a match (if such information is available), and / or optionally along with information about possible building modifications). After block 735, the routine continues to block 740 to display or otherwise present a current view of the retrieved information, and waits for a user selection in block 745. After the user selection in block 745, if it is determined in block 750 that the user selection corresponds to adjusting the current view of the current target building (e.g., changing one or more aspects of the current view), the routine continues to block 755 to update the current view in accordance with the user selection, and then returns to block 740 to update the displayed or otherwise presented information accordingly. User selections and corresponding updates to the current view may include, for example, changing the information displayed for one or more building structure attributes, and / or displaying or otherwise presenting a piece of associated linked information selected by the user (e.g., a particular image associated with the displayed visual indication of the determined acquisition location so that the associated linked information is overlaid on at least some of the previously displayed information; a particular other image that is linked to the current image and selected from the current image using a user-selectable control overlaid on the current image to represent the other image; etc.), and / or changing how the current view is displayed (e.g., zooming in or out; rotating the information as appropriate; selecting a new portion of the plan view to be displayed or otherwise presented, such as where some or all of the new portion was not previously visible, or where instead the new portion is a subset of the previously visible information; etc.).If it is determined in box 750 that the user chooses not to display further information about the current target building (e.g., display information for another building, end the current display operation, etc.), the routine continues to box 795, and if the user selection involves such further operation, returns to box 705 to perform the operation selected by the user.

[0101] If it is determined in block 710 that the instruction or other information received in block 705 is not to present information representing a building, the routine continues to block 782 to determine whether the instruction or other information received in block 705 is to perform a search for one or more matching buildings, such as based on building structural attribute information and / or other building attributes (e.g., based on an indication of the building structural attribute information and / or other building attributes using the indicated building), and if so, continues to block 982 to receive an indication of one or more target building attributes and / or other matching criteria (e.g., based on the information received in block 705). In block 984, the routine then identifies one or more buildings that match the specified criteria and displays or otherwise provides information about the identified buildings. After block 984, the routine continues to block 745. If it is determined in box 782 that the instructions or other information received in box 705 do not perform a search for one or more matching buildings, the routine continues to box 760 to determine whether the instructions or other information received in box 705 corresponds to identifying other images (if any) corresponding to one or more indicated target images, and if so, continues to boxes 765-770 to perform such activities. In particular, in box 765, the routine receives an indication of one or more target images for matching (e.g., from the information received in box 705 or based on one or more current interactions with the user) and one or more matching criteria (e.g., the amount of visual overlap), and in box 770, identifies one or more other images (if any) that match the indicated target images, such as by interacting with the ICA and / or MIGM system to obtain the other images. The routine then displays or otherwise provides information about the identified other images in box 770, so as to provide information about them as part of the search results, to display one or more of the identified other images, etc. If it is determined in block 760 that the instructions or other information received in block 705 are not to identify other images corresponding to one or more indicated target images, the routine continues to block 775 to determine whether the instructions or other information received in block 705 correspond to guided acquisition instructions to obtain and provide information about one or more indicated target images (e.g., most recently acquired images) during the image acquisition period, if so, continue to block 780, and otherwise continue to block 790. In block 780, the routine obtains information about one or more types of guided acquisition instructions, such as by interacting with the ICA system, and displays or otherwise provides information about the guided acquisition instructions in block 780, such as by overlaying the guided acquisition instructions on the partial plan view and / or on the most recently acquired image in a manner discussed in more detail elsewhere herein.

[0102] If it is determined in block 732 to display visualizations and / or other information about one or more determined building structure attributes, the routine proceeds to blocks 905-984. Specifically, in block 905, the routine determines whether the instructions or other information received in block 705 indicate displaying data for one or more accessibility attributes to provide information about at least one type of accessibility of the building (e.g., for at least one indicated user type), and if so, the routine proceeds to block 910 to optionally obtain an indication of one or more types of accessibility data (e.g., one or more accessibility attributes (if multiple accessibility attributes are available)), and / or one or more types of visualizations or other presentation techniques, and otherwise select one or more available or default accessibility attributes and visualization types. In block 915, the routine then retrieves or generates one or more visualizations of the indicated or otherwise selected (e.g., heat maps, text and / or numeric data, tabular data, etc.) for the indicated or otherwise selected one or more accessibility attributes of the one or more types. In box 920, the routine then displays or otherwise provides a current view of one or more visualizations for one or more accessibility attributes, optionally by overlaying or otherwise displaying the information on or adjacent to a displayed building floor plan and / or a visual representation of the property on which the building is located.

[0103] After block 920, or if it is determined in block 905 that the instructions or other information received in block 705 do not display data for one or more accessibility attributes, the routine continues to block 925 to determine whether the instructions or other information received in block 705 display data for one or more viewshed attributes to provide information about at least one of visibility or spaciousness of the building, and if so, continues to block 930 to optionally obtain an indication of one or more types of viewshed data (e.g., one or more viewshed attributes if multiple viewshed attributes are available) and / or an indication of one or more types of visualization or other presentation techniques, and otherwise select one or more available or default viewshed attributes and visualization types. In block 935, the routine then retrieves or generates one or more visualizations of one or more types, such as indicated or otherwise selected (e.g., heat maps, text and / or numeric data, tabular data, etc.) for the indicated or otherwise selected one or more viewshed attributes. In box 940, the routine then displays or otherwise provides a current view of one or more visualizations of one or more viewshed properties, optionally by overlaying or otherwise displaying information on or adjacent to a displayed visual representation of a building floor plan and / or property on which the building is located, and optionally simultaneously with data associated with one or more accessible properties if so performed in box 920.

[0104] After block 940, or if it is determined in block 925 that the instructions or other information received in block 705 do not display data for one or more viewshed attributes, the routine continues to block 945 to determine whether the instructions or other information received in block 705 displays data for one or more connectivity attributes to provide information about room-to-room connectivity and, optionally, other adjacency information for rooms of the building, and if so, continues to block 950 to optionally obtain an indication of one or more types of connectivity data (e.g., one or more connectivity attributes if multiple connectivity attributes are available) and / or an indication of one or more types of visualization or other presentation techniques, and otherwise select one or more available or default connectivity attributes and visualization types. In block 955, the routine then retrieves or generates one or more visualizations of the indicated or otherwise selected one or more types (e.g., heat maps, adjacency maps, visual indications of paths and / or straight lines between two or more locations, text and / or numerical data, tabular data, etc.) for the indicated or otherwise selected one or more connectivity attributes. In box 960, the routine then displays or otherwise provides a current view of one or more visualizations for one or more connectivity attributes, optionally by overlaying or otherwise displaying the information on or adjacent to a displayed visual representation of a building floor plan and / or the property on which the building is located, and optionally performed concurrently with displayed data associated with one or more accessibility attributes (if so performed in box 920) and / or concurrently with displayed data associated with one or more viewshed attributes (if so performed in box 940).

[0105] After block 960, or if it is determined in block 945 that the instructions or other information received in block 705 do not display data for one or more connectivity attributes, the routine continues to block 965 to determine whether the instructions or other information received in block 705 display data for one or more other building attributes, and if so, continues to block 970 to optionally obtain an indication of one or more other types of building attribute data and / or one or more types of visualizations or other presentation techniques, and otherwise select one or more available or default building attributes and visualization types. In block 975, the routine then retrieves or generates one or more visualizations of one or more types, as indicated or otherwise selected (e.g., heat maps, text and / or numeric data, tabular data, etc.) for the indicated or otherwise selected one or more other building attributes. In box 980, the routine then displays or otherwise provides a current view of one or more visualizations for one or more other building properties, optionally by overlaying or otherwise displaying the information on or adjacent to a displayed visual representation of a building floor plan and / or the property on which the building is located, and optionally concurrently with displayed data associated with one or more accessibility properties (if so performed in box 920) and / or concurrently with displayed data associated with one or more viewshed properties (if so performed in box 920) and / or concurrently with displayed data associated with one or more connectivity properties (if so performed in box 960).

[0106] After box 980, or if it is determined in box 965 that the instruction or other information received in box 705 does not display data for one or more other building attributes, the routine continues to box 745 to await one or more user selections regarding information displayed in one or more of boxes 920, 940, 960, and 980.

[0107] In box 790, the routine continues to perform other indicated operations, as appropriate, to configure parameters to be used in various operations of the system (e.g., based at least in part on information specified by users of the system, such as obtaining users of mobile devices within one or more buildings, operator users of BFPADUM and / or MIGM systems, etc., including for personalizing the display of information for particular users based on the preferences of particular users), to obtain and store other information about users of the system, to respond to requests for generated and stored information, to perform any housekeeping tasks, etc.

[0108] After box 770 or box 780 or box 790, or if it is determined in box 750 that the user selection does not correspond to the current building, the routine proceeds to box 795 to determine whether to continue, such as until an explicit indication to terminate is received, or only if an explicit indication to continue is received. If it is determined to continue (including whether the user makes a selection related to a new building to be presented in box 745), the routine returns to box 705 to wait for additional instructions or information (or if the user makes a selection related to a new building to be presented in box 745, then directly proceed to box 730), and if not, proceed to step 799 and end.

[0109] Non-exclusive exemplary embodiments described herein are further described in the following clauses.

[0110] A01. A computer-implemented method for one or more computing devices to perform an automated operation, comprising:

[0111] Obtaining, by one or more computing devices, information about a house having a plurality of rooms, the information comprising: a floor plan of the house having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms; and a plurality of house images acquired at a plurality of locations of the house;

[0112] analyzing, by the one or more computing devices, the floor plan and the image of the premises to determine, for each of a plurality of locations at the premises, an associated first value for an accessibility attribute at the location to represent one or more accessibility characteristics of an indicated user type located at the location;

[0113] analyzing, by the one or more computing devices, the floor plan and the image of the premises to determine, for each of the plurality of locations, an associated second value for the location for a viewshed attribute to represent one or more visibility characteristics from the location including an amount of the plurality of rooms visible from the location;

[0114] analyzing, by the one or more computing devices, the floor plan and the room image to determine, for each of a plurality of locations, an associated third value for a connectivity attribute for the location to represent one or more inter-room connectivity features from the location;

[0115] generating, by the one or more computing devices, a first visualization of the accessibility attribute for the associated first values ​​of the plurality of locations, a second visualization of the viewshed attribute for the associated second values ​​of the plurality of locations, and a third visualization of the connectivity attribute for the associated third values ​​of the plurality of locations;

[0116] presenting, by the one or more computing devices, the generated first visualization of the accessibility attribute overlaid on a first visual representation of the floor plan of the house;

[0117] presenting, by the one or more computing devices, the generated second visualization of the viewshed attribute superimposed on a second visual representation of the floor plan of the house; and

[0118] Presenting, by the one or more computing devices, a third visualization of the connectivity attribute generated overlaid on a third visual representation of the floor plan of the house.

[0119] A02. A computer-implemented method for one or more computing devices to perform an automated operation, comprising:

[0120] Obtaining, by one or more computing devices, information about an existing building having a plurality of rooms, the information comprising a floor plan of the existing building having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms;

[0121] analyzing, by the one or more computing devices, the floor plan to determine, for each of a plurality of locations at the existing building, an associated value for an accessibility attribute at the location to represent one or more accessibility characteristics for an indicated user type located at the location;

[0122] generating, by the one or more computing devices, a visualization of the accessibility attributes of the associated values ​​for the plurality of locations; and

[0123] The generated visualization of the accessibility attribute is presented, by the one or more computing devices, in association with the plurality of locations on the visual representation of the floor plan of the existing building.

[0124] A03. A computer-implemented method for one or more computing devices to perform an automated operation, comprising:

[0125] obtaining, by the one or more computing devices, information about an existing building having a plurality of rooms, the information comprising a floor plan of the existing building having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms;

[0126] analyzing, by the one or more computing devices, the floor plan, and determining, for each of a plurality of locations at the existing building, an associated value for an accessibility attribute of the location to represent one or more user accessibility characteristics at the location;

[0127] generating, by the one or more computing devices and for the accessibility attribute, one or more visual representations of the associated values ​​for the plurality of locations; and

[0128] The one or more visual representations of the accessibility attributes associated with the plurality of locations are provided, by the one or more computing devices.

[0129] A04. A computer-implemented method for one or more computing devices to perform an automated operation, comprising:

[0130] obtaining information about an existing building having a plurality of rooms, the information comprising a floor plan of the existing building having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms;

[0131] analyzing the floor plan and determining, for each of a plurality of locations at the existing building, an associated value for a viewshed attribute for the location to represent one or more visibility characteristics from the location, the one or more visibility characteristics including an amount of the plurality of rooms visible from the location;

[0132] generating a visual representation of the viewshed attribute for the associated values ​​of the plurality of locations; and

[0133] The generated visual representations of the viewshed attributes associated with the plurality of locations on the floor plan of the existing building are presented.

[0134] A05. A computer-implemented method as described in any of clauses A01-A04, wherein obtaining information about the house includes: generating the floor plan by analyzing visual data of the house image through the one or more computing devices to determine the at least two-dimensional house shape and the relative positions of the multiple houses, and determining additional building attributes of the house.

[0135] A06. The computer-implemented method of clause A05, further comprising:

[0136] generating, by the one or more computing devices and using information about the house, an adjacency graph representing the house and storing a first value for the association of the accessibility attribute, a second value for the association of the connectivity attribute, a third value for the association of the connectivity attribute, and information about a plurality of additional building attributes, the adjacency graph having a plurality of nodes, each node being associated with one of the plurality of rooms and storing information about one or more of the attributes corresponding to the associated room, and the adjacency graph further having a plurality of edges, each of the plurality of edges being between two nodes of the plurality of nodes and representing adjacency of the two nodes in the house of the associated rooms;

[0137] generating, by the one or more computing devices and using representation learning, a vector embedding for the house, the vector embedding representing information from the adjacency graph and encoding the information in a vector format including at least some of the plurality of properties of the house;

[0138] determining, by the one or more computing devices, a plurality of other houses that match the house by comparing the vector embedding of the house to additional vector embeddings of the plurality of other houses, the additional vector embeddings encoding information in a vector format including at least some additional attributes of the plurality of other houses, and measuring a distance between the vector embedding of the house and each of the additional vector embeddings of the plurality of other houses based at least in part on an amount of structural difference between the house and the other houses; and

[0139] Information about one or more of the determined plurality of other homes is provided, via the one or more computing devices.

[0140] A07. A computer-implemented method as described in any of clauses A05-A06, further comprising:

[0141] receiving, by the one or more computing devices, one or more specified criteria;

[0142] determining, by the one or more computing devices, that the home matches the one or more specified criteria based at least in part on the associated first value of the accessibility attribute and the associated second value of the viewshed attribute and the associated third value of the connectivity attribute and one or more additional building attributes; and

[0143] Information about the premises is provided, by the one or more computing devices, in response to the received one or more specified criteria.

[0144] A08. A computer-implemented method as described in any of clauses A05-A07, wherein determining the first value of the accessibility attribute, the second value of the viewshed attribute, and the third value of the connectivity attribute includes using one or more trained machine learning models to analyze at least the floor plan.

[0145] A09. A computer-implemented method as described in any of clauses A01-A08, wherein the plurality of locations includes at least one location in each of the plurality of rooms, wherein the indicated user type includes at least one of a user in a wheelchair or a user with limited mobility, and wherein, for each of the plurality of locations, determining the associated value for the accessibility attribute comprises the steps of: multiple times evaluating any structural obstacles to the building to travel between an entrance to the existing building and the location; evaluating at least one of the lengths or heights of multiple structural elements of the building in a defined area surrounding the location; and measuring distances and height changes from the location to multiple other building locations.

[0146] A10. A computer-implemented method as described in any of clauses A01-A09, further comprising: generating a grid of locations spanning the floor plan, wherein the plurality of locations include the locations of the grid, wherein the generated visualization is a heat map that represents the associated values ​​of the accessibility attributes of the locations using one of a plurality of visual attributes for each of the locations of the grid, and wherein presenting the generated visualization of the accessibility attributes includes overlaying the generated visualization on the visual representation of the floor plan.

[0147] A11. A computer-implemented method as described in any of clauses A01-A10, further comprising:

[0148] generating, by the one or more computing devices and based at least in part on analyzing the floor plan, associated additional values ​​for each of the plurality of locations for a viewshed property to represent one or more visibility characteristics from the location including an amount of the plurality of rooms visible from the location, and generating additional visualizations of the viewshed property of the associated additional values ​​for the plurality of locations;

[0149] generating, by the one or more computing devices and based at least in part on analyzing the floor plan, additional values ​​for each of a plurality of locations of a connectivity property to represent one or more inter-room connectivity features from the locations, and generating a further visualization of the connectivity property of the additional values ​​for the plurality of locations; and

[0150] The additional visualization of the viewshed attributes associated with the plurality of locations and the further visualization of the connectivity attributes associated with the plurality of locations are presented, by the one or more computing devices.

[0151] A12. A computer-implemented method as described in any of clauses A01-A11, wherein the associated values ​​of the accessibility attributes representing the one or more user accessibility features are determined with respect to an indicated user type, wherein generating one or more visual representations of the associated values ​​for the multiple locations includes generating visualizations of those associated values, and wherein providing the one or more visual representations associated with the multiple locations includes overlaying the generated visualizations on an additional displayed visual representation of the floor plan.

[0152] A13. A computer-implemented method as described in clause A12, wherein the plurality of locations includes at least one location in each of the plurality of rooms, wherein the indicated user type includes at least one of a user in a wheelchair or a user with limited mobility, and wherein, for each of the plurality of locations, determining the associated value of the accessibility attribute includes: evaluating any structural barriers of the existing building to travel between an entrance of the existing building and the location; and evaluating at least one of the lengths or heights of a plurality of structural elements of the existing building in a defined area surrounding the location.

[0153] A14. A computer-implemented method as described in any of clauses A12-A13, wherein the plurality of locations includes at least one location in each of the plurality of rooms, wherein the indicated user type includes at least one of a user in a wheelchair or a user with limited mobility, and wherein, for each of the plurality of locations, determining the associated value for the accessibility attribute includes measuring distance and altitude changes from the location to a plurality of other building locations.

[0154] A15. A computer-implemented method as described in any of clauses A12-A14, further comprising: generating a grid of locations across the floor plan, wherein the plurality of locations includes the locations of the grid, and wherein the generated visualization is a heat map using one of a plurality of visual attributes for each of the locations of the grid to represent the associated values ​​of the accessibility attribute of the locations.

[0155] A16. A computer-implemented method as described in any of clauses A01-A15, further comprising:

[0156] determining, by the one or more computing devices and based at least in part on further analyzing the floor plan, an associated additional value for each of the plurality of locations of a viewshed attribute to represent one or more visibility characteristics from the location including an amount of the plurality of rooms visible from the location;

[0157] generating, by the one or more computing devices and for the viewshed attribute, additional visual representations of the associated additional values ​​for the plurality of locations; and

[0158] The additional visual representations of the viewshed attributes associated with the plurality of locations are presented, by the one or more computing devices.

[0159] A17. A computer-implemented method as described in claim A16, wherein the plurality of locations includes at least one location in each of the plurality of rooms, wherein determining the associated additional values ​​of the plurality of locations includes at least one of the following: performing the further analysis of the floor plan by using a machine learning model trained to determine the value of the viewshed attribute; or performing the further analysis of the floor plan by using computer vision techniques to determine the line of sight from each of the plurality of locations in multiple directions from the location, and wherein the additional visual representation is a heat map that represents the associated additional value of the viewshed attribute of the location using one of the plurality of visual attributes of each of the plurality of locations.

[0160] A18. A computer-implemented method as described in any of clauses A01-A17, further comprising:

[0161] determining, by the one or more computing devices and based at least in part on further analyzing the floor plan, an associated additional value for each of a plurality of locations for a connectivity attribute to represent one or more inter-room connectivity characteristics from the location;

[0162] generating, by the one or more computing devices and for the connectivity attribute, an additional visual representation of the associated additional values ​​for the plurality of locations; and

[0163] The additional visual representations of the connectivity attributes associated with the plurality of locations are presented, by the one or more computing devices.

[0164] A19. A computer-implemented method as described in clause A18, wherein the plurality of locations includes a location in each of the plurality of rooms, wherein determining the associated additional values ​​of the plurality of locations includes at least one of the following: performing further analysis of the floor plan using a machine learning model trained to determine the value of the connectivity attribute; or performing further analysis of the floor plan using computer visualization techniques to identify walls that separate the plurality of rooms and to identify openings between the plurality of rooms; and determining, for each of the plurality of locations, at least one additional room that is adjacent to the room in which the location is located and separated from the room by at least one of the identified walls or openings, and wherein the additional visual representation is overlaid on the floor plan and includes a node for each of the plurality of rooms and includes an adjacency graph for edges for at least room-to-room connections between two rooms via at least one identified opening.

[0165] A20. A computer-implemented method as described in any of clauses A01-A19, wherein the plurality of locations includes at least one location in each of the plurality of rooms, and wherein determining the associated values ​​of the plurality of locations includes at least one of the following: performing an analysis of the floor plan by using a machine learning model trained to determine the values ​​of the accessibility attributes; or performing an analysis of the floor plan by using computer vision techniques to determine structural obstacles visible from each of the plurality of locations.

[0166] A21. A computer-implemented method as described in any of clauses A01-A20, further comprising: determining, by the one or more computing devices and before analyzing the floor plan, associations between multiple user types and multiple accessibility attributes using a trained machine learning model, the trained machine learning model evaluating data about multiple interactions involving multiple users of multiple types and multiple buildings, wherein the multiple interactions include at least one of acquiring interactions performed by at least some of the multiple users of at least some of the multiple buildings; or reconstructing interactions of at least some of the multiple users involving at least some of the multiple buildings; or browsing interactions of at least some of the multiple users utilizing online information about at least some of the multiple buildings,

[0167] and wherein, for each of the plurality of accessibility attributes, determining the associated value of the accessibility attribute is performed,

[0168] wherein providing the one or more visual representations comprises presenting the one or more visual representations to the indicated user, and

[0169] Wherein said generating and said providing are performed for one of said plurality of accessibility attributes that is associated with one of said plurality of user types selected for said indicated user.

[0170] A22. A computer-implemented method as described in any of clauses A01-A21, wherein, for each of the multiple accessibility attributes, determining the associated value of the accessibility attribute is performed, wherein, for one of the multiple accessibility attributes, generating the one or more visual representations is performed, and generating the one or more visual representations includes generating a heat map, wherein the method further includes: generating one or more additional visual representations for one or more additional accessibility attributes of the multiple accessibility attributes, the one or more additional accessibility attributes including one or more textual representations of at least one of the following items: one or more areas of the existing building; or one or more amounts of space within the existing building; or one or more ratios of the amount of space within the existing building to the total space, and wherein providing the one or more visual representations includes: presenting the heat map overlaid on the floor plan, and presenting the one or more textual representations simultaneously with presenting the heat map.

[0171] A23. A computer-implemented method as described in any of clauses A01-A22, further comprising: identifying the existing building as matching one or more specified criteria based at least in part on the associated values ​​of the accessibility attributes of the multiple locations, and wherein, in response to receiving the one or more specified criteria, providing the one or more visual representations is performed.

[0172] A24. The computer-implemented method of clause A23, wherein the one or more specified criteria include one or more additional building attributes that the existing building is determined to include.

[0173] A25. A computer-implemented method as described in any of clauses A01-A24, wherein providing the one or more visual representations also includes performing at least one of the following: evaluating the existing building against one or more defined evaluation criteria based at least in part on the associated values ​​of the accessibility attributes for the multiple locations, or recommending the existing building based at least in part on the associated values ​​of the accessibility attributes for the multiple locations, or determining user-specific information including the existing building based at least in part on the associated values ​​of the accessibility attributes for the multiple locations, or generating summary information about the existing building based at least in part on the associated values ​​of the accessibility attributes for the multiple locations.

[0174] A26. A computer-implemented method as described in any of clauses A01-A25, further comprising:

[0175] further analyzing the floor plan to determine, for each of the plurality of locations at the existing building, an additional associated value for an accessibility attribute of the location to represent one or more user accessibility characteristics at the location;

[0176] generating, for the accessibility attribute, one or more additional visual representations of the additional associated values ​​for the plurality of locations; and

[0177] The one or more additional visual representations of the accessibility attributes associated with the plurality of locations are provided.

[0178] A27. A computer-implemented method as described in any of clauses A01-A26, wherein the plurality of locations include locations of a grid overlaying the floor plan, wherein the generated visualization is a heat map using one of a plurality of visual attributes for each of the locations of the grid to represent the associated values ​​of the field of view attribute for the locations, and wherein presenting the generated visual representation includes overlaying the heat map on a displayed visual representation of the floor plan.

[0179] A28. A computer-implemented method as described in any of clauses A01-27, wherein the plurality of locations includes at least one location in each of the plurality of rooms, and wherein determining the associated values ​​of the plurality of locations includes at least one of the following: analyzing the floor plan by using a machine learning model trained to determine the value of the viewshed attribute; or analyzing the floor plan by using computer vision techniques to determine lines of sight from each of the plurality of locations in multiple directions from the location.

[0180] A29. A computer-implemented method comprising the steps of performing a plurality of automated operations that implement techniques substantially as disclosed herein.

[0181] B01. A non-transitory computer-readable medium having stored executable software instructions and / or other storage contents, wherein the stored executable software instructions and / or other storage contents enable one or more computing systems to perform automatic operations to implement the method of any of clauses A01-A29.

[0182] B02. A non-transitory computer-readable medium having stored executable software instructions and / or other storage contents, wherein the stored executable software instructions and / or other storage contents enable one or more computing systems to perform automatic operations that implement the techniques substantially as disclosed herein.

[0183] C01. One or more computing systems comprising one or more hardware processors and one or more memories having stored instructions, which, when executed by at least one of the one or more hardware processors, cause the one or more computing systems to perform automatic operations of a method implementing any of clauses A01-A29.

[0184] C02. One or more computing systems comprising one or more hardware processors and one or more memories having stored instructions that, when executed by at least one of the one or more hardware processors, cause the one or more computing systems to perform automated operations that implement techniques substantially as disclosed herein.

[0185] D01. A computer program adapted to perform the method of any one of clauses A01-A29 when said computer program is run on a computer.

[0186] Aspects of the present disclosure are described herein with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure.It should be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer-readable program instructions.It will be further understood that in some implementations, the functions provided by the above routines can be provided in an alternative manner, such as splitting between more routines, or merging into fewer routines.Similarly, in some implementations, the routines shown can provide more or less functions than described, such as when other routines shown lack or include such functions respectively, or when the amount of functions provided changes.In addition, although various operations can be shown as being performed in a particular manner (e.g., serial or parallel, or synchronous or asynchronous) and / or in a particular order, in other implementations, operations can be performed in other orders and other ways.Any data structure discussed above can also be constructed in different ways, such as by dividing a single data structure into multiple data structures and / or by merging multiple data structures into a single data structure. Similarly, in some implementations, illustrated data structures may store more or less information than depicted, such as when other illustrated data structures lack or include such information, respectively, or when the amount or type of information stored varies.

[0187] It will be appreciated from the above that, although specific embodiments are described herein for purposes of illustration, various modifications may be made without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited except by the corresponding claims and the elements cited by those claims. In addition, although certain aspects of the present invention may be presented in the form of certain claims at certain times, the inventors contemplate various aspects of the present invention in the form of any available claims. For example, although only some aspects of the present invention may be described as being embodied in a computer-readable medium at a particular time, other aspects may also be so embodied.

Claims

1. A computer-implemented method comprising: Obtaining, by one or more computing devices, information about an existing building having a plurality of rooms, the information comprising a floor plan of the existing building having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms; analyzing, by the one or more computing devices, the floor plan to determine, for each of a plurality of locations at the existing building, an associated value for an accessibility attribute at the location to represent one or more accessibility characteristics for an indicated user type located at the location; generating, by the one or more computing devices, a visualization of the accessibility attributes of the associated values ​​for the plurality of locations; as well as The generated visualization of the accessibility attribute is presented, by the one or more computing devices, in association with the plurality of locations on the visual representation of the floor plan of the existing building.

2. The computer-implemented method of claim 1, wherein: The plurality of locations include at least one location in each of the plurality of rooms, wherein the indicated user type includes at least one of a user in a wheelchair or a user with limited mobility, and wherein, for each of the plurality of locations, determining the associated value for the accessibility attribute includes the steps of: assessing any structural barriers to the building multiple times to travel between an entrance to the existing building and the location; assessing at least one of the length or height of multiple structural elements of the building in a defined area surrounding the location; and measuring distance and height changes from the location to multiple other building locations.

3. The computer-implemented method of claim 1 , further comprising: Generating a grid of locations across the floor plan, wherein the plurality of locations includes the location of the grid, wherein the generated visualization is a heat map representing the associated values ​​of the accessibility attribute of the location using one of a plurality of visual attributes for each of the locations of the grid, and wherein presenting the generated visualization of the accessibility attribute includes overlaying the generated visualization on the visual representation of the floor plan.

4. The computer-implemented method of claim 1 , further comprising: generating, by the one or more computing devices and based at least in part on analyzing the floor plan, associated additional values ​​for each of the plurality of locations for a viewshed property to represent one or more visibility characteristics from the location including an amount of the plurality of rooms visible from the location, and generating additional visualizations of the viewshed property of the associated additional values ​​for the plurality of locations; generating, by the one or more computing devices and based at least in part on analyzing the floor plan, additional values ​​for each of a plurality of locations of a connectivity property to represent one or more inter-room connectivity features from the locations, and generating a further visualization of the connectivity property of the additional values ​​for the plurality of locations; and The additional visualization of the viewshed attributes associated with the plurality of locations and the further visualization of the connectivity attributes associated with the plurality of locations are presented, by the one or more computing devices.

5. A non-transitory computer-readable medium having stored content, the stored content causing one or more computing devices to perform an automated operation, the automated operation comprising at least: obtaining, by the one or more computing devices, information about an existing building having a plurality of rooms, the information comprising a floor plan of the existing building having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms; analyzing, by the one or more computing devices, the floor plan, and determining, for each of a plurality of locations at the existing building, an associated value for an accessibility attribute of the location to represent one or more user accessibility characteristics at the location; generating, by the one or more computing devices and for the accessibility attribute, one or more visual representations of the associated values ​​for the plurality of locations; as well as The one or more visual representations of the accessibility attributes associated with the plurality of locations are provided, by the one or more computing devices.

6. The non-transitory computer readable medium of claim 5, wherein: The associated values ​​of the accessibility attributes representing the one or more user accessibility features are determined with respect to an indicated user type, wherein generating one or more visual representations of the associated values ​​for the multiple locations includes generating visualizations of those associated values, and wherein providing the one or more visual representations associated with the multiple locations includes overlaying the generated visualizations on an additional displayed visual representation of the floor plan.

7. The non-transitory computer readable medium of claim 6, wherein: The plurality of locations include at least one location in each of the plurality of rooms, wherein the indicated user type includes at least one of a user in a wheelchair or a user with limited mobility, and wherein, for each of the plurality of locations, determining the associated value of the accessibility attribute includes at least one of: measuring distances and height changes from the location to a plurality of other building locations; or assessing any structural barriers of the existing building to travel between an entrance of the existing building and the location; and assessing at least one of the lengths or heights of a plurality of structural elements of the existing building in a defined area surrounding the location.

8. The non-transitory computer readable medium of claim 6, wherein: The stored content includes software instructions that, when executed by the one or more computing devices, cause the one or more computing devices to perform further automatic operations, the further automatic operations including: generating a grid of locations across the floor plan, wherein the plurality of locations includes the locations of the grid, and wherein the generated visualization is a heat map that uses one of a plurality of visual attributes for each of the locations of the grid to represent the associated values ​​of the accessibility attributes of the locations.

9. The non-transitory computer readable medium of claim 5, wherein: The automatic operation also includes: determining, by the one or more computing devices and based at least in part on further analyzing the floor plan, an associated additional value for each of the plurality of locations of a viewshed attribute to represent one or more visibility characteristics from the location including an amount of the plurality of rooms visible from the location; generating, by the one or more computing devices and for the viewshed attribute, additional visual representations of the associated additional values ​​for the plurality of locations; and The additional visual representations of the viewshed attributes associated with the plurality of locations are presented, by the one or more computing devices.

10. The non-transitory computer readable medium of claim 9, wherein: The plurality of locations includes at least one location in each of the plurality of rooms, wherein determining the associated additional values ​​for the plurality of locations includes at least one of: performing the further analysis of the floor plan by using a machine learning model trained to determine the value of the viewshed attribute; or performing the further analysis of the floor plan by using computer vision techniques to determine lines of sight from each of the plurality of locations in multiple directions from the location, and wherein the additional visual representation is a heat map representing the associated additional value of the viewshed attribute for the location using one of the plurality of visual attributes for each of the plurality of locations.

11. The non-transitory computer readable medium of claim 5, wherein: The automatic operation also includes: determining, by the one or more computing devices and based at least in part on further analyzing the floor plan, an associated additional value for each of a plurality of locations for a connectivity attribute to represent one or more inter-room connectivity characteristics from the location; generating, by the one or more computing devices and for the connectivity attribute, an additional visual representation of the associated additional values ​​for the plurality of locations; and The additional visual representations of the connectivity attributes associated with the plurality of locations are presented, by the one or more computing devices.

12. The non-transitory computer readable medium of claim 11, wherein: The plurality of locations includes a location in each of the plurality of rooms, wherein determining the associated additional values ​​for the plurality of locations includes at least one of: performing further analysis of the floor plan using a machine learning model trained to determine the value of the connectivity attribute; or performing further analysis of the floor plan using computer vision techniques to identify walls that separate the plurality of rooms and to identify openings between the plurality of rooms; and determining, for each of the plurality of locations, at least one additional room that is adjacent to the room in which the location is located and separated from the room by at least one of the identified walls or openings, and wherein the additional visual representation is overlaid on the floor plan and includes a node for each of the plurality of rooms and includes an adjacency graph for edges of at least room-to-room connections between two rooms via at least one identified opening.

13. The non-transitory computer readable medium of claim 5, wherein: The plurality of locations includes at least one location in each of the plurality of rooms, and wherein determining the associated values ​​of the plurality of locations includes at least one of: performing an analysis of the floor plan using a machine learning model trained to determine the values ​​of the accessibility attributes; or performing an analysis of the floor plan using computer vision techniques to determine structural obstacles visible from each of the plurality of locations.

14. The non-transitory computer readable medium of claim 5, wherein: The automatic operation also includes: determining, by the one or more computing devices and prior to analyzing the floor plan, associations between a plurality of user types and a plurality of accessibility attributes using a trained machine learning model, the trained machine learning model evaluating data regarding a plurality of interactions involving a plurality of users of a plurality of types and a plurality of buildings, wherein the plurality of interactions include at least one of acquiring interactions by at least some of the plurality of users of at least some of the plurality of buildings; or reconstructing interactions involving at least some of the plurality of users of at least some of the plurality of buildings; or browsing interactions of at least some of the plurality of users utilizing online information regarding at least some of the plurality of buildings, and wherein, for each of the plurality of accessibility attributes, determining the associated value of the accessibility attribute is performed, wherein providing the one or more visual representations comprises presenting the one or more visual representations to the indicated user, and Wherein said generating and said providing are performed for one of said plurality of accessibility attributes that is associated with one of said plurality of user types selected for said indicated user.

15. The non-transitory computer readable medium of claim 5, wherein: For each of the multiple accessibility attributes, determining the associated value of the accessibility attribute is performed, wherein, for one of the multiple accessibility attributes, generating the one or more visual representations is performed, and generating the one or more visual representations includes generating a heat map, wherein the automatic operation also includes: generating one or more additional visual representations for one or more additional accessibility attributes of the multiple accessibility attributes, the one or more additional accessibility attributes including one or more textual representations of at least one of the following items: one or more areas of the existing building; or one or more amounts of space within the existing building; or one or more ratios of the amount of space within the existing building to the total space, and wherein providing the one or more visual representations includes: presenting the heat map overlaid on the floor plan, and presenting the one or more textual representations simultaneously with presenting the heat map.

16. The non-transitory computer readable medium of claim 5, wherein: The automatic operation also includes: identifying the existing building as matching one or more specified criteria based at least in part on the associated values ​​of the accessibility attributes of the multiple locations, wherein providing the one or more visual representations is performed in response to receiving the one or more specified criteria, and wherein the one or more specified criteria include one or more additional building attributes that the existing building is determined to include.

17. A system comprising: one or more hardware processors of one or more computing devices; as well as One or more memories having stored instructions, which, when executed by at least one of the one or more hardware processors, cause at least one of the one or more computing devices to perform an automatic operation, the automatic operation comprising at least: obtaining information about an existing building having a plurality of rooms, the information comprising a floor plan of the existing building having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms; analyzing the floor plan and determining, for each of a plurality of locations at the existing building, an associated value for a viewshed attribute for the location to represent one or more visibility characteristics from the location, the one or more visibility characteristics including an amount of the plurality of rooms visible from the location; generating a visual representation of the viewshed attribute for the associated values ​​of the plurality of locations; and The generated visual representations of the viewshed attributes associated with the plurality of locations on the floor plan of the existing building are presented.

18. The system of claim 17, wherein: The stored instructions include software instructions that, when executed by the at least one hardware processor, cause the at least one computing device to perform further automatic operations, the automatic operations including: further analyzing the floor plan to determine, for each of the plurality of locations at the existing building, an additional associated value for an accessibility attribute of the location to represent one or more user accessibility characteristics at the location; generating, for the accessibility attribute, one or more additional visual representations of the additional associated values ​​for the plurality of locations; and The one or more additional visual representations of the accessibility attributes associated with the plurality of locations are provided.

19. The system of claim 17, wherein: The plurality of locations comprises locations of a grid overlaying the floor plan, wherein the generated visualization is a heat map using one of a plurality of visual attributes for each of the locations of the grid to represent the associated values ​​of the viewshed attribute for the locations, wherein presenting the generated visual representation comprises overlaying the heat map on a displayed visual representation of the floor plan, and wherein determining the associated values ​​for the plurality of locations comprises at least one of: performing analysis of the floor plan using a machine learning model trained to determine the values ​​of the viewshed attribute; or performing analysis of the floor plan using computer vision techniques to determine lines of sight from each of the plurality of locations in a plurality of directions from the locations.

20. A computer-implemented method comprising: Obtaining, by one or more computing devices, information about a house having a plurality of rooms, the information comprising: a floor plan of the house having information about the plurality of rooms, the information about the plurality of rooms comprising at least two-dimensional room shapes and relative positions of the plurality of rooms; and a plurality of house images acquired at a plurality of locations of the house; analyzing, by the one or more computing devices, the floor plan and the image of the premises to determine, for each of a plurality of locations at the premises, an associated first value for an accessibility attribute at the location to represent one or more accessibility characteristics of an indicated user type located at the location; analyzing, by the one or more computing devices, the floor plan and the image of the premises to determine, for each of the plurality of locations, an associated second value for the location for a viewshed attribute to represent one or more visibility characteristics from the location including an amount of the plurality of rooms visible from the location; analyzing, by the one or more computing devices, the floor plan and the room image to determine, for each of a plurality of locations, an associated third value for a connectivity attribute for the location to represent one or more inter-room connectivity features from the location; generating, by the one or more computing devices, a first visualization of the accessibility attribute for the associated first values ​​of the plurality of locations, a second visualization of the viewshed attribute for the associated second values ​​of the plurality of locations, and a third visualization of the connectivity attribute for the associated third values ​​of the plurality of locations; presenting, by the one or more computing devices, the generated first visualization of the accessibility attribute overlaid on a first visual representation of the floor plan of the house; presenting, by the one or more computing devices, the generated second visualization of the viewshed attribute superimposed on a second visual representation of the floor plan of the house; and Presenting, by the one or more computing devices, a third visualization of the connectivity attribute generated overlaid on a third visual representation of the floor plan of the house.

Images